Information processing device and method

The information processing device and method address the issue of multipath reflections in 3D modeling by deriving a recommended polarization direction based on the object's shape to minimize specular reflections, improving image matching and texture quality in 3D modeling.

WO2024228328A1PCT designated stage expired Publication Date: 2024-11-07SONY GROUP CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/014971
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-01
Filing Date
2024-04-15
Publication Date
2024-11-07

Smart Images

  • Figure JP2024014971_07112024_PF_FP_ABST
    Figure JP2024014971_07112024_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure relates to an information processing device and a method that can suppress background reflections due to multipath. During polarization imaging, a recommended polarization direction of radiated light radiated from a light-emitting unit is derived in accordance with the shape of an object in three-dimensional space, and operation is guided so that a polarization direction of the radiated light approaches the recommended polarization direction. The polarization imaging is an imaging method in which an object onto which radiated light that has been polarized in a first direction is radiated is imaged by an imaging unit via a configuration that polarizes in a second direction. This second direction is a direction orthogonal to the first direction or a direction parallel to the first direction. The recommended polarization direction is a direction at which a mirror-reflected component of the radiated light mirror-reflected by a plurality of surfaces of the object and entering the imaging unit is at least average or less. The present disclosure can be applied to, for example, information processing devices, electronic apparatuses, information processing methods, and programs.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that are capable of suppressing glare due to multipath.

[0002] Conventional 3D modeling methods for 3D objects, such as photogrammetry, involve capturing images of the 3D object from multiple angles and generating 3D data based on the captured images. However, with this method, if the captured images contain specular reflection components (mirror reflection components) from the 3D object, there is a risk of errors in matching between viewpoints or reduced texture quality.

[0003] In response to this, there are imaging methods that use polarization sensors or polarization cameras to capture images of 3D objects (see, for example, Patent Documents 1 and 2). These imaging methods can obtain more accurate material information for 3D objects. However, because the ambient light of the 3D object cannot be controlled, it is sometimes impossible to sufficiently remove specular reflection components from the captured image.

[0004] Therefore, a method has been devised in which a 3D object is imaged by irradiating it with flash light polarized in a specific direction. For example, by irradiating a 3D object with flash light polarized in a first direction and then imaging the 3D object through a polarizing filter polarized in a second direction perpendicular to the first direction, the specular reflection component in the captured image can be further suppressed.

[0005] International Publication No. WO 2022 / 259466 International Publication No. WO 2019 / 069536

[0006] However, even with this imaging method, if the flash light is reflected off multiple surfaces and enters the imaging unit (so-called multi-path occurs), there is a risk of a phenomenon known as reflection, in which the colors or patterns of other surfaces are projected onto the surface of the 3D object.

[0007] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress glare due to multipath.

[0008] An information processing device according to one aspect of the present technology includes a recommended polarization direction derivation unit that derives a recommended polarization direction of illumination light emitted from a light-emitting unit in accordance with the shape of an object in three-dimensional space during polarization imaging, and a guide unit that guides an operation to bring the polarization direction of the illumination light closer to the recommended polarization direction, wherein the polarization imaging is an imaging method in which the object illuminated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction perpendicular to the first direction or a direction parallel to the first direction, and the recommended polarization direction is a direction in which the specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and enters the imaging unit is at least below average.

[0009] An information processing method according to one aspect of the present technology derives a recommended polarization direction of illumination light emitted from a light-emitting unit in accordance with the shape of an object in three-dimensional space during polarization imaging, and guides an operation for bringing the polarization direction of the illumination light closer to the recommended polarization direction, wherein the polarization imaging is an imaging method in which the object illuminated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction perpendicular to the first direction or a direction parallel to the first direction, and the recommended polarization direction being a direction in which the specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and incident on the imaging unit is at least below average.

[0010] In an information processing device and method according to one aspect of the present technology, when performing polarization imaging, a recommended polarization direction of the light emitted from the light-emitting unit is derived according to the shape of an object in three-dimensional space, and operations for bringing the polarization direction of the light emitted closer to the recommended polarization direction are provided.

[0011] 1 is a diagram for explaining an overview of photogrammetry; 2 is a diagram showing an example of cross-polarized light imaging; 3 is a diagram showing an example of multipath imaging; 4 is a diagram showing an example of brightness unevenness; 5 is a diagram showing a method of guiding manipulation of polarization direction; 6 is a diagram showing an example of polarization direction with respect to an incident surface; 7 is a diagram showing an example of reflectance; 8 is a diagram showing an example of specular reflection; 9 is a diagram showing an example of guidance display; 10 is a diagram showing an example of guidance display; 11 is a diagram showing an example of a normal map; 12 is a diagram showing an example of plane derivation; 13 is a block diagram showing an example of the main configuration of an imaging device; 14 is a block diagram showing an example of the main configuration of a recommended polarization direction derivation unit; 15 is a flowchart showing an example of the flow of imaging processing; 16 is a flowchart showing an example of the flow of polarization direction guidance processing; 17 is a flowchart showing an example of the flow of recommended polarization direction derivation processing; 18 is a flowchart showing an example of the flow of polarization direction guidance processing; 19 is a block diagram showing an example of the main configuration of a computer;

[0012] The following describes modes for carrying out the present disclosure (hereinafter referred to as embodiments). The description will be given in the following order: 1. Literature supporting technical content and technical terminology 2. 3D modeling and polarization imaging 3. Polarization direction operation guide 4. First embodiment (imaging device) 5. Supplementary notes

[0013] <1. Literature, etc. supporting technical content and technical terminology> The scope of what is disclosed in the present technology includes not only the content described in the embodiments, but also the content described in the following patent documents, etc. that were publicly known at the time of filing, and the content of other documents referenced in the following patent documents.

[0014] Patent Document 1: (described above) Patent Document 2: (described above)

[0015] In other words, the contents of the above-mentioned patent documents and the contents of other documents referenced in the above-mentioned patent documents are also used as the basis for determining the support requirements.

[0016] <2. 3D Modeling and Polarimetric Imaging> <3D Modeling> Conventionally, methods for 3D modeling of 3D objects with three-dimensional shapes, such as photogrammetry, involve capturing images of the 3D object from multiple directions and generating 3D data based on the multiple captured images. With this method, the three-dimensional position of the 3D object can be determined in a triangulation-like manner by capturing images of the same location from multiple viewpoints. Therefore, matching (detection of the same position on the 3D object) is performed between the multiple captured images.

[0017] Photogrammetry is a technique for reconstructing highly accurate three-dimensional models using the principles of triangulation from multiple images taken from various viewpoints. In this specification, the term "accuracy" in reference to 3D data (3D models) refers not only to the reproducibility (accuracy, definition, etc.) of the three-dimensional shape of the target 3D object, but also to the reproducibility (accuracy, definition, etc.) of the texture applied to the surface of the 3D model. For example, cameras 11-1 to 11-5 shown in FIG. 1 capture images of a 3D object 10 from multiple viewpoints to obtain multiple captured images. These captured images are then used to perform processes called SfM (Structure from Motion) and MVS (Multiview Stereo), followed by post-processing such as meshing and texturing to generate 3D data 15.

[0018] In SfM, for example, corresponding points are searched for between captured images, the position and orientation of the camera are derived using epipolar constraints, and the positions of each corresponding point in three-dimensional space are identified by triangulation based on the camera position and orientation. In this specification, these points in three-dimensional space are also referred to as three-dimensional points. In other words, three-dimensional points corresponding to each corresponding point are identified. The entire three-dimensional point cloud identified in this way is then optimized by bundle adjustment.

[0019] In MVS, for example, a dense corresponding point search is further performed using the 3D point group derived as described above, and 3D points are added.

[0020] As described above, photogrammetry performs a global optimization calculation called bundle adjustment to minimize errors, which allows it to obtain very accurate results, but it also places a heavy burden on the calculations.Furthermore, because it is based on geometric calculations rather than physical measurements, in principle, the higher the resolution of the images used, the more accurate the model can be restored.

[0021] <Influence of Specular Reflection Component> For example, when specular reflection (mirror reflection) occurs on a 3D object due to lighting conditions or other factors, the brightness of the portion of the 3D object where the specular reflection occurs may become saturated in the captured image. In such cases, matching between captured images may become difficult and may fail. Furthermore, even if the brightness does not become saturated, the specularly reflected light (specular reflection component or specular reflection component) travels in a specific direction (a limited direction). Therefore, the specular reflection component may cause the color, brightness, etc. of the 3D object to differ between captured images from each viewpoint. This may result in a failure to match between captured images. In this way, the specular reflection component may reduce the accuracy (precision) of matching between captured images.

[0022] Furthermore, in a captured image, the color, brightness, etc. of a 3D object may change locally due to the specular reflection component. In other words, the color, brightness, etc. of the part of the 3D object where the specular reflection occurs may differ from the surrounding area (i.e., the color, brightness, etc. may change from the color, brightness, etc. of the 3D object in its original state without the specular reflection component). In other words, there is a risk that the quality of the texture of the 3D object obtained from the captured image may be reduced.

[0023] In general, it is preferable that a 3D model can be used under any lighting conditions (i.e., the lighting conditions for the 3D model can be changed). Therefore, it is desirable that the texture of the 3D model does not include the effects of specific lighting conditions. However, the specular reflection component described above appears in the captured image as, for example, a "gloss" that is brighter than the surroundings. In other words, the effects of specific lighting conditions are included in the texture. As a result, the texture may not match the lighting conditions under which the 3D model is placed, which could reduce the sense of realism. In this sense, too, there is a risk that the quality of the texture of the 3D object obtained from the captured image may be reduced.

[0024] <Polarization Imaging> In response to this, imaging methods have been proposed that use a polarization sensor or polarization camera to capture images of 3D objects, such as those described in Patent Documents 1 and 2. Polarization sensors and polarization cameras have a configuration (e.g., a polarizing filter) that polarizes light from a subject entering an imaging unit in a predetermined direction, and generate a captured image by receiving the polarized component of the light from the subject. In this specification, capturing an image by receiving the polarized component of light from a subject is referred to as polarization imaging. This imaging method can obtain more accurate material information for a 3D object. However, because this method cannot control the ambient light that strikes the 3D object, it may not be possible to sufficiently remove the specular reflection component from the captured image.

[0025] Therefore, a method of capturing an image by irradiating a 3D object with flash light (irradiation light) polarized in a predetermined direction has been devised. For example, as shown in Fig. 2, a light-emitting unit 21 emits light, and the light from the light-emitting unit 21 is polarized in a predetermined direction (first direction) by a polarizing filter 22 or the like. The polarized irradiation light is then irradiated onto a 3D object 23, with the reflected light being dominant. The reflected light is then polarized in a predetermined direction (second direction) by a polarizing filter 24 or the like, and the transmitted light is received by an imaging unit 25, generating a captured image.

[0026] In this case, the specular reflection component of the reflected light from the 3D object is polarized in a first direction (the component polarized in the first direction is dominant). Therefore, by setting the second direction perpendicular to the first direction (the relative angle is 90 degrees), the specular reflection component incident on the imaging unit can be further reduced. This allows for more accurate matching between captured images. This allows for the generation of a more accurate 3D model. Furthermore, the original color (diffuse reflection component) and pattern of the 3D object can be more accurately detected, and the effects of specific lighting conditions on the captured image can be more effectively suppressed. This allows for more effective suppression of degradation in texture quality.

[0027] In this specification, this type of polarization imaging method (i.e., a polarization imaging method in which the polarization direction on the light-emitting unit 21 side (the polarization direction of the polarizing filter 22) and the polarization direction on the imaging unit 25 side (the polarization direction of the polarizing filter 24) are perpendicular (the relative angle is 90 degrees)) is referred to as a cross-polarization imaging method. Cross-polarization imaging can suppress the specular reflection component as described above, making it possible to detect more diffuse reflection components. Note that the first direction and the second direction may be parallel to each other. In this specification, this type of polarization imaging is also referred to as parallel polarization imaging. In the case of parallel polarization imaging, both the specular reflection component and the diffuse reflection component are received by the imaging unit 25, and a captured image is generated.

[0028] <Multipath> Depending on the shape of the 3D object, specular reflection components reflected from multiple surfaces of the 3D object may be received by the imaging unit. In the example shown in the upper part of Fig. 3, illumination light 32 emitted from light-emitting unit 21 is specularly reflected from surfaces 31A and 31B of 3D object 31, as indicated by the arrows, and is received by imaging unit 25. In this specification, the path of light specularly reflected from multiple surfaces is referred to as multipath.

[0029] In such a multipath reflection, a phenomenon known as "reflection" may occur, in which the pattern or color of one reflective surface is projected onto another reflective surface. For example, as shown in the lower part of Figure 3, when illumination light 35 travels along multiple paths indicated by arrows and is specularly reflected by surface 34A of 3D object 34 and surface 33A of 3D object 33, the illumination light 35 may project (reflect) the color, pattern, etc. of surface 34A onto (region 33B of) surface 33A. Assume that the color, pattern, etc. of the entire surface 33A (in terms of the diffuse reflection component) is uniform. In other words, assume that the color, pattern, etc. of region 33B is identical to the color, pattern, etc. of the rest of surface 33A. In this case, in a captured image generated by receiving this illumination light 35 using an imaging unit, the color, pattern, etc. of region 33B will differ from the color, pattern, etc. of the rest of surface 33A due to this reflection. In other words, there is a risk of texture quality being reduced due to the occurrence of multipath reflection. Furthermore, the occurrence of such reflections may cause the appearance (color, pattern, etc.) of the region 33B to differ in each captured image, which may reduce the accuracy of matching between captured images.

[0030] Even in the case of cross-polarized light imaging or parallel-polarized light imaging as described above, it is difficult to completely remove the specular reflection component, so there is a risk of being affected by the occurrence of such multipath light. In other words, there is a risk of glare due to multipath light. This may also reduce the accuracy of matching between captured images and degrade the quality of the texture.

[0031] <Brightness Unevenness Due to Imaging Distance> Furthermore, when imaging a 3D object by irradiating it with light from a light-emitting unit, as in the above-described cross-polarized light imaging and parallel-polarized light imaging, the intensity of the reflected light depends on the distance from the light-emitting unit or the imaging unit to the 3D object. For example, as shown in the upper part of FIG. 4 , when imaging the 3D object 41 from a close distance, i.e., when the distance 42 from the light-emitting unit 21 or the imaging unit 25 to the 3D object 41 is relatively short, the 3D object 41 is imaged at a relatively high brightness. In contrast, as shown in the middle part of FIG. 4 , when imaging the 3D object 41 from a long distance, i.e., when the distance 42 from the light-emitting unit 21 or the imaging unit 25 to the 3D object 41 is relatively long, the 3D object 41 is imaged at a relatively low brightness. As such, when 3D modeling is performed using multiple images captured at different distances to the subject, even if the brightness of the entire 3D object 41 is uniform, a 3D model 43 with different brightness levels may be generated, as shown in the middle part of FIG. 4 . This means that there is a risk of the so-called "brightness unevenness" occurring in the 3D model (texture), which means that the quality of the texture may be reduced.

[0032] <3. Guidance for Manipulating Polarization Direction> <Method 1> Therefore, as shown in the top row of the table in FIG. 5, guidance is provided for manipulating the polarization direction so as to reduce the multipath of s-polarized light based on the shape of the subject (Method 1).

[0033] For example, the information processing device may include a recommended polarization direction derivation unit that derives a recommended polarization direction of illumination light emitted from a light-emitting unit in accordance with the shape of an object in three-dimensional space during polarization imaging, and a guidance unit that guides an operation for bringing the polarization direction of the illumination light closer to the recommended polarization direction.For example, in an information processing method executed by the information processing device, a recommended polarization direction of illumination light emitted from a light-emitting unit in accordance with the shape of an object in three-dimensional space during polarization imaging is derived, and an operation for bringing the polarization direction of the illumination light closer to the recommended polarization direction is guided.

[0034] Polarized imaging is an imaging method in which an object is irradiated with illumination light polarized in a first direction and then imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction. The second direction is a direction perpendicular to the first direction or a direction parallel to the first direction. The recommended polarization direction is a direction in which the specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and incident on the imaging unit is at least equal to or less than the average.

[0035] That is, the recommended polarization direction is derived so as to reduce the specular reflection component of the multipath detected by the imaging unit. By reducing the specular reflection component of the multipath detected by the imaging unit in this way, it is possible to suppress the occurrence of glare due to the multipath in the captured image. Therefore, it is possible to suppress a reduction in the accuracy of matching between captured images and a reduction in texture quality due to the glare. Note that the occurrence of multipath depends on the shape of the object. Therefore, as described above, the recommended polarization direction may be derived according to the shape of the object.

[0036] For example, a polarization direction in which the specular reflection components of multipath light detected by the imaging unit are equal to or less than a predetermined standard may be derived as the recommended polarization direction. This standard may be any value. Since the purpose is to reduce the specular reflection components of multipath light, for example, a polarization direction in which the specular reflection components of multipath light detected by the imaging unit are at least equal to or less than the average for all polarization directions may be determined as the recommended polarization direction.

[0037] The smaller the value of this criterion, the more effectively it is possible to suppress the occurrence of reflections due to multipath reflection in the captured image. For example, the polarization direction in which the specular reflection component of the multipath reflected light detected by the imaging unit is estimated to be the smallest may be set as the recommended polarization direction. Also, for example, the polarization direction in which the specular reflection component of the multipath reflected light detected by the imaging unit is the smallest may be set as the recommended polarization direction. By doing so, it is possible to further suppress the occurrence of reflections due to multipath reflection in the captured image. Therefore, it is possible to further suppress the reduction in the accuracy of matching between captured images and the reduction in texture quality due to the reflection.

[0038] In this case, the recommended polarization direction may be a direction in which the s-polarized component of the specular reflection components of the multipath light detected by the imaging unit, whose electric field oscillation direction is perpendicular to the incident plane when reflected by the surface of the object, is estimated to be minimum.The recommended polarization direction may also be a direction in which the s-polarized component of the specular reflection components of the multipath light detected by the imaging unit, whose electric field oscillation direction is perpendicular to the incident plane when reflected by the surface of the object, is estimated to be minimum.

[0039] Specularly reflected light contains s-polarized and p-polarized components. As shown in the upper part of Figure 6, the polarization component whose electric field oscillates perpendicular to the plane (also called the plane of incidence) where the two straight lines of the incident light and reflected light exist is called s-polarized light (component). In the example of Figure 6, the plane of incidence is the paper surface, so the electric field of the s-polarized component oscillates perpendicular to the paper surface.

[0040] In contrast, a polarization component whose electric field vibration direction is parallel to the plane (incident plane) on which the two straight lines of incident light and reflected light exist is called p-polarized light (component), as shown in the lower part of Figure 6. In the example of Figure 6, the plane of incidence is the paper surface, so the electric field of the p-polarized light component vibrates parallel to this plane of paper (in the direction indicated by the double arrow).

[0041] The reflectance of the s-polarized component and the p-polarized component in this specular reflection differs. In the graph shown in Figure 7, the horizontal axis represents the angle of incidence and the vertical axis represents the reflectance. The solid line represents the s-polarized component and the dotted line represents the p-polarized component. In other words, the reflectance of the s-polarized component is generally greater than that of the p-polarized component.

[0042] For example, in the case of multipath light as shown in the example of FIG. 3, as shown in the upper part of FIG. 8, s-polarized light 101 is specularly reflected with a smaller attenuation rate than p-polarized light and enters the image capture unit 25. In contrast, as shown in the lower part of FIG. 8, p-polarized light 102 is attenuated more than s-polarized light 101 each time it is specularly reflected. In other words, the s-polarized light component contributes more to multipath reflections than the p-polarized light component. In other words, by reducing this s-polarized light component, it is possible to suppress the occurrence of multipath reflections in the captured image.

[0043] As described above, in the case of cross-polarized light imaging or parallel-polarized light imaging, the light emitted from the light-emitting unit is polarized in one direction, so increasing the p-polarized light can reduce the s-polarized light. For example, by aligning the polarization direction of the light emitted from the light-emitting unit with the vibration direction of the electric field of the p-polarized light component, the s-polarized light component can be minimized (or at least estimated to be minimized). In this way, the magnitude of the s-polarized light component can be controlled by the polarization direction of the light emitted from the light-emitting unit. Therefore, the polarization direction of the light emitted from the light-emitting unit may be set so that the s-polarized light component is minimized (or estimated to be minimized). In other words, this polarization direction may be set as the recommended polarization direction.

[0044] The directions of the s-polarized and p-polarized components (electric field oscillation direction) are determined relative to the incident surface as described above. That is, the directions of these components are determined depending on the orientation of the specularly reflecting surface. That is, since the electric field oscillation direction of the s-polarized component is determined depending on the shape of the object (the angle of the specularly reflecting surface in multipath), a recommended polarization direction may be derived depending on the shape of the object.

[0045] Any method may be used to guide the user through the operation to change the polarization direction of the light emitted from the light-emitting unit toward the recommended polarization direction. For example, in the information processing device described above, the guide unit may guide the user through the operation using visual information, which is information transmitted visually, such as images of characters, symbols, pictures, or light emission (color, brightness, blinking pattern, etc.). In this case, the guide unit may present the visual information guiding the user through the operation by superimposing it on a captured image obtained by the imaging unit. This captured image may be an image captured before the user or the like operates the shutter button (before an imaging instruction is input), or may be a captured image obtained by the user or the like operating the shutter button. This captured image may be a moving image or a generated image.

[0046] The guidance unit may also guide the user through the operation using auditory information, such as voice, music, etc. Alternatively, the guidance unit may guide the user through the operation using tactile information, such as vibration, shape change, etc.

[0047] Furthermore, the current polarization direction of the irradiated light may be detected, and a polarization direction manipulation direction for changing the current polarization direction to the recommended polarization direction may be provided. For example, the information processing device may further include a polarization direction detection unit that detects the current polarization direction of the irradiated light. The guidance unit may then provide guidance on a manipulation for changing the polarization direction of the irradiated light from the detected current polarization direction to closer to the recommended polarization direction.

[0048] The information processing device may further include a light emitting unit that emits irradiation light. The information processing device may further include an imaging unit that captures an image of the object illuminated with the irradiation light. The information processing device may further include a 3D model generation unit that generates a 3D model of the object using a captured image captured by the imaging unit.

[0049] <Method 1-1> As described above, any method may be used to guide the user on the operation to bring the polarization direction of the light emitted from the light-emitting unit closer to the recommended polarization direction. For example, the derived recommended polarization direction may be presented as shown in the second row from the top of the table in FIG. 5 (Method 1-1).

[0050] For example, in the information processing device described above, a guide unit may guide the operation by presenting a recommended polarization direction. For example, in the display unit 121 shown in the upper part of Fig. 9 , the recommended polarization direction is shown as a dotted line 123 superimposed on the captured image, and the current polarization direction is shown as a dashed-dotted line 122. As shown in the lower part of Fig. 9 , by operating the current polarization direction (dashed-dotted line 122) to match the recommended polarization direction (dotted line 123), it is possible to suppress the occurrence of glare due to multipath reflection in the captured image. Therefore, it is possible to suppress a reduction in the accuracy of matching between captured images and a reduction in texture quality due to the glare.

[0051] Note that the example in FIG. 9 is just an example, and the method of presenting the recommended polarization direction is not limited to this example.

[0052] <Method 1-2> Furthermore, as shown in the third row from the top of the table in FIG. 5, a method for manipulating the polarization direction may be presented (Method 1-2).

[0053] For example, in the above-described information processing device, the guide unit may guide the operation by presenting the direction of operation of the polarization direction. For example, in the display unit 121 shown in the upper part of FIG. 10 , the current polarization direction is shown as a dashed-dotted line 122 superimposed on the captured image, and the polarization direction operation direction (the operation direction for bringing the current polarization direction closer to the recommended polarization direction) is shown as arrows 131A and 131B. As shown in the lower part of FIG. 10 , by operating the polarization direction (dashed-dotted line 122) to change it in the direction of the arrows 131A and 131B (for example, by rotating the light-emitting unit (or an imaging device including a light-emitting unit) in this direction), it is possible to suppress the occurrence of reflections due to multipath reflections in the captured image. Therefore, it is possible to suppress the reduction in the accuracy of matching between captured images and the reduction in texture quality due to the reflections.

[0054] It should be noted that the example in FIG. 10 is merely an example, and the method of presenting the polarization direction manipulation direction is not limited to this example.

[0055] <Method 1-3> Alternatively, as shown in the fourth row from the top of the table in FIG. 5, the polarization direction in which the multipath of s-polarized light is minimized (or estimated to be minimized) may be derived based on the shape of the subject, and this may be used as a target (Method 1-3).

[0056] For example, in the information processing device described above, the recommended polarization direction deriving unit may derive the shape of an object based on the distance to the object, and derive the recommended polarization direction based on the shape of the object.

[0057] Any method may be used to derive the recommended polarization direction based on the object shape. For example, the recommended polarization direction derivation unit may generate a normal map that is distribution information of normals on the surface of the object based on the distance to the object, determine a representative normal based on the normal map, derive a representative plane based on the representative normal, and derive the recommended polarization direction based on the representative plane.

[0058] For example, the normals of the surface of an object may be derived based on the distance to the object, and a normal map, which is distribution information of the normals, may be generated, such as the graph shown in FIG. 11. The horizontal axis of the graph shown in FIG. 11 indicates the normal angle, and the vertical axis indicates the frequency. That is, in the example of FIG. 11, the frequencies of normal angles v0 and v1 are prominent. This indicates that the area of ​​the plane with these angles as the normal direction is large. For example, the normals of these normal angles may be determined as representative normals.

[0059] Furthermore, for example, as shown in the following formulas (1) and (2), the top two (h0 normals n0 and h1 normals n1) of the angle groups (normal groups) that are more frequent than a predetermined threshold in this normal map may be found, and the average value (average angle) of these two normal groups may be used as the representative normal (N0, N1). For example, in the example of the graph in Figure 11, the top two (thick line portions) of the parts that are greater than or equal to the threshold indicated by the dotted lines may be found, and the respective average values ​​may be used as the representative normal. ...(1) ... (2)

[0060] Then, for example, as shown in Figure 12, a plane including these representative normals is derived as a representative plane. In the example of Figure 12, a normal 151N (N0) of a surface 151 and a normal 152N (N1) of a surface 152 are set as representative normals, and a plane 153 (P) including these normals is set as a representative plane. For example, a plane P that passes through the origin (0,0,0) and is spanned by two vectors is calculated using the following equations (3) to (7). ...(3) ...(4) ...(5) ...(6) ... (7)

[0061] The above equation is solved to calculate the plane P as shown in the following equation (8). ...(8)

[0062] The recommended polarization direction is set based on the representative plane derived in this way. For example, the recommended polarization direction may be set so that this representative plane is the plane of incidence. In other words, when this representative plane is the plane of incidence, the polarization direction of the light emitted from the light-emitting unit may be set to "the direction in which the specular reflection component is minimized" or "the direction in which the specular reflection component is estimated to be minimized." In this case, the tilt of the camera's roll axis, which corresponds to the p-polarization direction, is displayed on the preview screen so that a line where z = 0 on plane P passes through the center of the camera's image height. By prompting the photographer to rotate the camera angle so that the flash polarization direction is parallel to this p-polarization direction, it becomes possible to capture images using the desired p-polarized flash.

[0063] The information processing device may further include a distance measuring unit that measures the distance to the object, and the polarization direction deriving unit may generate a normal map based on the distance to the object measured by the distance measuring unit.

[0064] The method for deriving the representative normal (specularly reflecting surface) may be any method and is not limited to the above example. For example, normals of two surfaces likely to have a large influence may be selected as the representative normal. For example, two wider surfaces may be selected as the two surfaces likely to have a large influence. Furthermore, two surfaces in a better position (e.g., closer to the center) may be selected as the two surfaces likely to have a large influence. Furthermore, two surfaces with a better angle (positional relationship) (e.g., within 180 degrees (concave shape), within 90 degrees, etc.) may be selected as the two surfaces likely to have a large influence. Furthermore, the material of the surfaces may be determined (e.g., a surface with high reflectivity). Furthermore, a comprehensive determination may be made using multiple conditions. Furthermore, multiple pairs of surfaces may be selected (e.g., the optimal polarization direction may be comprehensively determined based on the relationship between each of the multiple pairs). In this case, weighting may be performed according to, for example, area, position, angle, material, etc. Furthermore, the relationship between the shape of an object and the representative normal may be learned, and an appropriate representative normal may be derived from the learning results and the overall shape of the object.

[0065] Furthermore, the method of deriving the representative plane described above is arbitrary and is not limited to the above example. For example, the representative plane may be derived comprehensively from a plurality of two normals (normal pairs). In this case, each pair may be weighted. Furthermore, the relationship between the shape of the object and the representative plane may be learned, and an appropriate representative plane may be derived from the learning results and the overall shape of the object.

[0066] <Method 1-4> Furthermore, as shown in the bottom row of the table in FIG. 5, the light emission intensity may be controlled based on the subject distance (Method 1-4).

[0067] For example, the information processing device may further include a light emission intensity control unit that controls the light emission intensity of the light emitting unit based on the distance to the object. This can reduce the occurrence of brightness unevenness (differences in brightness between captured images) and reduce the deterioration of texture quality.

[0068] For example, the light emission intensity control unit may reduce the light emission intensity as the distance to the object becomes shorter. Generally, the closer the distance to the subject, the brighter the captured image becomes. Therefore, by controlling in this manner, it is possible to suppress the occurrence of brightness unevenness (differences in brightness between captured images) and to suppress deterioration of texture quality.

[0069] For example, the information processing device described above may further include a distance measurement unit that measures the distance to an object. The light emission intensity control unit may then control the light emission intensity based on the distance to the object measured by the distance measurement unit. This can reduce the occurrence of brightness unevenness (differences in brightness between captured images) and prevent degradation of texture quality.

[0070] <4. First Embodiment> <Imaging Device> FIG. 13 is a block diagram showing an example of the main configuration of an imaging device that is an embodiment of an information processing device to which the present technology described above in <3. Polarization Direction Operation Guide> is applied. The imaging device 200 shown in FIG. 13 is a device that captures a 3D object to generate a captured image and generates a 3D model using the captured image. In this case, the imaging device 200 can irradiate the 3D object with illumination light and capture an image of the 3D object illuminated with the illumination light. In this case, the imaging device 200 can polarize the illumination light in a predetermined first direction. Furthermore, the imaging device 200 can polarize light from the 3D object in a predetermined second direction and receive the light to generate a captured image. In this case, the imaging device 200 may be capable of performing cross-polarized imaging. That is, the first direction and the second direction may be perpendicular to each other. Furthermore, the imaging device 200 may be capable of performing parallel-polarized imaging. That is, the first direction and the second direction may be parallel to each other.

[0071] As described above, the imaging device 200 can apply the present technology described above in <3. Polarization Direction Operation Guidance>. For example, the imaging device 200 can apply the above-described Method 1. The imaging device 200 can also apply the above-described Method 1-1. The imaging device 200 can also apply the above-described Method 1-2. The imaging device 200 can also apply the above-described Method 1-3. The imaging device 200 can also apply the above-described Method 1-4.

[0072] As shown in FIG. 13, the imaging device 200 has a distance measurement unit 211, a recommended polarization direction derivation unit 212, a presentation unit 213, a polarization direction detection unit 214, an emission intensity control unit 215, an imaging control unit 216, an operation unit 221, an emission unit 222, an imaging unit 223, a memory unit 224, a 3D model generation unit 225, and an output unit 226.

[0073] The distance measurement unit 211 measures the distance to a subject (3D object) using, for example, LiDAR (Light Detection And Ranging) or infrared light, and supplies the measurement result (information indicating the distance to the subject) to the recommended polarization direction derivation unit 212 and the emission intensity control unit 215. Any method for measuring the distance by the distance measurement unit 211 may be used.

[0074] The recommended polarization direction derivation unit 212 derives a recommended polarization direction based on information indicating the distance to the subject supplied from the distance measurement unit 211, and supplies the recommended polarization direction to the presentation unit 213. In other words, the recommended polarization direction derivation unit 212 may derive the recommended polarization direction of the illumination light emitted from the light-emitting unit 222 during polarization imaging in accordance with the shape of an object in three-dimensional space.

[0075] In this case, the recommended polarization direction may be, for example, the polarization direction in which the specular reflection component of the multipath detected by the image capturing unit 223 is the smallest (or is estimated to be the smallest). In this case, the recommended polarization direction may be the direction in which the s-polarized component of the specular reflection component of the multipath detected by the image capturing unit 223 is the smallest (or is estimated to be the smallest), in which the oscillation direction of the electric field when reflected by the surface of the object is perpendicular to the plane of incidence.

[0076] The presentation unit 213 presents guidance regarding manipulation of the polarization direction based on the supplied recommended polarization direction. For example, the presentation unit 213 may provide guidance on manipulation for bringing the polarization direction of the irradiated light closer to the recommended polarization direction. Therefore, the presentation unit 213 can also be said to be a guidance unit.

[0077] For example, the operation may be guided using visual information. In this case, the presentation unit 213 may present the visual information guiding the operation by superimposing it on the captured image obtained by the imaging unit 223. The presentation unit 213 may also guide the operation using auditory information. The presentation unit 213 may also guide the operation using tactile information. The presentation unit 213 may also guide the operation to move the current polarization direction of the light-emitting unit 222 detected by the polarization direction detection unit 214 closer to the recommended polarization direction. The presentation unit 213 may also present the recommended polarization direction. The presentation unit 213 may also present the direction in which the polarization direction should be operated.

[0078] The polarization direction detection unit 214 detects the current polarization direction of the light emitting unit 222 and supplies the detection result (information indicating the current polarization direction) to the presentation unit 213 .

[0079] The light emission intensity control unit 215 controls the light emission intensity of the light emitting unit 222 based on information indicating the distance to the subject supplied from the distance measurement unit 211. For example, the light emission intensity control unit 215 may reduce the light emission intensity as the distance to the object becomes shorter. The light emission intensity control unit 215 supplies this control information to the imaging control unit 216.

[0080] The imaging control unit 216 controls the light emitting unit 222 and the imaging unit 223 to perform cross polarization imaging, parallel polarization imaging, etc. For example, the imaging control unit 216 causes the light emitting unit 222 to emit light and controls the imaging unit 223 to capture an image of a 3D object illuminated with the emitted light. In this case, the imaging control unit 216 may control the light emission intensity of the light emitting unit 222 according to the control of the light emission intensity control unit 215. In other words, the imaging control unit 216 may control the light emitting unit 222 to emit light at the light emission intensity set by the light emission intensity control unit 215. Furthermore, the imaging control unit 216 may control the above-described imaging according to instructions (e.g., imaging instructions) supplied from the operation unit 221.

[0081] The operation unit 221 accepts instructions from a user, such as an instruction to capture an image. The light emitting unit 222 has a light emitting device and emits light under the control of the imaging control unit 216. The imaging unit 223 has an imaging element and, under the control of the imaging control unit 216, captures an image of a subject, generates a captured image (or a captured image), and supplies the image to the storage unit 224 and the presentation unit 213.

[0082] The storage unit 224 has a storage medium and stores data of captured images supplied from the imaging unit 223 and data of 3D models generated by the 3D model generation unit 225. The 3D model generation unit 225 generates 3D models from captured images stored in the storage unit 224 using a technique such as photogrammetry. The 3D model generation unit 225 supplies the generated 3D model data to the storage unit 224 for storage. The output unit 226 reads out the captured images and 3D model data stored in the storage unit 224 and outputs them to the outside of the imaging device 200.

[0083] The recommended polarization direction derivation unit 212, the presentation unit 213, and the emission intensity control unit 215 shown in the dotted frame 231 may be combined into a single information processing device to which the present technology can be applied. The other components (processors) may be separate or may be omitted. Furthermore, if the emission intensity is not controlled, the emission intensity control unit 215 may be omitted.

[0084] <Recommended Polarization Direction Derivation Unit> The recommended polarization direction derivation unit 212 may derive the shape of an object based on the distance to the object, and may derive a recommended polarization direction based on the shape of the object. For example, the recommended polarization direction derivation unit 212 may generate a normal map that is distribution information of normals on the surface of the object based on the distance to the object, determine a representative normal based on the normal map, derive a representative plane based on the representative normal, and derive a recommended polarization direction based on the representative plane.

[0085] Fig. 14 is a block diagram showing an example of the main configuration of the recommended polarization direction derivation unit 212 in Fig. 13. As shown in Fig. 14, the recommended polarization direction derivation unit 212 has a normal map generation unit 251, a representative normal derivation unit 252, a plane derivation unit 253, and a recommended polarization direction determination unit 254.

[0086] The normal map generation unit 251 generates a normal map based on information indicating the distance to the subject, and supplies the normal map to the representative normal derivation unit 252. The representative normal derivation unit 252 derives a representative normal based on the normal map, and supplies the representative normal to the plane derivation unit 253. The plane derivation unit 253 derives a representative plane based on the representative normal, and supplies the representative plane to the recommended polarization direction determination unit 254. The recommended polarization direction determination unit 254 determines a recommended polarization direction based on the representative plane, and supplies the recommended polarization direction to the presentation unit 213.

[0087] That is, each processing unit of the recommended polarization direction derivation unit 212 can execute each process as described above in <Method 1-3>.

[0088] With the above-described configuration, the imaging device 200 can further suppress the occurrence of reflections due to multipath interference in the captured images. Therefore, the imaging device 200 can further suppress a decrease in the accuracy of matching between captured images and a decrease in texture quality due to reflections. Furthermore, the imaging device 200 can suppress the occurrence of brightness unevenness and a decrease in texture quality.

[0089] <Flow of Imaging Process> An example of the flow of imaging process executed by such imaging device 200 will be described with reference to the flowchart of FIG.

[0090] When the image capturing process starts, in step S101, the distance measuring unit 211 of the image capturing device 200 measures the distance to the subject.

[0091] In step S102, the recommended polarization direction derivation unit 212 and the presentation unit 213 execute a polarization direction guidance process to guide the polarization direction.

[0092] In step S103, the light emission intensity control unit 215 controls the light emission intensity in accordance with the distance to the subject.

[0093] In step S104, the light emitting section 222 emits light under the control of the imaging control section 216. The imaging section 223 captures an image of the subject under the control of the imaging control section 216.

[0094] In step S105, the storage unit 224 stores the captured image data generated by the processing in step S104.

[0095] In step S106, the imaging control unit 216 determines whether or not to end imaging. If it is determined not to end, the process returns to step S101, and the subsequent processes are repeated. The processes of steps S101 to S106 are repeatedly executed, and if it is determined in step S106 that imaging is to end, the process proceeds to step S107.

[0096] In step S107, the 3D model generation unit 225 generates a 3D model by a method such as photogrammetry using the captured image stored in step S105.

[0097] In step S108, the output unit 226 outputs the data of the 3D model generated in step S107.

[0098] When the process of step S108 is completed, the image capturing process ends.

[0099] <Flow of Polarization Direction Guidance Processing> The presentation unit 213 may present, for example, a recommended polarization direction, and thereby guide the user on an operation for bringing the polarization direction of the irradiated light closer to the recommended polarization direction. An example of the flow of the polarization direction guidance processing executed in step S102 of FIG. 15 in this case will be described with reference to the flowchart of FIG. 16 .

[0100] When the polarization direction guidance process is started, in step S121 the recommended polarization direction derivation unit 212 executes the recommended polarization direction derivation process and derives the recommended polarization direction based on the distance to the subject.

[0101] In step S122, the presentation unit 213 presents the derived recommended polarization direction.

[0102] When the process of step S122 is completed, the polarization direction guide process is completed, and the process returns to FIG.

[0103] <Flow of Recommended Polarization Direction Deriving Process> An example of the flow of the recommended polarization direction deriving process executed in step S121 of FIG. 16 will be described with reference to the flowchart of FIG.

[0104] When the recommended polarization direction derivation process starts, in step S141, the normal map generation unit 251 of the recommended polarization direction derivation unit 212 generates a normal map based on the distance measurement result.

[0105] In step S142, the representative normal derivation unit 252 derives a representative normal using the normal map.

[0106] In step S143, the plane derivation unit 253 derives a representative plane based on the representative normal.

[0107] In step S144, the recommended polarization direction determiner 254 determines the recommended polarization direction based on the representative plane.

[0108] When the process of step S144 ends, the process returns to FIG.

[0109] <Flow of Polarization Direction Guidance Processing> The presentation unit 213 may also present, for example, a direction for manipulating the polarization direction to bring the polarization direction of the irradiated light closer to the recommended polarization direction. In this case, an example of the flow of the polarization direction guidance processing executed in step S102 of FIG. 15 will be described with reference to the flowchart of FIG. 18.

[0110] When the polarization direction guidance process is started, in step S161, the recommended polarization direction derivation unit 212 executes the recommended polarization direction derivation process to derive the recommended polarization direction based on the distance to the subject. This process is executed in the same manner as described with reference to the flowchart in FIG.

[0111] In step S162, the presentation unit 213 derives and presents an operation method based on the derived recommended polarization direction and the current polarization direction.

[0112] When the process of step S162 is completed, the polarization direction guide process ends, and the process returns to FIG.

[0113] By performing each process as described above, the imaging device 200 can further reduce the occurrence of reflections due to multipath interference in the captured images. Therefore, the imaging device 200 can further reduce the reduction in matching accuracy between captured images and the reduction in texture quality due to reflections. Furthermore, the imaging device 200 can reduce the occurrence of brightness unevenness and the reduction in texture quality.

[0114] 5. Supplementary Notes Computer The above-described series of processes can be executed by hardware or software. When the series of processes are executed by software, the programs that make up the software are installed on a computer. Here, the term computer includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0115] FIG. 19 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0116] In a computer 900 shown in FIG. 19, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0117] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0118] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0119] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0120] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0121] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0122] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0123] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0124] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0125] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0126] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0127] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0128] <Others>

[0129] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0130] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0131] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0132] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0133] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0134] Furthermore, for example, the multiple technical elements included in the present technology can be implemented independently and individually, as long as no contradiction occurs. Of course, any multiple technical elements among them can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the present technology described above can be implemented in combination with other technologies not described above.

[0135] Note that the present technology can also be configured as follows. (1) An information processing device including: a recommended polarization direction deriving unit that derives a recommended polarization direction of illumination light emitted from a light-emitting unit in accordance with a shape of an object in three-dimensional space during polarization imaging; and a guide unit that guides an operation to bring the polarization direction of the illumination light closer to the recommended polarization direction, wherein the polarization imaging is an imaging method in which the object illuminated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction orthogonal to the first direction or a direction parallel to the first direction, and the recommended polarization direction being a direction in which a specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and enters the imaging unit is at least equal to or less than average. (2) The information processing device described in (1), wherein the guide unit guides the operation by presenting the recommended polarization direction. (3) The information processing device described in (1), wherein the guide unit guides the operation by presenting an operation direction for the polarization direction. (4) The information processing device according to any one of (1) to (3), wherein the guiding unit guides the user through the operation using visual information. (5) The information processing device according to (4), wherein the guiding unit presents the visual information guiding the user through the operation by superimposing it on a captured image obtained by the imaging unit. (6) The information processing device according to any one of (1) to (5), wherein the guiding unit guides the user through the operation using auditory information. (7) The information processing device according to any one of (1) to (6), wherein the guiding unit guides the user through the operation using tactile information. (8) The information processing device according to any one of (1) to (7), further comprising a polarization direction detection unit that detects a current polarization direction of the irradiated light, wherein the guiding unit guides the user through an operation to bring the polarization direction of the irradiated light closer to the recommended polarization direction from the detected current polarization direction. (9) The information processing device according to any one of (1) to (8), wherein the recommended polarization direction is a direction in which the specular reflection component is estimated to be minimum. (10) The information processing device according to (9), wherein the recommended polarization direction is a direction in which the s-polarized component of the specular reflection component, in which the vibration direction of the electric field when reflected by the surface of the object is perpendicular to the plane of incidence, is estimated to be minimum.(11) The information processing device according to (10), wherein the recommended polarization direction derivation unit derives a shape of the object based on a distance to the object, and derives the recommended polarization direction based on the shape of the object. (12) The information processing device according to (11), wherein the recommended polarization direction derivation unit generates a normal map that is distribution information of normals of the surface of the object based on the distance to the object, determines a representative normal based on the normal map, derives a representative plane based on the representative normal, and derives the recommended polarization direction based on the representative plane. (13) The information processing device according to (12), further comprising a distance measurement unit that measures the distance to the object, and wherein the recommended polarization direction derivation unit generates the normal map based on the distance to the object measured by the distance measurement unit. (14) The information processing device according to any of (1) to (13), further comprising a light emission intensity control unit that controls light emission intensity of the light emitter based on the distance to the object. (15) The information processing device according to (14), wherein the light emission intensity control unit reduces the light emission intensity as the distance to the object becomes shorter. (16) The information processing device according to (14) or (15), further comprising a distance measuring unit that measures a distance to the object, and the emission intensity control unit controls the emission intensity based on the distance to the object measured by the distance measuring unit. (17) The information processing device according to any of (1) to (16), further comprising the light emitting unit. (18) The information processing device according to any of (1) to (17), further comprising the imaging unit. (19) The information processing device according to any of (1) to (18), further comprising a 3D model generation unit that generates a 3D model of the object using a captured image captured and generated by the imaging unit.(20) An information processing method, in which, during polarized imaging, a recommended polarization direction of illumination light emitted from a light-emitting unit is derived according to the shape of an object in three-dimensional space, and an operation for bringing the polarization direction of the illumination light closer to the recommended polarization direction is guided, the polarized imaging is an imaging method in which the object illuminated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction perpendicular to the first direction or a direction parallel to the first direction, and the recommended polarization direction being a direction in which the specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and enters the imaging unit is at least below average.

[0136] 200 Imaging device, 211 Distance measurement unit, 212 Recommended polarization direction derivation unit, 213 Presentation unit, 214 Polarization direction detection unit, 215 Emission intensity control unit, 216 Imaging control unit, 221 Operation unit, 222 Emission unit, 223 Imaging unit, 224 Memory unit, 225 3D model generation unit, 226 Output unit, 251 Normal map generation unit, 252 Representative normal derivation unit, 253 Plane derivation unit, 254 Recommended polarization direction determination unit, 900 Computer

Claims

1. An information processing device comprising: a recommended polarization direction derivation unit that derives a recommended polarization direction of illumination light irradiated from a light emitting unit in accordance with the shape of an object in three-dimensional space during polarized imaging; and a guidance unit that guides an operation to bring the polarization direction of the illumination light closer to the recommended polarization direction, wherein the polarized imaging is an imaging method in which the object irradiated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction perpendicular to the first direction or a direction parallel to the first direction, and the recommended polarization direction being a direction in which the specular reflection component of the illumination light that is specularly reflected from multiple surfaces of the object and enters the imaging unit is at least below average.

2. The information processing device according to claim 1, wherein the guidance unit provides guidance for the operation by presenting the recommended polarization direction.

3. The information processing device according to claim 1, wherein the guidance unit provides guidance for the operation by presenting an operation direction in the polarization direction.

4. The information processing device according to claim 1, wherein the guidance unit provides guidance for the operation using visual information.

5. The information processing device according to claim 4, wherein the guidance unit presents the visual information guiding the operation by superimposing it on an image captured by the imaging unit.

6. The information processing device according to claim 1, wherein the guidance section provides guidance for the operation using auditory information.

7. The information processing device according to claim 1, wherein the guidance unit provides guidance for the operation using tactile information.

8. The information processing device according to claim 1, further comprising a polarization direction detection unit that detects the current polarization direction of the irradiated light, wherein the guidance unit guides an operation for changing the polarization direction of the irradiated light from the detected current polarization direction to closer to the recommended polarization direction.

9. The information processing device according to claim 1, wherein the recommended polarization direction is a direction in which the regular reflection component is estimated to be the smallest.

10. The information processing device according to claim 9, wherein the recommended polarization direction is a direction in which the s-polarized component of the specular reflection component, in which the vibration direction of the electric field when reflected by the surface of the object is perpendicular to the plane of incidence, is estimated to be minimal.

11. The information processing device according to claim 10, wherein the recommended polarization direction derivation unit derives a shape of the object based on a distance to the object, and derives the recommended polarization direction based on the shape of the object.

12. The information processing device described in claim 11, wherein the recommended polarization direction derivation unit generates a normal map which is distribution information of normals of the surface of the object based on the distance to the object, determines a representative normal based on the normal map, derives a representative plane based on the representative normal, and derives the recommended polarization direction based on the representative plane.

13. The information processing device according to claim 12, further comprising a distance measuring unit that measures a distance to the object, and the recommended polarization direction derivation unit generates the normal map based on the distance to the object measured by the distance measuring unit.

14. The information processing device according to claim 1, further comprising a light emission intensity control unit that controls the light emission intensity of the light emitting unit based on the distance to the object.

15. The information processing device according to claim 14, wherein the emission intensity control unit reduces the emission intensity as the distance to the object becomes shorter.

16. An information processing device according to claim 14, further comprising a distance measuring unit that measures a distance to the object, wherein the light emission intensity control unit controls the light emission intensity based on the distance to the object measured by the distance measuring unit.

17. The information processing device according to claim 1, further comprising the light emitting unit.

18. The information processing device according to claim 1, further comprising the imaging unit.

19. The information processing device according to claim 1, further comprising a 3D model generation unit that generates a 3D model of the object using a captured image captured and generated by the imaging unit.

20. An information processing method comprising: during polarized imaging, deriving a recommended polarization direction of illumination light irradiated from a light emitting unit according to the shape of an object in three-dimensional space; and guiding an operation for bringing the polarization direction of the illumination light closer to the recommended polarization direction; the polarized imaging is an imaging method in which the object irradiated with the illumination light polarized in a first direction is imaged by an imaging unit via a configuration in which the illumination light is polarized in a second direction, the second direction being a direction perpendicular to the first direction or a direction parallel to the first direction; and the recommended polarization direction being a direction in which the specular reflection component of the illumination light that is specularly reflected by multiple surfaces of the object and enters the imaging unit is at least below average.

Citation Information

Patent Citations

  • Information processing device, information processing method, and recording medium

    WO2019069536A1

  • Image processing device, and method for acquiring material information

    WO2022259466A1

  • Rotary polarization distance gating camera device

    CN111818251A

  • Optical system for detecting biological tissue using polarization property

    KR1020160096494A

  • Optical system polarizer calibration

    US8797534B2