Information processing device, information processing system, information processing method

Abstract

An information processing device for updating a vision-based artificial intelligence model, including circuitry configured to: obtain polarization image data of a scene; decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modify at least one scene parameter of the plurality of scene parameters; render one or more images based on the plurality of modified scene parameters; and update the vision-based artificial intelligence model based on the rendered one or more images.

Description

[0001] INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING METHOD

[0002] TECHNICAL FIELD

[0003] The present disclosure generally pertains to an information processing device, an information processing system and an information processing method for updating a vision-based artificial intelligence model.

[0004] TECHNICAL BACKGROUND

[0005] Generally, vision-based artificial intelligence models (“vision-based Al models”) are known which take one or more images as input, e.g., for detecting, recognizing and / or tracking objects in a scene.

[0006] It is known that such vision-based Al models typically need to be trained on a variety of images before being deployed.

[0007] However, once deployed, the vision-based Al models may become less accurate over time because their surrounding environment could change for various reasons (e.g., construction of new buildings, change of climate, demographic shifts or movement of the camera(s) which acquires the input images).

[0008] It is possible to fine-tune or update the (deployed) vision-based Al models with images captured by the cameras, but these images may suffer from a lack of variety or accuracy and may include personal information.

[0009] Although there exist techniques for updating a vision-based artificial intelligence model, it is generally desirable to improve the existing techniques.

[0010] SUMMARY

[0011] According to a first aspect, the disclosure provides an information processing device for updating a vision-based artificial intelligence model, comprising circuitry configured to: obtain polarization image data of a scene; decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modify at least one scene parameter of the plurality of scene parameters; render one or more images based on the plurality of modified scene parameters; and update the vision-based artificial intelligence model based on the rendered one or more images.

[0012] According to a second aspect, the disclosure provides an information processing system for updating a vision-based artificial intelligence model, wherein the information processing system comprises: a polarization camera system configured to acquire polarization image data of a scene; and an information processing device including circuitry configured to: obtain the polarization image data, decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data, modify at least one scene parameter of the plurality of scene parameters, render one or more images based on the plurality of modified scene parameters, update the vision-based artificial intelligence model based on the rendered one or more images.

[0013] According to a third aspect, the disclosure provides an information processing method for updating a vision-based artificial intelligence model, comprising: obtaining polarization image data of a scene; decomposing the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modifying at least one scene parameter of the plurality of scene parameters; rendering one or more images based on the plurality of modified scene parameters; and updating the vision-based artificial intelligence model based on the rendered one or more images.

[0014] Further aspects are set forth in the dependent claims, the drawings and the following description.

[0015] BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Embodiments are explained by way of example with respect to the accompanying drawings, in which:

[0017] Fig. 1 schematically illustrates in a block diagram an embodiment of an information processing system for updating a vision-based artificial intelligence model;

[0018] Fig. 2A schematically illustrates in a block diagram an embodiment of an information processing system for updating a vision-based artificial intelligence model; Fig. 2B schematically illustrates in a block diagram an embodiment of an information processing system for updating a vision-based artificial intelligence model;

[0019] Fig. 3 schematically illustrates in a flow diagram an embodiment of an information processing method;

[0020] Fig. 4 schematically illustrates an embodiment of polarization-based inverse rendering;

[0021] Fig. 5 schematically illustrates in a block diagram an embodiment of a polarization-based inverse rendering unit;

[0022] Fig. 6 schematically illustrates an embodiment of parameter modification;

[0023] Fig. 7 schematically illustrates in a block diagram an embodiment of a parameter modification unit;

[0024] Fig. 8 schematically illustrates in a block diagram an embodiment of a parameter modification unit;

[0025] Fig. 9 schematically illustrates an embodiment of image rendering; and

[0026] Fig. 10 schematically illustrates in a flow diagram an embodiment of an information processing method.

[0027] DETAILED DESCRIPTION OF EMBODIMENTS

[0028] Before a detailed description of the embodiments under reference of Fig. 1 is given, general explanations are made.

[0029] As mentioned in the outset, generally, vision-based artificial intelligence models (“vision-based Al models”) are known which take one or more images as input, e.g., for detecting, recognizing and / or tracking objects in a scene.

[0030] It is known that such vision-based Al models typically need to be trained on a variety of images before being deployed.

[0031] However, once deployed, the vision-based Al models may become less accurate over time because their surrounding environment could change for various reasons (e.g., construction of new buildings, change of climate, demographic shifts or movement of the camera(s) which acquires the input images).

[0032] It is possible to fine-tune or update the (deployed) vision-based Al models with images captured by the cameras, but these images may suffer from a lack of variety or accuracy and may include personal information. It has been recognized that polarization cameras and polarization-based inverse may be utilized to generate training images for (deployed) vision-based Al models.

[0033] It has been recognized that this may provide an accurate decomposition of the polarization images into parameters which may be modified to generate images with enough variety and accuracy and free from personal information.

[0034] Hence, some embodiments pertain to an information processing device for updating a visionbased artificial intelligence model, wherein the information processing device includes circuitry configured to: obtain polarization image data of a scene; decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modify at least one scene parameter of the plurality of scene parameters; render one or more images based on the plurality of modified scene parameters; and update the vision-based artificial intelligence model based on the rendered one or more images.

[0035] The information processing device may be a server, computer, board-computer, a mobile device (e.g., a laptop, smartphone, tablet etc.), a data processing module or the like.

[0036] The vision-based artificial intelligence model (“vision-based Al model”) may be any Al model which takes images as input for detecting, recognizing and / or tracking objects in a scene. For example, the Al model may be used in robotics, autonomous driving, surveillance applications, medical applications etc. and may be deployed separately from the information processing device.

[0037] Thus, in some embodiments, the vision-based Al model is deployed.

[0038] In other embodiments, the vision-based Al model is in a training stage on a server or the like.

[0039] The circuitry may be or may be implemented by or may include a CPU (central processing unit), an application processor, a graphical processing unit (GPU), a microcontroller, an FPGA (field programmable gate array), an ASIC (application specific integrated circuit) or the like. The functionality may be implemented by software executed by a processor such as an application processor or the like. The circuitry may be based on or may include or may be implemented by typical electronic components configured to achieve the functionality as described herein. The circuitry may be or may be implemented by or may include in parts by typical electronic components and integrated circuitry logic and in parts by software. The circuitry may include data storage capabilities to store data such as memory which may be based on semiconductor storage technology (e.g., RAM, EPROM, etc.) or magnetic storage technology (e.g., a hard disk drive) or the like.

[0040] The circuitry may include one or more interfaces (e.g., (data) bus interface, point-to-point connections, network interfaces etc.) for communication and exchange of data with external electronic components or devices.

[0041] Some embodiments pertain to an information processing system for updating a vision-based artificial intelligence model, wherein the information processing system includes: a polarization camera system configured to acquire polarization image data of a scene; and an information processing device including circuitry configured to: obtain the polarization image data, decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data, modify at least one scene parameter of the plurality of scene parameters, render one or more images based on the plurality of modified scene parameters, update the vision-based artificial intelligence model based on the rendered one or more images.

[0042] The information processing system may be deployed in a single entity (e.g., a robot, a vehicle or the like) or may be distributed over several locations (e.g., the polarization camera may be deployed in a photo studio and the information processing device is deployed in a computer network or data center or the like).

[0043] In some embodiments, the vision-based Al model is deployed in the information processing system, and the circuitry is further configured to execute the vision-based Al model.

[0044] The polarization image data may represent one or more polarization images acquired at one or more viewpoints with a polarization camera system, which may include one or more polarization cameras.

[0045] Thus, in some embodiments, the polarization camera system includes a first polarization camera configured to acquire first polarization image data of the scene from a first viewpoint and a second polarization camera configured to acquire second polarization image data of the scene from a second viewpoint; and wherein the polarization image data include the first and second polarization image data. Generally, polarization cameras are known. A polarization camera typically includes, in some embodiments, an image sensor (e.g., a CCD (“Charge-Coupled Device” sensor or a CMOS (“Complementary Metal Oxide Semiconductor”) sensor) and a polarizing optical element placed in front of the image sensor such that the image sensor detects an incident light amount depending on the polarization of the light.

[0046] The polarizing optical element may be a single adaptable (e.g., by rotation) polarizer (e.g., wiregrid polarizer) for the whole image sensor.

[0047] The polarizing optical element may include pixel-wise polarizers, for example, each group of four pixels may include a polarizer aligned at 0° with respect to a reference axis, one aligned at 90°, one aligned at -45° and one aligned at 45°. In such embodiments, various polarization images are obtained from a single acquisition, in particular as the polarization images corresponding to other polarization angles can be calculated from the four obtained polarization images by interpolation, as generally known.

[0048] The polarization camera may further include optical parts (e.g., lenses and color filters), mechanical parts (e.g., housing, holders etc.), electrical parts (e.g., power supply) and an illumination (e.g., photodiode, laser diode etc.).

[0049] As mentioned above, the circuitry decomposes the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data.

[0050] Generally, polarization-based inverse rendering is known. The circuitry may include or may use any known physical-based or neural network-based polarization-based inverse Tenderer.

[0051] The polarization of light is sensitive to the material and the surface normal of objects such that its detection allows to obtain shapes and materials of objects in a scene, as well as the illumination of the scene.

[0052] In some embodiments, the plurality of scene parameters includes a shape and a material of an object and an illumination of the scene.

[0053] The plurality of scene parameters may include additionally or alternatively diffuse and specular reflections, albedo and shading.

[0054] The materials may be represented by its diffuse and specular albedo, its roughness and / or the parameters of the Disney BRDF (“bidirectional reflectance distribution function”).

[0055] As mentioned above, the circuitry modifies at least one scene parameter of the plurality of scene parameters. This allows to provide a variety of training images for updating the vision-based Al model.

[0056] In some embodiments, modifying at least one scene parameter of the plurality of scene parameters includes at least one of changing the shape and material of the object and the illumination of the scene.

[0057] In some embodiments, the circuitry is further configured to detect personal information based on the plurality of scene parameters.

[0058] The personal information may be any kind of information that is associated with a specific person. For example, a face, a license plate or the like.

[0059] In some embodiments, the circuitry is further configured to modify the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy-free information.

[0060] This may allow to avoid overfitting and personalization of the (deployed) vision-based Al model. For example, a face may be replaced with a dummy face and a license plate may be replaced with a dummy license plate from a database.

[0061] In some embodiments, the circuitry is further configured to modify the at least one scene parameter of the plurality of scene parameters further based on feedback from the vision-based Al model.

[0062] This may allow to modify the scene parameters which influence the accuracy of the (deployed) vision-based Al model most.

[0063] In some embodiments, the polarization camera is configured to adapt its configuration based on feedback from the vision-based Al model.

[0064] As mentioned above, the circuitry renders one or more images based on the plurality of modified scene parameters.

[0065] Generally, image rendering is known. The circuitry may include or may use any known physicalbased or neural network-based image Tenderer.

[0066] The circuitry updates the vision-based Al model based on the rendered one or more images by providing the one or more images to the vision-based Al model. The circuitry may include an instruction in the message indicating the vision-based Al model to perform training based on the provided images for updating itself. The circuitry may perform the training of the vision-based Al model for updating it. Some embodiments pertain to an information processing method for updating a vision-based artificial intelligence model, wherein the information processing method includes: obtaining polarization image data of a scene; decomposing the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modifying at least one scene parameter of the plurality of scene parameters; rendering one or more images based on the plurality of modified scene parameters; and updating the vision-based artificial intelligence model based on the rendered one or more images.

[0067] The information processing method may be performed by the information processing device or the information processing system as described herein.

[0068] The methods as described herein are also implemented in some embodiments as a computer program causing a computer and / or a processor to perform the method, when being carried out on the computer and / or processor. In some embodiments, also a non-transitory computer- readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.

[0069] Returning to Fig. 1, there is schematically illustrated in a block diagram an embodiment of an information processing system 1 for updating a vision-based Al model 50, which is discussed in the following.

[0070] The information processing system 1 includes a polarization camera 10, an information processing device (not shown) and the vision-based Al model 50. In other embodiments, the vision-based Al model 50 is not part of the information processing system 1.

[0071] The information processing device includes a polarization-based inverse rendering unit 20, a parameter modifying unit 30 and a rendering unit 40, which will be discussed in more detail under reference of Figs. 4 to 9.

[0072] Fig. 2A schematically illustrates in a block diagram an embodiment of an information processing system 1-A for updating the vision-based Al model 50, which is discussed in the following.

[0073] In this embodiment, the polarization camera 10 is deployed in a photo studio in which realistic scenes 11 are recreated for generating training images for the vision-based Al model 50. The polarization camera 10 acquires polarization image data which represent a polarization image 12 which is transmitted to a server 13-A, wherein the server 13-A is an embodiment of an information processing device.

[0074] The server 13-A includes the polarization-based inverse rendering unit 20, the parameter modifying unit 30, the rendering unit 40, which will be discussed in more detail under reference of Figs. 4 to 9, and the vision-based Al model 50.

[0075] The vision-based Al model 50 is already deployed in this embodiment such that it is executed by the server 13-A in the inference stage in which it takes images from a camera 15-A as input for object detection, recognition and / or tracking. For example, the camera 15-A is an RGB (“red- green-blue”) camera deployed for surveillance applications at a public location.

[0076] Fig. 2B schematically illustrates in a block diagram an embodiment of an information processing system 1-B for updating the vision-based Al model 50, which is discussed in the following.

[0077] In this embodiment, the information processing system 1-B is deployed in a vehicle 16 and the polarization camera 10 is deployed in the vehicle 16 for acquiring polarization images of the real surrounding of the vehicle for generating training images for the vision-based Al model 50.

[0078] The polarization images are transmitted to a board-computer 13-B which is an embodiment of an information processing device.

[0079] The board-computer 13-B includes the polarization-based inverse rendering unit 20, the parameter modifying unit 30, the rendering unit 40, which will be discussed in more detail under reference of Figs. 4 to 9, and the vision-based Al model 50.

[0080] The vision-based Al model 50 is already deployed in this embodiment such that it is executed by the board-computer 13-B in the inference stage in which it takes images from a camera 15-B as input for object detection, recognition and / or tracking. For example, the camera 15-A is an RGB (“red-green-blue”) camera deployed for monitoring the environment of the vehicle 16 for advanced driver assistance functionality.

[0081] Fig. 3 schematically illustrates in a flow diagram an embodiment of an information processing method 900, which is discussed in the following under reference of Figs. 3 to 9.

[0082] The information processing method 900 may be performed by any information processing system as described herein such as the information processing system 1 of Fig. 1, the information processing system 1-A of Fig. 2A and the information processing system 1-B of Fig. 2B.

[0083] At 910, one or more polarization images are captured. At 920, the polarization images are decomposed into a plurality of scene parameters by using polarization-based inverse rendering.

[0084] Referring to Fig. 4, which schematically illustrates an embodiment of polarization-based inverse rendering.

[0085] A polarization image, e.g. the polarization image 12 of Fig. 2A, is input into the polarizationbased inverse rendering unit 20 which outputs the 3D shapes of the objects, the materials of the objects and the illumination(s) of the scene.

[0086] Referring to Fig. 5, which schematically illustrates in a block diagram an embodiment of the polarization-based inverse rendering unit 20.

[0087] The polarization-based inverse rendering unit 20 includes a decomposition unit 210, a polarization rendering unit 220 and a loss calculation unit 230.

[0088] The decomposition unit 210 decomposes the polarization image illustrating a scene into a plurality of scene parameters such as 3D shapes of objects, materials of objects and illumination(s) of the scene. The plurality of scene parameters may include additionally or alternatively diffuse and specular reflections, albedo and shading.

[0089] The decomposition unit 210 may be or may include or may use any known physical -based or neural network-based polarization-based inverse Tenderer.

[0090] The materials may be represented by its diffuse and specular albedo, its roughness and / or the parameters of the Disney BRDF (“bidirectional reflectance distribution function”).

[0091] The polarization rendering unit 220 renders, based on the plurality of scene parameters, a new rendered polarization image. The polarization rendering unit 220 may be, in some embodiments, the same as the rendering unit 40.

[0092] The loss calculation unit 230 calculates a difference between the original polarization image and the new rendered polarization image for estimating whether the scene parameter decomposition is accurate enough. The polarization rendering unit 220 may be or may include or may use any known physical-based or neural network-based Tenderer.

[0093] The loss calculation 230 provides feedback to the decomposition unit 210 for adapting the scene parameter decomposition.

[0094] Once the scene parameter decomposition is accurate enough, the plurality of scene parameters is output.

[0095] Referring to Fig. 3, at 930, the plurality of scene parameters is modified. Referring to Fig. 6, which schematically illustrates an embodiment of parameter modification.

[0096] The plurality of scene parameters of the polarization image 12 is input to the parameter modifying unit 30 which changes the shape and material of the object and the illumination of the scene.

[0097] Moreover, the parameter modifying unit 30 detects personal information based on the plurality of scene parameters and modifies the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy-free information.

[0098] Referring to Fig. 7, which schematically illustrates in a block diagram an embodiment of the parameter modification unit 30.

[0099] The parameter modification unit 30 includes a personal information detection unit 310, a personal information modifying unit 320, a database and an other information modifying unit 301.

[0100] The database 330 may include a set of artificial 3D models of human faces, dummy license plates, parameters sampled from a generative Al or the like for concealing the personal information.

[0101] The personal information detection unit 310 detects personal information based on the plurality of scene parameters, for example, faces and license plates or the like. The personal information detection unit 310 may use any known data-driven or analytical method.

[0102] The personal information modifying unit 320 modifies the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy -free information. For example, the personal information modifying unit 320 uses the database 330 for modification.

[0103] The other information modifying unit 301 changes the shape and material of the object and the illumination of the scene.

[0104] Referring to Fig. 8, which schematically illustrates in a block diagram an embodiment of the parameter modification unit 30.

[0105] The parameter modification unit 30 may use feedback from the vision-based Al model 50 to modify the at least one scene parameter of the plurality of scene parameters.

[0106] This may allow to modify the scene parameters which influence the accuracy of the (deployed) vision-based Al model 50 most. The vision-based Al model 50 includes a deep neural network unit 510 and a loss calculation unit 520.

[0107] The vision-based Al model 50 provides the loss information back to the other information modifying unit 301.

[0108] For example, if the loss calculated by the loss calculation unit 520 is worse when Parameter 1 is modified, the other information modifying unit 301 adds more varieties for Parameter 1 than for others.

[0109] As another example, the other information modifying unit 301 is differentiable such that the calculated by the loss calculation unit 520 can be backpropagated to the other information modifying unit 301 using, e.g., gradient-based methods.

[0110] Referring to Fig. 3, at 940, the images are rendered based on the modified images, as schematically illustrated in Fig. 9.

[0111] As illustrated in Fig. 9, a variety of combinations between the modified scene parameters are possible such that a variety of different images can be generated for training the vision-based Al model 50.

[0112] Referring to Fig. 3, at 950, the vision-based Al model 50 is trained by using the rendered images.

[0113] At 960, it is determined whether the vision-based Al model 50 is accurate enough.

[0114] If yes, then the update procedure is terminated.

[0115] If not, the scene parameters are further modified.

[0116] Fig. 10 schematically illustrates in a flow diagram an embodiment of an information processing method 1000, which is discussed in the following.

[0117] The information processing method 1000 may be performed by the information processing device or the information processing system as described herein.

[0118] At 1010, polarization image data of a scene is obtained, as discussed herein.

[0119] At 1020, the scene is decomposed into a plurality of scene parameters by using inverse rendering based on the polarization image data, as discussed herein.

[0120] At 1030, personal information is detected based on the plurality of scene parameters, as discussed herein.

[0121] At 1040, at least one scene parameter of the plurality of scene parameters is modified, as discussed herein. At 1050, the at least one scene parameter of plurality of scene parameters is modified such that the detected personal information is concealed by replacing the detected personal information with privacy-free information, as discussed herein.

[0122] At 1060, the at least one scene parameter of the plurality of scene parameters is further rendered based on feedback from the vision-based artificial intelligence model, as discussed herein.

[0123] At 1070, one or more images are rendered based on the plurality of modified scene parameters, as discussed herein.

[0124] At 1080, the vision-based artificial intelligence model is updated based on the rendered one or more images, as discussed herein.

[0125] At 1090, the vision-based artificial intelligence model is executed, as discussed herein.

[0126] It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding.

[0127] Please note that the division of the information processing device into units 20 to 40 is only made for illustration purposes and that the present disclosure is not limited to any specific division of functions in specific units. For instance, the information processing device could be implemented by a respective programmed processor, field programmable gate array (FPGA) and the like.

[0128] All units described in this specification can, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such units and entities can, if not stated otherwise, be implemented by software.

[0129] In so far as the embodiments of the disclosure described above are implemented, at least in part, using software-controlled data processing apparatus, it will be appreciated that a computer program providing such software control and a transmission, storage or other medium by which such a computer program is provided are envisaged as aspects of the present disclosure.

[0130] Note that the present technology can also be configured as described below.

[0131] (1) An information processing device for updating a vision-based artificial intelligence model, wherein the information processing device includes circuitry configured to: obtain polarization image data of a scene; decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modify at least one scene parameter of the plurality of scene parameters; render one or more images based on the plurality of modified scene parameters; and update the vision-based artificial intelligence model based on the rendered one or more images.

[0132] (2) The information processing device of (1), wherein the vision-based artificial intelligence model is deployed.

[0133] (3) The information processing device of (1) or (2), wherein the plurality of scene parameters includes a shape and a material of an object and an illumination of the scene.

[0134] (4) The information processing device of (3), wherein modifying at least one scene parameter of the plurality of scene parameters includes at least one of changing the shape and material of the object and the illumination of the scene.

[0135] (5) The information processing device of anyone of (1) to (4), wherein the circuitry is further configured to detect personal information based on the plurality of scene parameters.

[0136] (6) The information processing device of (5), wherein the circuitry is further configured to modify the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy- free information.

[0137] (7) The information processing device of anyone of (1) to (6), wherein the circuitry is further configured to modify the at least one scene parameter of the plurality of scene parameters further based on feedback from the vision-based artificial intelligence model.

[0138] (8) An information processing system for updating a vision-based artificial intelligence model, wherein the information processing system includes: a polarization camera system configured to acquire polarization image data of a scene; and an information processing device including circuitry configured to: obtain the polarization image data, decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data, modify at least one scene parameter of the plurality of scene parameters, render one or more images based on the plurality of modified scene parameters, update the vision-based artificial intelligence model based on the rendered one or more images. (9) The information processing system of (8), wherein the polarization camera system includes a first polarization camera configured to acquire first polarization image data of the scene from a first viewpoint and a second polarization camera configured to acquire second polarization image data of the scene from a second viewpoint; and wherein the polarization image data include the first and second polarization image data.

[0139] (10) The information processing system of (8) or (9), wherein the vision-based artificial intelligence model is deployed in the information processing system, and wherein the circuitry is further configured to execute the vision-based artificial intelligence model.

[0140] (11) An information processing method for updating a vision-based artificial intelligence model, wherein the information processing method includes: obtaining polarization image data of a scene; decomposing the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modifying at least one scene parameter of the plurality of scene parameters; rendering one or more images based on the plurality of modified scene parameters; and updating the vision-based artificial intelligence model based on the rendered one or more images.

[0141] (12) The information processing method of (11), wherein the vision-based artificial intelligence model is deployed.

[0142] (13) The information processing method of (11) or (12), wherein the plurality of scene parameters includes a shape and a material of an object and an illumination of the scene.

[0143] (14) The information processing method of (13), wherein modifying at least one scene parameter of the plurality of scene parameters includes at least one of changing the shape and material of the object and the illumination of the scene.

[0144] (15) The information processing method of anyone of (11) to (14), further including: detecting personal information based on the plurality of scene parameters.

[0145] (16) The information processing method of (15), further including: modifying the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy-free information.

[0146] (17) The information processing method of anyone of (11) to (16), further including: modifying the at least one scene parameter of the plurality of scene parameters further based on feedback from the vision-based artificial intelligence model.

[0147] (18) The information processing method of anyone of (11) to (17), further including: acquiring the polarization image data. (19) The information processing method of (18), further including: acquiring first polarization image data of the scene from a first viewpoint; acquiring second polarization image data of the scene from a second viewpoint; and wherein the polarization image data include the first and second polarization image data.

[0148] (20) The information processing method of (18) or (19), further including: executing the vision-based artificial intelligence model.

[0149] (21) A computer program comprising program code causing a computer to perform the method according to anyone of (11) to (20), when being carried out on a computer.

[0150] (22) A non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the method according to anyone of (11) to (20) to be performed.

Claims

CLAIMS1. An information processing device for updating a vision-based artificial intelligence model, comprising circuitry configured to: obtain polarization image data of a scene; decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modify at least one scene parameter of the plurality of scene parameters; render one or more images based on the plurality of modified scene parameters; and update the vision-based artificial intelligence model based on the rendered one or more images.

2. The information processing device of claim 1, wherein the vision-based artificial intelligence model is deployed.

3. The information processing device of claim 1, wherein the plurality of scene parameters includes a shape and a material of an object and an illumination of the scene.

4. The information processing device of claim 3, wherein modifying at least one scene parameter of the plurality of scene parameters includes at least one of changing the shape and material of the object and the illumination of the scene.

5. The information processing device of claim 1, wherein the circuitry is further configured to detect personal information based on the plurality of scene parameters.

6. The information processing device of claim 5, wherein the circuitry is further configured to modify the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy- free information.

7. The information processing device of claim 1, wherein the circuitry is further configured to modify the at least one scene parameter of the plurality of scene parameters further based on feedback from the vision-based artificial intelligence model.

8. An information processing system for updating a vision-based artificial intelligence model, wherein the information processing system comprises: a polarization camera system configured to acquire polarization image data of a scene; and an information processing device including circuitry configured to: obtain the polarization image data,decompose the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data, modify at least one scene parameter of the plurality of scene parameters, render one or more images based on the plurality of modified scene parameters, update the vision-based artificial intelligence model based on the rendered one or more images.

9. The information processing system of claim 8, wherein the polarization camera system includes a first polarization camera configured to acquire first polarization image data of the scene from a first viewpoint and a second polarization camera configured to acquire second polarization image data of the scene from a second viewpoint; and wherein the polarization image data include the first and second polarization image data.

10. The information processing system of claim 8, wherein the vision-based artificial intelligence model is deployed in the information processing system, and wherein the circuitry is further configured to execute the vision-based artificial intelligence model.

11. An information processing method for updating a vision-based artificial intelligence model, comprising: obtaining polarization image data of a scene; decomposing the scene into a plurality of scene parameters by using inverse rendering based on the polarization image data; modifying at least one scene parameter of the plurality of scene parameters; rendering one or more images based on the plurality of modified scene parameters; and updating the vision-based artificial intelligence model based on the rendered one or more images.

12. The information processing method of claim 11, wherein the vision-based artificial intelligence model is deployed.

13. The information processing method of claim 11, wherein the plurality of scene parameters includes a shape and a material of an object and an illumination of the scene.

14. The information processing method of claim 13, wherein modifying at least one scene parameter of the plurality of scene parameters includes at least one of changing the shape and material of the object and the illumination of the scene.

15. The information processing method of claim 11, further comprising: detecting personal information based on the plurality of scene parameters.

16. The information processing method of claim 15, further comprising: modifying the at least one scene parameter of plurality of scene parameters such that the detected personal information is concealed by replacing the detected personal information with privacy-free information.

17. The information processing method of claim 11, further comprising: modifying the at least one scene parameter of the plurality of scene parameters further based on feedback from the vision-based artificial intelligence model.

18. The information processing method of claim 11, further comprising: acquiring the polarization image data.

19. The information processing method of claim 18, further comprising: acquiring first polarization image data of the scene from a first viewpoint; acquiring second polarization image data of the scene from a second viewpoint; and wherein the polarization image data include the first and second polarization image data.

20. The information processing method of claim 18, further comprising: executing the vision-based artificial intelligence model.

Images