Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises: an acquisition means for acquiring a striped image representing a three-dimensional shape of a measurement target as a change in a two-dimensional pattern; a generation means for generating three-dimensional model data on the basis of shape characteristics common to a plurality of measurement targets; a calculation means for calculating the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the striped image; and a restoration means for restoring the three-dimensional shape of the measurement target by adding the three-dimensional model data to the difference.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include one that generates data representing a three-dimensional shape from images captured by a camera. For example, Patent Document 1 discloses a technology in which a person's head is captured by a right and left camera unit, and the three-dimensional shape is calculated from the images captured by each camera unit and output to a display or the like.

[0003] International Publication No. 2022 / 034694

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises an acquisition means for acquiring a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern; a generation means for generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects; a calculation means for calculating the difference between wrapping model data that has been phase wrapped on the three-dimensional model data and a phase image obtained by analyzing the fringe image; and a restoration means for restoring the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference.

[0006] One aspect of the information processing method disclosed herein involves using at least one computer to acquire a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, generating three-dimensional model data based on shape characteristics common to multiple measurement objects, calculating the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and adding the three-dimensional model data to the difference to restore the three-dimensional shape of the measurement object.

[0007] One aspect of the recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which acquires a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, generates three-dimensional model data based on shape characteristics common to multiple measurement objects, calculates the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and adds the three-dimensional model data to the difference, thereby restoring the three-dimensional shape of the measurement object.

[0008] 1 is a block diagram showing the hardware configuration of a first information processing device. 2 is a perspective view showing the configuration of the first information processing device. 3 is a top view showing the configuration of the first information processing device. 4 is a graph showing changes in data due to phase wrapping. 5 is a graph showing problems caused by differences in unwrapping paths in phase unwrapping. 6 is a schematic diagram showing defects caused in three-dimensional shape data due to phase unwrapping. 7 is a block diagram showing the functional configuration of the first information processing device. 8 is a flowchart showing the flow of operations of the first information processing device. 9 is a chart showing a specific example of operations of the first information processing device. 10 is a flowchart showing the flow of operations of the second information processing device. 11 is a chart showing a specific example of operations of the second information processing device. 12 is a block diagram showing the functional configuration of a third information processing device. 13 is a flowchart showing the flow of operations of the third information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 9. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, the first information processing device 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, an output device 16, a first unit 21, and a second unit 22. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, first unit 21, and second unit 22 are connected to each other via a data bus 17. Note that the data bus 17 may be an interface other than a data bus (e.g., a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the first information processing device 1 via a network interface. The processor 11 executes the loaded computer program to perform various processes. When the processor 11 executes the loaded computer program, functional blocks related to the processes performed by the first information processing device 1 are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the first information processing device 1.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the first information processing device 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the first information processing device 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the first information processing device 1 to the outside. For example, the output device 16 may be a display device (e.g., a display or a monitor) that can display information related to the first information processing device 1. The output device 16 may also be a speaker or the like that can output information related to the information processing device 1 as audio.

[0020] The first unit 21 is configured to include a first camera 211 and a first irradiating unit 212. The first camera 211 is arranged to capture an image of the target from a first direction. The first irradiating unit 212 is arranged to irradiate light onto the target from the first direction (i.e., the shooting direction of the first camera 211). Similarly, the second unit 22 is configured to include a second camera 221 and a second irradiating unit 222. The second camera 221 is arranged to capture an image of the target from a second direction. The second irradiating unit 222 is arranged to irradiate light onto the target from the second direction (i.e., the shooting direction of the second camera 221). The specific arrangement of the first unit 21 and the second unit 22 will be described in detail later.

[0021] The first camera 211 and the second camera 221 may include a solid-state imaging element such as a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc. The first camera 211 and the second camera 221 may also include an optical system that forms an image of the subject on the imaging surface of the solid-state imaging element, a signal processing circuit that processes the output of the solid-state imaging element to obtain a luminance value for each pixel, etc.

[0022] The first irradiating unit 212 and the second irradiating unit 222 may be configured to project a periodic light pattern. For example, the first irradiating unit 212 and the second irradiating unit 222 may be configured to project sinusoidal wave patterns with different periods. The first irradiating unit 212 and the second irradiating unit 222 may be, for example, a DLP (Digital Light Processing) projector or a liquid crystal projector.

[0023] 1. For example, the first information processing device 1 may be configured to include only the processor 11, RAM 12, and ROM 13 among the above-mentioned components. In this case, the storage device 14, input device 15, output device 16, first unit 21, and second unit 11 may be provided as devices external to the first information processing device 1. Furthermore, some of the calculation functions of the first information processing device 1 may be realized by an external server, a cloud, or the like.

[0024] (Specific Device Configuration) Next, a specific configuration of the first information processing device 1 (particularly, an example of the arrangement of the first unit 21 and the second unit 22) will be described with reference to Figures 2 and 3. Figure 2 is a perspective view showing the configuration of the first information processing device. Figure 3 is a top view showing the configuration of the first information processing device.

[0025] 2 and 3 , the first information processing device 1 is configured as a device that photographs the head of the subject 50 (specifically, a facial region including the face of the subject 50 and a neck region including the neck of the subject 50). The subject 50 may be photographed, for example, with the subject sitting in front of the first information processing device 1. Alternatively, the subject 50 may be photographed with the subject standing in front of the first information processing device 1.

[0026] The first unit 21 in the first information processing device 1 is disposed on the left front side as viewed from the target 50. Therefore, the first camera 211 photographs the head of the target 50 from the left front side. Furthermore, the first irradiating section 212 irradiates light from the left front side toward the head of the target 50. The second unit 22 in the first information processing device 1 is disposed on the right front side as viewed from the target 50. Therefore, the second camera 221 photographs the head of the target 50 from the right front side. Furthermore, the second irradiating section 222 irradiates light from the right front side toward the head of the target 50.

[0027] The first information processing device 1 includes a third camera 300 that captures an image of the head of the target 50 from the front of the target 50. The third camera 300 may be a camera similar to the first camera 211 and the second camera 221 described above. However, the third camera 300 is not an essential component of the first information processing device 1.

[0028] (Phase Unwrapping) Next, the phase unwrapping performed in the first information processing device 1 will be described with reference to Fig. 4. Fig. 4 is a graph showing changes in data due to phase wrapping.

[0029] As shown in Figure 4, the true shape data of the object to be measured is represented by a straight line in which z monotonically increases as x increases. When such an object is measured by the first information processing device 1, folding back of the measurement value (i.e., phase wrapping) occurs in the part exceeding the height (specifically, the phase period) specified by the device. Therefore, the measurement data measured by the first information processing device 1 is data in which the measurement value is folded at a predetermined phase period (here, 2π).

[0030] When restoring the folded data described above, a process called phase unwrapping is performed. For example, adjacent measurement points are observed, and phase unwrapping is performed according to the phase difference between their values. Specifically, if the phase difference between adjacent points is less than half the phase period (i.e., π), it is determined that no phase value wrapping has occurred, and phase unwrapping is not performed. On the other hand, if the phase difference between adjacent points is equal to or greater than half the phase period, it is determined that phase value wrapping has occurred, and phase unwrapping is performed. When performing phase unwrapping, for example, a process of adding a value equivalent to the phase period to the phase value may be performed.

[0031] (Problems with Phase Unwrapping) Next, problems that may arise in the above-described phase unwrapping will be specifically described with reference to Fig. 5 and Fig. 6. Fig. 5 is a graph showing problems that arise due to differences in unwrapping paths in phase unwrapping. Fig. 6 is a schematic diagram showing defects that arise in three-dimensional shape data due to phase unwrapping.

[0032] In Fig. 5, when performing two-dimensional phase unwrapping, unlike in the one-dimensional case, it is required to appropriately determine the path along which phase unwrapping should be performed. For example, in the example of measuring the object shown in Fig. 5, when phase unwrapping is performed along a clockwise path and when phase unwrapping is performed along a counterclockwise path, the restored shape will differ. Specifically, when phase unwrapping is performed along a counterclockwise path, the same shape as the true shape is restored, whereas when phase unwrapping is performed along a clockwise path, a shape different from the true shape (a shape concave downward) is restored.

[0033] The above-mentioned problems can also occur when measuring the head of the subject 50 in FIG. 6 . For example, in a human head, there is a relatively large step between the chin and neck. Such a large step exceeds the measurable step (in other words, the phase period), and the portions before and after the step may appear to be of the same height. Therefore, if the connection path is not properly set, phase unwrapping may not be performed properly. For example, when generating three-dimensional shape data by performing phase unwrapping on a phase image, defects may occur in the generated three-dimensional shape data. Specifically, as shown in the example of FIG. 6 , there is a risk that the generated three-dimensional shape data may show a protruding neck region.

[0034] The first information processing device 1 is configured as a device that can solve the above-mentioned problem (that is, the problem in phase unwrapping).

[0035] (Functional Configuration) Next, the functional configuration of the first information processing device 1 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of the first information processing device.

[0036] 7, the first information processing device 1 is configured to include, as components for realizing its functions, an image acquisition unit 110, a three-dimensional model data generation unit 120, a difference calculation unit 130, and a restoration unit 140, in addition to the above-mentioned first camera 211 and second camera 221. Each of the image acquisition unit 110, the three-dimensional model data generation unit 120, the difference calculation unit 130, and the restoration unit 140 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0037] The image acquisition unit 110 is configured to acquire a fringe image of the object 50. The fringe image here is an image that represents the three-dimensional shape of the object 50 as a change in a two-dimensional pattern. The image acquisition unit 110 may be configured to use coherent light such as a laser as a light source, and acquire an interference fringe image by interfering the reflected or transmitted light of the light irradiated on the object 50 with the reference light, and the light obtained without passing through the object 50 with the reference light. The first unit 21 and the second unit 22 in the first information processing device 1 may have the function for acquiring such a fringe image. For example, the image acquisition unit 110 may be configured to acquire an image captured by the first camera 221 (hereinafter referred to as the "first image") and an image captured by the second camera 221 (hereinafter referred to as the "second image") as fringe images of the object 50. The image acquisition unit 110 may be configured to perform image analysis processing on the acquired fringe images. For example, the image acquisition unit 110 may be configured to acquire a phase image by performing image analysis processing on the fringe image. The image acquisition unit 110 may also be configured to acquire an amplitude image by performing image analysis processing on the fringe image. Information about each image acquired by the image acquisition unit 110 is configured to be output to each of the three-dimensional model data generation unit 120 and the difference calculation unit 130.

[0038] The three-dimensional model data generation unit 120 is configured to generate three-dimensional model data based on shape characteristics common to multiple measurement targets. The three-dimensional model data is a reference model that reflects the shape characteristics of the target 50. For example, the three-dimensional model data may be a model having an average shape of the head of the target 50. Alternatively, the three-dimensional model data may be a model that exaggerates the shape characteristics of the head of the target 50. More specifically, the three-dimensional model data may be a model having a shape that combines a sphere corresponding to the head and a cylinder corresponding to the neck. The three-dimensional model data generation unit 120 may generate three-dimensional model data in advance according to the type of measurement target. Alternatively, the three-dimensional model data generation unit 120 may generate three-dimensional model data each time based on an image of the target 50 acquired by the image acquisition unit 110.

[0039] The 3D model data generation unit 120 may generate (estimate) 3D model data using an estimation model trained by machine learning. For example, the 3D model data generation unit 120 may use a first estimation model trained by machine learning to estimate the 3D shape of the measurement object. The first estimation model estimates the 3D shape of the object from an image of the object 50. The 3D model data generation unit 120 then generates 3D model data from the 3D shape estimated by the first estimation model. In this case, 3D model data based on the shape characteristics of the measurement object can be appropriately generated. Alternatively, a second estimation model trained by machine learning to estimate the depth of an image captured with a monocular camera may be used. The second estimation model estimates the depth in the image (in other words, the distance to the object 50) from the image of the object 50. The 3D model data generation unit 120 then generates 3D model data based on the depth estimated by the second estimation model. In this case, 3D model data based on the shape characteristics of the measurement object can be appropriately generated.

[0040] The difference calculation unit 130 is configured to be able to calculate the difference between data obtained by performing phase wrapping on the 3D model data generated by the 3D model data generation unit 120 (hereinafter referred to as "wrapped model data") and a phase image obtained by analyzing a fringe image. Phase wrapping is a process of folding the phase values ​​of the 3D model data according to a phase period. The difference calculation unit 130 may have a function of performing phase wrapping on the 3D model data to generate wrapped model data. Alternatively, the difference calculation unit 130 may be configured to acquire wrapped data that has undergone phase wrapping without performing phase wrapping itself.

[0041] The restoration unit 140 is configured to be able to execute a process of adding three-dimensional model data to the difference calculated by the difference calculation unit 130 (i.e., the difference between the wrapping model data and the phase image). The restoration unit 140 restores the three-dimensional shape of the object 50 by adding the three-dimensional model data to the difference. In other words, the restoration unit 140 restores the three-dimensional shape of the object 50 indicated by the original phase image.

[0042] (Flow of Operation) Next, the flow of operation of the first information processing device 1 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of operation of the first information processing device.

[0043] 8, when the operation of the first information processing device 1 is started, the image acquisition unit 110 first acquires a fringe image (step S101), and then performs image analysis processing on the fringe image to acquire a phase image (step S102).

[0044] Next, the three-dimensional model data generating unit 120 generates three-dimensional model data (step S103). Note that if three-dimensional model data has been generated in advance, the three-dimensional model data generating unit 120 may execute a process of reading out the generated three-dimensional model data.

[0045] Next, the difference calculation unit 130 performs phase wrapping on the three-dimensional model data generated by the three-dimensional model data generation unit 120 to generate wrapping model data (step S104). Then, the difference calculation unit 140 calculates the difference between the wrapping model data and the phase image (step S105).

[0046] Next, the restoration unit 140 adds the three-dimensional model data to the difference calculated by the difference calculation unit 130 to restore the three-dimensional shape of the object 50 (step S106). The restoration unit 140 may perform a process of outputting data indicating the restored three-dimensional shape of the object 50.

[0047] (Example of Operation) Next, a specific example of operation of the first information processing device 1 will be described with reference to Fig. 9. Fig. 9 is a chart showing a specific example of operation of the first information processing device.

[0048] 9 , in the first information processing device 1, the image acquisition unit 110 acquires a fringe image. The phase indicated by the phase image obtained from the fringe image is represented as the device-acquired phase. The device-acquired phase is in a folded state in which the portion exceeding the phase period is folded.

[0049] On the other hand, in the first information processing device 1, the three-dimensional model data generation unit 120 generates three-dimensional model data. The phase indicated by the three-dimensional model data is expressed as the model phase before wrapping. The model phase before wrapping indicates the general shape of the head of the target 50, and no aliasing occurs according to the phase cycle.

[0050] The three-dimensional model data is subjected to phase wrapping to generate wrapped model data. The phase indicated by the wrapped model data is expressed as a model phase after wrapping. The model phase after wrapping is in a state where the portion exceeding the phase period is folded, similar to the device-acquired phase.

[0051] Once the wrapping model data is generated, the difference calculation unit 130 calculates the difference between the wrapping model data and the phase image. The phase indicated by the difference is expressed as the phase obtained after model subtraction. As shown in the figure, the phase obtained after model subtraction has few irregularities. Specifically, it does not have any steps that exceed the phase period.

[0052] Once the difference is calculated, the restoration unit 140 adds the three-dimensional model data to the difference to restore three-dimensional shape data of the object 50. The phase corresponding to the restored three-dimensional shape is represented as the post-restoration acquired phase. The post-restoration acquired phase accurately reflects the three-dimensional shape of the object 50. Specifically, it is the same as the phase obtained by appropriately phase unwrapping the device-acquired phase.

[0053] (Technical Effects) Next, technical effects obtained by the first information processing device 1 will be described.

[0054] As described with reference to FIGS. 1 to 9 , the first information processing device 1 calculates the difference between the phase image of the object 50 and the wrapping model data. Then, the three-dimensional model data is added to the calculated difference to reconstruct the three-dimensional shape of the object 50. In this manner, even if the phase image of the object 50 contains an unmeasurable step (i.e., a step exceeding the phase period) due to the measurement device, it is possible to properly reconstruct the three-dimensional shape of the object 50. Note that when performing phase unwrapping on a phase image, problems such as those described with reference to FIGS. 5 and 6 may occur. Furthermore, when using a method for estimating a three-dimensional shape by depth estimation using deep learning or the like, there is a risk that the estimation accuracy is insufficient and a shape different from the actual shape is estimated. However, according to the first information processing device, by using three-dimensional model data, it is possible to achieve measurement of a three-dimensional shape with high accuracy and few problems.

[0055] Second Embodiment A second information processing device 1 will be described with reference to Figures 10 and 11. The second information processing device 1 differs in some configurations and operations from the first information processing device 1 described above, but other parts may be similar to the first information processing device 1. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0056] (Operation Flow) First, the operation flow of the second information processing device 1 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operation flow of the second information processing device. Note that in Fig. 10, the same processes as those described in Fig. 8 are denoted by the same reference numerals.

[0057] 10 , when the operation of the second information processing device 1 is started, the image acquisition unit 110 first acquires a fringe image (step S101), and then performs image analysis processing on the fringe image to acquire a phase image (step S102).

[0058] Next, the three-dimensional model data generating unit 120 generates three-dimensional model data (step S103). Note that if three-dimensional model data has been generated in advance, the three-dimensional model data generating unit 120 may execute a process of reading out the generated three-dimensional model data.

[0059] Next, the difference calculation unit 130 performs phase wrapping on the three-dimensional model data generated by the three-dimensional model data generation unit 120 to generate wrapping model data (step S104). Then, the difference calculation unit 140 calculates the difference between the wrapping model data and the phase image (step S105).

[0060] Next, the restoration unit 140 performs phase unwrapping on the difference calculated by the difference calculation unit 130 (step S201). Then, the restoration unit 140 adds the three-dimensional model data to the phase-unwrapped difference to restore the three-dimensional shape of the object 50 (step S202). The restoration unit 140 may perform processing to output data indicating the restored three-dimensional shape of the object 50.

[0061] (Operation Example) Next, a specific operation example of the second information processing device 1 will be described with reference to Fig. 11. Fig. 11 is a chart showing a specific operation example of the second information processing device.

[0062] 11 , in the second information processing device 1, the image acquisition unit 110 acquires a fringe image. The phase indicated by the phase image obtained from the fringe image is represented as the device-acquired phase. The device-acquired phase is in a folded state in the portion exceeding the phase period.

[0063] On the other hand, in the first information processing device 1, the three-dimensional model data generation unit 120 generates three-dimensional model data. The phase indicated by the three-dimensional model data is expressed as the model phase before wrapping. The model phase before wrapping indicates the general shape of the head of the target 50, and no aliasing occurs according to the phase cycle.

[0064] The three-dimensional model data is subjected to phase wrapping to generate wrapped model data. The phase indicated by the wrapped model data is expressed as a model phase after wrapping. The model phase after wrapping is in a state where the portion exceeding the phase period is folded, similar to the device-acquired phase.

[0065] Once the wrapping model data is generated, the difference calculation unit 130 calculates the difference between the wrapping model data and the phase image. The phase indicated by the difference is expressed as the phase obtained after model subtraction. As shown in the figure, the phase obtained after model subtraction has few irregularities. Specifically, it does not have any steps that exceed the phase period.

[0066] Once the difference is calculated, the restoration unit 140 performs phase unwrapping on the difference. Therefore, even if a portion exceeding the phase period exists in the phase obtained after model subtraction, the folded portion will be restored. Note that, since the phase obtained after model subtraction has few steps, typically, no portion exceeds the phase period. However, for example, if the head of the subject 50 is tilted obliquely, the tilt may cause a portion exceeding the phase period. By performing phase wrapping, the appropriate phase value can be restored even in such a case.

[0067] After phase unwrapping is performed on the difference, the restoration unit 140 adds the three-dimensional model data to the difference to restore three-dimensional shape data of the object 50. The phase corresponding to the restored three-dimensional shape is represented as the post-restoration acquired phase. The post-restoration acquired phase accurately reflects the three-dimensional shape of the object 50. Specifically, it is the same as the phase obtained by appropriately phase unwrapping the device-acquired phase.

[0068] (Technical Effects) Next, technical effects obtained by the second information processing device 1 will be described.

[0069] 10 and 11 , in the second information processing device 1, phase unwrapping is performed on the difference calculated by the difference calculation unit 130. In this way, it becomes possible to appropriately restore steps that do not depend on the shape characteristics of the object 50 (i.e., unevenness that is not expressed in the three-dimensional model data).

[0070] <Third embodiment> A third information processing device 1 will be described with reference to Figures 12 and 13. The third information processing device 1 differs in some configurations and operations from the first and second information processing devices 1 described above, but other parts may be similar to the first and second information processing devices 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0071] (Functional Configuration) First, the functional configuration of the third information processing device 1 will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 12, the same elements as those shown in Fig. 7 are denoted by the same reference numerals.

[0072] 12, the third information processing apparatus 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, an image acquisition unit 110, a three-dimensional model data generation unit 120, a difference calculation unit 130, a restoration unit 140, and a guide unit 150. That is, the third information processing apparatus 1 further includes a guide unit 150 in addition to the configuration of the first embodiment (see FIG. 7). Note that the guide unit 150 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0073] The guide unit 150 is configured to be able to determine whether or not both a face region and a neck region are included in the stripe image acquired by the image acquisition unit 110. Alternatively, the guide unit 150 may be configured to be able to determine whether or not both a face region and a neck region are included in the phase image or the amplitude image. If the image acquired by the image acquisition unit 110 does not include both a face region and a neck region, the guide unit 150 outputs guide information for capturing an image including both a face region and a neck region. Note that if the image acquired by the image acquisition unit 110 includes both a face region and a neck region, the guide unit 150 does not need to output guide information.

[0074] The guide information may be guidance information that prompts the user to move to an appropriate position (i.e., a position where both the face region and the neck region are within the imaging range of the first camera 211 and the second camera 221). For example, the guide unit 150 may output guide information that outputs a message such as "Please move a little further away from the camera" or "Please move your face up a little more." Such a message may be displayed as text on a display or the like, or as an illustration. It may also be output as audio by a speaker or the like.

[0075] Furthermore, the guide information may be control information that changes the imaging range by controlling the first camera 211 and the second camera 221. For example, the guide unit 150 may output guide information that controls the zoom or pan of the first camera 211 and the second camera 221. Alternatively, the guide unit 150 may output guide information that changes the angle or position of the first camera 211 and the second camera 221.

[0076] (Operation Flow) Next, the operation flow of the third information processing device 1 will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 13, the same processes as those shown in Fig. 8 are denoted by the same reference numerals.

[0077] As shown in FIG. 13, when the operation of the third information processing apparatus 1 is started, the image acquisition unit 110 acquires a stripe image (step S101).

[0078] Next, the guide unit 140 determines whether or not the striped image includes both a face region and a neck region (step S301). If the striped image does not include both a face region and a neck region (step S301: NO), the guide unit 140 outputs guide information (step S302).

[0079] After the guide information is output, the process of step S101 may be executed again. That is, after the guide information has been used to place both the face region and the neck region within the imaging ranges of the first camera 211 and the second camera 221, a striped image of the target 50 may be captured again.

[0080] On the other hand, if the stripe image includes both a face region and a neck region (step S301: YES), the image acquisition unit 110 performs image analysis processing on the stripe image to acquire a phase image (step S102).

[0081] Next, the three-dimensional model data generating unit 120 generates three-dimensional model data (step S103). Note that if three-dimensional model data has been generated in advance, the three-dimensional model data generating unit 120 may execute a process of reading out the generated three-dimensional model data.

[0082] Next, the difference calculation unit 130 performs phase wrapping on the three-dimensional model data generated by the three-dimensional model data generation unit 120 to generate wrapping model data (step S104). Then, the difference calculation unit 140 calculates the difference between the wrapping model data and the phase image (step S105).

[0083] Next, the restoration unit 140 adds the three-dimensional model data to the difference calculated by the difference calculation unit 130 to restore the three-dimensional shape of the object 50 (step S106). The restoration unit 140 may perform a process of outputting data indicating the restored three-dimensional shape of the object 50.

[0084] (Technical Effects) Next, technical effects obtained by the third information processing device 1 will be described.

[0085] As described with reference to FIGS. 12 and 13 , the third information processing device 1 outputs guide information when the image acquired by the image acquisition unit 110 does not include both the face region and the neck region of the target 50. In this way, even if the position of the target 50 is shifted, the position can be adjusted using the guide information, and an image including the face region and the neck region can be appropriately acquired. That is, an image of the human head, which is the measurement target, can be appropriately acquired. Note that, as already described, when performing phase unwrapping on an image of the human head, there is a risk of problems occurring due to the step between the face region and the neck region. However, the third information processing device 1 can appropriately restore the original shape of an image of the human head including the face region and the neck region by processing using three-dimensional model data.

[0086] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0087] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0088] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0089] (Appendix 1) The information processing device described in Appendix 1 is an information processing device that includes an acquisition means for acquiring a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, a generation means for generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects, a calculation means for calculating the difference between wrapping model data that has been phase-wrapped on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and a restoration means for restoring the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference.

[0090] (Supplementary Note 2) The information processing device described in Supplementary Note 2 is the information processing device described in Supplementary Note 1, in which the restoration means restores the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference after phase unwrapping.

[0091] (Supplementary Note 3) The information processing device described in Supplementary Note 3 is the information processing device described in Supplementary Note 1 or 2, wherein the generation means generates the three-dimensional model data based on an estimation result of a first estimation model that has been machine-learned to estimate the three-dimensional shape of the measurement object.

[0092] (Supplementary Note 4) The information processing device described in Supplementary Note 4 is the information processing device described in Supplementary Note 1 or 2, wherein the generation means generates the three-dimensional model data using an estimation result of a second estimation model that has been machine-learned to estimate the depth of an image captured by a monocular camera.

[0093] (Supplementary Note 5) The information processing device according to Supplementary Note 5 is the information processing device according to any one of Supplementary Notes 1 to 4, in which the measurement target is a human head including the face region and the neck region.

[0094] (Appendix 6) The information processing device described in Appendix 6 is the information processing device described in Appendix 5, further comprising a guide means for outputting guide information for capturing an image including both the face area and the neck area when the striped image does not include both the face area and the neck area.

[0095] (Supplementary Note 7) The information processing method described in Supplementary Note 7 is an information processing method that, using at least one computer, acquires a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, generates three-dimensional model data based on shape characteristics common to a plurality of the measurement objects, calculates the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and adds the three-dimensional model data to the difference, thereby restoring the three-dimensional shape of the measurement object.

[0096] (Appendix 8) The recording medium described in Appendix 8 is a recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, which includes acquiring a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects, calculating a difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and adding the three-dimensional model data to the difference, thereby restoring the three-dimensional shape of the measurement object.

[0097] (Supplementary Note 9) The computer program described in Supplementary Note 9 is a computer program that causes at least one computer to execute an information processing method, which acquires a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern, generates three-dimensional model data based on shape characteristics common to a plurality of the measurement objects, calculates a difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image, and adds the three-dimensional model data to the difference, thereby restoring the three-dimensional shape of the measurement object.

[0098] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0099] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 21 First unit 211 First camera 212 First irradiation unit 22 Second unit 221 Second camera 222 Second irradiation unit 300 Third camera 50 Object 110 Image acquisition unit 120 Three-dimensional model data generation unit 130 Difference calculation unit 140 Restoration unit 150 Guide unit

Claims

1. An information processing device comprising: an acquisition means for acquiring a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern; a generation means for generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects; a calculation means for calculating the difference between wrapping model data that has been subjected to phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image; and a restoration means for restoring the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference.

2. The information processing device according to claim 1, wherein the restoration means restores the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference after phase unwrapping.

3. The information processing device according to claim 1 or 2, wherein the generating means generates the three-dimensional model data based on an estimation result of a first estimation model that has been machine-learned to estimate the three-dimensional shape of the measurement object.

4. The information processing device according to claim 1 or 2, wherein the generating means generates the three-dimensional model data using the estimation results of a second estimation model that has been machine-trained to estimate the depth of an image captured by a monocular camera.

5. The information processing device according to claim 1 or 2, wherein the measurement object is a human head including the face region and neck region.

6. An information processing device according to claim 5, further comprising a guide means for outputting guide information for capturing an image including both the face region and the neck region when the striped image does not include both the face region and the neck region.

7. An information processing method comprising: acquiring, by at least one computer, a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern; generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects; calculating the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image; and reconstructing the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference.

8. A recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, which comprises: acquiring a fringe image that represents the three-dimensional shape of a measurement object as a change in a two-dimensional pattern; generating three-dimensional model data based on shape characteristics common to a plurality of the measurement objects; calculating the difference between wrapping model data obtained by performing phase wrapping on the three-dimensional model data and a phase image obtained by analyzing the fringe image; and restoring the three-dimensional shape of the measurement object by adding the three-dimensional model data to the difference.

Images