Three-dimensional shape measuring device and program thereof

Abstract

To provide a three-dimensional shape measuring device that has high three-dimensional shape quality and has a less operation amount.SOLUTION: A three-dimensional shape measuring device 1 measures a three-dimensional shape of a subject by using images photographed by multiple fixed cameras 4 installed in different positions, and has concerned region cut-out means 2 for generating a cutout image in which a concerned region is cut out of an inputted cutout object image while the cutout object image is inputted from cutout object cameras 41, 42 that are one portion of the fixed cameras 4, and three-dimensional shape measurement means 3 for measuring a three-dimensional shape of a subject on the basis of a cutout image, and a camera parameter corresponding to the cutout image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional shape measuring device and a program therefor. [Background technology]

[0002] Photogrammetry is a known technique that estimates the three-dimensional shape of a subject based on a sequence of images taken from different viewpoints (for example, see Patent Document 1). Photogrammetry is a technique that simultaneously estimates the three-dimensional shape and shooting position of a subject based on a sequence of images of a stationary subject taken from various angles.

[0003] There is a volumetric capture technology that estimates the three-dimensional shape of a subject moment by moment based on multiple images captured by a multi-view camera.A volumetric capture technology is known that uses a combination of high-resolution, low-frame-rate images and low-resolution, high-frame-rate images to estimate the three-dimensional shape of a subject moment by moment in a dynamic and highly accurate manner (for example, Patent Document 2).

[0004] Furthermore, in volumetric capture technology, a technology has been proposed that adds a subject tracking function to a camera in order to expand the measurable area and improve the accuracy and resolution of the subject's three-dimensional shape and texture information (Non-Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6652253 [Patent Document 2] Patent No. 7235215 Publication [Non-patent literature]

[0006] [Non-Patent Document 1] Yamada, Suginoshi, Morioka, Misu, Mitsumine, and Seni, "3D Object Region Estimation Method for Automatic Object Tracking Shooting," Proceedings of the ITE Annual Conference, 21B-2 (2022) Summary of the Invention [Problem to be solved by the invention]

[0007] The aforementioned photogrammetry technology is designed for stationary subjects because it requires time to calibrate the camera, and has the problem of being unable to handle moving subjects. For this reason, with photogrammetry, if part or all of the subject moves while a series of images are being captured, the calculation process cannot be completed in time, and the correct three-dimensional shape cannot be obtained for the moving area, resulting in a deterioration in the shape accuracy of the stationary area. There is a demand for a reduction in the amount of calculation so that it can also handle moving subjects.

[0008] In volumetric capture technology, in order to cover a wider range of movement of the subject, it is necessary to increase the number of cameras or use wide-angle lenses. Increasing the number of cameras not only directly leads to an increase in cost, but also to an increase in the memory and time required for calculations. Furthermore, if the sensor resolution is the same, the wider the angle of view of the lens used, the larger the solid angle per pixel. As a result, it becomes difficult to capture the fine shape and texture of the subject (resolution decreases). On the other hand, if one tries to maintain a constant resolution, a higher-resolution sensor is required, which increases the amount of calculation.

[0009] The method of Patent Document 2 generates high-resolution, high-frame-rate video using super-resolution technology to improve the quality of the three-dimensional shape of a subject. In other words, the method of Patent Document 2 has the problem of requiring a huge amount of calculations because it processes high-resolution, high-frame-rate video and also performs super-resolution processing.

[0010] The method of Non-Patent Document 1 allows measurement of a wide area using a telephoto lens by controlling the camera to point at the subject. However, the method of Non-Patent Document 1 has a problem in that when a pan-tilt-zoom camera (PTZ camera, robot camera) is used, the quality of the three-dimensional shape is reduced due to mechanical errors of the camera.

[0011] Therefore, an object of the present invention is to provide a three-dimensional shape measuring device and a program therefor that can measure a high-quality three-dimensional shape with a small amount of calculation. [Means for solving the problem]

[0012] In order to solve the above problem, the three-dimensional shape measurement device of the present invention is a three-dimensional shape measurement device that measures the three-dimensional shape of a subject using images captured by multiple fixed cameras installed at different positions, and is configured to include: a region of interest cut-out means that receives a cut-out target image from a cut-out target camera that is some or all of the fixed cameras, and generates a cut-out image in which a region of interest is cut out from the input cut-out target image; and a three-dimensional shape measurement means that measures the three-dimensional shape of the subject based on the cut-out image and camera parameters corresponding to the cut-out image.

[0013] The region of interest cutting means is configured to include a region of interest detection means that detects a region having predetermined image characteristics as a region of interest in the image to be cut out, an image cutting means that generates a cut-out image by cutting out the region of interest detected by the region of interest detection means from the image to be cut out, and a parameter correction means that receives as input camera parameters of the camera to be cut out and converts the input camera parameters of the camera to be cut out in accordance with a method for transforming the region of interest, thereby calculating camera parameters corresponding to the cut-out image.

[0014] According to this configuration, the three-dimensional shape measuring device uses images taken by a fixed camera whose orientation does not change, so no mechanical errors occur and the quality of the three-dimensional shape can be improved. Furthermore, the three-dimensional shape measuring device processes a cut-out image that is a part of the cut-out target image, rather than the entire cut-out target image, and therefore the amount of calculation can be reduced.

[0015] The present invention can also be realized by a program for causing a computer to function as the above-mentioned three-dimensional shape measuring device. [Effects of the Invention]

[0016] According to the present invention, it is possible to improve the quality of a three-dimensional shape and reduce the amount of calculation. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing the configuration of a three-dimensional shape measurement system according to a first embodiment. [Figure 2] FIG. 2 is an explanatory diagram illustrating a region of interest in the first embodiment. [Figure 3] FIG. 2 is an explanatory diagram illustrating a region of interest in the first embodiment. [Figure 4] 4 is a flowchart showing the operation of the three-dimensional shape measuring apparatus according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of a three-dimensional shape measurement system according to a second embodiment. [Figure 6] FIG. 10 is a block diagram showing the configuration of a three-dimensional shape measurement system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. However, each embodiment described below is intended to embody the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following. Furthermore, the same means will be given the same reference numerals, and their description may be omitted.

[0019] (First embodiment) [Overview of the 3D shape measurement system] An overview of a three-dimensional shape measurement system 100 according to the first embodiment will be described with reference to FIG. The three-dimensional shape measurement system 100 measures the three-dimensional shape of a subject. As shown in FIG.

[0020] The three-dimensional shape measuring device 1 measures the three-dimensional shape of a subject using images taken by multiple fixed cameras 4 installed at different positions. The configuration of the three-dimensional shape measuring device 1 will be described in detail later.

[0021] The fixed cameras 4 are wide-angle cameras whose postures (pan, tilt, zoom) are fixed during shooting. The fixed cameras 4 may capture not only still images of the subject but also moving images of the subject. In the example of Fig. 1, four fixed cameras 4 are installed at different positions.

[0022] Here, among the multiple fixed cameras 4, those that capture the cropping target image described below, i.e., those connected to the region of interest cropping means 2, will be referred to as "cropping target cameras." Also, among the multiple fixed cameras 4, those that are directly connected to the three-dimensional shape measurement means 3 without going through the region of interest cropping means 2 will be referred to as "directly connected cameras."

[0023] Some of the fixed cameras 4 are cropping target cameras. In this embodiment, of the four fixed cameras 4, two are cropping target cameras 41 and 42, and the remaining two are directly-attached cameras 43 and 44. The resolution of the cropping target cameras 41 and 42 is higher than the resolution of the directly-attached cameras 43 and 44. Therefore, the resolution of the cropped image captured by the cropping target cameras 41 and 42 is higher than the resolution of the captured image captured by the directly-attached cameras 43 and 44. For example, the cropping target cameras 41 and 42 are 8K or 16K ultra-high definition wide-angle cameras, and the directly-attached cameras 43 and 44 are 4K wide-angle cameras.

[0024] The cropping target cameras 41 and 42 output the captured cropping target images and the camera parameters used when the cropping target images were captured to the region of interest cropping means 2. In addition, the directly connected cameras 43 and 44 output the captured images of the subject and the camera parameters used when the captured images were captured to the three-dimensional shape measurement means 3.

[0025] The three-dimensional shape of the subject is a three-dimensional model that represents the three-dimensional shape of the subject, such as a point cloud that represents the subject. The subject whose three-dimensional shape is to be measured is not particularly limited. For example, the subject may be a person, an animal, or an object. The fixed camera 4 may capture either a still image or a moving image of the subject. In this embodiment, the entire subject is described as being the subject of measurement of the three-dimensional shape, but only a portion of the subject may be the subject of measurement of the three-dimensional shape.

[0026] [Configuration of 3D shape measurement device] The configuration of the three-dimensional shape measuring device 1 will be described below. 1, the three-dimensional shape measurement device 1 includes a region of interest extraction means 2 and a three-dimensional shape measurement means 3. In this embodiment, the three-dimensional shape measurement device 1 includes region of interest extraction means 21 and 22 corresponding to the extraction target cameras 41 and 42, respectively.

[0027] The region of interest cutting means 2 (21, 22) receives a cutout target image from cutout target cameras 41, 42, which are part or all of the fixed camera 4, and generates a cutout image by cutting out a region of interest from the input cutout target image. As shown in FIG. 1, the region of interest cutting means 2 includes a region of interest detection means 20, an image cutting means 21, and a parameter correction means 22.

[0028] Since the region of interest extraction means 21 and 22 are the same except for the extraction target image to be processed, only the region of interest extraction means 21 will be described, and a description of the region of interest extraction means 22 will be omitted. Also, for the region of interest extraction means 22, illustration of each means has been omitted to make the drawing easier to understand.

[0029] The image to be cropped from the camera 41 to be cropped may be input directly to the region of interest detection means 20, or may be input after undergoing a predetermined conversion process. This conversion process may include, for example, reducing the image to be cropped, reducing the frame rate, reducing the bit depth, changing the color subsampling, grayscaling, binarization, image super-resolution, edge enhancement, histogram equalization, etc.

[0030] The region of interest detection means 20 detects a region having a predetermined image characteristic as a region of interest in the image to be cut out. That is, the region of interest detection means 20 detects a characteristic region (hereinafter referred to as a region of interest) from the image to be cut out, and outputs information regarding the position and size of the region of interest.

[0031] The region of interest detection means 20 may detect a part of the subject (face, hands, legs) as the region of interest. In the example of Fig. 2, the region of interest detection means 20 detects the face region of the subject 9 (e.g., a mascot character) included in the image 90 to be cut out as the region of interest 91. Alternatively, the region of interest detection means 20 may detect the entire subject as the region of interest. In the example of Fig. 3, the region of interest detection means 20 detects the entire body of the subject 9 included in the image 90 to be cut out as the region of interest 91.

[0032] Here, the region of interest detection means 20 can detect a region of interest based on spatial features of the image to be cropped. For example, the region of interest detection means 20 identifies pixels whose amplitude values ​​satisfy a predetermined condition (e.g., exceed a threshold) after applying filtering in the spatial frequency domain to the image to be cropped. Examples of such filtering include a high-pass filter, a low-pass filter, and a band-pass filter. The region of interest detection means 20 then detects a region of interest based on the spatial distribution of the identified pixels (hereinafter, "important pixels"). In this case, the region of interest detection means 20 may calculate the center of gravity and variance of the important pixels, determine the center of the region of interest based on the center of gravity, and calculate the size of the region of interest based on the variance. Alternatively, the region of interest detection means 20 may detect a region of interest based on the smallest rectangle that encompasses the important pixels.

[0033] Alternatively, the region of interest detection means 20 may detect a region of interest based on a feature amount in the time domain of the image to be extracted. For example, the region of interest detection means 20 may detect a local region with large motion as a region of interest according to the magnitude of a motion vector (including optical flow).

[0034] Alternatively, region of interest detection means 20 may use artificial intelligence (AI) such as a deep neural network (DNN) to detect the region where a subject exists as the region of interest. Alternatively, region of interest detection means 20 may use statistical processing or AI to detect the region of interest based on a saliency map that quantifies regions that people tend to focus on.

[0035] Here, the region of interest is detected as a two-dimensional region within the image to be cropped. The shape of the region of interest may be rectangular or non-rectangular. For example, the region of interest is a rectangular region and is represented by four variables: a central horizontal coordinate, a central vertical coordinate, a width, and a height. The region of interest is represented by four variables: an upper left horizontal coordinate and a vertical coordinate, and a lower right horizontal coordinate and a vertical coordinate. The region of interest may also be an N-sided polygonal region and be represented by N vertices (2N variables). Furthermore, the silhouette obtained when a three-dimensional region in which the subject is located (for example, the eight vertices of a rectangular parallelepiped) is projected onto the image to be cropped may be defined as the region of interest.

[0036] Even if the region of interest includes an extraneous region other than the subject (for example, the background), this does not cause a problem in measuring the three-dimensional shape of the subject. In this case, the quality is improved even for the extraneous region other than the subject.

[0037] The region of interest detection means 20 outputs the image to be cut out and information relating to the detected region of interest to the image cut-out means 21.

[0038] The image cropping means 21 generates a cropped image by cropping the region of interest detected by the region of interest detection means 20 from the cropping target image. In other words, the image cropping means 21 crops out a partial image belonging to the region of interest from the cropping target image input from the region of interest detection means 20 as a cropped image.

[0039] Here, the image cropping means 21 may crop a portion that matches the region of interest. Alternatively, the image cropping means 21 may crop a minimum rectangular region having a predetermined aspect ratio that includes the region of interest. Alternatively, the image cropping means 21 may crop a portion that is larger (or smaller) than the region of interest by a predetermined width or height. Furthermore, if the region of interest is not arc-connected, the image cropping means 21 may crop it based on one of the arc-connected regions that satisfies a predetermined condition (for example, maximum area or minimum area).

[0040] For example, the position of the region of interest is determined by the origin position (O x ,O y ) The origin position can be any vertex of the image to be cut out (for example, the upper left vertex) or the center of the image. The origins of both the image to be cut out and the region of interest are set using the same standard (for example, the upper left vertex or the center of the image is used as the origin for both).

[0041] Here, a method for transforming the cut-out region of interest will be described. The image cropping means 21 may generate a cropped image without transforming the cut-out region of interest. For example, the image cropping means 21 may generate a cropped image that maintains the size of the subject image included in the cut-out region of interest (at the same pixel size). On the other hand, the image cropping means 21 may transform the cut-out region of interest and generate a cropped image. For example, the image cropping means 21 may enlarge, reduce, or change the aspect ratio of the cut-out region of interest to a predetermined size and generate a cropped image.

[0042] The image cutting means 21 outputs the generated cut-out target image to the three-dimensional shape measuring means 3.

[0043] Parameter correction means 22 receives the camera parameters of cropping target camera 41 and converts the received camera parameters of cropping target camera 41 in accordance with the deformation method of the region of interest to calculate camera parameters corresponding to the cropped image. Hereinafter, the camera parameters of cropping target camera 41 may be referred to as "uncorrected camera parameters," and the camera parameters corresponding to the cropped image may be referred to as "corrected camera parameters."

[0044] The pre-correction camera parameters correspond to the camera parameters when the entire region of interest is photographed by the fixed camera 4, and indicate how the cropped image is cropped from the cropping target image. For example, the pre-correction camera parameters include the position (e.g., the position of the first principal point of the lens) and attitude of the cropping target camera 41, information on the angle of view (e.g., the focal length of the lens), and information on the size of the pixel of the sensor (e.g., the horizontal length r of the pixel x , vertical length r y ) and information about the optical axis deviation (e.g., the image coordinates where the optical axis intersects the image plane (c x ,c y )) and.

[0045] For example, the corrected camera parameters include the position of the cropping target camera 41 (for example, the position of the first principal point of the lens), the attitude, information about the angle of view (for example, the focal length of the lens), and information about the size of the pixel of the sensor (for example, the horizontal length R of the pixel x , vertical length R y ) and information about the optical axis deviation (e.g., the image coordinates where the optical axis intersects the image plane (C x ,C y )) and.

[0046] <Calculating corrected camera parameters: Example 1> The calculation of the corrected camera parameters will be explained below using two specific examples. In this first example, the region of interest is extracted while maintaining its size.

[0047] Of the corrected camera parameters, only the information about the optical axis deviation needs to be corrected for the uncorrected camera parameters. The information about the optical axis deviation in the corrected camera parameters is used to correct the image coordinates (C x ,C y ), it can be calculated using the following formula (1).

[0048]

number

[0049] It should be noted that, except for the information regarding the deviation of the optical axis, the corrected camera parameters are the same as the uncorrected camera parameters.

[0050] <Calculating corrected camera parameters: Example 2> In this second example, the region of interest is cut out by enlarging or reducing it or changing the aspect ratio so that it has a predetermined size.

[0051] The number of horizontal pixels in the partial image is s x , the number of pixels in the vertical direction is s y After cutting out this partial image, it is enlarged or reduced in the horizontal and vertical directions, and the horizontal resolution of the obtained cut-out image is set to S. x , vertical resolution S y Let's say.

[0052] Among the corrected camera parameters, information about the optical axis deviation and information about the pixel size of the sensor must be corrected for the uncorrected camera parameters. The information about the optical axis deviation in the corrected camera parameters is used to calculate the image coordinates (C x ,C y ) and the information about the size of the sensor pixel is expressed as the horizontal length of the pixel R x , vertical length R y When expressed as above, it can be calculated using the following formula (2).

[0053]

number

[0054] It should be noted that the corrected camera parameters are the same as the uncorrected camera parameters except for the information regarding the deviation of the optical axis and the information regarding the pixel size of the sensor.

[0055] The parameter correction means 22 outputs the calculated corrected camera parameters to the three-dimensional shape measurement means 3.

[0056] The three-dimensional shape measurement means 3 measures the three-dimensional shape of the subject based on the cut-out image, the camera parameters corresponding to the cut-out image, the captured image input from the directly connected camera, and the camera parameters of the directly connected camera.

[0057] The three-dimensional shape measurement means 3 receives input of cropped images and corrected camera parameters from at least one region of interest cropping means 2. The three-dimensional shape measurement means 3 also receives input of captured images and camera parameters from fixed cameras 4, bypassing the region of interest cropping means 2. In this embodiment, the three-dimensional shape measurement means 3 receives input of cropped images and corrected camera parameters from region of interest cropping means 21 and 22, and captured images and camera parameters from directly connected cameras 43 and 44. The three-dimensional shape measurement means 3 then uses these four sets of images and camera parameters to measure the three-dimensional shape of the subject.

[0058] The three-dimensional shape measurement means 3 can measure the three-dimensional shape using any method, such as the stereo method or the visual volume intersection method. For example, the three-dimensional shape measurement means 3 can use the method described in Japanese Patent Application Laid-Open No. 2022-76850. Note that this method uses a mask image to distinguish the subject area, but the method for generating this is common and therefore will not be described here. Furthermore, the three-dimensional shape measurement means 3 may measure texture information (color and pattern) and material information (texture information such as reflection coefficients) in addition to the three-dimensional shape of the subject.

[0059] That is, the three-dimensional shape measurement device 1 measures the three-dimensional shape of the entire subject from the images captured by the directly connected cameras 43 and 44. At this time, the three-dimensional shape of the subject can be ensured to have sufficient quality if it is only observed from a distance. Furthermore, the three-dimensional shape measurement device 1 measures the three-dimensional shape of the region of interest, which is an important part of the subject, at high resolution from the cutout target images captured by the cutout target cameras 41 and 42. For example, the three-dimensional shape of the subject is a point cloud in which the texture density and resolution of the region of interest are higher than those of other regions. This allows the three-dimensional shape of the subject to be of high quality enough to withstand close-up observation.

[0060] [Operation of the 3D shape measurement device] The operation of the three-dimensional shape measuring device 1 will be described with reference to FIG. As shown in FIG. 4, in step S1, the region of interest detection means 20 detects a region of interest from the cut-out target image captured by the cut-out target cameras 41 and 42. In step S2, the image cutting means 21 cuts out the region of interest detected in step S1 from the cut-out target image to generate a cut-out image.

[0061] In step S3, the parameter correction means 22 converts the camera parameters of the cropping target cameras 41 and 42 in accordance with the deformation method of the region of interest, thereby calculating camera parameters corresponding to the cropped image. The order of steps S2 and S3 may be reversed, or steps S2 and S3 may be executed simultaneously.

[0062] In step S4, the three-dimensional shape measurement means 3 measures the three-dimensional shape of the subject based on the cut-out image, the camera parameters corresponding to the cut-out image, the captured image input from the directly connected cameras 43, 44, and the camera parameters of the directly connected cameras 43, 44.

[0063] [Actions and Effects] As described above, the three-dimensional shape measuring device 1 uses images taken by the fixed camera 4 whose posture does not change, so no mechanical errors occur and the quality of the three-dimensional shape can be improved. Furthermore, the three-dimensional shape measuring device 1 processes not the entire image to be cut out but a cut-out image that is a part of the image to be cut out, thereby reducing the amount of calculation.

[0064] That is, the three-dimensional shape measurement device 1 narrows down the image region to be measured for three-dimensional shape to the region of interest, so image information outside the region of interest is not input to the three-dimensional shape measurement means 3, reducing the amount of calculation. For example, when measuring the three-dimensional shape of a cropped image, the amount of calculation can be reduced to about one-tenth of that required when measuring the three-dimensional shape of the entire captured image. Furthermore, the three-dimensional shape measurement device 1 can increase the resolution of the cropping target cameras 41, 42 by the amount of reduction in calculation, thereby improving the accuracy and resolution of the three-dimensional shape measurement of the region of interest.

[0065] (Second embodiment) [Overview of the 3D shape measurement system] With reference to FIG. 5, a three-dimensional shape measuring system 100B according to the second embodiment will be described, focusing on differences from the first embodiment. In the first embodiment, there is a one-to-one correspondence between the cutout target cameras 41 and 42 and the region of interest cutout means 2 (21 and 22) (FIG. 1). On the other hand, in the second embodiment, multiple region of interest cutout means 2B (2B1 and 2B2) correspond to a single cutout target camera 41.

[0066] As shown in FIG. 5, the three-dimensional shape measuring system 100B includes a three-dimensional shape measuring device 1B and a fixed camera 4. In this embodiment, of the three fixed cameras 4, one is the cropping target camera 41, and the remaining two are directly-coupled cameras 43 and 44. The cropping target camera 41 outputs the captured cropping target image and the camera parameters used when the cropping target image was captured to the region of interest cropping means 2B1 and 2B2. That is, the cropping target image and camera parameters of the cropping target camera 41 are output to the three-dimensional shape measurement means 3 via the multiple region of interest cropping means 2B. Furthermore, the directly-coupled camera 44 outputs the captured image and the camera parameters used when the captured image was captured to the three-dimensional shape measurement means 3.

[0067] [Configuration of 3D shape measurement device] The configuration of the three-dimensional shape measuring apparatus 1B will be described below. 5, three-dimensional shape measurement device 1B includes region of interest extraction means 2B and three-dimensional shape measurement means 3. In this embodiment, three-dimensional shape measurement device 1B includes two region of interest extraction means 2B (2B1, 2B2) corresponding to one extraction target camera 41. Note that three-dimensional shape measurement means 3 is the same as in the first embodiment, and therefore description thereof will be omitted.

[0068] Region of interest cutting means 2B (2B1, 2B2) receives a cutout target image from the same cutout target camera 41 and generates cutout images in which different regions of interest are cut out from the input cutout target image. Region of interest cutting means 2B (2B1, 2B2) includes region of interest detection means 20B (20B1, 20B2), image cutting means 21 (211, 212), and parameter correction means 22 (221, 222).

[0069] Region of interest detection means 20B (20B1, 20B2) detect different regions of interest from the same cropping target image. For example, region of interest detection means 20B1 and 20B2 may detect regions of interest using different detection methods. Specifically, region of interest detection means 20B1 detects a region of interest based on spatial features of the cropping target image, while region of interest detection means 20B2 detects a region of interest based on temporal features of the cropping target image.

[0070] Furthermore, although region of interest detection means 20B1 and 20B2 detect regions of interest using the same detection method, their detection parameters may be different (for example, the center frequencies of band-pass filters may be different).

[0071] The region of interest detection means 20B1 and 20B2 may detect regions of interest with different sizes and aspect ratios, for example, the region of interest detection means 20B1 detects a region of interest with a larger size than the region of interest detection means 20B2.

[0072] Image cropping means 211 generates a cropped image by cropping the region of interest detected by region of interest detection means 20B1 from the cropping target image. Also, image cropping means 212 generates a cropped image by cropping the region of interest detected by region of interest detection means 20B2 from the cropping target image. Note that the processing contents of image cropping means 211 and 212 are similar to those of image cropping means 21 in FIG. 1, and therefore description thereof will be omitted.

[0073] Parameter correction means 221 calculates camera parameters corresponding to the cropped image by converting the camera parameters of cropping target camera 41 in accordance with the deformation method of the region of interest detected by region of interest detection means 20B1. Parameter correction means 222 calculates camera parameters corresponding to the cropped image by converting the camera parameters of cropping target camera 41 in accordance with the deformation method of the region of interest detected by region of interest detection means 20B2. The processing contents of parameter correction means 221 and 222 are the same as those of parameter correction means 22 in FIG. 1, and therefore description thereof will be omitted.

[0074] [Actions and Effects] As described above, the three-dimensional shape measuring apparatus 1B can improve the quality of the three-dimensional shape and reduce the amount of calculation, similar to the first embodiment. Furthermore, three-dimensional shape measurement device 1B can cut out multiple subjects or different parts of the same subject from the same cut-out target image using multiple region-of-interest cut-out means 2B1, 2B2. This allows three-dimensional shape measurement device 1B to cut out multiple subjects that are far apart within the screen of cut-out target camera 41 in the smallest size for each subject, further reducing the proportion of image areas that do not show the subjects that are input to three-dimensional shape measurement means 3, thereby reducing the amount of calculations.

[0075] (Third embodiment) [Overview of the 3D shape measurement system] With reference to FIG. 6, a three-dimensional shape measuring system 100C according to the third embodiment will be described, focusing on differences from the first embodiment. In the first embodiment, some of the fixed cameras 4 are cutout target cameras 41 and 42 (FIG. 1). In contrast to this, in the third embodiment, all of the fixed cameras 4 are cutout target cameras 41 to 44. In other words, the three-dimensional shape measurement system 100C does not include the directly connected cameras 43 and 44 of FIG.

[0076] As shown in FIG. 6, the three-dimensional shape measuring system 100C includes a three-dimensional shape measuring device 1C and a fixed camera 4. As described above, all four fixed cameras 4 are cropping target cameras 41-44. At least one of cropping target cameras 41-44 may capture an image of the entire measurement target area of ​​the subject, and the remaining cameras may capture an image of the entire measurement target area of ​​the subject. In this embodiment, cropping target camera 41 captures an image of the entire measurement target area of ​​the subject. Each of cropping target cameras 41-44 outputs the captured cropping target image and the camera parameters used when capturing the cropping target image to region of interest cropping means 2C1-2C4.

[0077] The measurement target area of ​​a subject is the area of ​​the subject that is the target for measuring the three-dimensional shape. The measurement target area may be the entire subject (for example, the entire body of a person), or a part of the subject (for example, the head of a person).

[0078] [Configuration of 3D shape measurement device] The configuration of the three-dimensional shape measuring apparatus 1C will be described below. 6, three-dimensional shape measuring device 1C includes region of interest extraction means 2C and three-dimensional shape measuring means 3. In this embodiment, three-dimensional shape measuring device 1C includes four region of interest extraction means 2C (2C1 to 2C4) corresponding to four extraction target cameras 41 to 44.

[0079] The region of interest extraction means 2C (2C1 to 2C4) and the three-dimensional shape measurement means 3 are the same as those in the first embodiment, and therefore their explanation will be omitted. Also, for the region of interest extraction means 2C2 to 2C4, illustration of each means has been omitted to make the drawings easier to understand.

[0080] [Actions and Effects] As described above, the three-dimensional shape measuring apparatus 1C can improve the quality of the three-dimensional shape and reduce the amount of calculation, similar to the first embodiment. That is, three-dimensional shape measurement device 1C can measure the three-dimensional shape of a subject because at least one of the target cameras 41-44 captures an image of the entire subject. Furthermore, three-dimensional shape measurement device 1C narrows down the image area to be measured for three-dimensional shape to a region of interest, so image information outside the region of interest is not input to three-dimensional shape measurement means 3, reducing the amount of calculation. Furthermore, three-dimensional shape measurement device 1C can increase the resolution of target cameras 41-44 by the amount of reduction in calculation, thereby improving the accuracy and resolution of three-dimensional shape measurement of the region of interest.

[0081] (Variation) Although the embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design modifications and the like within the scope of the present invention.

[0082] In the above-described embodiments, the number of fixed cameras is four, but two or more fixed cameras may be used. The three-dimensional shape measurement system includes 20 to 30 directly connected cameras and two to four cropping target cameras. These directly connected cameras and cropping target cameras are arranged to surround the subject, as in conventional volumetric capture. In this case, the cropping target cameras should be arranged so as to directly face the subject, but they may also be arranged at an angle to the subject.

[0083] In the above-described embodiments, the resolution of the camera to be cropped (image to be cropped) is described as being higher than the resolution of the directly-connected camera (captured image), but this is not limited to this. For example, the resolution of the camera to be cropped and the directly-connected camera may be the same. Even in this case, the three-dimensional shape measurement device measures the three-dimensional shape by capturing images of the subject from various directions, which makes it less likely that the three-dimensional shape will be lost, and the quality of the three-dimensional shape can be improved.

[0084] In the above-described embodiments, the three-dimensional shape measurement device has been described as an independent piece of hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, memory, and hard disk of a computer to function as the three-dimensional shape measurement device. This program may be distributed via a communication line or written to a recording medium such as a CD-ROM or flash memory. [Explanation of symbols]

[0085] 1, 1B, 1C Three-dimensional shape measurement device 2, 21, 22, 2B, 2B1, 2B2, 2C, 2C1 to 2C4 Region of interest extraction means 3. Three-dimensional shape measurement method 4 Fixed Cameras 41~44 Cameras for cropping 43,44 Directly connected camera 20, 20B, 20B1, 20B2 region of interest detection means 21,211,212 Image extraction means 22,221,222 Parameter correction means 100,100B 3D shape measurement system

Claims

1. A three-dimensional shape measurement device that measures the three-dimensional shape of a subject using images taken by multiple fixed cameras installed at different positions, a region of interest cutout means for inputting a cutout target image from a cutout target camera that is a part or all of the fixed cameras, and for generating a cutout image by cutting out a region of interest from the input cutout target image; a three-dimensional shape measurement means for measuring a three-dimensional shape of the subject based on the extracted image and camera parameters corresponding to the extracted image, The region of interest extraction means a region of interest detection means for detecting a region having a predetermined image feature as a region of interest in the image to be cut out; an image cropping means for generating a cropped image by cropping the region of interest detected by the region of interest detection means from the cropping target image; a parameter correction means for receiving camera parameters of the cropping target camera and converting the received camera parameters of the cropping target camera in accordance with a deformation method of the region of interest to calculate camera parameters corresponding to the cropped image; A three-dimensional shape measuring device comprising:

2. a part of the fixed cameras is used as the cropping target cameras, and the rest of the fixed cameras are used as direct-coupled cameras that output captured images to the three-dimensional shape measuring means, 2. The three-dimensional shape measuring device according to claim 1, wherein the three-dimensional shape measuring means measures the three-dimensional shape of the subject based on the cut-out image, camera parameters corresponding to the cut-out image, the captured image input from the directly-connected camera, and the camera parameters of the directly-connected camera.

3. 3. The three-dimensional shape measuring apparatus according to claim 2, wherein the resolution of the cutout target image is higher than the resolution of the captured image.

4. The three-dimensional shape measurement device according to claim 1, characterized in that the region of interest cutout means inputs the cutout target image from the same cutout target camera, and generates the cutout image in which a different region of interest is cut out from the input cutout target image.

5. 2. The three-dimensional shape measuring device according to claim 1, wherein all of the fixed cameras are used as the cutout target cameras, and at least one of the cutout target cameras photographs the entire measurement target area of ​​the subject.

6. A program for causing a computer to function as the three-dimensional shape measuring device according to any one of claims 1 to 5.

Images