Measurement device, imaging device, control method and program

Abstract

To present an imaging direction for obtaining a measurement result of suitable accuracy to a user.SOLUTION: A measurement device performs measurement relating to a predetermined item with respect to an object subject to be measured. The measurement device includes: an acquisition means for acquiring distance information indicating a distribution of subject distances from an imaging device which has taken a captured image obtained by capturing an imaging range including a target subject, and normal information indicating at least the normal of a surface of the target subject; and a presentation means for presenting a notification indicating an imaging direction of the imaging device for measuring the predetermined item based on the normal information acquired by the acquisition means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a measurement apparatus, an imaging apparatus, a control method, and a program, and more particularly to a measurement technique based on an image of an object within an imaging range where imaging is performed. [Background technology]

[0002] There is an imaging device that can obtain the three-dimensional position of the subject at each pixel position by deriving the distance (subject distance) from the imaging device based on a group of images that are in a parallax relationship for a subject present in the imaging range, and can measure the spatial distance between two specified points (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-232330 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in Patent Document 1, depending on the positional relationship between the imaging device and the two points to be measured, the measurement accuracy may decrease. For example, when a so-called perspective expression is used as a composition for imaging the measurement target, a difference may occur in the positional relationship in the depth direction between the two points to be measured in the imaging range, and measurement results with suitable accuracy may not be obtained. This is because when imaging the measurement target subject, the more distant the subject is, the lower the spatial resolution appears in the image, so even if two points with different depth directions are used as measurement targets and measurement is performed based on the image, an error may occur in the measurement result.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a measurement device, an imaging device, a control method, and a program that present to a user an imaging direction that will produce measurement results with suitable accuracy. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the measurement apparatus of the present invention is a measurement apparatus that performs measurements related to predetermined items on a target subject that is the object of measurement, and includes an acquisition means that acquires, for an image obtained by imaging an imaging range including the target subject, distance information that indicates a distribution of subject distances from the imaging device that performed the imaging, and normal information that indicates at least a normal to a surface of the target subject, and a determination means that determines whether or not to measure the predetermined items based on the normal information; a notification means for presenting a notification indicating an imaging direction of the imaging device when the determination means determines that the measurement of the predetermined item is not to be performed; The determination means determines to perform measurements of the specified items when it is determined that the state of the imaging device when the image was captured is directly facing the surface of the target subject, and determines not to perform measurements of the specified items when it is determined that the state of the imaging device when the image was captured is not directly facing the surface of the target subject. Effect of the Invention

[0007] With this configuration, the present invention makes it possible to present to the user an imaging direction that will provide measurement results with suitable accuracy. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the functional configuration of a digital camera 100 according to an embodiment and a modification of the present invention. [Diagram 2] FIG. 1 is a diagram for explaining the configuration of an image sensor 111 according to an embodiment and a modification of the present invention; [Diagram 3] 1 is a flowchart illustrating a measurement process executed by the digital camera 100 according to an embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating a distance derivation process and a normal derivation process according to an embodiment and a modification of the present invention; [Diagram 5] FIG. 1 is a diagram for explaining an image-forming relationship of an imaging optical system 112 according to an embodiment and a modified example of the present invention; [Figure 6] FIG. 13 is a diagram illustrating an example of a notification mode of an image capturing direction suitable for measuring a target subject in the digital camera 100 according to an embodiment of the present invention. [Figure 7] FIG. 13 is a diagram illustrating an example of a presentation mode of measurement points for measuring a target subject in the digital camera 100 according to the embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [Embodiment] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] In the embodiment described below, an example of the present invention is applied to a digital camera that can derive a subject distance based on a pair of captured images having a parallax relationship, as an example of a measurement device. However, the present invention can be applied to any device that is configured to be able to derive a normal to a surface of a subject included in an imaging range.

[0011] Configuration of digital camera 100 FIG. 1 is a block diagram showing the functional configuration of a digital camera 100 according to an embodiment of the present invention.

[0012] The system control unit 101 is a control device, such as a microcomputer, that controls each block of the digital camera 100. The system control unit 101 reads out an operation program for each block of the digital camera 100 from a recording medium 102, expands it into a memory 103, and executes it to control the operation of each block of the digital camera 100.

[0013] The recording medium 102 is a rewritable non-volatile memory such as a flash ROM. The recording medium 102 stores parameters and the like required for the operation of each block in addition to the operation programs of each block included in the digital camera 100. On the other hand, the memory 103 is a rewritable volatile memory. The memory 103 is used not only as an area for loading the operation programs of each block included in the digital camera 100, but also as a temporary storage area for intermediate data output by the operation of each block.

[0014] The imaging unit 104 captures an image of a subject included in an imaging range, and outputs image data of the captured image. In this embodiment, the imaging unit 104 is configured to include at least an imaging element 111 and an imaging optical system 112. The imaging optical system 112 forms an image of a subject on the imaging element 111. The imaging optical system 112 includes, for example, a fixed lens, a variable magnification lens that changes the focal length, and a focus lens that adjusts the focus. The imaging optical system 112 also includes an aperture, and the aperture adjusts the aperture diameter of the optical system to adjust the amount of light during shooting.

[0015] The imaging element 111 is a photoelectric conversion element such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor) sensor. The imaging element 111 photoelectrically converts an optical image formed on an imaging surface by an imaging optical system 112 to obtain an analog image signal. The imaging unit 104 obtains digital image data (hereinafter simply referred to as a captured image) by applying an A / D conversion process to the obtained analog image signal. The imaging unit 104 outputs the obtained image data to the memory 103 for storage.

[0016] In this embodiment, the imaging unit 104 is described as outputting a captured image by applying a simple A / D conversion process, but the imaging unit 104 may be configured to perform other image processing so that the captured image can be suitably used in various processes. The imaging unit 104 may perform image processing such as white balance adjustment, color interpolation, and reduction / enlargement processing.

[0017] Configuration of the image sensor 111 In this embodiment, the image sensor 111 is configured to capture a pair of images having a parallax relationship so as to enable derivation of the distance (object distance) between the object and the principal point of the imaging optical system 112 and the surface normal of the object based on the distribution of the object distance. Details of this will be described below with reference to FIG.

[0018] As shown in Fig. 2(a), the image sensor 111 has a two-dimensional pixel structure arranged perpendicular to the optical axis 114 of the image sensor 112. More specifically, the image sensor 111 is configured in a so-called Bayer array, in which a pixel group 201 of 2 rows x 2 columns is regarded as one unit and the pixel group is arranged two-dimensionally. Color filters are applied to four pixels included in the pixel group 201, and pixels 202G1 and G2 that detect the amount of light of the green wavelength component are arranged in the diagonal direction. In addition, pixel 202R that detects the amount of light of the red wavelength component and pixel 202B that detects the amount of light of the blue wavelength component are arranged in the remaining two pixels.

[0019] Further, in order to capture the pair of captured images described above, a photoelectric conversion element provided in one pixel has a structure as shown in FIG. 2(b). FIG. 2(b) is a diagram illustrating an example of a pixel structure in a cross section taken along line I-I' of one pixel group 201 in FIG. 2(a). As shown in the figure, each of the pixels 202G1 and 202R is configured to have a light receiving layer 203 and a light guide layer 204, and a light flux incident via a microlens 205 is photoelectrically converted in the light receiving layer 203. Here, the light receiving layer 203 of each pixel is provided with two photoelectric conversion elements 206a and 206b that photoelectrically convert the received light, and each of the photoelectric conversion elements 206a and 206b photoelectrically converts a light flux that has passed through different regions of the exit pupil 113 of the imaging optical system 112 out of the incident light flux. That is, with such a configuration, the image sensor 111 of this embodiment is capable of imaging by pupil-dividing the incident light beam, and a pair of captured images can be obtained by integrating the outputs from the photoelectric conversion elements 206 on the same side (a or b) of all pixels.

[0020] More specifically, the photoelectric conversion element 206 of one pixel and the exit pupil 113 of the imaging optical system 112 are in an optically conjugate relationship via the microlens 205. With this configuration, the light beam incident on each pixel is received by each of the photoelectric conversion elements 206a and 206b by the microlens 205, thereby realizing pupil division, and a group of captured images (a pair of captured images) related to the incident light beams from two different directions can be obtained in one imaging. In other words, of the light beams passing through the exit pupil 113 of the imaging optical system 112, the light beams indicated by dots in FIG. 2C are received by the photoelectric conversion element 206a, and the light beams indicated by hatching are received by the photoelectric conversion element 206b. Here, FIG. 2C shows the exit pupil 113 of the imaging optical system 112 as viewed from the intersection (center image height) of the optical axis 114 and the imaging element 111. In other words, the pair of captured images are equivalent to images captured by a group of imaging devices relating to different viewpoints, with the baseline length being the distance between the center of gravity position 207a at the exit pupil 113 of the light beam received by the photoelectric conversion element 206a and the center of gravity position 207b relating to the photoelectric conversion element 206b.

[0021] In this embodiment, the photoelectric conversion elements 206 are configured to perform pupil division in the x-axis direction of the image sensor 111 as shown in the drawing, but the implementation of the present invention is not limited to this. That is, the pupil division direction may be determined in the y-axis direction, or may be determined on two axes, the x-axis and the y-axis, with four photoelectric conversion elements 206, or may be divided in any direction and number.

[0022] The distance derivation unit 105 derives the subject distance for the subject in the imaging range based on the captured image obtained by the imaging unit 104. As described above, a pair of captured images obtained by the imaging unit 104 of this embodiment are in a parallax relationship, and the distance derivation unit 105 derives a defocus amount based on a relative image shift amount between the images, and can derive the subject distance based on this and the imaging conditions. For example, for each pixel of one captured image, the distance derivation unit 105 derives the distance to the subject present at that pixel, and configures distance information indicating a two-dimensional distribution of the subject distance.

[0023] The normal derivation unit 106 derives a normal to a surface constituting a subject (target subject) to be measured, for example, based on the distance information obtained by the distance derivation unit 105. Although details will be described later, the digital camera 100 of this embodiment is configured to be able to measure, for example, the distance between two points set on the subject, the peripheral distance of a specific portion (surface of the target subject), the area of ​​an area set on the subject, and the like, for the target subject among the subjects in the imaging range. Since the measurement is performed based on a captured image, in this embodiment, the normal derivation unit 106 is described as deriving a normal based on distance information obtained from a pair of captured images by a passive stereo method, but the implementation of the present invention is not limited to this. In other words, this method effectively utilizes distance information used for measurement to realize a device configuration with a minimum configuration, and it goes without saying that other methods may be adopted for deriving the normal.

[0024] The direction derivation unit 107 derives an imaging direction suitable for imaging the target subject in order to improve the measurement accuracy of the above-mentioned measurement. That is, when imaging a target subject in a composition in which the target subject extends in the depth direction in the imaging range, the spatial resolution of the subject image appearing in the captured image changes, and the measurement accuracy may decrease. For this reason, in the digital camera 100 of this embodiment, the direction derivation unit 107 derives an imaging direction that enables imaging suitable for measurement based on information on the normal of the target subject derived by the normal derivation unit 106.

[0025] The measurement unit 108 measures the items determined to be measured for the target subject based on the distance information acquired for the preferred imaging direction.

[0026] The display unit 109 is a display device such as an LCD, and presents various types of information in the digital camera 100. The display unit 109 functions as a digital viewfinder by displaying (through display) A / D converted image data while the imaging unit 104 is capturing an image. In this embodiment, the display unit 109 is also used to display a GUI for notifying a suitable imaging direction and to display measurement results.

[0027] The operation input unit 110 is a user input interface provided in the digital camera 100, such as a release switch, a setting button, a mode setting dial, etc. When the operation input unit 110 detects an operation input performed on various user input interfaces, it outputs a control signal corresponding to the operation input to the system control unit 101. In addition, in an aspect in which the display unit 109 is equipped with a touch panel sensor, the operation input unit 109 also functions as an interface that detects a touch operation performed on the display unit 109.

[0028] In this embodiment, the processing is described as being realized by circuits and processors corresponding to each block of the hardware provided in the digital video camera 100. However, the embodiment of the present invention is not limited to this, and the processing of each block may be realized by a program that performs the same processing as that of the corresponding block.

[0029] <<Measurement processing>> A specific measurement process for measuring a relevant item of a target subject based on distance information obtained from a pair of captured images in the digital camera 100 of this embodiment having such a configuration will be described with reference to the flowchart in Fig. 3. The process corresponding to the flowchart can be realized by the system control unit 101 reading out a corresponding processing program stored in, for example, the recording medium 102, and executing the program by expanding it in the memory 103. This measurement process will be described as being started when, for example, a measurement mode for performing image-based measurement of the target subject is set in the digital camera 100. Note that, in the following, for ease of understanding of the present invention, it is assumed that the measurement item is to measure a dimension between two points set on the target subject.

[0030] In S301, the imaging unit 104 captures an image of the imaging range under the control of the system control unit 101. When the imaging unit 104 captures a pair of captured images, it stores the captured images in the memory 103. The processing of this step may be performed in response to detection of an operation input related to an image capture instruction, or may be performed without an operation input after performing a predetermined process such as focus detection or focus adjustment that is performed before shooting.

[0031] In S302, the distance derivation unit 105, under the control of the system control unit 101, derives the subject distance in the imaging range based on the pair of captured images acquired in S301, and executes distance derivation processing to configure distance information.

[0032] <Distance Derivation Processing> Here, the distance derivation process in this step will be described in detail with reference to the flowchart in FIG.

[0033] In S401, the distance derivation unit 105 performs various correction processes before distance derivation on the pair of captured images used to derive the subject distance. More specifically, the distance derivation unit 105 performs correction to reduce bias in the amount of received light between the photoelectric conversion elements 206a and 206b caused by vignetting of the imaging optical system 112, etc. The distance derivation unit 105 also applies a band-pass filter to reduce the influence of noise included in the pair of captured images.

[0034] In S402, the distance derivation unit 105 derives the image shift amount for the subject of each pixel based on the pair of captured images. First, the distance derivation unit 105 sets a pixel of interest and a reference area centered on the pixel of interest for one image (image A) of the pair of captured images. Then, the distance derivation unit 105 sequentially sets a reference pixel and a reference area centered on the reference pixel for the other image (image B), and identifies the position of the reference pixel with the highest correlation while deriving the correlation value between the images of the reference area and the reference area. As the correlation value, a known method can be used, and for example, an evaluation value called SSD that evaluates the sum of squares of the difference between the pixel value in the reference area and the pixel value in the reference area can be used. In SSD, the degree of difference is evaluated, and the smaller the evaluation value, the higher the degree of correlation. The reference pixel and the reference area may be set by sequentially moving them in the x direction, which is the pupil division direction. This enables the distance derivation unit 105 to identify in which of the images B an object corresponding to a pixel of interest in the image A is captured, and to derive the amount of image shift between the images A and B.

[0035] In S403, the distance derivation unit 105 derives the subject distance of the subject based on the image shift amount of the subject for each pixel derived in S402. Here, the subject distance may be derived from the image shift amount using an existing method. For example, the image shift amount can be converted into the defocus amount of the subject based on a geometric relationship using the base line length, so that the subject distance can be derived by using the defocus amount and the imaging relationship of the imaging optical system 112.

[0036] In S404, the distance derivation unit 105 configures the information on the subject distance derived for each pixel as two-dimensional information, generates distance information, and stores it in the memory 103. After the distance derivation unit 105 performs the distance derivation process in this manner and generates the distance information, the system control unit 101 proceeds to S303.

[0037] In S303, under the control of the system control unit 101, the normal derivation unit 106 executes a normal derivation process to derive normal information indicating the surface normal of the target subject, using the distance information generated in S302.

[0038] <Normal derivation process> Here, the normal derivation process in this step will be described in detail with reference to the flowchart in FIG.

[0039] In S411, the normal derivation unit 106 sets an area (target area) in which an image of a target object for which a surface normal is to be derived exists. For simplicity, in this embodiment, it is assumed that the target object that the user desires to measure is located near the center of the angle of view, and the normal derivation unit 106 sets an area of ​​a predetermined size that exists at the center of the angle of view as the target area.

[0040] In this embodiment, a subject located near the center of the angle of view is set as the target area in which the image of the target subject exists, but the present invention is not limited to this. The target area may be set based on a touch operation input such as tracing or other operation input for specifying an area, performed while a captured image is displayed on the display unit 109, and any area may be selectable as the target area.

[0041] In S412, the normal derivation unit 106 derives a plane that approximates a surface constituted by the object present in the target area. The normal derivation unit 106 randomly samples (or references) pixels included in the target area, and derives the three-dimensional coordinates of the target object present in these pixels based on the distance information.

[0042] Here, the derivation of the three-dimensional coordinates based on the distance information is performed by considering the imaging relationship via the imaging optical system 112 shown in FIG. 5. For example, assume that the subject image of point 511 of the target subject is formed at point 501 on the imaging element 111. In this case, the subject distance stored in the distance information for the pixel related to point 501 indicates distance 512 from the principal point of the imaging optical system 112 to point 511. In addition, distance 502 from the principal point of the imaging optical system 112 to the imaging surface of the imaging element 111 is determined from the imaging setting. As a result, the x coordinate of the target subject related to point 511 (distance 513 from the optical axis 114 in the x-axis direction) can be derived from the imaging relationship equation by using distance 503 from the imaging element 111 where the image of the optical center is formed to point 501, and the ratio between distance 502 and distance 512. The y-axis direction can be derived in a similar manner, so that the three-dimensional coordinates of any point on the target subject in a coordinate system with the principal point of the imaging optical system 112 as the origin can be derived.

[0043] Then, the normal derivation unit 106 derives a plane that approximates the distribution of the derived three-dimensional coordinates by the least squares method. More specifically, the normal derivation unit 106 derives a least squares plane of the three-dimensional coordinate group of points distributed on the target object derived in relation to the pixels in the target area as an approximate plane of the object present in the target area. The normal derivation unit 106 stores in the memory 103 information on parameters required to define the derived approximate plane.

[0044] In S413, the normal derivation unit 106 selects, from the approximate plane derived in S412, an area that includes a point whose spatial distance from an arbitrary point of the target subject is smaller than a predetermined threshold and is continuous with the point, as the measurement plane of the target subject.

[0045] In S414, the normal derivation unit 106 derives the normal of the measurement plane selected in S413, and stores it as normal information in the memory 103. Since the approximation plane derived in S412 can be expressed by a plane equation (ax+by+cz+d=0), the normal derivation unit 106 can derive the normal of the measurement plane based on the coefficients of the equation. After the normal derivation process is performed in this manner and the normal derivation unit 106 stores the normal information, the system control unit 101 moves the process to S304.

[0046] In S304, the direction derivation unit 107, under the control of the system control unit 101, judges whether or not the image is captured from a direction appropriate for measuring the target subject based on the normal to the measurement plane derived in S303. Here, the direction appropriate for measuring the target subject is the direction in which the digital camera 100 faces the measurement plane, that is, the direction in which the image is captured in a state in which the points on the measurement plane are uniformly distributed in the vicinity of a certain object distance. Therefore, the judgment in this step is made depending on whether or not the angle between the normal to the measurement plane derived in S303 and the optical axis 114 is smaller than a predetermined angle that is judged to be the same. If the direction derivation unit 107 judges that the image is captured from a direction appropriate for measuring the target subject, the process proceeds to S306. If the direction derivation unit 107 judges that the image is not captured from a direction appropriate for measuring the target subject, the direction derivation unit 107 stores information on the appropriate direction or information on the change in attitude of the digital camera 100 to change the direction to the appropriate direction in the memory 103, and proceeds to S305.

[0047] In S305, the system control unit 101 creates a notification to guide the user to change the imaging direction to a direction appropriate for measuring the target subject, transmits the notification to the display unit 109, and displays the notification superimposed on the captured image. As described above, the direction appropriate for measuring the target subject is the direction in which the angle between the normal to the measurement plane and the optical axis becomes small. In this step, the display unit 109 may provide guidance by displaying a notification in a display form as shown in FIG. 6 to encourage the user to move or change the attitude of the digital camera 100.

[0048] Here, the display mode of the above notification is exemplified for a state in which the target subject 600 is captured in a captured image in a state where it is not facing the digital camera 100 as shown in FIG. 6(a) and the normal 601 of the measurement surface is not oriented in the direction of the optical axis 114. The notification may be a mode in which a moving direction 602 of the digital camera 100 is superimposed on the captured image to set the imaging direction of the digital camera 100 to a direction appropriate for measuring the target subject 600 as shown in FIG. 6(b). The notification may also be a mode in which a grid 603 that changes according to the inclination and indicates the inclination state of the measurement surface of the target subject is superimposed and presented as shown in FIG. 6(c). At this time, the grid 603 is a GUI that three-dimensionally shows the measurement surface, and is displayed in a rectangular shape as it approaches a state of facing directly, so that the user can grasp whether or not the target subject is facing directly and the moving direction for facing directly according to the distortion of the grid 603. Alternatively, the notification may be presented by superimposing the normal 601 of the measurement plane itself in the form of a three-dimensional vector, as shown in FIG. 6(d).

[0049] The captured image to be superimposed may be one of the pair of captured images acquired in S301. Alternatively, an image generated by synthesizing the pair of captured images, i.e., an image equivalent to an image captured without pupil division, in which the outputs of the photoelectric conversion elements 206a and 206b are summed at each pixel, may be used.

[0050] 6, the embodiment of the present invention is not limited to the embodiment in which a predetermined notification is simply superimposed on the captured image. In order to more clearly notify the user that the image capturing situation of the target subject is not in a direction appropriate for measurement, the deviation from the appropriate direction may be indicated by different display modes such as the color and thickness of the GUI.

[0051] After making such a presentation, the system control unit 101 returns the process to S301 when the imaging direction is changed and an operation input related to another imaging instruction is made.

[0052] On the other hand, if it is determined in S304 that the image is captured from a direction appropriate for measuring the target object, the system control unit 101 holds the captured image displayed on the display unit 109 in S306. Then, the system control unit 101 accepts an operation input related to the designation of two measurement points that are targets for measuring dimensions on the surface of the target object. The operation input may be made based on a touch operation on the display unit 109, for example. If the system control unit 101 determines that the two measurement points have been designated, it stores information on the measurement points in the memory 103 and moves the process to S307. At this time, when the measurement points are designated, the system control unit 101 may control the screen transition of the display unit 109 so as to display an image in which the measurement points 701 are superimposed on the captured image, as shown in FIG. 7.

[0053] In S307, under the control of the system control unit 101, the measurement unit 108 measures the dimension (length) between the two measurement points specified in S306. More specifically, the measurement unit 108 may measure the dimension (spatial distance) between the two measurement points based on the distance to the measurement surface based on the distance information, the lateral magnification of the imaging optical system 112, and the coordinates of each measurement point on the captured image. Alternatively, the measurement unit 108 may first derive the three-dimensional coordinates of each measurement point based on the distance information, and measure the dimension between the two measurement points based on the coordinate values. Then, the measurement unit 108 stores the information on the dimensions obtained by the measurement in the memory 103 as the measurement result, and completes the main measurement process.

[0054] By doing this, when performing measurements of a target subject based on an captured image, it is possible to avoid a situation where the measurement surface is not directly opposed, which could reduce measurement accuracy, and to perform measurements under ideal conditions.

[0055] In the normal derivation process of this embodiment, the approximation plane is derived simply based on the three-dimensional coordinate distribution of the object included in the target area in S412, but the implementation of the present invention is not limited to this. That is, considering that multiple surfaces may exist in the target area, an appropriate approximation plane may be derived based only on the object to be measured among them. More specifically, the normal derivation unit 106 may be configured to exclude from the target area, after performing plane approximation, pixels included in the target area whose object distance deviates from the approximation plane beyond a predetermined range, and derive the approximation plane again for the pixels in the target area after the exclusion. Such processing may be configured to be performed recursively, and may be repeated a predetermined number of times, or until the number of pixels deviating from the approximation plane beyond a predetermined range becomes equal to or less than a threshold value.

[0056] Alternatively, the target area may be divided into a plurality of small areas, and the target area may be divided into areas whose distance from the approximation plane derived in S412 is greater than a predetermined value and areas whose distance is smaller than a predetermined value. In this case, the user may select the measurement plane from among the small areas.

[0057] In addition, in this embodiment, notification of the appropriate direction for measuring the target subject and presentation of the measurement results are described as being performed using the display unit 109, but the manner in which this information is notified to the user does not need to be limited to displaying the information on the display unit 109.

[0058] In the present embodiment, the notification of the change of the imaging direction is given only when it is determined that the imaging is not performed from a direction appropriate for measuring the target subject in S304 of the measurement process, but the present invention is not limited to this. The notification indicating the current imaging direction of the digital camera 100 may be given regardless of whether or not a change is necessary for the measurement. It is determined whether or not the mode is appropriate. In other words, when the imaging is performed from a direction appropriate for the measurement, a notification that there is no need to change the imaging direction may be given.

[0059] In the present embodiment, the image sensor 111 is configured to be pupil-divided and recordable to acquire distance information, but the acquisition of distance information is not limited to this. For example, instead of using one image sensor 111 to record a pupil-divided image, a pair of captured images may be acquired using multiple image sensors 111 provided in one digital camera 100 or multiple digital cameras 100. In addition, the distance information may be acquired using a time-of-flight method that converts the time it takes for light emitted from a light-emitting unit to be reflected by a target subject and reach the image sensor 111 into distance information using the speed of light.

[0060] [Variations] In the above-described embodiment, when it is determined that the image is not captured from a direction appropriate for measuring the target subject, a notification is issued to prompt a user to change the image capturing direction of the digital camera 100. However, depending on the image capturing environment, it may be difficult to move the digital camera 100 to a suitable image capturing direction due to obstacles or the like existing in the real space, or the accuracy of deriving the distance information may decrease due to changes in the illumination conditions.

[0061] For this reason, for example, when it is determined in S304 of the measurement process that the image is not captured from a direction appropriate for measuring the target subject, and a notification is presented in S305 to guide the user to change the direction, the measurement may be performed without the need for change in response to a predetermined input. More specifically, even if the image is captured from a direction different from the direction appropriate for measurement, when a predetermined input is made, the measurement unit 108 may generate an image in a state where the measurement plane is directly facing the digital camera 100 by, for example, affine transforming the captured image. Then, the measurement unit 108 may derive the distance between the two measurement points based on the image generated by the transformation, the distance of the measurement plane taking the transformation into account, and the lateral magnification of the imaging optical system 112. In this way, the degree of decrease in measurement accuracy when the measurement plane is not directly facing the digital camera 100 can be reduced without the need for changing the imaging direction.

[0062] Note that the process of performing pseudo-measurement using such a transformation may not be performed in response to a predetermined input, but may be a measurement mode prepared in advance that is different from the mode that issues a notification to prompt a change in the imaging direction. Also, the application of the affine transformation in the process of performing pseudo-measurement is not limited to the captured image, but may be applied to at least one of the captured image and the distance information, and the measurement unit 108 may perform measurement based on the transformed information.

[0063] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

[0064] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0065] 100: digital camera, 101: system control unit, 102: recording medium, 103: memory, 104: imaging unit, 105: distance derivation unit, 106: normal derivation unit, 107: direction derivation unit, 108: measurement unit, 109: display unit, 110: operation input unit, 111: imaging element, 112: imaging optical system, 113: exit pupil, 114: optical axis

Claims

1. A measurement apparatus for performing measurements relating to predetermined items on a target subject that is a measurement target, an acquisition means for acquiring distance information indicating a distribution of subject distances from an imaging device that has captured an image obtained by capturing an image of an imaging range including the target subject, and normal information indicating at least a normal to a surface of the target subject; a determination means for determining whether or not to measure the predetermined item based on the normal information; a notification means for presenting a notification indicating an imaging direction of the imaging device when the determination means determines that the measurement of the predetermined item is not to be performed; having The determination means is if it is determined that the state of the imaging device when the imaging is performed is directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is to be performed; If it is determined that the state of the imaging device when the imaging is performed is not directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is not to be performed. A measuring device characterized by:

2. The measurement device according to claim 1, characterized in that the judgment means judges whether the state of the imaging device when the image was captured was directly facing the surface of the target subject, based on an angle between a normal to the surface of the target subject and an optical axis of an imaging optical system of the imaging device.

3. The measurement device according to claim 2, characterized in that the judgment means judges that the state of the imaging device when the image was captured is directly facing the surface of the target subject when an angle between a normal to the surface of the target subject and an optical axis of the imaging optical system of the imaging device falls within a predetermined range.

4. 4. The measuring device according to claim 1, wherein the notification presented by the presenting means is a notification that prompts the user to change the imaging direction to the direction determined by the determining means when the measurement of the predetermined item is performed.

5. 5. The measuring device according to claim 4, wherein the notification presented by the presenting means includes a moving direction in an imaging direction that the determining means determines when the measurement of the predetermined item is to be performed.

6. 6. The measurement apparatus according to claim 4, wherein the notification presented by the presentation means includes a display of the captured image on which at least one of a surface of the target subject and a normal to the surface is superimposed.

7. an input means for receiving an instruction to measure the predetermined item without changing an imaging direction of the imaging device when the determination means determines not to measure the predetermined item; a conversion means for converting at least one of the captured image and the distance information when the instruction is received by the input means; a measuring means for measuring the predetermined item based on the information converted by the converting means when the determining means determines that the measurement of the predetermined item is not to be performed and the instruction is accepted; The measuring device according to any one of claims 1 to 6, further comprising:

8. a first constructing means for constructing the normal information based on the distance information; The acquiring means acquires the normal information configured by the first configuring means.

8. The measuring device according to claim 1, wherein the measuring device is a measuring device for measuring a temperature of the measuring object.

9. The measurement device according to claim 8, characterized in that the first construction means derives an approximation plane from a distribution of subject distances in an area corresponding to the target subject from the distance information, and constructs the normal information by taking a normal to the approximation plane as a normal to a surface of the target subject.

10. the measurement device includes the imaging device, the imaging device has an output unit that outputs a plurality of captured images obtained by capturing images of the target subject from a plurality of different viewpoints; the measurement apparatus further includes a second configuration means for configuring the distance information based on the plurality of captured images output by the output means, The acquiring means acquires the distance information configured by the second configuring means.

10. The measuring device according to claim 1, wherein the measuring device is a measuring device for measuring a temperature of the measuring object.

11. 11. The measuring apparatus according to claim 1, further comprising a selection unit for selecting the target subject from among the subjects captured in the captured image.

12. 12. The measurement device according to claim 1, wherein the specified items include at least one of a dimension between set measurement points on a surface of the target subject, a peripheral distance of the surface, and an area of ​​a set region.

13. An imaging apparatus for performing measurements relating to predetermined items on a target subject that is a measurement target, an imaging means for imaging an imaging range including the target subject from a plurality of different viewpoints and outputting a plurality of captured images; a generating means for generating distance information indicating a distribution of subject distances from the imaging device for subjects in the imaging range based on the plurality of captured images, and normal information indicating a normal to at least a surface of the target subject; a determination means for determining whether or not to measure the predetermined item based on the normal information generated by the generation means; a notification means for presenting a notification indicating an imaging direction of the imaging device when the determination means determines that the measurement of the predetermined item is not to be performed; having The determination means is if it is determined that the state of the imaging device when the imaging is performed is directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is to be performed; If it is determined that the state of the imaging device when the imaging is performed is not directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is not to be performed.

1. An imaging device comprising:

14. 1. A method for controlling a measurement apparatus that performs measurements relating to predetermined items on a target subject that is a measurement target, comprising: an acquisition step of acquiring distance information indicating a distribution of subject distances from an imaging device that has captured an image of an imaging range including the target subject, and normal information indicating at least a normal to a surface of the target subject; a determination step of determining whether or not to measure the predetermined item based on the normal information; a presentation step of presenting a notification indicating an imaging direction of the imaging device when it is determined in the determination step that the measurement of the predetermined item is not to be performed; having In the determination step, if it is determined that the state of the imaging device when the imaging is performed is directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is to be performed; If it is determined that the state of the imaging device when the imaging is performed is not directly facing the surface of the target subject, it is determined that the measurement of the predetermined item is not to be performed. A method for controlling a measuring device comprising the steps of:

15. A program for causing a computer to function as each of the means of the measuring device according to any one of claims 1 to 12.

Images