Estimation device, estimation system, estimation method, and program

Abstract

The estimation device (2) according to the present disclosure comprises a reference object region detection unit (20) that detects a reference object region which is a region indicating at least a portion of an image of a reference object in a captured image generated by capturing an image of a reference object extending in a real space, a reference object shape specification unit (32) that generates a shaping region obtained by shaping the reference region to a shape that corresponds to the shape of the reference object, and a pixel resolution calculation unit (45) that calculates a pixel resolution which is the ratio of the length of a subject to the length of a pixel that constitutes the captured image on the basis of the shaping region.

Description

[Technical Field]

[0001] This disclosure relates to estimation devices, estimation systems, estimation methods, and programs. [Background technology]

[0002] Image processing is being considered to improve the efficiency of structural inspections. Specifically, techniques have been proposed to estimate deteriorated areas of structures by detecting images of cracks, exposed rebar, etc., from captured images. In recent years, it has also become known that deep learning segmentation methods can be used to estimate structural deterioration.

[0003] Furthermore, in structural inspections, it is not sufficient to merely estimate the deteriorated parts of a structure from captured images. In order to restore the deteriorated parts, it is necessary to understand the extent of the damage and the scope of repair in those deteriorated areas. For this reason, for example, Non-Patent Document 1 describes a technique for extracting crack width using a crack scale captured under the same conditions as cracks on the concrete surface. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Yusuke Fujita, et al., "Image Synthesis and Semi-Automatic Crack Evaluation for Visual Inspection of Concrete Structures," Journal of Japan Society of Civil Engineers, Series F3 (Civil Engineering Informatics), Vol. 74, No. 1, 18-32, 2018. [Overview of the project] [Problems that the invention aims to solve]

[0005] However, it is necessary to evaluate the size of the degraded areas detected from the captured images with greater precision.

[0006] In view of these circumstances, the purpose of this disclosure is to provide an estimation device, estimation system, estimation method, and program that can calculate pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the captured image, with high accuracy. [Means for solving the problem]

[0007] To solve the above problems, the estimation device according to the present disclosure includes: a reference object region detection unit that detects a reference object region, which is a region showing at least a part of the image of a reference object in an image generated by imaging a reference object extending in real space; a reference object shape standardization unit that generates a shaped region by shaping the reference object region to a shape corresponding to the shape of the reference object; and a pixel resolution calculation unit that calculates a pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the image, based on the shaped region.

[0008] Furthermore, in order to solve the above problems, the estimation system according to the present disclosure comprises an imaging device that captures an image of a subject including a reference object extending in real space and generates an image, and an estimation device, the estimation device having a reference object region detection unit that detects a reference object region which is a region in the image of the reference object that shows at least a part of the image of the reference object, a reference object shape standardization unit that generates a shaped region which is a shaped reference object region which is a shape corresponding to the shape of the reference object, and a pixel resolution calculation unit which calculates a pixel resolution which is the ratio of the length of the subject in real space to the length of the pixels constituting the image, based on the shaped region.

[0009] Furthermore, in order to solve the above problems, the estimation method according to this disclosure includes the step of detecting a reference object region, which is a region that shows at least a part of the image of a reference object in an image generated by imaging a reference object that extends in real space, in an estimation method performed by an estimation device, The method includes the steps of: generating a shaped region by reshaping the reference region to a shape corresponding to the shape of the reference object; and calculating the pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the captured image, based on the shaped region.

[0010] Also, to solve the above problems, the program according to the present disclosure causes a computer to function as the above-described estimation device.

Advantages of the Invention

[0011] According to the estimation device, estimation system, estimation method, and program according to the present disclosure, it is possible to calculate the pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the captured image, with high accuracy.

Brief Description of the Drawings

[0012] [Figure 1] It is a schematic diagram showing an example of the estimation system according to the present embodiment. [Figure 2A] It is a diagram showing an example of a captured image generated by the imaging device shown in FIG. 1. [Figure 2B] It is a diagram showing an example of a reference object region in the captured image shown in FIG. 2A. [Figure 3] It is a schematic diagram showing in detail the invisible region complementing unit shown in FIG. 1. [Figure 4A] It is a diagram showing another example of the reference object region. [Figure 4B] It is a diagram showing an example of a shaped region obtained by shaping the reference object region shown in FIG. 4A. [Figure 4C] It is a diagram showing an example of an extended shaped region obtained by extending the shaped region shown in FIG. 4B. [Figure 5A] It is a diagram showing still another example of the reference object region. [Figure 5B] It is a diagram showing an example of a shaped region obtained by shaping the reference object region shown in FIG. 5A. [Figure 5C] It is a diagram showing an example of an extended shaped region obtained by extending the shaped region shown in FIG. 5B. [Figure 6] It is a diagram showing still another example of the reference object region. [Figure 7] It is a schematic diagram showing in detail the pixel resolution estimation unit shown in FIG. 1. [Figure 8A]Figure 7 illustrates an example of the number of interval pixels counted by the interval pixel counting unit. [Figure 8B] Figure 7 illustrates another example of the number of interval pixels counted by the interval pixel counting unit. [Figure 9] Figure 1 is a flowchart illustrating an example of the operation of the estimation device. [Figure 10] Figure 9 is a flowchart detailing steps S4 and S5. [Figure 11] This is a hardware block diagram of the estimation device. [Modes for carrying out the invention]

[0013] As shown in Figure 1, the estimation system 100 according to this embodiment comprises an imaging device 1 and an estimation device 2.

[0014] The imaging device 1 may consist of a camera equipped with an optical element, an image sensor, and an output interface. The output interface is an interface for outputting information.

[0015] The imaging device 1 generates an image of a subject that includes a reference object S extending in real space, as shown in Figure 2A. The subject is an object, etc., that is imaged by the imaging device 1. The reference object S is a predetermined object attached to a structure among the subjects. The structure can be, for example, a tunnel used for laying communication cables. Alternatively, the structure may be a tunnel for laying gas pipes, power lines, etc., a manhole, etc. In a configuration where the structure is a tunnel, the reference object S can be a flat plate-shaped reinforcing bar fixed to the wall surface of the tunnel. The imaging device 1 may also be mounted on a mobile device such as a drone.

[0016] As shown in Figure 2A, the captured image has pixels arranged in the y-axis direction (first direction) and the x-axis direction (second direction) which is perpendicular to the y-axis direction. The y-axis direction is the direction in the captured image that corresponds to the extension direction of the reference object S. The "direction in the captured image that corresponds to the extension direction of the reference object S" is the direction in the captured image where the x-axis direction and the y-axis direction, in which pixels are arranged, have a smaller angle with respect to the extension direction of the reference object S. The format of the captured image may be arbitrary.

[0017] Furthermore, the imaging device 1 outputs the captured image data, which represents the captured image, to the estimation device 2.

[0018] <Configuration of the estimation device> The estimation device 2 comprises an input unit 10, a reference object region detection unit 20, an invisible region interpolation unit 30, a pixel resolution estimation unit 40, and an output unit 50.

[0019] The input unit 10 is comprised of an input interface. The input interface can be an interface that accepts information output from other devices. The input interface may also include a communication interface that receives information from other devices. Standards such as Ethernet®, FDDI (Fiber Distributed Data Interface), and Wi-Fi® may be used for the communication interface. The reference object area detection unit 20, the invisible area interpolation unit 30, and the pixel resolution estimation unit 40 are comprised of a controller. The controller may be comprised of dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array), or it may be comprised of a processor, or it may include both. The output unit 50 is comprised of an output interface. The output interface may include a communication interface that transmits information to other devices.

[0020] The input unit 10 receives captured image data, which represents the captured image generated by the imaging device 1, as input.

[0021] The reference object region detection unit 20 detects the reference object region (the region shown in white in the example of Figure 2B), which is the region in the captured image data received as input by the input unit 10 that represents at least a part of the image of the reference object S. Specifically, the reference object region detection unit 20 can detect the reference object region by using machine learning, deep learning, etc. Hereafter, regions in the captured image that were not detected as the reference object region will be referred to as non-reference object regions.

[0022] The reference object region is composed of multiple pixels from among the pixels that make up the image. It is preferable that all of the multiple pixels that make up the reference object region are included in the reference object region. In other words, it is preferable that the reference object region is not configured such that a portion of any one of the multiple pixels that make up the reference object S is included in the reference object region, while the remaining portion is not included in the reference object region.

[0023] As shown in Figure 3, the invisible area completion unit 30 includes an area determination unit 31, a reference object shape standardization unit 32, and a shape extension unit 33.

[0024] In real space, other objects may be placed between the reference object S and the imaging device 1. For example, in a configuration where the structure is a tunnel and the reference object S is a reinforcing bar, the other objects may be support brackets, cables, etc. In such cases, as shown in Figure 2A, a portion of the image of the reference object S is not shown in the captured image, and the image representing a single reference object S is fragmented. For this reason, the reference object region detection unit 20 described above divides the reference object region constituting the image of a single reference object into regions enclosed by thick circles, as shown in Figure 2B, and detects them as multiple reference object regions R0.

[0025] Therefore, the region determination unit 31 determines one or more reference object regions that constitute an image of a single reference object S, based on the positional relationship of one or more reference object regions. Specifically, first, the region determination unit 31 determines whether or not the multiple reference object regions are arranged adjacent to each other in a direction corresponding to the extension direction of the reference object S.

[0026] If the region determination unit 31 determines that multiple reference object regions are not adjacent to each other in a direction corresponding to the extension direction of the reference object S, it determines that one reference object region constitutes an image of a single reference object S. If the region determination unit 31 determines that multiple reference object regions are adjacent to each other in a direction corresponding to the extension direction of the reference object S, it determines that the multiple reference object regions constitute an image of a single reference object S based on the distance between the multiple reference object regions.

[0027] As an example, the region determination unit 31 determines whether the length between the extensions of the centerlines of each of the multiple reference object regions, which are arranged in a direction corresponding to the extension direction of the reference object S, is less than or equal to a predetermined value. Here, the centerline is a straight line passing through the coordinates obtained by averaging the coordinates (x coordinates) of the midpoint of the reference object region in the x-axis direction.

[0028] To illustrate with an example from Figure 4A, the region determination unit 31 determines which reference regions constitute an image of a single reference object S based on the positional relationship of multiple reference regions R01 and R02. Specifically, the region determination unit 31 determines whether the length d01 between the extensions of the centerlines LN1 and LN2 of reference regions R01 and R02, which are adjacent to each other in the direction corresponding to the extension direction of the reference object S (the y-axis direction in the example of Figure 4A), is less than or equal to a predetermined value. If the region determination unit 31 determines that the length d01 is less than or equal to the predetermined value, it determines that the multiple reference regions R01 and R02 constitute an image of a single reference object S. If the region determination unit 31 determines that the length d01 is longer than the predetermined value, it determines that the multiple reference regions R01 and R02 do not constitute an image of a single reference object S.

[0029] Similarly, the region determination unit 31 determines whether the length d02 between the extensions of the centerlines LN3 and LN4 of the reference object regions R03 and R04, which are adjacent to each other in a direction corresponding to the extension direction of the reference object S (the y-axis direction in the example of Figure 4A), is less than or equal to a predetermined value. If the region determination unit 31 determines that the length d02 is less than or equal to the predetermined value, it determines that the multiple reference object regions R03 and R04 constitute an image of a single reference object S. If the region determination unit 31 determines that the length d02 is longer than the predetermined value, it determines that the multiple reference object regions do not constitute an image of a single reference object S.

[0030] As another example, the region determination unit 31 determines whether the length between multiple reference object regions that are adjacent to each other in a direction corresponding to the extension direction of the reference object S is shorter than the length between multiple reference object regions that are adjacent to each other in a direction orthogonal to the extension direction of the reference object S.

[0031] To illustrate with an example from Figure 4A, the region determination unit 31 determines whether the length d1 between multiple reference object regions R01 and R02, which are adjacent to each other in the direction corresponding to the extension direction of the reference object S (the y-axis direction in the example of Figure 4A), is shorter than the length d2 between multiple reference object regions R01 and R03, which are adjacent to each other in a direction orthogonal to the direction corresponding to the extension direction of the reference object S (the x-axis direction in the example of Figure 4A).

[0032] The region determination unit 31 then determines that if length d1 is shorter than length d2, the multiple reference object regions R01 and R02 constitute an image of a single reference object S. Alternatively, the region determination unit 31 determines that the multiple reference object regions R01 and R02 do not constitute an image of a single reference object S.

[0033] The reference object shape standardization unit 32 generates a shaped region by shaping the reference object region detected by the reference object region detection unit 20 based on the shape of the reference object S in real space. Specifically, as shown in Figure 4A, if it is determined that multiple reference object regions constitute an image of a single reference object S, the reference object shape standardization unit 32 generates multiple shaped regions, each shaped for a different reference object region, as shown in Figure 4B. Also, as shown in Figure 5A, if it is determined that one reference object region constitutes an image of a single reference object S, the reference object shape standardization unit 32 generates one shaped region, each shaped for a single reference object region, as shown in Figure 5B.

[0034] As an example, the reference object shape standardization unit 32 can take a reference object region as shown in Figure 4A and generate a shaped region as shown in Figure 4B, which is circumscribing the reference object region and has a rectangle with the smallest area as its periphery. Note that Figure 4A is an example where it is determined that multiple reference object regions constitute the image of a single reference object S. However, even if it is determined that one reference object region constitutes the image of a single reference object S, as shown in Figure 5A, the reference object shape standardization unit 32 will generate a shaped region as shown in Figure 5B, which is circumscribing the reference object region and shaped into a rectangle with the smallest area.

[0035] As another example, the standard object shape specification section 32, as shown in Figure 6, is located at the maximum coordinate y in the y-axis direction from the standard object region (the region shown in white in the example in Figure 6). max and minimum coordinate y min It detects the maximum coordinate y in the y-axis direction. Then, the reference object shape standardization section 32 determines the maximum coordinate y max The maximum coordinate in the x-axis direction is x max1 and minimum coordinate x min1 The two points P1 and P2 located at each of these points, and the minimum coordinate y in the y-axis direction. min The maximum coordinate in the x-axis direction is x max2 and minimum coordinate x min2Extract two points P3 and P4 that result. Then, the reference object shape standardization unit 32 can generate a shaped region by shaping the reference object region into a trapezoid having the extracted four points P1, P2, P3, and P4 as vertices. In the example shown in FIG. 6, the reference object shape standardization unit 32 shapes the reference object region into a shaped region that is shaped into a trapezoid, but it is not limited to this, and the maximum coordinate y in the y-axis direction max and the minimum coordinate y min , and a shaped region shaped into a quadrilateral having points based on the maximum coordinate x in the x-axis direction max and the minimum coordinate x min as vertices can be generated.

[0036] Note that the reference object shape standardization unit 32 may generate a shaped region according to one of the above examples based on the shape information received by the input unit 10, or may generate a shaped region according to one of the other examples above. That is, the reference object shape standardization unit 32 can generate a quadrilateral shaped region, and when the shape shown in the shape information is a rectangle, a shaped region obtained by shaping the reference object region into a rectangle is generated. Further, when the shape shown in the shape information is a trapezoid, the reference object shape standardization unit 32 generates a shaped region obtained by shaping the reference object region into a trapezoid. Further, when the shape shown in the shape information is another quadrilateral, the reference object shape standardization unit 32 generates a shaped region obtained by shaping the reference object region into the other quadrilateral.

[0037] The shape extension unit 33 generates an extended shaped region by extending the shaped region in the direction in the captured image corresponding to the extending direction of the reference object S in the real space. In the examples shown in FIGS. 4A to 4C, the direction corresponding to the extending direction of the reference object S is the y-axis direction, and the shape extension unit 33 extends the shaped region in the y-axis direction.

[0038] As described above, in the configuration in which the reference object shape standardization unit 32 corrects the reference object region into a rectangle, when it is determined that a plurality of reference object regions constitute an image of a single reference object S as shown in FIG. 4A, the shape extension unit 33, as shown in FIG. 4C, to a plurality of shaped regions R11 and R12 that constitute an image of a single reference object S, a first complementary region R located between the plurality of shaped regions R11 and R12 +1And a second complementary region R located between the reshaping region R11 and the edge of the captured image. +2 The extended shaping region RA is generated by adding the following. Similarly, the shape extension 33 adds a first complementary region R to the multiple shaping regions R13 and R14 that constitute the image of a single reference object S, which are located between the multiple shaping regions R13 and R14. +1 And a second complementary region R located between the reshaping region R13 and the edge of the captured image. +2 The extended shaping region RB is generated by adding the following.

[0039] Furthermore, as shown in Figure 5A, if it is determined that one reference object region constitutes an image of a single reference object S, the shape extension unit 33 extends a second complementary region R located between the shaping region R11 and the edge of the captured image, as shown in Figure 5C, to the shaping region R11 generated by the reference object shape standardization unit 32. +2 The shape extension section 33 adds an extended shaping region to the shaping region R12 generated by the reference object shape standard section 32, as shown in Figure 5C, which is located between the shaping region R12 and the edge of the captured image. +2 This generates an extended and reshaped region with the added element.

[0040] As described above, even in a configuration where the reference object shape standardization section 32 corrects the reference object area to a trapezoid, the shape extension section 33 extends the shaping area in a direction corresponding to the extension direction of the reference object. Specifically, the shape extension section 33 consists of two opposing line segments L, as shown in Figure 6, which are not parallel to each other. 13 and line segment L 24 Extend each of them (the extension line L shown in Figure 6). 13 'and L 24 '). Line segment L 13 L is the line segment connecting vertices P1 and P2. 24 This is a line segment connecting vertices P2 and P4. The extended shape 33 is formed by two extension lines L 13 'and L 24 A trapezoidal region enclosed by the line segment that forms the edge of the captured image is generated as the extended shaping region.

[0041] As shown in Figure 7, the pixel resolution estimation unit 40 includes a reference pixel count unit 41, a width pixel count unit 42, a thinning unit 43, an interval pixel count unit 44, and a pixel resolution calculation unit 45.

[0042] The reference object counting unit 41 counts the number of extended shaping regions generated by the invisible region completion unit 30.

[0043] The width pixel count unit 42 counts the width pixels in the captured image, which corresponds to the length of the reference object S in the direction orthogonal to the extension direction (width direction). The width pixels may also be the number of pixels in the direction orthogonal to the extension direction of the reference object region in the captured image. As another example, the width pixels may be the number of pixels in the x-axis direction of the captured image.

[0044] The thinning unit 43 determines the center line in each of the multiple extended shaping regions when the number of extended shaping regions counted by the reference number counting unit 41 is multiple. The center line determined by the thinning unit 43 is a line segment formed by the set of midpoints of both ends in the x-axis direction of the extended shaping region. In the example shown in Figure 4C, the thinning unit 43 determines the center line LNA in the extended shaping region RA and the center line LNB in ​​the extended shaping region RB.

[0045] As shown in Figure 8A, the spacing pixel count unit 44 counts the spacing pixel count c, which is the number of pixels between the multiple center lines LNA and LNB determined by the thinning unit 43.

[0046] Furthermore, as shown in Figure 8B, the interval pixel count unit 44 can count the interval pixel count c at a plurality of predetermined positions in the y-axis direction. Specifically, the interval pixel count unit 44 may measure the interval pixel counts c1, c2, and c3 at a first position y1 in the y-axis direction, a second position y2 different from the first position, and a third position y3 different from the first position y1 and the second position y2.

[0047] The pixel resolution calculation unit 45 calculates the pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the captured image, based on the shaping region. The pixel resolution calculation unit 45 may also calculate the pixel resolution based on the extended shaping region generated by the shape extension unit 33 extending the shaping region.

[0048] Specifically, the pixel resolution calculation unit 45 can calculate the pixel resolution Rp based on the width of the pixels a and the width b of the portion of the reference object S corresponding to the width of the pixels a. For example, the pixel resolution calculation unit 45 can calculate the pixel resolution Rp by dividing the width b by the width of the pixels a, as shown in the following equation (1). The width b of the reference object S is a known value and may be a value received as input through the input interface.

number

[0049] Furthermore, the pixel resolution calculation unit 45 may calculate a pixel resolution Rp, which represents the length in real space relative to the length of a pixel, based on the number of spacing pixels c between a plurality of center lines LNA and LNB determined by the thinning unit 43, and the length d of the subject in real space corresponding to the number of spacing pixels c.

[0050] For example, the interval pixel count unit 44 calculates the pixel resolution Rp by dividing the length d of the subject in real space corresponding to the interval pixel count c by the interval pixel count c, as shown in equation (2) below. The length d is a known value, and may be a value received as input by the input interface, for example.

number

[0051] As described above, in a configuration in which the interval pixel count unit 44 counts the interval pixel counts c1 to cn (n=3 in the example of Figure 8B described above) at multiple predetermined positions in the y-axis direction, the pixel resolution calculation unit 45 may calculate the pixel resolution Rp1 to Rpn for each of the multiple predetermined positions by dividing the interval pixel counts c1 to cn by the length d. In real space, even if the interval between multiple reference objects S is constant, the interval pixel count c between the center lines LNA and LNB may differ depending on the position in the captured image, as shown in Figure 8B, depending on the installation angle of the imaging device relative to the reference objects S. In such cases as well, the pixel resolution calculation unit 45 can calculate the pixel resolution Rp for each position in the captured image by having the interval pixel count unit 44 count the interval pixel count c for each position in the captured image. Furthermore, as shown in Figure 8A, in a configuration in which multiple reference objects are arranged substantially parallel in real space and multiple center lines LNA and LNB in ​​the captured image are substantially parallel, the pixel resolution calculation unit 45 can calculate the pixel resolution Rp for the entire captured image. Therefore, the estimation device 2 can calculate the pixel resolution Rp with high accuracy, and the magnitude of the deformation in the subject can be estimated with high accuracy by using this pixel resolution Rp.

[0052] As described above, if there is only one extended shaping region, the width pixel count unit 42 counts the width pixels a, and the pixel resolution calculation unit 45 calculates the pixel resolution Rp based on the width pixels a. In contrast, if there are multiple extended shaping regions, the width pixel count unit 42 may count the width pixels a, and the pixel resolution calculation unit 45 may calculate the pixel resolution Rp based on the width pixels a. Alternatively, the thinning unit 43 may determine the center line, the spacing pixel count unit 44 may count the spacing pixels c, and the pixel resolution calculation unit 45 may calculate the pixel resolution Rp based on the spacing pixels c. In real space, the spacing between multiple reference objects S is generally longer than the width of the reference objects S. Therefore, the pixel resolution calculation unit 45 can calculate the pixel resolution Rp based on the spacing pixels c with higher accuracy than calculating the pixel resolution Rp based on the width pixels a. Therefore, if there are multiple extended shaping regions, it is preferable that the pixel resolution calculation unit 45 calculates the pixel resolution Rp based on the number of interval pixels c rather than the number of width pixels a. For this reason, it is preferable that the thinning unit 43 and the number of interval pixels count unit 44 perform the processing rather than the number of width pixels count unit 42.

[0053] The output unit 50 outputs pixel resolution information, including the pixel resolution Rp calculated by the pixel resolution calculation unit 45. The pixel resolution information may further include information indicating the position of the pixel where the image of the subject is displayed at the pixel resolution Rp, along with the pixel resolution Rp. The pixel resolution information output by the output unit 50 may be stored in a storage device equipped with memory.

[0054] <Operation of the Estimation Device> Here, the operation of the estimation device 2 according to this embodiment will be described with reference to Figures 9 and 10. Figures 9 and 10 are flowcharts illustrating an example of the operation of the estimation device 2 according to this embodiment. The operation of the estimation device 2 described with reference to Figures 9 and 10 corresponds to an example of the estimation method of the estimation device 2 according to this embodiment.

[0055] As shown in Figure 9, in step S1, the input unit 10 receives captured image data, which represents an image generated by the imaging device 1 capturing a reference object S that extends in one direction in real space.

[0056] In step S2, the reference object region detection unit 20 detects a reference object region, which is a region that represents at least a part of the image of the reference object S in the captured image generated by imaging the reference object S that extends in real space.

[0057] In step S3, the region determination unit 31 determines one or more reference object regions that constitute an image of a single reference object S.

[0058] In step S4, the reference object shape standardization unit 32 generates a shaping region by shaping the reference object region to a shape corresponding to the shape of the reference object S.

[0059] In step S5, the shape extension section 33 generates an extended shaping region by extending the shaping region in the direction in the captured image that corresponds to the extension direction of the reference object S in real space.

[0060] Here, with reference to Figure 10, the process of generating the shaping region in step S4 and the process of generating the extended shaping region in step S5 will be described in detail.

[0061] In step S41, the reference object shape standardization unit 32 determines whether the region determination unit 31 has determined that multiple reference object regions constitute an image of a single reference object S.

[0062] If, in step S41, the region determination unit 31 determines that multiple reference object regions constituting the image of a single reference object S have been determined, then in step 42, the reference object shape standardization unit 32 generates multiple shaped regions by shaping each of the multiple reference object regions.

[0063] In step S51, the shape extension unit 33 generates an extended shaping region by adding a first complementary region located between the multiple shaping regions and a second complementary region located between the shaping region and the edge of the captured image to the multiple shaping regions.

[0064] Furthermore, if it is determined in step S41 that one reference object region constituting the image of a single reference object S has been selected, in step S43 the reference object shape standardization unit 32 generates one shaped region by shaping one reference object region.

[0065] In step S52, the shape extension unit 33 generates an extended shaping region by adding a second complementary region located between the first shaping region and the edge of the captured image to one shaping region.

[0066] In steps S41 and S42, the standard object shape standardization section 32 may generate a shaping region that is circumscribing the standard object region and has a rectangle with the smallest area as its periphery, or, as described above, it may generate a shaping region in which the standard object region is shaped into a trapezoid.

[0067] Returning to Figure 9, in step S6, the pixel resolution calculation unit 45 calculates the pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the captured image, based on the shaping region. The pixel resolution calculation unit 45 may also calculate the pixel resolution based on the extended shaping region.

[0068] In step S7, the output unit 50 outputs pixel resolution information, including the pixel resolution calculated by the pixel resolution calculation unit 45.

[0069] As described above, according to this embodiment, the estimation device 2 includes a reference object region detection unit 20 that detects a reference object region, which is a region showing at least a part of the image of a reference object S in an image generated by imaging a reference object S that extends in real space; a reference object shape standardization unit 32 that generates a shaped region by shaping the reference object region to a shape corresponding to the shape of the reference object S; and a pixel resolution calculation unit 45 that calculates the pixel resolution, which is the ratio of the length of the subject to the length of the pixels constituting the image, based on the shaped region.

[0070] As a result, the estimation device 2 can calculate the pixel resolution, which is the ratio of the length of the subject to the length of the pixels that make up the captured image, with high accuracy. Therefore, by utilizing the pixel resolution, the size of the deformation shown in the captured image in real space can be estimated with high accuracy.

[0071] For example, even if, in real space, other objects such as mounting brackets or cables are placed between the imaging device 1 and the reference object S, and a portion of the image of the reference object S is not shown in the captured image, the estimation device 2 of this embodiment generates a shaping region corresponding to the entire portion corresponding to the reference object S. This allows the pixel resolution Rp to be calculated using the lengths (width and spacing in the example above) related to the reference object S in real space.

[0072] Furthermore, if a corrected region is not generated with the reference object region corrected, multiple center lines may be extracted when extracting the center line in the reference object region used to calculate the pixel resolution Rp. In this case, it may be impossible to determine which of the multiple center lines should be used to calculate the pixel resolution Rp. In contrast, the estimation device 2 of this embodiment generates a corrected region with the reference object region corrected, and can appropriately extract one center line from the reference object region corresponding to one reference object S. Accordingly, the estimation device 2 can appropriately calculate the pixel resolution Rp.

[0073] Furthermore, according to this embodiment, the estimation device 2 further includes a shape extension unit 33 that generates an extended shaping region by extending the shaping region in the direction in the captured image that corresponds to the direction of extension of the reference object S in real space. The pixel resolution calculation unit 45 calculates the pixel resolution based on the extended shaping region. As a result, the estimation device 2 can calculate the pixel resolution in the portion of the captured image where the image of the reference object S is not shown. Therefore, even in the portion of the captured image where the image of the reference object S is not shown, the size in real space of the deformation shown in that portion can be calculated. Moreover, even when multiple reference object regions constitute an image of a single reference object S because a part of the image of the reference object S is not shown in the captured image, the estimation device 2 generates an extended shaping region that constitutes an image of a single reference object S by generating an extended shaping region. Therefore, the estimation device 2 can accurately determine the number of reference objects S whose images are shown in the captured image.

[0074] Furthermore, the estimation device 2 may also include a storage unit, which stores the captured image data received as input by the input unit 10. In this configuration, the reference object region detection unit 20 may detect the reference object region indicated by the captured image data stored in the storage unit.

[0075] <Program> The estimation device 2 described above can be implemented by a computer 601. A program for enabling the estimation device 2 to function may also be provided. This program may be stored on a storage medium or provided via a network. Figure 11 is a block diagram illustrating the schematic configuration of a computer 601 functioning as the estimation device 2. Here, the computer 601 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. Program instructions may be program code, code segments, etc., for executing the required tasks.

[0076] As shown in Figure 11, the computer 601 comprises a processor 610, a ROM (Read Only Memory) 620, a RAM (Random Access Memory) 630, storage 640, an input unit 650, a display unit 660, and a communication interface (I / F) 670. Each component is connected to the others via a bus 680 so as to be able to communicate with each other. The processor 610 is specifically a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), SoC (System on a Chip), etc., and may be composed of multiple processors of the same or different types.

[0077] The processor 610 controls each configuration and performs various arithmetic operations. Specifically, the processor 610 reads a program from the ROM 620 or storage 640 and executes the program using the RAM 630 as a working area. The processor 610 controls each configuration and performs various arithmetic operations according to the program stored in the ROM 620 or storage 640. In the embodiment described above, the program according to this disclosure is stored in the ROM 620 or storage 640.

[0078] The program may be stored on a storage medium readable by computer 601. Using such a storage medium, the program can be installed on computer 601. Here, the storage medium on which the program is stored may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, but may include, for example, a CD-ROM, DVD-ROM, or USB (Universal Serial Bus) memory. Alternatively, the program may be downloaded from an external device via a network.

[0079] ROM620 stores various programs and data. RAM630 temporarily stores programs or data as a working area. Storage640 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data, including the operating system.

[0080] The input unit 650 includes one or more input interfaces that receive user input operations and acquire information based on the user operations. For example, the input unit 650 is a pointing device, a keyboard, a mouse, etc., but is not limited to these.

[0081] The display unit 660 includes one or more output interfaces for outputting information. For example, the display unit 660 is a display that outputs information as video, or a speaker that outputs information as audio, but is not limited to these. If the display unit 660 is a touch panel display, it also functions as an input unit 650.

[0082] Communication interface (I / F) 670 is an interface for communicating with external devices.

[0083] The following additional information is disclosed regarding the embodiments described above.

[0084] (Additional note 1) The controller is equipped with a controller, A reference object region is detected in an image generated by imaging a subject that includes a reference object extending in real space, which is a region that shows at least a part of the image of the reference object. A shaping region is generated by reshaping the aforementioned reference region to a shape corresponding to the shape of the reference object. An estimation device that calculates the pixel resolution, which is the ratio of the length of the subject in real space to the length of the pixels constituting the captured image, based on the shaping region. (Additional note 2) The aforementioned controller, An extended shaping region is generated by extending the shaping region in the direction in the captured image that corresponds to the direction of extension of the reference object in the real space. An estimation apparatus according to Appendix 1, which calculates the pixel resolution based on the extended shaping region. (Additional note 3) The aforementioned controller, Based on the positional relationship of one or more of the above reference object regions, one or more reference object regions that constitute an image of a single reference object are determined. If it is determined that multiple reference object regions constitute an image of a single reference object, Multiple shaping regions are generated by shaping each of the aforementioned multiple reference region regions. An extended shaping region is generated by adding a first complementary region located between the multiple shaping regions and a second complementary region located between the shaping region and the edge of the captured image to the multiple shaping regions. If it is determined that one reference region constitutes an image of a single reference object, A single shaped region is generated by shaping the aforementioned single reference region. The estimation device according to Appendix 2, which generates an extended shaping region by adding a second complementary region located between the shaping region and the edge of the captured image to the aforementioned shaping region. (Additional note 4) The controller generates the shaping region having a minimum area rectangle as its periphery, which is circumscribing the reference region, according to the estimation device described in Appendix 1 or 2. (Additional note 5) In the captured image, pixels are arranged in a first direction and a second direction perpendicular to the first direction, and the first direction is the direction in the captured image that corresponds to the extension direction of the reference object S. The estimation device according to Appendix 1 or 2, wherein the controller generates a shaped region that is shaped into a quadrilateral whose vertices are points based on the maximum and minimum coordinates in the first direction and the maximum and minimum coordinates in the second direction. (Additional note 6) The system comprises an imaging device that captures a subject including a reference object extending in real space and generates an image, and an estimation device, The estimation device includes a controller, and the controller is A reference object region is detected in the captured image, which is a region showing at least a part of the image of the reference object. A shaping region is generated by reshaping the aforementioned reference region to a shape corresponding to the shape of the reference object. An estimation device that calculates the pixel resolution, which is the ratio of the length of the subject in real space to the length of the pixels constituting the captured image, based on the shaping region. (Additional note 7) In the estimation method performed by the estimation device, A reference object region is detected in an image generated by imaging a subject that includes a reference object extending in real space, which is a region that shows at least a part of the image of the reference object. A shaping region is generated by reshaping the aforementioned reference region to a shape corresponding to the shape of the reference object. An estimation method for calculating the pixel resolution, which is the ratio of the length of the subject in real space to the length of the pixels constituting the captured image, based on the reshaping region. (Additional note 8) A non-temporary storage medium storing a program executable by a computer, the non-temporary storage medium storing a program that causes the computer to function as an estimation device as described in any one of the appendices 1 to 5.

[0085] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually described as being incorporated by reference.

[0086] Although the embodiments described above are representative examples, it will be apparent to those skilled in the art that many modifications and substitutions are possible within the spirit and scope of this disclosure. Therefore, the present invention should not be construed as being limited by the embodiments described above, and various modifications or changes are possible without departing from the claims. For example, it is possible to combine multiple component blocks shown in the configuration diagram of the embodiments into one, or to divide one component block. [Explanation of Symbols]

[0087] 1. Imaging device 2 Estimation device 10 Input section 20 Reference Object Area Detection Unit 30 Invisible area complement part 31 Area determination section 32 Reference object shape specification section 33 Shape extension 40 Pixel Resolution Estimation Unit 41 Reference Quantity Counting Unit 42 Pixel count section 43 Thinning section 44 Pixel count section 45 Pixel resolution calculation unit 50 Output section 100 Estimation Systems 601 Computer 610 Processor 620 ROM 630 RAM 640 storage 650 Input section 660 Output section 670 Communication Interface 680 bus

Claims

1. A reference object region detection unit detects a reference object region in an image generated by imaging a subject that includes a reference object extending in real space, which is a region that shows at least a part of the image of the reference object. A reference object shape standardization unit that generates a shaping region by shaping the reference object region to a shape corresponding to the shape of the reference object, An estimation device comprising: a pixel resolution calculation unit that calculates a pixel resolution, which is the ratio of the length of the subject in real space to the number of pixels corresponding to the length of the subject in the captured image, based on the shaping region.

2. The system further comprises a shape extension unit that generates an extended shaping region by extending the shaping region in the direction in the captured image that corresponds to the extending direction of the reference object in the real space, The estimation apparatus according to claim 1, wherein the pixel resolution calculation unit calculates the pixel resolution based on the extended shaping region.

3. The region determination unit further comprises a region determination unit that determines one or more reference object regions that constitute an image of a single reference object based on the positional relationship of one or more reference object regions, If it is determined that multiple reference object regions constitute an image of a single reference object, The standard object shape standardization unit generates a plurality of shaping regions by shaping each of the plurality of standard object regions, The shape extension unit generates an extended shaping region by adding a first complementary region located between the plurality of shaping regions and a second complementary region located between the shaping region and the edge of the captured image to the plurality of shaping regions. If it is determined that one reference region constitutes an image of a single reference object, The aforementioned standard object shape standardization unit generates one shaping region by shaping one standard object region, The estimation device according to claim 2, wherein the shape extension portion generates an extended shaping region by adding a second complementary region located between the shaping region and the edge of the captured image to the one shaping region.

4. The estimation device according to claim 1 or 2, wherein the standard object shape specification section generates the shaping region having a rectangle with the smallest area as its periphery, which is circumscribing the standard object region.

5. In the captured image, pixels are arranged in a first direction and a second direction perpendicular to the first direction, and the first direction is the direction in the captured image that corresponds to the extension direction of the reference object S. The estimation device according to claim 1 or 2, wherein the standard object shape specification section generates a shaped region shaped into a quadrilateral whose vertices are points based on the maximum and minimum coordinates in the first direction and the maximum and minimum coordinates in the second direction.

6. The system comprises an imaging device that captures a subject including a reference object extending in real space and generates an image, and an estimation device, The estimation device is, A reference object region detection unit detects a reference object region in the captured image, which is a region showing at least a part of the image of the reference object. A reference object shape standardization unit that generates a shaping region by shaping the reference object region to a shape corresponding to the shape of the reference object, An estimation system having a pixel resolution calculation unit that calculates a pixel resolution, which is the ratio of the length of the subject in real space to the number of pixels corresponding to the length of the subject in the captured image, based on the shaping region.

7. In the estimation method performed by the estimation device, A step of detecting a reference object region, which is a region showing at least a part of the image of the reference object in an image generated by imaging a subject that includes a reference object extending in real space, The steps include generating a shaped region by shaping the reference region to a shape corresponding to the shape of the reference object, An estimation method comprising the step of calculating a pixel resolution, which is the ratio of the length of the subject in real space to the number of pixels corresponding to the length of the subject in the captured image, based on the shaping region.

8. A program for causing a computer to function as an estimation device according to any one of claims 1 to 3.

Images