Measurement system, measurement method, measurement program, and object

Abstract

To provide a measurement system, a measurement method, a measurement program, and an object capable of easily and accurately measuring a change in positions and postures in multiple places on the same object.SOLUTION: A measurement system 1 comprises: markers 100 provided in multiple places of an object; a photographing section 10 for photographing the markers 100; and an arithmetic section 20 for calculating information on positions and postures of each of the multiple markers 100 on the basis of the image information photographed by the photographing section 10. The photographing section 10 photographs the markers 100 at each time of time t1 and t2 (t1<t2), and the arithmetic section 20 calculates information on positions and postures of each of the markers 100 at each time of time t1 and t2 and calculates a change in positions and postures of each of the markers 100 on a rocket 200 between time t1 and t2.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a measurement system, a measurement method, a measurement program, and an object. [Background technology]

[0002] It is desirable to accurately measure changes in the relative positions of multiple locations. For example, if it were possible to accurately measure the deformation of each part of a flying object such as a rocket or the loosening of fasteners, it would be possible to prevent malfunctions from occurring. Furthermore, if it were possible to periodically measure the relative displacement of specific parts of buildings such as bridges, it would be easy to determine whether maintenance is required.

[0003] Patent Document 1 describes a control method for calculating the position and orientation of a workpiece using position and orientation data of the workpiece acquired from a captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-11339 Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 1 does not disclose or suggest a technique for easily and accurately measuring changes in the positions and orientations of multiple locations on the same object.

[0006] An object of the present disclosure is to provide a measurement system, a measurement method, a measurement program, and an object that can easily and accurately measure changes in position and orientation at multiple locations on the same object. [Means for solving the problem]

[0007] The present disclosure solves the above problems by the following means. For ease of understanding, reference numerals corresponding to embodiments of the present disclosure are attached for explanation, but it is not limited thereto.

[0008] A first disclosure is a measurement system (1) including markers (100, 110) provided at a plurality of locations on an object (200, 300), an imaging unit (10) that images the markers (100, 110), and a calculation unit (20) that calculates information on the position and orientation of each of the plurality of markers (100, 110) based on the image information captured by the imaging unit (10). The imaging unit (10) images the markers (100, 110) at each of times t1 and t2 (t1 < t2), the calculation unit (20) calculates information on the position and orientation of each of the markers (100, 110) at each of times t1 and t2, and calculates the change in the position and orientation of each of the markers (100, 110) on the object (200, 300) between time t1 and time t2.

[0009] A second disclosure is the measurement system (1) according to the first disclosure, wherein at least one (110) of the plurality of markers (100, 110) is provided at a part of the object (200, 300) that is less likely to be displaced than the part where the other markers (100) are provided.

[0010] A third disclosure is the measurement system (1) according to the second disclosure, wherein the calculation unit (20) calculates the change in the position and orientation of the other markers (100) based on the marker (110) provided at a part of the object (300) that is less likely to be displaced.

[0011] The fourth disclosure is a measurement system (1) as described in the second disclosure or the third disclosure, wherein the marker (110) provided at a site on the object (300) that is difficult to displace is provided in any form of embedding, painting, printing, or adhesion with respect to the object (300).

[0012] The fifth disclosure is a measurement method in a measurement system (1) including markers (100, 110) provided at a plurality of locations on an object (200, 300), an imaging unit (10) that images the markers (100, 110), and a calculation unit (20) that calculates information on the position and orientation of each of the plurality of markers (100, 110) based on the image information imaged by the imaging unit (10), the method including: a step in which the imaging unit (10) images the markers (100, 110) at respective times of time t1 and time t2 (t1 < t2); a step in which the calculation unit (20) calculates information on the position and orientation of each of the markers (100, 110) at respective times of time t1 and time t2; and a step in which the calculation unit (20) calculates changes in the position and orientation of each of the markers (100, 110) on the object (200, 300) between time t1 and time t2.

[0013] The sixth disclosure is a measurement program used in a measurement system (1) including markers (100, 110) provided at multiple locations on an object (200, 300), an imaging unit (10) that images the markers (100, 110), and a calculation unit (20) that calculates information on the position and orientation of each of the multiple markers (100, 110) based on the image information captured by the imaging unit (10). The measurement program causes a computer to perform the steps of: the imaging unit (10) imaging the markers (100, 110) at each of times t1 and t2 (t1 < t2); the calculation unit (20) calculating information on the position and orientation of each of the markers (100, 110) at each of times t1 and t2; and the calculation unit (20) calculating the changes in the position and orientation of each of the markers (100, 110) on the object (200, 300) between time t1 and time t2.

[0014] The seventh disclosure is an object (300) having a main body portion (310) serving as a reference for the entire object (300) and a displaceable portion (320) that can be displaced or is likely to be displaced relative to the main body portion (310), and including a marker (110) provided on the main body portion (310) and a marker (100) provided on the displaceable portion.

Advantages of the Invention

[0015] According to the present disclosure, it is possible to provide a measurement system, a measurement method, a measurement program, and an object that can easily and accurately measure changes in the positions and orientations of multiple locations on the same object.

Brief Description of the Drawings

[0016] [Figure 1] It is a diagram showing a first embodiment of a measurement system according to the present disclosure. [Figure 2] It is a diagram showing an example of a marker 100. [Figure 3] It is a flowchart showing the flow of a measurement operation by the measurement system 1 of the first embodiment. [Figure 4] FIG. 2 shows a second embodiment of a measurement system according to the present disclosure, with markers 100, 110 attached to a spacecraft 300. [Figure 5] 10 is a flowchart showing the flow of a measurement operation by the measurement system 1 of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0018] (First embodiment) Fig. 1 is a diagram showing a first embodiment of a measurement system according to the present disclosure. Note that Fig. 1 and the other figures shown below are schematic diagrams, and the size and shape of each part are exaggerated or omitted as appropriate for ease of understanding. In the following explanation, specific numerical values, shapes, materials, etc. are used for explanation, but these can be changed as appropriate.

[0019] The measurement system 1 of the first embodiment is a system that measures changes in the relative position and attitude of each part of a rocket 200, and includes a marker 100, an imaging unit 10, and a calculation unit 20. Note that while the rocket 200 is used as an example here, the system may be a movable object such as an airplane, ship, or vehicle, or may be an object that does not normally move such as a bridge, bridge pier, or building.

[0020] The markers 100 are attached to different parts of the rocket so as to be integrated with the target object. Here, the target object is the object to be measured for which relative movement and changes in attitude are to be measured. Examples include bolts, nuts, screws, etc., which are fastening members exposed on the outside of the rocket. Another example is the outer panel member of the rocket.

[0021] By providing the marker 100 so that it is integral with the object, changes in the position and orientation of the marker 100 can be measured, thereby enabling changes in the position and orientation of the object to be measured. Therefore, it is desirable that the marker 100 be provided so that it does not easily move relative to the object. The marker 100 may take a variety of specific forms. For example, when the marker 100 is prepared as a separate component from the object, it can be attached to the object by gluing. When attaching the marker 100 by gluing, it is desirable to use an adhesive with high adhesive strength and durability, and for example, an aircraft structural adhesive can be suitably used. Specific examples include Henkel's LOCTITE® EA 9696 AERO, EA 7000 AERO, EA 9695 AERO, EA 9658 AERO, and EA 9895 WPP AERO, but other adhesives may also be used.

[0022] Furthermore, the method of attaching the marker 100 to the object is not limited to adhesive. The marker 100 may be embedded in a recess formed by recessing a portion of the surface of the object to match the outer diameter shape of the marker 100. In this case, an adhesive or the like may be filled around the recess and on the back side. Furthermore, the marker may not be a separate component, but may be directly formed on the surface of the object by painting or printing.

[0023] FIG. 2 is a diagram showing an example of the marker 100. Various known markers can be used as specific forms of the marker 100. The marker 100 of this embodiment is configured as a plate having a substantially square shape when viewed from the normal direction of the plate surface as shown in FIG. 2, and has a plurality of marks 111 arranged thereon. In this embodiment, the marker 100 has a substantially square shape of 100 mm x 100 mm when viewed from the front side, and circular marks 111 are arranged at intervals, one near each of the four corners of the marker 100. The size of the marker 100 may be changed as appropriate depending on the object and shooting distance.

[0024] It is desirable that at least three marks 111 are arranged discretely so that a plane can be identified. If there are two marks 111, or if there are three or more marks 111 arranged in a straight line, the plane cannot be identified, which will hinder coordinate identification, as described below. From the observation results of the marks 111, for example, from the size and relative position of the marks 111, the relative position, tilt, and attitude of the imaging unit 10 and the markers 100 can be accurately detected. Furthermore, if the number of marks 111 is more than three, for example, even if some of the marks 111 are obscured due to some obstacle, position detection is possible from the observation results of the remaining marks 111. Furthermore, by using multiple marks 111, the accuracy of position detection can be improved.

[0025] Furthermore, in this embodiment, the mark 111 is configured to have a circular shape, but is not limited to a circular shape and may be a polygonal shape such as a triangle or a rectangle, or may be another shape. From the viewpoint of improving detection accuracy, the shape of the mark 111 is preferably a circle or a regular polygon, and a circle is particularly preferable. The marker 100 is used to detect the relative positional relationship between the shooting position and the marker 100 (hereinafter also simply referred to as position detection) depending on how the mark 111 is observed.

[0026] When the marker 100 is a separate component from the target object, the marker 100 may be made of any material, but it is desirable to select the material of the marker 100 depending on the required detection accuracy. In particular, when high detection accuracy is required, it is advisable to use, for example, a glass plate as the substrate, which is less susceptible to shape deformation due to heat, etc. The marker 100 can be formed using methods with high positional accuracy, such as photolithography and inkjet. Photolithography is particularly desirable from the standpoint of accuracy. The mark 111 can be formed by arranging multiple layers of different colors formed from a resist material in a pattern on the substrate.

[0027] The marker 100 also has an identification mark 112 in the center surrounded by the marks 111. The identification mark 112 is a pattern graphic (graphic for identification) that is associated with a specific meaning and displays unique information by its pattern depending on the pattern of the mark. For example, each different pattern of the identification mark 112 is associated with a unique number (unique value) or alphabet. The identification mark 112 can be a two-dimensional barcode, a three-dimensional barcode, a QR code (registered trademark), ArUco, or the like. As described above, the identification mark 112 can be any of various known identification codes, but by using the identification mark 112 of this embodiment, which has a reduced number of patterns and a larger pattern, it can be easily detected by a camera.

[0028] FIG. 1 illustrates an example in which markers 100 are provided at four different locations on rocket 200. The identification marks 112 of these four markers each indicate a different number. Therefore, no matter what attitude rocket 200 is in, all four markers 100 can be individually identified. Here, the explanation will be given assuming that each marker 100 is attached to the head of a bolt (not shown) exposed to the outside of rocket 200. Note that markers 100 with different unique values ​​for the identification marks 112 are also provided on the back side of rocket 200, which is not visible in FIG. 1.

[0029] The imaging unit 10 is a camera that captures the rocket 200 together with the marker 100. The calculation unit 20 calculates information on the position and orientation of each of the plurality of markers 100 based on the image information captured by the imaging unit 10. The calculation unit 20 is configured, for example, by installing a computer program in a general-purpose computer. More specifically, the calculation unit 20 of the present embodiment is a computer device such as a personal computer or a smartphone, which is a general-purpose product, and has installed an application program for the measurement system of the present invention. Note that the calculation unit 20 may be a larger computer device or a dedicated device for the measurement system 1 of the present invention. The method for calculating the position and orientation of the mark 111 using the captured image of the mark 111 performed by the calculation unit 20 uses the method described in Hideyuki Tanaka, "Fundamentals and Latest Trends of AR Marker Technology," Transactions of the Institute of Electronics, Information and Communication Engineers, Vol. 97, No. 8, 2014, pp. 734-740. The direction of the mark 111, that is, the direction in which the front of the mark 111 (marker 100) faces (the orientation of the mark 111) can be represented, for example, by roll, yaw, and pitch.

[0030] Figure 3 is a flowchart showing the flow of the measurement operation by the measurement system 1 of the first embodiment. When the measurement by the measurement system 1 is started, in step (hereinafter referred to as S) 11, at time t1, the imaging unit 10 captures the rocket 200 together with each marker 100, and stores the captured image in a storage unit (not shown) as the captured image at time t1.

[0031] In S12, at time t2 (t1 < t2) when a predetermined time has elapsed from time t1 in S11, the imaging unit 10 captures the rocket 200 together with each marker 100, and stores the captured image in a storage unit (not shown) as the captured image at time t2. Note that the time between t1 and t2 is set to an appropriate time according to the object to be measured. In the present embodiment, since the imaging is performed during the takeoff of the rocket 200, for example, t2 - t1 = 0.025 seconds, and this may be continuously performed. Also, when the object is a structure such as a bridge that usually does not move, for example, t2 - t1 = 30 days or 360 days may be set.

[0032] In S13, the calculation unit 20 calculates information on the position and orientation of each of the multiple markers 100 at time t1 and time t2.

[0033] In S14, the calculation unit 20 calculates the change in position and attitude of each marker 100 on the rocket 200 between time t1 and time t2. Because the image capture unit 10 is fixed to the ground and the rocket 200 moves, information on the position and attitude of each marker 100 was obtained in S13 at times t1 and t2. Then, by comparing the positions and attitudes of the markers 100, if there is no relative (hereinafter simply referred to as "relative") change between times t1 and t2, it can be determined that none of the markers 100 has moved relatively. On the other hand, if, for example, three of the four markers 100 have not changed relative to each other in position or attitude, but only the remaining marker 100 has changed relative to each other, it can be determined that the remaining marker 100 is behaving in a manner other than movement or attitude change due to the movement of the rocket 200. In this embodiment, each marker 100 is attached to the head of a bolt. Therefore, it can be determined that there is a high possibility that the bolt on which the marker 100 whose position and posture have changed relatively is attached is loose.

[0034] In the above example, a simple example in which there are only four markers 100 has been described. However, it is also possible to provide more markers 100, such as 50 or 100. Even in such a case, by statistically processing the position and orientation of each marker, it is possible to identify multiple markers 100 whose positions and orientations have changed relatively.

[0035] As described above, the measurement system 1, measurement method, measurement program, and object (in this embodiment, rocket 200) equipped with marker 100 of this embodiment can easily and accurately measure changes in position and attitude at multiple locations on the same object.

[0036] (Second embodiment) FIG. 4 is a diagram showing a second embodiment of a measurement system according to the present disclosure, in which markers 100 and 110 are attached to a spacecraft 300. The second embodiment is similar to the first embodiment except that the target object is the spacecraft 300, a reference marker 110 is provided, and the calculation method and contents of the calculation program in the calculation unit 20 are partially different. Therefore, parts that perform the same functions as those in the first embodiment described above are given the same reference numerals, and duplicate explanations will be omitted as appropriate. Note that the image capture unit 10 and calculation unit 20 are not shown in FIG. 4.

[0037] In this embodiment, the object of measurement is a spacecraft 300. The spacecraft 300 can repeatedly take off and land, and legs (displaceable parts) 320 used during landing can be deployed from a main body 310. FIG. 4 shows a state in which the legs 320 are deployed. The main body 310 serves as a reference for the entire spacecraft 300.

[0038] In the example of FIG. 4, four legs 320 are provided, each with one marker 100. These markers 100 are similar to the markers 100 of the first embodiment. In the second embodiment, the spacecraft 300 is provided with a reference marker 110. The reference marker 110 itself is similar to the other markers 100. However, the reference marker 110 is provided in a different location from the other markers 100.

[0039] The markers 100 are provided on parts of the spacecraft 300, which is the object to be measured, that may move or deform. This is to measure changes in the position and attitude of these parts that may move or deform. In this embodiment, the markers 100 are provided on the legs 320, which move and deform. In the first embodiment, the markers 100 are provided on the heads of bolts that may move and be displaced (loosened).

[0040] On the one hand, the reference marker 110 is provided on the main body 310, which is a part of the spacecraft 300 that is less likely to be displaced than the part where the marker 100 is provided on the spacecraft 300. That is, the reference marker 110 is not provided on a movable part of the spacecraft 300, nor on the head of a bolt or the like, and is usually provided on an outer plate part that does not displace. Therefore, the reference marker 110 moves integrally with the spacecraft 300 unless an accident or the like occurs. Therefore, based on the position of this reference marker 110, changes in the position and orientation of other markers 100 can be determined more accurately. That is, in the measurement method of the previous first embodiment, there was a possibility that correct measurement could not be performed in an abnormal state where all the bolts provided with the markers 100 were loosened. However, by providing the reference marker 110, it becomes possible to perform correct measurement. Also, in this embodiment, the marker 100 is provided on the leg part 320, and all the leg parts 320 are normally displaced (deployed and stored). Even in such a second embodiment, it is possible to confirm by measurement that the leg part 320 is appropriately deployed and stored. The identification between the reference marker 110 and the normal marker 100 is performed by the identification mark 112.

[0041] Figure 5 is a flowchart showing the flow of the measurement operation by the measurement system 1 of the second embodiment. When the measurement is started, in S21, at time t1, the imaging unit 10 captures the spacecraft 300 together with each marker 100 and the reference marker 110, and stores the captured image as the captured image at time t1 in a storage unit (not shown).

[0042] In S22, at time t2 (t1 < t2) after a predetermined time has elapsed from time t1 in S21, the imaging unit 10 captures the spacecraft 300 together with each marker 100 and the reference marker 110, and stores the captured image as the captured image at time t2 in a storage unit (not shown).

[0043] In S23, the calculation unit 20 calculates the information on the position and orientation of each of the marker 100 and the reference marker 110 at each of time t1 and time t2.

[0044] In S23, the calculation unit 20 calculates the changes in the position and attitude of each marker 100 on the spacecraft 300 between time t1 and time t2, using the information on the position and attitude of the reference marker 110 as a reference. The position and attitude of the reference marker 110 can be regarded as the position and attitude of the spacecraft 300 itself. Therefore, the changes in the position and attitude of the other markers 100 relative to the spacecraft 300 can be correctly measured. Therefore, the changes in the position and attitude of the leg 320 relative to the spacecraft 300 can be correctly measured.

[0045] As described above, according to the measurement system 1, measurement method, measurement program, marker 100, and object (spacecraft 300 in this embodiment) provided with reference marker 110 of this embodiment, the reference marker 110 can be used as a reference, so that changes in position and attitude at multiple locations on the same object can be measured easily and more accurately than in the first embodiment.

[0046] (Variations) The present disclosure is not limited to the above-described embodiments, and various modifications and variations are possible, and these are also within the scope of the present disclosure.

[0047] (1) In each embodiment, the rocket 200 and the spaceship 300 have been described as examples. The measurement targets are not limited to these moving bodies, and may be, for example, bridges, buildings, or other objects that are basically stationary.

[0048] (2) In each embodiment, the photographing unit 10 is described as being installed on the ground away from the rocket 200 or spacecraft 300, which is the object of measurement. This is not limiting, and for example, the photographing unit 10 may be attached to the object of measurement itself. Also, for example, the photographing unit 10 may be provided on a moving body, such as an airplane, separate from the rocket 200. Even if the photographing unit 10 moves, as long as it can photograph the markers 100 and reference markers 110 on the object, which is the object of measurement, the relative relationship between the positions and attitudes of the markers 100 and reference markers 110 on the object can be properly measured.

[0049] (3) In each embodiment, the marker 100 and the reference marker 110 are described as having both the mark 111 and the identification mark 112. However, the present invention is not limited to this, and for example, a marker and a reference marker having only the identification mark 112 may be used. For example, if ArUco is used, although the accuracy is lower than when the mark 111 is also used, it is possible to measure the position and orientation of the marker and also to read the unique number.

[0050] (4) In each embodiment, the marker 100 and the reference marker 110 have a shape of 100 mm x 100 mm as viewed from the surface side, but if the purpose is to monitor the loosening of small bolts inside a spacecraft, for example, a smaller marker such as 10 mm x 10 mm can be used. Also, while a glass plate has been exemplified as the material for the marker 100 and the reference marker 110, titanium, which is less susceptible to shape deformation due to heat, etc., may also be used.

[0051] The embodiments and modifications may be used in combination as appropriate, but detailed description thereof will be omitted. The present disclosure is not limited to the embodiments described above. [Explanation of symbols]

[0052] 1. Measurement System 10. Photography Department 20 Arithmetic section 100 markers 110 fiducial markers 111 marks 112 Identification Mark 200 rockets 300 Spaceship 310 Main body 320 Leg (displaceable part)

Claims

1. Markers provided at multiple locations on the object; an imaging unit that images the marker; a calculation unit that calculates information on the position and orientation of each of the plurality of markers based on image information captured by the imaging unit; A measurement system comprising: the photographing unit photographs the marker at time t1 and time t2 (t1<t2), The calculation unit calculates information on the position and orientation of each of the markers at each of the times t1 and t2, and calculates changes in the position and orientation of each of the markers on the object between the time t1 and the time t2.

2. 2. The measurement system of claim 1, At least one of the plurality of markers is provided at a portion of the object that is less likely to be displaced than portions at which the other markers are provided; A measurement system characterized by:

3. 3. The measurement system according to claim 2, the calculation unit calculates changes in the positions and orientations of the other markers based on the marker provided at a portion of the object that is difficult to displace; A measurement system characterized by:

4. The measurement system according to claim 2 or 3, The marker provided at a portion of the object that is difficult to displace is provided on the object in any one of the following forms: embedded, painted, printed, or adhered; A measurement system characterized by:

5. Markers provided at multiple locations on the object; an imaging unit that images the marker; a calculation unit that calculates information on the position and orientation of each of the plurality of markers based on image information captured by the imaging unit; A measurement method in a measurement system comprising: a step in which the photographing unit photographs the marker at time t1 and time t2 (t1<t2); a step in which the calculation unit calculates position and orientation information of each of the markers at each of the time t1 and the time t2; a step in which the calculation unit calculates changes in position and orientation of each of the markers on the object between the time t1 and the time t2; A measurement method in a measurement system comprising:

6. Markers provided at multiple locations on the object; an imaging unit that images the marker; a calculation unit that calculates information on the position and orientation of each of the plurality of markers based on image information captured by the imaging unit; A measurement program for use in a measurement system comprising: On the computer, a step in which the photographing unit photographs the marker at time t1 and time t2 (t1<t2); a step in which the calculation unit calculates position and orientation information of each of the markers at each of the time t1 and the time t2; a step in which the calculation unit calculates changes in position and orientation of each of the markers on the object between the time t1 and the time t2; A measurement program for executing the above.

7. a main body that serves as the reference for the entire object; a displaceable portion that is displaceable or likely to be displaced relative to the main body portion; and a marker provided on the main body; a marker provided on the displaceable portion; An object having:

Images