Calibration method for motion capture system for moving object

The method addresses the calibration challenges of motion capture systems on moving construction machines by aligning coordinate systems and converting camera coordinates, enabling precise capture and display of object positions and movements.

JP2024156335APending Publication Date: 2024-11-06SHIMIZU CORP +4
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023070709
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-24
Publication Date
2024-11-06

AI Technical Summary

Technical Problem

Existing motion capture systems struggle to accurately capture the position and movement of elements attached to the tip of a moving construction machine arm due to difficulties in calibrating the camera coordinate system with the mechanical system, leading to imprecise real-time measurement of construction processes.

Method used

A method for calibrating a motion capture system with multiple cameras on a moving object involves setting the relationship between mechanical and calibration coordinate systems, aligning axis directions, and converting camera coordinates to calibration coordinates, using a total station to track the position and posture of the moving object.

Benefits of technology

This method enables accurate determination and display of the position and movement of objects captured by the motion capture system in a calibration coordinate system, enhancing precision in construction tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156335000001_ABST
    Figure 2024156335000001_ABST
Patent Text Reader

Abstract

To provide a calibration method for a motion capture system for a moving object and a motion capture system utilizing the same.SOLUTION: Positions and angles of multiple cameras installed on a moving object are determined, then the X-axis, Y-axis, and Z-axis of the camera coordinate system are set so that their directions coincide with those of the X-axis, Y-axis, and Z-axis of the calibration coordinate system, and the relationship between an origin of the calibration coordinate system and an origin of the camera coordinate system is obtained to set the camera coordinate system. By measuring the position and orientation of the moving object, the calibration coordinates of the object captured by a motion capture system can be determined.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a calibration method for motion capture for a moving object, and a motion capture system utilizing the same. [Background technology]

[0002] For example, construction machinery is typically equipped with various arms (which may be extendable booms) for construction work, and works are performed via operating parts such as arms attached to the ends of the arms.

[0003] Construction machinery that can be brought into the tunnel includes so-called tunnel jumbos, which are used to drill holes in the tunnel face, spraying machines, and steel support erection equipment.

[0004] When spraying concrete using a sprayer, there is no way to measure the finished shape in real time, so workers spray the concrete by eye. There is a demand for technology that can automatically spray the concrete while measuring the finished shape in real time.

[0005] Patent Document 1 relates to a method and system for erecting shoring, and relates to a technology that can grasp the position of the shoring in real time, in which motion capture markers installed at multiple predetermined fixed points on the steel shoring are photographed with multiple motion capture cameras, the three-dimensional coordinates of each motion capture marker are obtained based on the images captured by each motion capture camera and the three-dimensional coordinates of each motion capture camera, and the steel shoring is erected in the designed position based on the obtained three-dimensional coordinates of each motion capture marker. Here, the three-dimensional coordinates of the motion capture marker are obtained by measuring the position of the target installed on the erector device using a total station, and calculating the three-dimensional coordinates of the motion capture camera based on the relative positional relationship between the motion capture camera installed at a predetermined position and the target set at a predetermined position. In this invention, the camera coordinate system of the object is specified and not converted to a stationary coordinate system in order to obtain the three-dimensional coordinates of the object. Also, the camera is installed in a camera holder on the erector device, but it is difficult to install it accurately at the expected three-dimensional X, Y, Z coordinate position, so it is difficult to obtain the three-dimensional coordinates of the object captured by the motion capture camera with high accuracy. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-173393 A Summary of the Invention [Problem to be solved by the invention]

[0007] Such motion capture technology, including motion capture cameras, is useful. Based on this background technology, the present inventors have attempted various developments in search of a technology for automatically performing spraying while measuring the finished shape in real time in spraying work. In other words, we attempted a demonstration experiment based on the idea that by attaching a reflector near the sprayed concrete nozzle or boom and continuously photographing it with a motion capture camera and analyzing the images, it might be possible to calculate the position and movement of the spray nozzle in real time.

[0008] During this process, it was discovered that when performing construction work while moving the arm of the construction machine and the element attached to the tip of the arm, in order to accurately capture the position and movement of the element, it is necessary to calibrate the motion capture system with high precision.

[0009] The main object of the present invention is to provide a highly accurate calibration method for a motion capture system used, for example, when performing construction work while moving an arm of a construction machine and an element attached to the tip of the arm, to accurately capture the position and movement of the element so as to enable the execution of required operations, and also to provide a method for displaying an object captured by a camera of the motion capture system in a calibration coordinate system. [Means for solving the problem]

[0010] A first aspect of the means for solving the above problems is as follows. A method for calibrating a motion capture system equipped with multiple cameras on a moving object, comprising the steps of: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; A step of matching the X-axis, Y-axis, and Z-axis directions of the camera coordinate system and the machine coordinate system with the X-axis, Y-axis, and Z-axis directions of a calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A method for calibrating a motion capture system, comprising:

[0011] The second aspect is as follows. A method for displaying an object captured by a camera of a motion capture system equipped with multiple cameras on a moving object in a calibration coordinate system, comprising: A step of setting the relationship between the origin of the mechanical coordinate system of the moving body and the origin of the calibration coordinate system. determining an arrangement of the plurality of cameras; A step of matching the X-axis, Y-axis, and Z-axis directions of the camera coordinate system and the machine coordinate system with the X-axis, Y-axis, and Z-axis directions of a calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A step of calculating the degree of rotation of the camera coordinate system based on changes in the position and orientation of the moving object; A step of converting coordinates of an object captured by the motion camera in a camera coordinate system into coordinates in a calibration coordinate system; A display method comprising:

[0012] The third aspect is as follows. A method for calibrating a motion capture system equipped with multiple cameras on a moving object, comprising the steps of: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; a step of aligning the X-axis direction, the Y-axis direction and the Z-axis direction of the camera coordinate system with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively, in a state where the machine body axial direction is aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the machine coordinate system are aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A method for calibrating a motion capture system, comprising:

[0013] The fourth aspect is as follows. A method for displaying an object captured by a camera of a motion capture system equipped with multiple cameras on a moving object in a calibration coordinate system, comprising: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; a step of aligning the X-axis direction, the Y-axis direction and the Z-axis direction of the camera coordinate system with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively, in a state where the machine body axial direction is aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the machine coordinate system are aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A step of calculating the degree of rotation of the camera coordinate system based on changes in the position and orientation of the moving object; A step of converting coordinates of an object captured by a motion capture camera in a camera coordinate system into coordinates in a calibration coordinate system; A display method comprising: Effect of the Invention

[0014] As described above, according to the present invention, it is possible to provide a calibration method that can accurately capture the position, posture, and movement of an object captured by a camera in a motion capture system using multiple cameras provided on a moving body, for example, a construction machine, etc. Also, it is possible to display the object captured by the camera of the motion capture system in a calibration coordinate system. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 is an explanatory diagram of the entire calibration. [Diagram 2] FIG. 1 is an explanatory diagram of camera calibration. [Diagram 3] This is the relationship between the coordinate systems during calibration of the moving object and when it moves. [Figure 4] FIG. 2 is a schematic diagram for explaining the operation of a construction machine installed in a tunnel. [Diagram 5] (a) is a plan view and (b) is a side view of the boom section equipped with a motion capture target. [Figure 6] 1 is an explanatory diagram of an example of a camera installation state and each field of view. [Figure 7] FIG. 1 is an explanatory diagram of a camera, where (a) is a front view and (b) is a perspective view. [Figure 8] FIG. 2 is a front view showing an example of a motion capture target. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with respect to a method for calibrating a motion capture system when a camera is mounted on a construction machine.

[0017] An outline of the embodiment is shown in FIG. (First embodiment) That is, a machine body 10, three prisms 19 provided on the machine body 10, a camera group 11G to which seven cameras 11 are fixed, a total station TS, and three LED markers 60A, 60B, and 60C used for system calibration, which are attached to tripods so as to form an L-shape in a plan view within the camera field of view, are arranged. Two or more cameras are sufficient, but in this embodiment, seven cameras are used.

[0018] Detailed examples of these are as follows: The calibration coordinate system is a coordinate system set in the total station TS. The camera coordinate system is a coordinate system set in the motion capture camera, and is determined as described below. The machine coordinate system is a coordinate system set in the construction machine, and the position of the origin is associated with the positions of the three prisms placed on the moving body, and as described below, the directions of the X-axis, Y-axis, and Z-axis of the machine coordinate system are set to match the directions of the X-axis, Y-axis, and Z-axis of the camera coordinate system, respectively.

[0019] (A) First, the three prisms 19 mounted on the machine body 10 are measured by the total station TS to obtain the positional relationship of the three prisms in the calibration coordinate system. This makes it possible to set the relationship between the origin of the machine coordinate system, whose positional relationship with the three prisms on the machine body is known, and the origin of the calibration coordinate system. The origin of the machine coordinate system may be set, for example, at the center of a base plate (not shown) at the base of the spray device.

[0020] (a) Next, it is determined how the seven cameras 11 are attached to the frame of the camera group. For this purpose, for example, a T-shaped jig 50 as shown in Fig. 2 is used. This jig 50 has self-luminous LED markers 51, 52, 53 installed at positions where there is a difference in the distance between the left and right sides, avoiding both ends and the vicinity of the center (the installation positions and the distance between them are known). Within the camera field of view, a worker swings jig 50 around (the support rod part to which LED markers 51, 52, and 53 are attached is swung like a metronome within the swinging plane, and the support rod part of jig 50 is made to move back and forth across its longitudinal plane like the rolling trajectory of a wheel, etc.), and during this process, image information is captured by each camera, and based on the image information from each camera, the difference in the position of the individual cameras is calculated backwards to determine the position and angle of each camera in camera group 11G.

[0021] (c) Next, to set the camera coordinate system, three LED markers 60A, 60B, and 60C are attached to tripods within the field of view of the camera and are arranged horizontally in an L-shape in plan view, as shown in Figure 1. At this time, the longitudinal direction of the L-shape arrangement (the direction connecting LED markers 60B and 60C) is aligned with the X-axis direction of the calibration coordinate system using a total station TS. In addition, the perpendicular direction of the L-shape arrangement (the direction connecting LED markers 60B and 60A) is aligned with the Y-axis direction of the calibration coordinate system using a total station TS. The Z-axis direction is the vertical direction of the L-shape arrangement. The corner of the L shape (LED marker 60B in Figure 1) is the origin (0,0,0) of the camera coordinate system, the longitudinal direction (the direction connecting LED markers 60B and 60C) is the X-axis of the camera coordinate system, and the perpendicular direction is the Y-axis of the camera coordinate system. Furthermore, the X-axis, Y-axis, and Z-axis directions of the machine coordinate system are set to coincide with the X-axis, Y-axis, and Z-axis directions of the camera coordinate system, respectively. This allows the roll, pitch, and yawing of the machine coordinate system to coincide with the roll, pitch, and yawing of the camera coordinate system.

[0022] (d) The LED marker 60B at the corner of the L shape, which is the origin of the camera coordinate system, is measured by the total station TS, and the relationship in the calibration coordinate system between the origin of the camera coordinate system and the origin of the machine coordinate system measured in (a) is recorded. Since the position of the origin of the machine coordinate system can be tracked by the total station TS, the coordinate in the calibration coordinate system of the origin of the camera coordinate system at any point in time can be calculated.

[0023] (E) When the construction machinery moves and the X-axis, Y-axis, and Z-axis directions of the machine coordinate system and the calibration coordinate system no longer match, the roll, pitch, and yaw of the machine coordinate system relative to the calibration coordinate system can be obtained from the positional relationship of the three prisms in the calibration coordinate system obtained in (A) and the measurement results of the three prisms by the total station TS at said arbitrary point in time.

[0024] As a result, by following the procedure described above, it is possible to convert the coordinates of the camera coordinate system calculated when the motion capture camera views an object at any point while the construction machine is moving into the calibration coordinate system. In other words, the position of the object in the calibration coordinate system can be determined by rotating the coordinate system based on the roll, pitch, and yaw of the camera coordinate system relative to the calibration coordinate system (which corresponds to the roll, pitch, and yaw of the machine coordinate system relative to the calibration coordinate system), and then transforming the origin of the camera coordinate system to the origin in the calibration coordinate system. The roll, pitch, and yaw relative to the calibration coordinates of the camera coordinate system can be calculated by using the positions of the three prisms measured by the total station TS as the reference (a) (shown in the upper part of Figure 3) and measuring the positions of the three prisms with the total station TS at any time (shown in the lower part of Figure 3).

[0025] That is, by performing the above-mentioned motion capture calibration, the position of the object captured by the camera in the calibration coordinate system can be identified by combining the roll, pitch, and yaw relative to the calibration coordinates of the camera coordinate system, which are determined from the positions of the three prisms measured by the total station TS at any point in time, with the position of the object captured by the camera in the camera coordinates. As a result, it becomes possible to display the position information captured by the motion capture camera on a moving object in the calibration coordinate system.

[0026] (Second embodiment) (A) First, the three prisms 19 mounted on the machine body 10 are measured by the total station TS to obtain the relationship between the position of the origin of the machine coordinate system and the coordinates in the calibration coordinate system, and the positional relationship of the three prisms in the calibration coordinate system. This makes it possible to set the relationship between the origin of the machine coordinate system, whose positional relationship with the three prisms on the machine body is known, and the origin of the calibration coordinate system. The origin of the machine coordinate system may be set, for example, at the center of a base plate (not shown) at the base of the spraying device. At this time, the machine main body is adjusted so that the body is horizontal, and as described below, the axial direction of the body is aligned with the longitudinal direction of the L-shaped arrangement (the direction connecting LED markers 60B and 60C).

[0027] (a) Next, it is determined how the seven cameras 11 are attached to the frame of the camera group. For this purpose, for example, a T-shaped jig 50 as shown in Fig. 2 is used. This jig 50 has self-luminous LED markers 51, 52, 53 installed at positions where there is a difference in the distance between the left and right sides, avoiding both ends and the vicinity of the center (the installation positions and the distance between them are known). Within the camera field of view, a worker swings jig 50 around (the support rod part to which LED markers 51, 52, and 53 are attached is swung like a metronome within the swinging plane, and the support rod part of jig 50 is made to cross back and forth across its longitudinal plane, giving it a movement resembling the rolling trajectory of a wheel, etc.), and during this process, image information is captured by each camera, and based on the image information from each camera, the difference in the position of the individual cameras is calculated backwards to determine the position and angle of each camera in camera group 11G.

[0028] (c) Next, to set the camera coordinate system, three LED markers 60A, 60B, and 60C are mounted on tripods and placed horizontally in an L-shaped arrangement in a plan view within the camera's field of view, as shown in Figure 1. At this time, the longitudinal direction of the L-shaped arrangement (the direction connecting LED markers 60B and 60C) is aligned with the X-axis direction of the calibration coordinate system using a total station TS. The axial direction of the machine body is also aligned with the longitudinal direction of the L-shaped arrangement. In addition, the perpendicular direction of the L-shaped arrangement (the direction connecting LED markers 60B and 60A) is aligned with the Y-axis direction of the calibration coordinate system using a total station TS. The Z-axis direction is the vertical direction of the L-shaped arrangement. The corner of the L shape (LED marker 60B in Figure 1) is the origin (0,0,0) of the camera coordinate system, the longitudinal direction (the direction connecting LED markers 60B and 60C) is the X-axis of the camera coordinate system, and the perpendicular direction is the Y-axis of the camera coordinate system. As shown in Fig. 1, two prisms 19 attached to the sides of the machine body so as to be parallel to the body axis are observed by a total station TS, so that the body axis direction coincides with the longitudinal direction of the L-shaped arrangement (the direction connecting LED markers 60B and 60C). Because the machine body is adjusted so as to be horizontal, this causes the X-axis direction (body axis direction), Y-axis direction, and Z-axis direction (vertical direction) of the machine coordinate system to coincide with the X-axis direction, Y-axis direction, and Z-axis direction of the calibration coordinate system, respectively, and further coincide with the X-axis direction, Y-axis direction, and Z-axis direction of the camera coordinate system. This allows the roll, pitch, and yawing of the machine coordinate system to coincide with the roll, pitch, and yawing of the camera coordinate system.

[0029] In this second embodiment, the X-axis direction of the machine coordinate system coincides with the machine body axle, and the Z-axis coincides with the vertical upward direction with respect to the machine body. Therefore, when the origin of the machine coordinate system set on the machine body is displayed in the calibration coordinate system, it is easy to calculate. The machine body axle coincides with the direction in which the machine moves straight, and may be set to pass through the center of the base plate, for example. When acquiring the position of each component installed on the machine body, if the positional relationship between each component and a target point (motion capture target) on the machine body is acquired in advance, the position of each component can be easily obtained as a coordinate in the camera coordinate system by measuring the target point by motion capture.

[0030] (d) The LED marker 60B at the corner of the L shape, which is the origin of the camera coordinate system, is measured by the total station TS, and the relationship in the calibration coordinate system between the origin of the camera coordinate system and the origin of the machine coordinate system measured in (a) is recorded. Since the position of the origin of the machine coordinate system can be tracked by the total station TS, the coordinate in the calibration coordinate system of the origin of the camera coordinate system at any point in time can be calculated.

[0031] (E) When the construction machinery moves and the X-axis, Y-axis, and Z-axis directions of the machine coordinate system and the calibration coordinate system no longer match, the roll, pitch, and yaw of the machine coordinate system relative to the calibration coordinate system can be obtained from the positional relationship of the three prisms in the calibration coordinate system obtained in (A) and the measurement results of the three prisms by the total station TS at said arbitrary point in time.

[0032] As a result, by following the procedure described above, it is possible to convert the coordinates of the camera coordinate system calculated when the motion capture camera views an object at any point while the construction machine is moving into the calibration coordinate system. In other words, the position of the object in the calibration coordinate system can be determined by rotating the coordinate system based on the roll, pitch, and yaw of the camera coordinate system relative to the calibration coordinate system (which corresponds to the roll, pitch, and yaw of the machine coordinate system relative to the calibration coordinate system), and then transforming the origin of the camera coordinate system to the origin in the calibration coordinate system. The roll, pitch, and yaw relative to the calibration coordinates of the camera coordinate system can be calculated by using the positions of the three prisms measured by the total station TS as the reference (a) (shown in the upper part of Figure 3) and measuring the positions of the three prisms with the total station TS at any time (shown in the lower part of Figure 3).

[0033] That is, by performing the above-mentioned motion capture calibration, the position of the object captured by the camera in the calibration coordinate system can be specified by combining the roll, pitch, and yaw relative to the calibration coordinates of the camera coordinate system, which are obtained from the positions of the three prisms measured by the total station TS at any point in time, and the position of the object captured by the camera in the camera coordinates. As a result, it becomes possible to display the position information captured by the motion capture camera on a moving body in the calibration coordinate system.

[0034] Furthermore, according to the calibration of the second aspect, the X-axis of the machine coordinate system and the camera coordinate system coincides with the axle of the machine body, and the Z-axis coincides with the vertical upward direction relative to the body. Therefore, when acquiring the position of each component installed on the machine body, if the positional relationship between each component and a target point (motion capture target) on the machine body is acquired in advance, the position of each component can be advantageously easily obtained as a coordinate in the camera coordinate system by measuring the target point by motion capture.

[0035] Next, an example using motion capture according to the calibration method of the present invention will be described. The machine body 10 located inside the tunnel T shown in Figures 4 and 5 is a spraying device that sprays concrete against the interior surface of a tunnel using, for example, the NATM construction method, and its spraying boom 22 is attached to a support 16 that rotates horizontally around a vertical axis 15 at the front end of the machine body 10, and is adapted to swing up and down by a hoisting cylinder 17. It has a spray boom 22 (FIG. 5) in the center, a spray nozzle 26, a support 25 for holding the spray nozzle 26, and a radar range finder 27 fixed to the support 25 as required. The three-dimensional position and direction of the spray nozzle 26 can be moved in three dimensions. The three-dimensional position and direction of the spray nozzle 26 can be controlled by accurately capturing the position of the spray nozzle 26 and the movement of the moving part 20 including the boom 22.

[0036] In the above example, the spray nozzle 26 and the spray boom 22, as well as the various moving elements described above, are mechanically mounted on the machine body 10 and constitute the machinery for construction, and the connecting parts belong to the movable parts that move in pairs and are components of the moving part 20. A support 25 for the spray nozzle 26 (the support 25 in this embodiment includes a support seat plate for a radar range finder 27 in addition to a holding frame as shown in Figure 5, and holds a passage for the spray material that communicates with the spray nozzle 26) is installed.

[0037] At least two cameras 11 are fixed to the machine body 10 facing the motion capture target 100, and in this embodiment, as shown in Figure 6, for example, six cameras 11 are fixed with different optical axis angles to form a camera group 11G. As shown in FIG. 1, the camera group 11G is fixed to the front portion of the machine body 10.

[0038] An example of the camera 11 is shown in Figure 7. It is preferable that the camera 11 has a wide-angle lens. In view of the dusty environment inside the tunnel T, it is preferable to use a camera 11 having multiple LED light emitters 11A around it and covered with a dust-proof hood 11B so that the motion capture target 100 can be clearly captured.

[0039] Information on the current position of the machine body 10 of the construction machine within the tunnel can be obtained using, for example, a total station TS installed within the tunnel T, the position of which is known by surveying from outside the tunnel. By capturing, for example, a general-purpose prism target 19 mounted at an appropriate position on the machine body 10 using a total station TS whose position is known, the position of the target 19 can be obtained based on surveying, and at the same time, the position of the machine body 10 within the tunnel can be set which can be related to the position of the target 19.

[0040] Based on the position of the machine body 10 inside the tunnel, the position of the spray nozzle 26 can be grasped from motion data such as the current position and angle from an extension distance detector, a rotation angle detector, etc. (not shown as they would be obvious to a person skilled in the art) provided on the moving part 20. The three-dimensional position and direction of the spray nozzle 26 can be moved in three dimensions. In addition, data on the current distance to the inner surface of the tunnel T (the current distance to the concrete surface if concrete has been sprayed) can be obtained from a signal from a radar range finder 27 (see Figure 5) fixed to the support body 25.

[0041] Now, due to factors thought to be influenced by the length of the boom 22, a phenomenon occurs in which the tip of the boom 22 twists (rotates) around its axis.

[0042] When these rotations occur, even if the position of the tip of the spray nozzle 26 remains the same, the direction in which the opening at the tip of the spray nozzle 26 faces will shift from the desired direction, making it difficult to spray the desired wall position with the desired spray thickness. A target consisting of one marker is sufficient to obtain the position of the spray nozzle 26, but in order to detect these rotations, it is necessary to provide a motion capture target 100 having at least three self-luminous parts, as shown in Figure 8 as an example, and obtain the roll, pitch, and yaw of the boom 22. By only determining the XYZ coordinates of one marker 21, the roll, pitch and yaw are unknown.

[0043] The tunnel motion capture target 100 of the embodiment is connected to the machine body 10 of a concrete sprayer and is provided at the tip of an extended spraying boom 22. This motion capture target 100 is provided to capture the position and movement of the spray nozzle 26 when the spray nozzle 26 connected to the boom 22 is moved and concrete is sprayed from the spray nozzle 26 onto the inner wall surface of the tunnel T.

[0044] The motion capture target 100 of the embodiment has an attachment portion 111 for mounting to the boom 22, a plate-shaped substrate portion 110 joined to the attachment portion 111, and at least four LED portions 121, 122, 123, 124 attached to one surface of the substrate portion 110. The substrate portion 110 may be integrated with the attachment portion 111 so as to be perpendicular to the attachment portion 111 or so as to be inclined slightly upward toward the tip, for example, at an angle ranging from more than 90 degrees to less than 80 degrees.

[0045] The LED units 121 to 124 each have a self-emitting LED and a spherical cover that radiates the light emitted by the LED isotropically. The LED preferably uses near-infrared light (for example, a peak wavelength of 850 nm).

[0046] It is preferable that the diameter of the LED part is 5 to 20 mm. By setting it in this range, capture is ensured, and the accuracy of the position of the center point O can be improved. If the diameter is smaller than 5 mm, there is a high probability that it will be lost due to dust, and there is a possibility that the three LED parts will not be captured. If the diameter is larger than 20 mm, the accuracy of the position of the center of the LED part will deteriorate, and the accuracy of the position of the center point O will also deteriorate.

[0047] It is desirable that all four LED units 121-124 are disposed at equal distances R (see FIG. 8) from the center point O of the surface of the substrate unit 110. The distance R is preferably 60-80 mm. By setting the distance R within this range, the accuracy of the position of the center point O can be improved. If the distance is less than 60 mm or more than 80 mm, the accuracy of the calculated position of the center point O deteriorates. By setting the diameter of the LED portion to 5 to 20 mm and the distance R to 60 to 80 mm, the nozzle position can be calculated with an accuracy of 10 mm or less.

[0048] It is desirable to arrange the LED units 121 to 124 so that the angles α, β, γ, and δ between the adjacent LED units 121 to 124 are different. For example, the angles α, β, γ, and δ between adjacent LED units on the surface of the substrate unit 110 are set to 90 degrees, 50 degrees, 72 degrees, and 148 degrees. It has been confirmed that using these angles can improve the probability of calculating the center point O in real time with high accuracy.

[0049] If the number of LED parts is three or more, it is possible to define the center point O. However, it is preferable that the number is four or more in order to be able to define the center point O even in cases where some of the LED parts cannot be detected due to the influence of dust in the tunnel, or some of the LED parts have stopped emitting light.

[0050] The positions of the LED units 121, 122, 123, and 124 on the board unit 110 are known, and are pre-stored in the motion capture software with numbers assigned thereto for distinction.

[0051] Then, the stereo consecutive images seen from the camera side are analyzed to calculate the X, Y and Z coordinates of the center point O of the LED part, as well as the rotation (roll, pitch and yaw). The three-dimensional coordinates of the center point O are calculated based on the three-dimensional coordinates of each of the three LED parts.

[0052] By performing the above operations, the rotation, i.e., roll, pitch, and yaw, of the motion capture target 100 from its initial position is obtained, and the degree of twisting at the boom tip position is grasped, making it possible to obtain three-dimensional position and directional data of the spray nozzle 26 based on the position and attitude of the machine body 10 captured by the total station TS. [Industrial Applicability]

[0053] The calibration method for the motion capture system of the present invention can be applied to cases where high-precision construction is carried out using a motion capture system in general construction machinery used in the NATM method, such as concrete spraying machines (equipment) located inside tunnels for construction, as well as machines for installing shoring and tunnel drill jumbos. [Explanation of symbols]

[0054] 10: Machine body of concrete sprayer (construction machine for NATM method), 11: Motion capture camera, 19: Prism, 60A, 60B, 60C: LED marker, 50: Jig

Claims

1. A method for calibrating a motion capture system equipped with multiple cameras on a moving object, comprising the steps of: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; aligning the X-axis, Y-axis, and Z-axis directions of the camera coordinate system and the machine coordinate system with the X-axis, Y-axis, and Z-axis directions of a calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A method for calibrating a motion capture system, comprising:

2. A method for displaying an object captured by a camera of a motion capture system equipped with multiple cameras on a moving object in a calibration coordinate system, comprising: A step of setting the relationship between the origin of the mechanical coordinate system of the moving body and the origin of the calibration coordinate system. determining an arrangement of the plurality of cameras; aligning the X-axis, Y-axis, and Z-axis directions of the camera coordinate system and the machine coordinate system with the X-axis, Y-axis, and Z-axis directions of a calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A step of calculating the degree of rotation of the camera coordinate system based on changes in the position and orientation of the moving object; A step of converting coordinates of an object captured by a motion capture camera in a camera coordinate system into coordinates in a calibration coordinate system; A display method comprising:

3. A method for calibrating a motion capture system equipped with multiple cameras on a moving object, comprising the steps of: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; a step of aligning the X-axis direction, the Y-axis direction and the Z-axis direction of the camera coordinate system with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively, in a state where the machine body axial direction is aligned with the X-axis direction of the machine coordinate system and the X-axis direction, the Y-axis direction and the Z-axis direction of the machine coordinate system are aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A method for calibrating a motion capture system, comprising:

4. A method for displaying an object captured by a camera of a motion capture system equipped with multiple cameras on a moving object in a calibration coordinate system, comprising: setting a relationship between an origin of a mechanical coordinate system of the moving body and an origin of a calibration coordinate system; determining an arrangement of the plurality of cameras; a step of aligning the X-axis direction, the Y-axis direction and the Z-axis direction of the camera coordinate system with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively, in a state where the machine body axial direction is aligned with the X-axis direction of the machine coordinate system and the X-axis direction, the Y-axis direction and the Z-axis direction of the machine coordinate system are aligned with the X-axis direction, the Y-axis direction and the Z-axis direction of the calibration coordinate system, respectively; Setting the relationship between the origin of the camera coordinate system and the origin of the calibration coordinate system; A step of calculating the degree of rotation of the camera coordinate system based on changes in the position and orientation of the moving object; A step of converting coordinates of an object captured by a motion capture camera in a camera coordinate system into coordinates in a calibration coordinate system; A display method comprising:

5. 4. The method for calibrating a motion capture system according to claim 1 or 3, wherein the moving object is a construction machine.

6. 5. The method for displaying an object captured by a camera of a motion capture system in a calibration coordinate system according to claim 2 or 4, wherein the moving object is a construction machine.

Citation Information

Patent Citations

  • Ambient display system and work machines equipped with the same

    JP2020147962A

  • Timbering construction system

    JP2023037188A

  • Steel support work building method and building system

    JP2019173393A