Coordinate alignment system for range image sensors and other measurement systems

Abstract

To enable coordinate system matching between a motion analysis system using a ranging image sensor and another measurement system (such as a pressure distribution sheet).SOLUTION: A ranging jig designed to be used with a ranging image sensor using infrared rays is provided, the jig having a target surface consisting of a first reflective surface 6 in the center and a second reflective surface 7 provided to surround the first reflective surface 6 and made of an infrared reflective material. The center of the target surface acts as the ranging target.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to coordinate alignment between a motion measurement system using a distance image sensor and another measurement system. [Background technology]

[0002] In the fields of sports and rehabilitation, three-dimensional motion analysis using motion capture is performed to record and analyze the movements of subjects. Conventional motion capture technologies include optical motion capture and motion capture using inertial sensors. However, in recent years, so-called markerless motion capture, which does not use markers or inertial sensors, has emerged. For example, a markerless motion capture technique uses infrared or LiDAR (Light Detection and Ranging) to acquire a single distance image (depth image) using a time-of-flight (TOF) method, and then estimates joint positions from the acquired depth image (Non-Patent Document 1). Furthermore, skeletal information estimation based on distance images is also described in, for example, Patent Documents 1 to 3. In recent years, relatively inexpensive motion capture devices using distance imaging have appeared on the market (Non-Patent Document 2), and their use in clinical settings is expected (Non-Patent Document 3).

[0003] Meanwhile, measurement systems using force plates and pressure distribution sensors, which are known as measurement systems for analyzing the gait of subjects, are known (Patent Documents 4 and 5). It would be useful to synchronize measurements between a motion analysis system using a distance image sensor and another system (e.g., a pressure distribution sensor or force plate) and use the information from each other to gain a deeper understanding of the subject's movements. For example, in the case of rehabilitation, a comprehensive understanding of the subject's movements using a distance image sensor and other devices can provide useful information for determining an appropriate treatment plan.

[0004] However, ordinary distance image sensors do not have a means for aligning coordinates with other devices, and therefore cannot perform measurements in the same coordinate system. Patent Document 6 is an example of prior art for a function for aligning a motion analysis device with other external devices. This method can be used not only with force plates but also with other devices. However, for the following reasons, this method cannot be applied to motion measurement systems that use infrared distance image sensors.

[0005] First, in a range image sensor, it is impossible to obtain the distance from an infrared reflective marker, and therefore the position of the marker cannot be identified. The inventors conducted experiments and found that when attempting to obtain the distance from an infrared reflective material using a range image sensor, it is not possible to calculate an appropriate distance. Second, since range image sensors do not use the DLT method (adopted in Patent Document 6), the calculation formula used in Patent Document 6 cannot be used. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6617830 [Patent Document 2] Patent No. 5211069 [Patent Document 3] Patent No. 6433149 [Patent Document 4] Patent Publication No. 2001-29329 [Patent Document 5] Patent Publication No. 2006-284404 [Patent Document 6] Patent Publication No. 2013-162911 [Non-patent literature]

[0007] [Non-Patent Document 1] J. Shotton et al., “Real-Time Human Pose Recognition in Parts from a Single Depth Image,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), IEEE CS Press, 2011, pp. 1297-1304. [Non-patent document 2] Z. Zhang. Microsoft kinect sensor and its effect. IEEE MultiMedia, 19(2):4-10, Feb 2012. [Non-patent document 3] Application of 3D motion analysis using markerless motion capture Koichi Haruna, Keisuke Kon, Jun Inagaki, Yoichiro Sato Journal of the Japanese Society of Prosthetics and Orthotics, Vol. 35, No. 1, pp. 17-23, 2019 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0008] The present invention aims to solve the above-mentioned problems when realizing coordinate alignment between a motion analysis system using a distance image sensor and another measurement system (such as a pressure distribution sheet). [Means for solving the problem]

[0009] The first technical means adopted by the present invention is: a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface; the second reflecting surface is formed from an infrared reflecting material; It is a distance measuring tool used in conjunction with an infrared distance image sensor. In one embodiment, the target surface and the first reflecting surface are concentric circles, and the second reflecting surface is ring-shaped. In one embodiment, the orientation of the target surface is variable.

[0010] The second technical means adopted by the present invention is: A distance measurement method using a camera, a distance image sensor using infrared rays, and a distance measurement jig, The distance measuring jig is a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface; the second reflecting surface is formed from an infrared reflecting material; detecting an area of ​​the target surface by image processing based on the image acquired by the camera, and detecting a specific point on the first reflecting surface; acquiring a distance to the specific point on the first reflecting surface using infrared rays emitted from the distance image sensor; A ranging method including: In one embodiment, the image processing comprises: acquiring a first image including a distance measuring jig installed at a predetermined position using a camera whose position and orientation are fixed; acquiring a second image with the camera without the distance measuring jig installed; detecting a region of the target surface based on a difference in brightness between the first image and the second image, and detecting a specific point on the first reflecting surface; It consists of: In one embodiment, the particular point is the center of the target surface. In one embodiment, the camera is an infrared camera.

[0011] The third technical means adopted by the present invention is to obtain the coordinate values ​​of a specific point on the target using a distance measurement method; The 3D coordinate values ​​(U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values ​​(X, Y, Z) of a three-dimensional coordinate system, Using the distance image sensor and the distance measurement method, obtain 3D coordinate values ​​(U, V, D) on a depth image of the specific point on the target surface; The 3D coordinate values ​​(X, Y, Z) in the three-dimensional coordinate system corresponding to the 3D coordinate values ​​(U, V, D) are obtained.

[0012] The fourth technical means adopted by the present invention is: A coordinate alignment system in a composite measurement system including a first measurement system having a distance image sensor using infrared rays and a second measurement system, the coordinate alignment system includes three calibration jigs detachably installed at predetermined positions of the second measurement system, and a camera; Each calibration jig is a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface; the second reflecting surface is formed from an infrared reflecting material; a specific point on the first reflecting surface is a target, the three calibration jigs are installed in the second measurement system in a positional relationship such that the first target, the second target, and the third target form vertices of a scalene triangle; In the first measurement system, 3D coordinate values ​​(U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values ​​(X, Y, Z) of a first coordinate system; the coordinate alignment system includes a processing unit and a storage unit; the storage unit stores reference coordinate values ​​of each target in the first coordinate system (X, Y, Z) of the range image sensor when the first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned; The processing unit detecting an area of ​​the target surface by image processing based on the image acquired by the camera, and detecting a specific point on the first reflecting surface as a target; Using the range image sensor, obtain measurement coordinate values ​​(X, Y, Z) of each target on a first coordinate system; The system is configured to associate measurement coordinate values ​​with reference coordinate values ​​for each target and obtain a transformation formula that matches the measurement coordinate values ​​with the reference coordinate values. [Effects of the Invention]

[0013] The present invention makes it possible to provide a system that implements a function for aligning the coordinates of a motion analysis system using a distance image sensor with another measurement system (such as a pressure distribution sheet). By aligning the coordinate systems of the motion analysis system using a distance image sensor with those of another measurement system (such as a pressure distribution sheet), the usefulness of the combined system can be increased. For example, in rehabilitation, gait analysis using the combined system can obtain information useful for determining treatment guidelines for patients. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is an overall view of a composite measurement system consisting of a first measurement system and a second measurement system using a range image sensor. [Figure 2] 10A and 10B are diagrams showing a jig for acquiring coordinate values ​​of a distance measuring jig according to the present embodiment. [Figure 3] FIG. 2 is a diagram showing an installation jig for the distance measuring jig according to the embodiment. [Figure 4] This figure shows a distance measurement jig related to this embodiment, where the upper figure is a different oblique view of the distance measurement jig, the lower figure (left) shows an installation jig, and the lower figure (right) shows the state in which a coordinate value acquisition jig is attached to the installation jig. [Figure 5] FIG. 10 is a diagram showing the attached state of the installation jig. [Figure 6] 10A to 10C are diagrams illustrating steps for acquiring coordinate values ​​of a target using a distance measuring jig according to the present embodiment. [Figure 7] 1 is a schematic diagram of a coordinate alignment system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] [A] Composite measurement system The composite measurement system according to this embodiment will now be described. As shown in Fig. 1, the measurement system according to this embodiment comprises a first measurement system and a second measurement system. The first measurement system is a motion measurement system using a distance image sensor, and the second measurement system is a pressure distribution sheet equipped with a pressure distribution sensor.

[0016] The first measurement system includes a distance image sensor that acquires a depth image of the object, and a 3D joint position estimation means that acquires the 3D joint positions of the object based on the depth image of the object acquired by the distance image sensor, and measures the movement of the object walking on the pressure distribution sheet.

[0017] The 3D joint position estimation means according to this embodiment can be configured from a computer equipped with a memory and a processor. The 3D joint position estimation means according to the first measurement system acquires a person area based on depth data (range image data) and estimates joint positions by inputting distance information of the person area into a classifier consisting of a trained model. A program for estimating the 3D joint positions of an object based on the depth data of the object is stored in the computer's memory, and the program is executed by the processor to estimate and output the 3D joint positions of the object. Estimating the 3D coordinate values ​​of the joints of an object based on a depth image of the object is itself well known and is described in Patent Documents 1 to 3 and Non-Patent Document 1.

[0018] In this specification, the 3D coordinate values ​​of the range image sensor are defined as the coordinate values ​​(Xc Yc Zc) obtained using the range image sensor. The 2D coordinate values ​​of the range image sensor are defined as the coordinate values ​​(U, V, D) obtained when the coordinate values ​​are displayed in 2D using the range image sensor. The coordinate values ​​(U, V) are the coordinate values ​​on the monitor, and D is the depth distance from the sensor. The range image sensor of the first measurement system implements a conversion function from the 2D coordinate values ​​(U, V, D) to the 3D coordinate values ​​(Xc, Yc, Zc). The first coordinate system of the first measurement system in Figure 1 is the 3D coordinate values ​​(Xc Yc Zc) of the range image sensor.

[0019] The first measurement system may include a distance image sensor that captures a depth image (distance image) of the object, as well as a video camera (RGB camera) that captures a color image (RGB image) of the object. In this case, it is desirable that the coordinate values ​​of the depth image and the 3D coordinate values ​​of the video camera are mutually convertible. In one embodiment, a 2D joint position estimation program that estimates 2D joint positions based on an RGB image is stored in computer memory, and pose estimation (2D joint position estimation) is performed to acquire the 2D joint positions of the object based on the color image (RGB image) of the object captured by the video camera. The 3D joint positions estimated from the distance image sensor may be corrected using the 2D joint positions estimated from the camera image. In this case, the distance image sensor and the video camera may be integrated into a composite sensor, or the distance image sensor and the video camera may be provided separately. Kinect (registered trademark) is a typical example of such a composite sensor system. Kinect is composed of a composite sensor consisting of an infrared projector, a depth camera, and a color image camera, and the distance image sensor (depth sensor) is composed of the infrared projector and the infrared camera (it does not have a pose estimation means based on an RGB image).

[0020] The pressure distribution sheet serving as the second measurement system acquires the sole load distribution of a subject walking on the sheet. The sole load distribution is acquired according to a second coordinate system (xy plane). In this embodiment, the 3D coordinate system of the second measurement system is assumed to be (Xs, Ys, 0) because the system (pressure distribution sheet) is placed on the floor for use. The second coordinate system of the second measurement system in FIG. 1 is the 3D coordinate value (Xs, Ys, 0) of the pressure distribution sheet. The second measurement system includes a processor that processes data acquired by the load sensor of the pressure distribution sheet and a computer including memory for storing the data. The second measurement system has an external input function (in one embodiment, an external input terminal of a synchronization cable) and is configured to input a synchronization signal from the first measurement system (master). Note that the second measurement system may also be configured as the master to output a synchronization signal to the first measurement system. Although the second measurement system according to this embodiment is a pressure distribution sheet, the second measurement system is not limited to a pressure distribution sheet and may be, for example, a force plate.

[0021] The first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned, and the measurement data acquired by the first measurement system and the measurement data acquired by the second measurement system are synchronized. The subject's gait analysis is performed based on the subject's walking movement data (including posture data at a certain point in time) acquired by the first measurement system and the time and distance factors acquired by the second measurement system.

[0022] One synchronization method is to prepare a jig that outputs a synchronization signal simultaneously with illuminating an LED at the moment measurement starts, and synchronize the start of measurement with the moment the LED lights up on the color camera of the range image sensor of the first measurement system (which is synchronized with the range image sensor in advance) and the moment the second measurement system receives the synchronization signal. In this embodiment, synchronization is achieved by sounding a buzzer with a speaker. Various known means can be used as appropriate to synchronize the first and second measurement systems.

[0023] [B] Coordinate Alignment System The coordinate alignment system includes a composite measurement system consisting of a first measurement system and a second measurement system each equipped with a distance image sensor, as well as a camera (infrared camera), three distance measurement jigs (calibration jigs), and a computer equipped with a memory and a processor. The computer constituting the composite measurement system and the computer constituting the coordinate alignment system may be the same computer.

[0024] [B-1] Distance measurement jig (calibration jig) The distance measuring jig according to this embodiment will be described. The distance measuring jig according to this embodiment is a distance measuring jig used together with a distance image sensor that uses infrared rays. The distance image sensor according to this embodiment is a distance measuring means that irradiates an object with light from a built-in infrared LED and measures the time it takes for the light to reflect and return to determine the distance. The distance measuring jig according to this embodiment is composed of a coordinate value acquisition jig 1 and an installation jig 2 for the coordinate value acquisition jig (hereinafter referred to as the "installation jig"). The distance measuring jig according to this embodiment is a calibration jig used to align the coordinates of the first coordinate system of the first measurement system and the second coordinate system of the second measurement system.

[0025] As shown in FIG. 2, the coordinate value acquisition jig 1 comprises a main body 3 including a target surface, a support portion 4 of the main body 3, and a base 5 located at the lower end of the support portion 4. The target surface comprises a first reflective surface 6 in the center and a second reflective surface 7 surrounding the first reflective surface 6, with the second reflective surface 7 being made of an infrared-reflective material. Through research by the inventors, we have discovered that when attempting to acquire distance from an infrared-reflective material using a range image sensor, an appropriate distance cannot be calculated. Therefore, even if the spherical infrared reflective marker disclosed in Patent Document 6 is used as a calibration jig, the distance D of the coordinate values ​​(U, V, D) of the range image cannot be acquired. To solve this problem, we developed a distance measurement jig (calibration jig) with a target surface consisting of two reflective surfaces (the first reflective surface 6 and the second reflective surface 7) with different physical properties.

[0026] The infrared reflective material of the second reflecting surface 7 is a material that produces a high-brightness area when projected with infrared light and used with a camera (infrared camera) (a material from which the distance cannot be measured by an infrared distance image sensor). Infrared reflective materials are well known, and for example, the second reflecting surface 7 can be formed by applying commercially available infrared reflective tape to a predetermined location on the target surface.

[0027] The material of the first reflecting surface 6 surrounded by the second reflecting surface 7 must be a material that allows the infrared distance image sensor to obtain distance by reflecting infrared rays (it is not a material that absorbs or scatters infrared rays). Due to the nature of infrared rays, it is understood that many materials around us in our daily lives meet this condition.

[0028] In the distance measuring jig according to this embodiment, the target surface and the first reflecting surface 6 are concentric circles, and the second reflecting surface 7 is ring-shaped. The center of the target surface, i.e., the center of the first reflecting surface 6, is the distance measurement target, and the distance measuring jig measures the distance to the distance measurement target. In this embodiment, a strip of infrared reflective tape is attached as the second reflecting surface 7, and the distance to the second reflecting surface 7 (the area of ​​the infrared reflective tape) on the target surface cannot be obtained by the range image sensor. Here, a feature is that the infrared reflective tape is not attached to the central region of the target surface (first reflecting surface 6). The target surface appears as a high-brightness area due to halation in the infrared reflective camera, but the center of the high-brightness area does not have the infrared reflective tape attached, so the distance to the center can be obtained by the range image sensor. In other words, the range image sensor can obtain the coordinate values ​​(U, V, D) of the target.

[0029] In one embodiment, the width of the ring-shaped second reflective surface 7 is set so that the brightness is high all the way to the center. Specifically, the infrared reflective tape is set so that it reflects the infrared light output from the distance image sensor, causing halation in the infrared reflective camera, resulting in high brightness all the way to the center. As described below, when detecting the target surface area through image processing, the difference in brightness between images acquired before and after placing the jig is taken, and the area with the largest difference is detected as the target surface. Since the center position of the constant brightness area is calculated in this process, it is desirable that the area be illuminated all the way to the center. However, it will be understood by those skilled in the art that the target surface can be detected even if the area is not illuminated all the way to the center by designing the image processing program.

[0030] The main body 3 of the distance measuring jig is a disk-shaped body in a vertical position, the base 5 is a disk-shaped body in a horizontal position, and the support part 4 consists of a raised part 40 and a square plate-shaped base part 41 at the lower end of the raised part 40, with the base part 41 being integrated with the upper surface of the base 5.

[0031] The installation jig 2 is a jig for aligning the first coordinate system of the first measurement system (distance image sensor) with the second coordinate system of the second measurement system (pressure distribution sheet) by combining with the coordinate value acquisition jig 1. As shown in Fig. 3, the installation jig 2 has a main body 8 that is rectangular in plan view, a circular opening 80 formed in the center of the main body 8, and a fixing part 9 extending from one side of the rectangular main body 8, and can be attached to a predetermined position on the pressure distribution sheet via the fixing part 9.

[0032] The main body 8 of the installation jig 2 according to this embodiment is a sheet-like or plate-like body of a predetermined thickness. The fixing portion 9 is thinner than the main body 8, and the end face of the main body 8 is abutted against the edge of the pressure distribution sheet, the main body 8 is placed on the floor, and the fixing portion 9 is placed over the pressure distribution sheet and fixed to the pressure distribution sheet. The installation jig 2 is fixed to a predetermined position with good reproducibility by utilizing the configuration and shape of the second measurement system that constitutes the composite measurement system. The installation jig 2 according to this embodiment is configured to be fixed to a specific position on the pressure distribution sheet with good reproducibility. In this embodiment, the shape is designed to be fixed to the connection between the pressure distribution sensors, allowing it to be attached to a specific position on the pressure distribution sheet with good precision. The fixing part 9 of the installation jig 2 according to this embodiment is formed with a groove 90 that is U-shaped in plan view to receive a pair of protruding elements of the pressure distribution sheet, and the position of the installation jig (i.e., the distance measurement jig or calibration jig) relative to the pressure distribution sheet is determined by receiving the protruding elements in the groove and abutting the end face of the main body 8 against the edge of the pressure distribution sheet. Note that the configuration of the installation jig 2 shown in the figure (particularly the configuration of the fixing part 9) is merely one example, and it will be understood by those skilled in the art that the fixing part 9 can be designed to correspond to the specific configuration of the second measurement system that constitutes the composite measurement system, for example.

[0033] By installing the base 5 of the coordinate value acquisition jig 1 in the circular opening 80 of the main body 8 of the installation jig 2, the coordinate value acquisition jig 1 is installed at a predetermined position on the pressure distribution sheet. The bottom surface of the base 5 of the coordinate value acquisition jig 1 is placed on the floor surface on which the pressure distribution sheet is placed. Because the disk-shaped base 5 is located inside the circular opening 80, the base 6 can rotate within the opening 80. In other words, with the distance measurement jig attached at a predetermined position on the pressure distribution sheet, the orientation of the target surface of the coordinate value acquisition jig 1 can be changed.

[0034] As described above, the base 5 of the coordinate value acquisition jig 1 is rotatable within the opening 80 of the main body 8 of the installation jig 2, making the coordinate value acquisition jig 1 rotatable relative to the main body 8 of the installation jig 2. That is, the target surface of the coordinate value acquisition jig 1 is rotatably supported, and the orientation of the target surface can be changed by rotating the coordinate value acquisition jig 1. Therefore, the orientation of the target surface can be changed to match the orientation of the range image sensor. Note that the coordinate value acquisition jig and the installation jig may be integrated (the base of the coordinate value acquisition jig doubles as the installation jig) to form a ranging jig, and the support part of the coordinate value acquisition jig may be rotatable relative to the base. Furthermore, although the target surface is vertical in this embodiment, the inclination of the target surface may be variable.

[0035] In this embodiment, a concentric target surface and first reflecting surface 6 are used, but the shapes of the target surface and first reflecting surface 6 are not limited to circles. The target surface is made up of a central first reflecting surface 6 and a second reflecting surface 7 provided to surround the first reflecting surface 6, and the second reflecting surface 7 is made of an infrared reflective material. Any shape may be used as long as it allows a specific point (e.g., the center) on the target surface (first reflecting surface 6) to be calculated based on the area of ​​the target surface extracted from the image, and for example, the target surface and first reflecting surface 6 may be rectangular (concentric quadrangles).

[0036] In this embodiment, the base 5 of the coordinate value acquisition jig 1 is disk-shaped, and by rotating the coordinate value acquisition jig 1 relative to the installation jig 2, the target surface can be made to face the lens surface of the camera, and a mechanism is provided so that the position of the midpoint does not change even when calculated for the planar projection image onto the camera.

[0037] [B-2] Camera In this embodiment, the target surface of the distance measuring jig is detected by image processing based on images acquired using an infrared camera. Note that the target surface can be extracted using a normal camera because its brightness is stronger than that of other areas. While an infrared camera is not essential, it is advantageous to use an image from an infrared camera to more effectively detect the target surface area. The coordinate system of the range image sensor and the coordinate system of the infrared camera (camera) are matched in advance. Conversion information between the 2D coordinate system of the range image sensor and the 2D coordinate system of the camera is obtained in advance, making conversion possible. Furthermore, the 2D coordinate system of the range image sensor (the coordinate system of the depth image) and the 3D coordinate system are convertible.

[0038] [B-3] Acquisition of target coordinates for the distance measurement jig (calibration jig) With reference to FIG. 6, the steps for acquiring the coordinate values ​​of the target of the distance measuring jig (calibration jig) will be described. (a) An infrared camera is used to obtain an image including a distance measuring jig installed at a predetermined position on the pressure distribution sheet as a first image. (a) Using the infrared camera (at the same position and orientation), an image of the pressure distribution sheet without the distance measurement jig is acquired as the second image. The order in which the first and second images are acquired is not limited. (c) By image processing, the area of ​​the target surface of the distance measuring jig is detected based on the difference in brightness between the first image and the second image. (d) Based on the detected area, the coordinate values ​​(U, V) of a specific point (center) on the first reflecting surface 6 of the target surface are obtained. (e) Using a distance image sensor, the distance D to a specific point (center) on the first reflecting surface 6 is acquired. (f) The coordinate values ​​(U, V, D) of a specific point on the target surface are converted to coordinate values ​​(Xc, Yc, Zc).

[0039] [B-4] Coordinate alignment step The coordinate alignment step will now be explained. Three calibration jigs are placed in predetermined positions on the pressure distribution sheet in advance, and the three-dimensional coordinate values ​​of the three targets (first target, second target, third target) when coordinate alignment is performed are set as reference coordinate values. During measurement, the coordinates of the three targets (first target, second target, third target) on the three calibration jigs are measured, and the measured coordinate values ​​of the three targets are obtained. Coordinate alignment is performed by translating and rotating the targets so that their measured coordinate values ​​match the set reference coordinate values.

[0040] The following contents are set in advance and stored in the memory of the computer. (i) Information about where the origin of the pressure distribution sheet is located on the distance image sensor In the setting file, the amount of parallel translation in the X direction, the amount of parallel translation in the Y direction (for example, in mm), and the amount of rotation (deg) are set. (ii) Information about the coordinate values ​​of the calibration jig when the pressure distribution sheet and the coordinate system of the range image sensor are aligned. In the configuration file, set the coordinate values ​​of the 3D coordinate system of the range image sensor of the three targets as reference coordinate values.

[0041] The three points set in each coordinate system are P s c1 =(X s c1 , Y s c1 , Z s c1 ), P s c2 =(X s c2 , Y s c2 , Z s c2 ), P s c3 = (X s c3 , Y s c3 , Z s c3) (coordinate values ​​after movement). At this time, the length of each side of the triangle formed by these three points should be different. 3 points, P s c1 =(X s c1 , Y s c1 , Z s c1 ), P s c2 =(X s c2 , Y s c2 , Z s c2 ), P s c3 = (X s c3 , Y s c3 , Z s c3 ) can be set reproducibly on the sheet and consist of any three points that are not on a straight line. In this embodiment, P s c1 is a point that can be fixed on the sheet in the sheet coordinate system, and P s c2 HA P s c1 and the sheet coordinate system, the Y coordinate value is the same at one point, and P s c3 HA P s c2 In the sheet coordinate system, a point with the same X coordinate value is used.

[0042] Before measurement, the 3D coordinate values ​​of the three calibration jigs are obtained. An infrared image is obtained as the first image without the calibration jigs in place. The three calibration jigs are placed in their designated positions, and an infrared image is obtained as the second image. The difference in brightness between the first and second images is taken, and positions above a certain value are identified as target surfaces, and the coordinate values ​​(U, V) of the center of each target surface are obtained. The 2D image of the range image sensor is used to obtain (U, V, D), and conversion is performed to obtain the coordinate values ​​(X c1 , Y c1 , Z c1 ), coordinate value (X c2 , Y c2 , Z c2), coordinate value (X c3 , Y c3 , Z c3 ) is obtained. For these three coordinate values, the correspondence with the first target, second target, and third target is obtained based on the length of the sides of the triangle consisting of the three points. In this way, the coordinate values ​​P of the three points before conversion are obtained. c1 =(X c1 , Y c1 , Z c1 ), P c2 =(X c2 , Y c2 , Z c2 ), P c3 =(X c3 , Y c3 , Z c3 ) (the coordinate value before movement).

[0043] Measurement coordinate values ​​P of the three targets c1, P c2 ,P c3 The coordinate values ​​P of the setting file for the three targets are s c1, P s c1, P s c1 Specifically, the following calculations are performed: (1) Regarding the coordinate values ​​before movement, V s e1 = V s 2→1 = ( P s c2 -P s c1 ) / | P s c2 -P s c1 | V s 3→1 = ( P s c3 -P s c1 ) / | P s c3 -P s c1 |And, V s2→1 and V s 3→1 The unit normal vector to V s e2 Let's say. V s e2 and V s e1 The normal vector of V s e3 Let's say. matrix M s M s = ( V s e1 , V s e2 , V s e3 )

[0044] (2) Regarding the coordinate values ​​after movement, V c e1 = V 2→1 = ( P c2 -P c1 ) / | P c2 -P c1 | V 3→1 = ( P c3 -P c1 ) / | P c3 -P c1 |And, V 2→1 and V 3→1 The unit normal vector to V c e2 Let's say. V c e2 and V c e1 The normal vector of V c e3 Let's say. matrix M c M c = ( V c e1 , V c e2 , V c e3 )

[0045] (3) Coordinate value before movement P0 = (Xp , Y p , Z p ) T In contrast, P c1 is at the origin, and the transformation that makes the three points coincide with the floor is P0' = (M s ) -1 (P0-P C1 ) (4) The formula for converting the coordinates converted in (3) to the converted coordinate value P0” is P0” = M c P0'+ P C1 This becomes: If (3) and (4) are processed simultaneously, P0” = M c (M s ) -1 (P0-P C1 )+ P C1 It can be calculated as follows. This calculation makes it possible to align the coordinate system of the pressure distribution sheet with the coordinate system of the distance image sensor for any point. After that, by performing a parallel translation using the "information about where the origin position of the pressure distribution sheet corresponds on the distance image sensor (information in the setting file)" in the setting file, it becomes possible to place the origin of the pressure distribution sheet at any location relative to the origin of the distance image sensor. Explanation of symbols

[0046] 1 Coordinate value acquisition jig 2. Coordinate value acquisition jig installation jig 3. Coordinate acquisition jig body 4 Support part 5. Pedestal 6 1st reflective surface 7 Second reflective surface 8 Installation jig body

Claims

1. A coordinate alignment system in a composite measurement system including a first measurement system, which is a motion measurement system equipped with a distance image sensor that irradiates an object with infrared light and measures the time it takes for the infrared light to reflect and return to determine a distance, and a second measurement system made of a pressure distribution sheet, the coordinate alignment system comprises three calibration jigs detachably installed at predetermined positions on the pressure distribution sheet, an infrared camera, and the distance image sensor; Each calibration jig is a target surface including a first reflecting surface at the center and a second reflecting surface provided so as to surround the first reflecting surface; the second reflecting surface is formed from an infrared reflecting material; the first reflecting surface is formed of a material that allows the distance image sensor to acquire distance by reflecting infrared rays, and a specific point on the first reflecting surface serves as a target; Each calibration jig is placed on the pressure distribution sheet with the target surface in a vertical position, the three calibration jigs are installed on the pressure distribution sheet in a positional relationship such that the first target, the second target, and the third target are vertices of a scalene triangle; In the first measurement system, 3D coordinate values ​​(U, V, D) of the depth image acquired by the distance image sensor can be converted into 3D coordinate values ​​(X, Y, Z) of a first coordinate system; the coordinate alignment system includes a processing unit and a storage unit; the storage unit stores reference coordinate values ​​of each target in the first coordinate system (X, Y, Z) of the range image sensor when the first coordinate system of the first measurement system and the second coordinate system of the second measurement system are coordinate-aligned; The processing unit detecting an area of ​​the target surface by image processing based on the image acquired by the infrared camera, and detecting a specific point on the first reflecting surface as a target; Using the 3D coordinate values ​​(U, V, D) of each target acquired by the range image sensor, measurement coordinate values ​​(X, Y, Z) of each target on a first coordinate system are acquired; and obtaining a transformation formula for matching measurement coordinate values ​​with reference coordinate values ​​of each target and making the measurement coordinate values ​​coincide with the reference coordinate values. Coordinate alignment system.

2. the target surface and the first reflecting surface are concentric circles, and the second reflecting surface is ring-shaped; The coordinate alignment system of claim 1 .

3. The orientation of the target surface is variable. The coordinate alignment system of claim 1 .

4. The inclination of the target surface is variable. The coordinate alignment system of claim 1 .

5. The specific point is the center of the target surface. The coordinate alignment system of claim 2 .

6. The image processing acquiring a first image including the calibration jig installed at a predetermined position using an infrared camera whose position and orientation are fixed; acquiring a second image with the infrared camera without the calibration jig installed; detecting a region of the target surface based on a difference in brightness between the first image and the second image, and detecting a specific point on the first reflecting surface; Consists of: The coordinate alignment system according to any one of claims 1 to 5.

Images