Measurement system, measurement method, and measurement program

Abstract

To efficiently perform a survey on an operation state of an on-site resource in a construction site.SOLUTION: A measurement system using a surveying device 100 having a camera and a positioning function using laser light comprises: imaging means which continuously images heavy machines 201 to 204 being on-site resources performing the work in a construction site with the camera; recognition means which recognizes the on-site resources from the photographed images obtained by imaging; tracking means which tracks the image of the recognized on-site resources in the photographed images obtained by the continuous imaging; and positioning means which performs collimation to the on-site resources being the tracking target and performs positioning of the on-site resources by using the positioning function, the positioning is performed a plurality of times with an interval.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for grasping the operating status of on-site resources such as heavy machinery at a construction site. [Background technology]

[0002] There is a demand for improving the operating efficiency of on-site resources such as heavy machinery at construction sites. To achieve this, it is necessary to check the operating status of on-site resources. For example, Patent Document 1 describes a technology that identifies heavy machinery using images taken from a flying UAV and tracks its movement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-159565 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the investigation of the operation status of on-site resources such as heavy machinery at a construction site, it is desirable to minimize the need for manpower and to automate the investigation. In this context, the present invention aims to provide a technology for efficiently investigating the operation status of on-site resources at a construction site. [Means for solving the problem]

[0005] The present invention relates to a method for performing work at a construction site. stomach , Heavy machinery having a base for traveling and a movable part that moves on the base A camera for continuously photographing the above-mentioned Heavy machinery a recognition means for recognizing the recognized Heavy machinery A tracking means for tracking an image of the Heavy machinery The laser beam is collimated to the target. The heavy machinery A positioning means for measuring the position of the a moving center calculation means for calculating a point fixed to the base unit as a moving center, instead of the measured point, based on a result of the positioning;and the positioning is performed multiple times at intervals.

[0006] In the present invention, Heavy machinery In one embodiment, the present invention further comprises a map generating means for displaying the progress of the movement of the Heavy machinery The recognition is performed by using the image captured by the camera and a previously prepared recognition target. Heavy machinery One example of such a method is to compare a three-dimensional model with images taken from a plurality of different viewpoints.

[0008] In the present invention, the relationship between the exterior orientation elements of the camera and the exterior orientation elements of the positioning means is known, and based on said relationship, the position of the reflection point of the laser light in the image of the heavy equipment taken by the camera is obtained as a first relationship, and by the comparison, a three-dimensional model corresponding to the image of the heavy equipment taken by the camera is obtained as a second relationship, and based on the first relationship and the second relationship, the position of the reflection point of the laser light in the three-dimensional model of the heavy equipment is obtained, and the center of movement is calculated based on the position of the reflection point of the laser light in the three-dimensional model of the heavy equipment.

[0009] In the present invention, Heavy machinery is a plurality of Heavy machinery The tracking is performed for each of the target objects, and the continuous photographing is continued while the collimation is being performed. The heavy machinery In one embodiment, the tracking is performed to the

[0010] The present invention involves performing work at a construction site, Heavy machinery having a base for traveling and a movable part that moves on the base a step of continuously photographing the object by a camera, and selecting the object from among the photographed images obtained by the photographing. Heavy machinery a recognition step of recognizing the recognized Heavy machinery a tracking step of tracking an image of the Heavy machinery The laser beam is collimated to the target. The heavy machinerya positioning step for performing positioning of the a movement center calculation step of calculating a point fixed to the base unit as a movement center, instead of the measured point, based on the result of the positioning; The positioning method may also be understood as a measurement method in which the positioning is performed multiple times at intervals.

[0011] The present invention is a measurement program executed by a computer, which causes the computer to carry out operations at a construction site, Heavy machinery having a base for traveling and a movable part that moves on the base a step of continuously photographing the object by a camera, and selecting the object from among the photographed images obtained by the photographing. Heavy machinery a recognition step of recognizing the recognized Heavy machinery a tracking step of tracking an image of the Heavy machinery The laser beam is collimated to the target. The heavy machinery a positioning step for performing positioning of the a movement center calculation step of calculating a point fixed to the base unit as a movement center, instead of the measured point, based on the result of the positioning; and performing the positioning a plurality of times at intervals. Effect of the Invention

[0012] According to the present invention, the operating status of on-site resources at a construction site can be efficiently investigated. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a conceptual diagram showing an overview of a measurement system utilizing the present invention. [Diagram 2] 1A is a perspective view of the surveying device seen from the front, and FIG. 1B is a perspective view of the surveying device seen from the rear. [Diagram 3] FIG. 2 is a block diagram of the surveying instrument. [Figure 4] 11 is a flowchart illustrating an example of a processing procedure. [Diagram 5] 11 is a flowchart illustrating an example of a processing procedure. [Figure 6] FIG. [Figure 7] 11 is a flowchart illustrating an example of a processing procedure. [Figure 8] Diagrams (A), (B), and (C) plot the positions of heavy equipment and workers. [Figure 9] This is a diagram plotting the positions of heavy equipment and workers. [Figure 10] 13A, 13B, and 13C are model diagrams showing images of a moving heavy machine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] 1. First embodiment (overview) FIG. 1 shows an example of a measurement system using the present invention. FIG. 1 shows a surveying device 100. The surveying device 100 is a total station having a camera, a laser positioning function, and an automatic tracking function for a surveying target. The surveying device 100 identifies a tracking target from an image captured by its own camera, tracks the identified target in the captured image, and measures the position of the target using a laser beam. Details of the surveying device 100 will be described later.

[0015] 1 shows heavy machinery 201-204 and workers 301-304 performing civil engineering work. In this example, the heavy machinery 201-204 and the workers 301-304 are on-site resources. The operating status of these on-site resources is measured by the surveying device 100. On-site resources are machines and people engaged in construction work at a construction site, and are mobile. Machines treated as on-site resources include various types of mobile heavy machinery, power sources, concrete mixers, compressors, etc.

[0016] The surveying device 100 captures images of the work site using a camera. Site resources to be tracked (heavy machinery 111-114 and workers 121-124) are recognized from the captured images, and each recognized site resource is identified.

[0017] Each of the site resources identified above is tracked and positioned by the surveying device 100. The position information of each site resource positioned by the surveying device 100 is plotted on a map (e.g., FIG. 9). By observing the change in the position of each heavy machine over time, the progress of the operation status of the site resources can be known.

[0018] The above process is performed automatically, so that the operation status of on-site resources at a construction site can be efficiently investigated.

[0019] (Surveying equipment) 2 is a perspective view (A) and (B) of the surveying instrument 100. (A) is a perspective view seen from the front side, and (B) is a perspective view seen from the rear side. The surveying instrument 100 comprises a base unit 102 fixed on a tripod 101, a horizontal rotation unit 103 capable of horizontal rotation on the base unit 102, and a vertical rotation unit 104 held on the horizontal rotation unit 103 in a state capable of vertical rotation (elevation angle control and depression angle control).

[0020] Horizontal and vertical rotations are performed by motors. The horizontal angle of the horizontal rotation unit 103 (the horizontal direction of the optical axis of the telescope 105) and the vertical angle of the vertical rotation unit 104 (the elevation or depression angle of the optical axis of the telescope 105) are precisely measured by encoders.

[0021] A telescope 105 and a wide-angle camera 106 are arranged in front of the vertical rotation unit 104, and a docking unit 107 for the telescope 105 and a touch panel display 108 are arranged on the back. The telescope 105 also serves as the optical system of the narrow-angle (telephoto) camera 111 shown in Fig. 3. A laser beam for measuring distances is emitted to the outside via the objective lens of the telescope 105, and the reflected light is received.

[0022] The touch panel display 108 is an operation panel and display for the surveying instrument 100. On the touch panel display 108, various information related to the operation of the surveying instrument 100 and information related to the surveying results are displayed.

[0023] 3 is a block diagram of the surveying device 100. The surveying device 100 includes a narrow-angle camera 111, a wide-angle camera 106, a shooting range setting unit 112, a camera control unit 113, an image acquisition unit 114, a tracked object recognition unit 115, a tracking unit 116, a positioning object selection unit 117, a positioning unit 118, a movement center calculation unit 119, a mapping unit 120, a drive control unit 121, a data storage unit 122, and a communication unit 123.

[0024] Each functional unit, namely the shooting range setting unit 112, the camera control unit 113, the image acquisition unit 114, the tracked object recognition unit 115, the tracking unit 116, the positioning object selection unit 117, the moving center calculation unit 119, the mapping unit 120, and the data storage unit 122, is realized by a computer provided in the surveying device 100.

[0025] The computer includes a CPU, a storage device, and an interface. An operating program for executing the functions of the functional units is read and executed by the computer, thereby realizing each of the functional units. It is also possible to realize some or all of the functional units by dedicated hardware. It is also possible to realize some or all of the functional units by FPGA or the like.

[0026] Narrow-angle camera 111 captures a relatively narrow range through telescope 105. This captures magnified images at a telephoto distance, and can capture images with greater detail than wide-angle camera 106. Wide-angle camera 106 captures a relatively wide range (wide angle).

[0027] The relationship between the exterior orientation elements (position and attitude) of the narrow-angle camera 111, the wide-angle camera 106, and the optical system of the positioning unit 118 described later in the surveying device 100 is known. In addition, the optical axes of the narrow-angle camera 111 and the optical system of the positioning unit 118 are on the same axis (on the optical axis of the telescope 105). The optical axis of the wide-angle camera 106 is in a positional relationship parallel to the optical axis of the narrow-angle camera 111 and the optical system of the positioning unit 118 (the optical axis of the telescope 105).

[0028] The shooting range setting unit 112 is involved in the process of step S101 described later, and sets the shooting range of the wide-angle camera 106. The camera control unit 113 controls the shooting operations of the wide-angle camera 106 and the narrow-angle camera 111. The image acquisition unit 114 acquires image data of images captured by the wide-angle camera 106 and the narrow-angle camera 111.

[0029] The tracked object recognition unit 115 performs the process of step S104, which will be described later. The tracker 116 performs the process of step S105, which will be described later. The positioning object selection unit 117 performs the process of step S106, which will be described later.

[0030] The positioning unit 118 performs the process of step S107 described later. The positioning unit 118 includes an emitter of distance measurement light (laser light for distance measurement) and an irradiation optical system thereof, a light receiving optical system and light receiving element for the distance measurement light reflected and returned from the target object, a calculator of the distance measurement to the reflection point of the distance measurement light, and a calculator of the position of the reflection point of the distance measurement light based on the direction of the optical axis and the distance measurement value.

[0031] The distance to the reflection point is calculated using the principle of optical distance measurement. There are two methods for calculating the distance: one that uses the phase difference of the received distance measurement light, and one that uses the propagation time. In this example, the distance is measured using the phase difference method.

[0032] In the method using phase difference, a reference optical path is provided in the surveying instrument 100, and the distance to the object is calculated from the difference (phase difference) between the reception timing of the distance measurement light propagated through this reference optical path and the reception timing of the distance measurement light reflected from the object. In the method using propagation time, the distance to the object is calculated from the time it takes for the distance measurement light to hit the object, be reflected, and return.

[0033] The position of the reflection point is calculated with the surveying device 100 as the origin based on the distance to the reflection point of the measuring light and its direction. If the exterior orientation elements of the surveying device 100 in the absolute coordinate system are known, the position of the reflection point in the absolute coordinate system can be obtained. The absolute coordinate system is a coordinate system used in GNSS and maps. For example, the position in the absolute coordinate system is described by latitude, longitude, and altitude.

[0034] The moving center calculation unit 119 performs the process of step S108 described below. For example, consider a hydraulic shovel 600 as shown in Fig. 6. The hydraulic shovel 600 has a moving base 601 that is equipped with caterpillar tracks 601a and moves on the ground by the caterpillar tracks 601a. ​​A rotating unit 604 that is equipped with a driver's seat and supports the base of an arm 603 is disposed on the moving base 601. A bucket 602 is connected to the tip of the arm 603.

[0035] When the hydraulic shovel 600 operates without moving, the positions of the rotating part 604, arm 603, and bucket 602 change, but the hydraulic shovel 600 does not move as a vehicle. In this case, if the arm 603 or bucket 602 is used as the object of position measurement, the movement state of the hydraulic shovel 600 cannot be properly grasped. In this case, the movement of the hydraulic shovel 600 can be properly grasped by using the moving base 601 of the hydraulic shovel 600 as the object of position measurement.

[0036] The mapping unit 120 describes the position of the tracked object and its transition on a map. It is also possible to describe the time at the same time as the position. The function of the mapping unit 120 allows the operation map of FIG. 8 or FIG. 9 to be obtained.

[0037] The drive control unit 121 controls the direction of the optical axis of the surveying instrument 100. Specifically, it controls the drive of the horizontal rotation unit 103 and the drive of the vertical rotation unit 104. The data storage unit 122 stores data and operating programs required for the operation of the surveying instrument 100, surveying data, and data obtained as a result of various processes.

[0038] The communication device 123 communicates with external devices. The communication is performed using a wireless LAN or a mobile phone communication line. Using the communication device 123, data of the operation map shown in Fig. 8 and Fig. 9 is transmitted to the external devices.

[0039] (Example of processing) Fig. 4 shows an example of the procedure of the operation performed by the surveying instrument 100. A program for executing the flowchart shown in Fig. 4 is stored in an appropriate storage medium or storage area, and is executed by a computer provided in the surveying instrument 100. It is also possible to perform at least a part of the processing in Fig. 4 by an external control computer or processing server. This also applies to the flowcharts in Figs. 5 and 7.

[0040] First, prior to processing, exterior orientation elements of the surveying instrument 100 in the coordinate system used in the operational map are obtained. For example, exterior orientation elements of the surveying instrument 100 in the absolute coordinate system are obtained. In addition, the range (measurement range) for tracking and positioning on-site resources is determined in advance.

[0041] Also, a shooting range for the above measurement range is determined. For example, it is assumed that measurement is performed in the horizontal direction, the angular range of the measurement range is 90°, shooting is performed by wide-angle camera 106, and the shooting range of wide-angle camera 106 is 25°. In this case, the shooting range is set by dividing the 90° range into four. In this case, a first shooting range, a second shooting range, a third shooting range, and a fourth shooting range are set, each with a center direction shifted by 90° / 4. Of course, the shooting ranges may partially overlap. Also, the order of shooting of the shooting ranges is determined. Note that a form in which narrow-angle camera 111 is used for shooting is also possible.

[0042] In the following process, the site resources are heavy machinery 201 to 204 and workers (people) 301 to 304 as shown in FIG.

[0043] When the process is started, the optical axis of the surveying instrument 100 is directed to the first imaging range among the previously set imaging ranges (step S101). Next, continuous imaging using the wide-angle camera 106 is started (step S102). This imaging is repeated at specific intervals. For example, imaging is performed at intervals of 0.5 seconds or 1 second. A video may be captured, and the frame images that make up the video may be used as images from the repeated imaging.

[0044] Once continuous photography has started, one of the images is acquired as a reference image (step S103). Next, site resources are recognized from the reference image acquired in step S103. For example, assume that the heavy equipment 201 and workers 301, 302, and 303 in FIG. 1 are captured in the reference image acquired in step S103. In this case, the images of the heavy equipment 201 and the workers 301, 302, and 303 are recognized using image analysis technology.

[0045] The process of step S104 will be described in detail below. Fig. 5 is a flowchart showing the process of step S104 in detail. The process of Fig. 5 is performed on the reference image acquired in step S103.

[0046] First, an image processing algorithm that recognizes people from captured images is used to recognize people from the target image (step S201). The image processing technology for recognizing people from captured images utilizes technology developed in the fields of security and automatic driving technology.

[0047] Next, an image of the heavy equipment is recognized from the reference image acquired in step S103 (step S202). The recognition of the image of the heavy equipment is described below. Here, a method of recognizing the image of the heavy equipment from the captured image using a three-dimensional model is described. In this example, the target heavy equipment is listed in advance as candidates, and a three-dimensional model of each of the heavy equipment is acquired in advance. Methods for obtaining a three-dimensional model of the heavy equipment include a method of obtaining it from design data, a method of obtaining it from three-dimensional photo measurement, and a method of obtaining it by laser scanning.

[0048] The recognition of images of heavy machinery is carried out in stages as follows. First, elements related to the attributes of heavy machinery, such as "tracks present," "wheels present," "crane arm present," and "bucket arm present," are recognized, and images of candidate heavy machinery are recognized as a first candidate group image. Next, the first candidate group image is compared with the above-mentioned 3D model prepared in advance.

[0049] This comparison is performed as follows: (1) one of the prepared 3D models of heavy machinery is selected, and (2) an image of the selected 3D model viewed from a specific viewpoint is acquired as a comparison image.

[0050] For example, a comparison image obtained from a three-dimensional model is an image of the three-dimensional model viewed from a specific viewpoint, as shown in FIG.

[0051] Next, (3) the acquired comparison image is compared with the first group of candidates. Here, the process from (2) to (3) is repeated while changing the position of the viewpoint in (2) to search for an image that matches or is similar to the comparison image from among the first group of candidates. This search is also repeated while changing the scale.

[0052] For the 3D model selected in (1), if a matching or similar comparison image is found in the first candidate group as a result of performing the processes in (2)→(3), the 3D model of the heavy machinery on which the comparison image is based is determined to be the recognized object. On the other hand, if a matching or similar comparison image is not found in the first candidate group, another 3D model is selected and the same process is repeated.

[0053] In this way, an image corresponding to the three-dimensional model prepared in advance is searched for among the captured images obtained in step S103. Then, when an image corresponding to the three-dimensional model prepared in advance is found, it is recognized as an image of heavy machinery. The process of Fig. 5 is performed in step S104, and images of the worker and heavy machinery are recognized from the reference image obtained in step S103.

[0054] 4, after step S104, step S105 is executed. In step S105, tracking of each of the tracking objects (on-site resources) image-recognized in step S104 (there may be only one, but here it is assumed that there are multiple) is started.

[0055] Step S105 will be described in detail below. First, from among the reference images acquired in step S103, a photographed image (next photographed image) photographed immediately after the reference image (or as close as possible thereafter) is acquired, and from that image, on-site resources that have already been recognized are recognized.

[0056] For example, suppose that the heavy machinery shown in Fig. 6 is recognized in the reference image. In this case, even if the heavy machinery is in operation, the appearance of the heavy machinery on the screen does not differ significantly in the next captured image after the reference image. Therefore, the image of the heavy machinery in Fig. 6 can be easily identified in the next captured image.

[0057] As photographs are taken continuously, the position and appearance of the heavy machinery in the photographed image may gradually change. However, when comparing the nth and n+1th photographed images, for the reasons described above, it is easy to recognize the image of the heavy machinery recognized in the nth photographed image in the n+1th photographed image. Based on this principle, the image of the heavy machinery recognized in step S104 continues to be recognized in the continuously photographed images.

[0058] Using the same principle, the worker recognized in the reference image is also tracked. In this way, the tracking target (heavy machinery and the worker in this case) is tracked in the images captured continuously. In other words, the same tracking target is sequentially recognized along the time axis in multiple images that are distributed discretely on the time axis obtained by continuously capturing images.

[0059] Fig. 10 shows an example of a model diagram of three images taken continuously with the optical axis fixed. Here, (A) is taken first, then (B), and then (C). That is, the example shows the images taken in the order (A) ⇒ (B) ⇒ (C) on the time axis. Here, the heavy machinery 204 is moving from right to left on the shooting screen. Therefore, as the images progress from (A) ⇒ (B) ⇒ (C), the heavy machinery 204 gradually moves leftward.

[0060] Here, what is image-recognized as heavy machinery 204 in Fig. 10(A) is image-recognized as the same heavy machinery 204 in a screen position shifted slightly to the left in Fig. 10(B), and is image-recognized as the same heavy machinery 204 in a screen position shifted further to the left in Fig. 10(C). This is the tracking of the tracking target in the continuously captured images described above.

[0061] Furthermore, in the positioning in step S107 described later, the optical axis of the surveying instrument 100 is directed toward each positioning target. At this time, the direction of the optical axis of the wide-angle camera 106 changes, and the position of the tracked object in the photographed screen changes. However, as described above, when the nth and n+1th photographed images are compared, there is a similarity between the image of the identified object captured in the nth photographed image and the image of the identified object captured in the n+1th photographed image.

[0062] Using this, the tracking object is tracked when the direction of the optical axis of the wide-angle camera 106 changes. That is, the correspondence between the tracking object in the n-th captured image and the tracking object in the n+1-th captured image in a situation where the direction of the optical axis of the wide-angle camera 106 is changing is specified, and the tracking object of interest is tracked. The above process starts in step S105.

[0063] The tracking of the tracking object in the continuous images described above is performed for each of the multiple on-site resources. Also, during positioning, which will be described later, the optical axis of the surveying device 100 is aimed at the positioning point (the operation of aiming the surveying device 100 at the point where positioning is performed), but at that time, tracking of other tracking objects that are not the target of the collimation is performed continuously. In other words, while collimation and positioning are performed for the tracking objects one after another, the tracking of the other multiple tracking objects that have been recognized is performed without interruption.

[0064] Next, a selection is made of tracking objects that have not yet been positioned at that time (step S106). For example, in the reference image of interest at this time, workers 301-303 and heavy equipment 201 in Fig. 1 are recognized as tracking objects, and are tracked in the subsequent continuous shooting. Then, it is assumed that workers 301 and 302 have been positioned.

[0065] In this case, the worker 303 and the heavy machinery 201 are the tracked objects whose positions have not been measured. In this case, the worker 303 or the heavy machinery 201 is selected in step S106.

[0066] Next, the position of the tracked object selected in step S106 and not yet determined is determined (step S107). The position is determined by using the laser positioning function of the surveying instrument 100.

[0067] The positioning points of the tracked object are determined as follows: If the tracked object is a person, the waist area is selected as the positioning point; if the tracked object is heavy machinery, the center of gravity of the image or the largest area of ​​the body is selected as the positioning point.

[0068] Next, the position of the center of movement of the tracked object measured in step S107 is calculated (step S108). The position of the tracked object is measured on its surface, but does not necessarily reflect the movement of the tracked object. Therefore, when it is desired to grasp the movement (travel) of the heavy equipment 600 on the ground, it is desirable to obtain a parameter that can appropriately evaluate the movement of the heavy equipment 600. The center of movement is adopted as this parameter.

[0069] When the tracking target is a person, the center of movement is calculated by referring to a skeleton model prepared in advance. For example, the position of the center of gravity of the skeleton model is calculated as the center of movement. When the tracking target is heavy machinery, the center of movement is calculated by the following method.

[0070] Fig. 7 is a flowchart showing the details of the process of calculating the center of movement of the heavy equipment. In the process of Fig. 7, first, a three-dimensional model of the heavy equipment being tracked, which was used to recognize the tracked object, is obtained (step S301). Next, the positional relationship between the positioning data obtained in step S107 and the three-dimensional model is obtained (step S302).

[0071] Here, when performing laser positioning in step S107, an image is captured by narrow-angle camera 111. Since the optical axis of positioning unit 118 and the optical axis of the narrow-angle camera are on the same axis line, the center of the image captured by narrow-angle camera 111 is the positioned point. Therefore, the position of the positioned point on the tracked object is known.

[0072] Here, the relationship between the exterior orientation elements of the narrow-angle camera 111 and the wide-angle camera 106 is known. Therefore, the corresponding relationship between the image captured by the narrow-angle camera 111 and the image captured by the wide-angle camera 106 can be obtained. Also, the relationship between the image of the tracked object captured by the wide-angle camera 106 and its three-dimensional model is determined in step S104. Therefore, the relationship between the point positioned in step S107 and the three-dimensional model is known. In other words, it is known which part on the three-dimensional model the point positioned in step S107 corresponds to. The positional relationship between the positioned point and the three-dimensional model is acquired in step 302.

[0073] That is, the relationship between the exterior orientation elements of the narrow-angle camera 111 and the wide-angle camera 106 and the exterior orientation elements of the optical system of the positioning unit 118 is known, and therefore the position (element 2) of the reflection point of the positioning light in the image (element 1) of the heavy equipment captured by the wide-angle camera 106 is obtained as the first relationship. On the other hand, from the relationship of the above exterior orientation elements, a three-dimensional model (element 3) corresponding to the image (element 1) of the heavy equipment captured by the wide-angle camera 106 is obtained as the second relationship.

[0074] That is, the relationship between element 1 and element 2 is determined, and the relationship between element 1 and element 3 is determined. This determines the relationship between element 2 and element 3. That is, the relationship between the position of the reflection point of the positioning light in the heavy machinery of interest (element 2) and the three-dimensional model (element 3) is obtained. That is, it is determined which part of the three-dimensional model (element 3) the point (element 2) positioned in step S107 corresponds to.

[0075] Next, based on the positional relationship between the positioning point obtained in step S302 and the three-dimensional model, the position of the center of movement of the three-dimensional model of the target object is calculated (step S303).

[0076] As is clear from Fig. 6, the part that serves as the mobile base of the heavy equipment can be grasped in the three-dimensional model. In the case of Fig. 6, the part equipped with the caterpillar tracks is mobile base 601. The center of movement can be any point fixed to the mobile base of the heavy equipment, but for example, the position of the center of gravity of the mobile base is calculated as the center of movement.

[0077] According to step S108, the position information of the base part moving on the caterpillar track (for example, the moving base 601 of the heavy equipment 600) is obtained, and the information of the movement (travel) of the heavy equipment can be accurately evaluated. The position of this moving center is treated as the position data of the tracked object.

[0078] Once the position of the center of movement of the tracked object is obtained, the time when the position is obtained is acquired (step S109). Here, the time when the positioning process of step S107 is performed is acquired.

[0079] Next, the presence or absence of an unlocated tracked object is determined in the reference image acquired in step S103 (step S110). If an unlocated tracked object is present, the latest captured image at that point in time is acquired (step S111), and the processes from step S106 onwards are repeated.

[0080] In step S110, if there is no tracked object whose position has not been determined, the surveying instrument 100 is directed to the next shooting range, a reference image of the next shooting range is acquired (step S103), and the processes from step S103 onwards are repeated.

[0081] For example, suppose that a first shooting range, a second shooting range, and a third shooting range are set as the shooting ranges. In this case, the process from step 103 in Fig. 4 onwards is repeated in the order of the first shooting range ⇒ the second shooting range ⇒ the third shooting range ⇒ the first shooting range ⇒ the second shooting range ⇒ the third shooting range ⇒...

[0082] As a result, for each of the tracked objects in the first shooting range to the third shooting range, position information on the time axis can be obtained, albeit discontinuously, such as the position of the moving center at the first time, the position of the moving center at the second time, the position of the moving center at the third time, etc.

[0083] FIG. 8(A) shows a diagram in which the positions of each tracked object obtained by the first round of processing are plotted on a map, FIG. 8(B) shows a diagram in which the positions of each tracked object obtained by the second round of processing are plotted on a map, and FIG. 8(C) shows a diagram in which the positions of each tracked object obtained by the third round of processing are plotted on a map.

[0084] In Figure 8(A)-(C), the positions of heavy equipment (□), heavy equipment (△), heavy equipment (○), heavy equipment (▽), and heavy equipment (◇)) are plotted on a map. The shift in the plotted points in Figure 8(A)-(C) indicates that the heavy equipment has moved.

[0085] The time when the positions of the heavy equipment □ and the heavy equipment △ in Figure 8(A) were acquired is different. This is because the positioning is performed for the tracked objects in order, and the movement center is calculated. This is the same for the other heavy equipment.

[0086] Fig. 9 is a superimposition of Fig. 8 (A) to (C). On the map in Fig. 9, arrows are displayed so that the movement of each piece of heavy machinery can be seen over time. The movement of each piece of heavy machinery can be seen from Fig. 9. In other words, Fig. 9 shows the movement transition of each tracked object, that is, the operating status.

[0087] 2. Second embodiment On-site resources may be identified using code indication. For example, workers wear helmets and reflective vests (vests with reflective material) to ensure safety when they work. An identification mark is attached to the helmet or reflective vest, and the worker is identified by image recognition. Identification marks may be in the form of letters, bar codes, figures, or colors.

[0088] In the case of heavy machinery, the identification mark shall be displayed in a conspicuous part of the machinery. The form of the identification mark shall be the same as that of the reflective vest. In addition, a pole with an identification mark may be installed on the top of the heavy machinery so that it can be identified from any direction.

[0089] 3. Third embodiment In the second and subsequent measurements, the image obtained in the first round may be used to identify the tracked object.

[0090] 4.Other FIG. 1 shows a civil engineering work site as an example of a construction site, but the present invention is applicable to acquiring the operating status of on-site resources at construction sites such as buildings, factories, and various facilities. [Explanation of symbols]

[0091] 100: surveying equipment, 101: tripod, 102: base, 103: horizontal rotation section, 104: vertical rotation section, 105: telescope, 106: wide-angle camera, 107: docking section, and 108: touch panel display.

Claims

1. A camera that continuously photographs a heavy machine that performs work at a construction site and has a base part for traveling and a movable part that moves on the base part; A recognition means for recognizing the heavy machinery from the captured image obtained by the photographing; A tracking means for tracking the recognized image of the heavy machinery in the plurality of captured images obtained by the continuous photographing; A positioning means for aiming at the heavy machinery to be tracked and measuring the position of the heavy machinery using a laser light; a moving center calculation means for calculating a point fixed to the base unit as a moving center, instead of the measured point, based on a result of the positioning; having A measurement system in which the positioning is performed multiple times at intervals.

2. The measurement system according to claim 1, further comprising a map generating means for displaying a progress of the movement of the heavy machinery.

3. The recognition of the heavy machinery is An image captured by the camera; and A 3D model of the heavy machinery to be recognized is prepared in advance and images are taken from multiple different viewpoints. The measurement system according to claim 1 , wherein the measurement is performed by comparing:

4. The relationship between the exterior orientation elements of the camera and the exterior orientation elements of the positioning means is known; Based on the relationship, the position of the reflection point of the laser light in the image of the heavy equipment captured by the camera is calculated as a first relationship; By the comparison, a three-dimensional model corresponding to the image of the heavy equipment captured by the camera is obtained as a second relationship; A position of a reflection point of the laser light in a three-dimensional model of the heavy equipment is obtained based on the first relationship and the second relationship; The measurement system according to claim 3 , wherein the center of movement is calculated based on the position of a reflection point of the laser light on a three-dimensional model of the heavy machinery.

5. The heavy machinery is a plurality of machines, The tracking is performed for each of the plurality of heavy machinery; The continuous imaging is continued while the collimation is being performed, The measurement system of claim 1 , wherein the tracking is performed on the heavy equipment that is not the target of the collimation while the collimation is being performed.

6. A step of continuously photographing a heavy machine that performs work at a construction site and has a base part for traveling and a movable part that moves on the base part, using a camera; a recognition step of recognizing the heavy machinery from among the captured images obtained by the photographing; a tracking step of tracking the recognized image of the heavy machinery in a plurality of captured images obtained by the continuous photographing; a positioning step of aiming at the heavy machinery to be tracked and measuring the position of the heavy machinery using a laser light; a movement center calculation step of calculating a point fixed to the base unit as a movement center, instead of the measured point, based on the result of the positioning; having A measurement method in which the positioning is performed multiple times at intervals.

7. A measurement program executed by a computer, To your computer A step of continuously photographing a heavy machine that performs work at a construction site and has a base part for traveling and a movable part that moves on the base part, using a camera; a recognition step of recognizing the heavy machinery from among the captured images obtained by the photographing; a tracking step of tracking the recognized image of the heavy machinery in a plurality of captured images obtained by the continuous photographing; a positioning step of aiming at the heavy machinery to be tracked and measuring the position of the heavy machinery using a laser light; a movement center calculation step of calculating a point fixed to the base unit as a movement center, instead of the measured point, based on the result of the positioning; Run the command, A measurement program in which the positioning is performed multiple times at intervals.

Images