Measuring data processing apparatus, measuring data processing method, and program for measuring data processing

Abstract

To realize safety management of heavy machinery with versatility and simplicity.SOLUTION: Heavy machines 201 to 204 and workers 301 to 304 are photographed by a camera of a surveying device 100; image recognition of heavy machines and workers in a captured image is performed; the image-recognized heavy machines and workers are positioned by the surveying device 100 using laser light; the distance between the positioned heavy machine and the worker is calculated; when the calculated separation distance is less than the threshold, the warning is issued to heavy machines 201 to 204 and workers 301 to 304.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for ensuring the safety during the operation of heavy machinery.

Background Art

[0002] In the operation of heavy machinery (construction machinery), it is necessary to ensure the safety of the surrounding area. Generally, personnel for safety monitoring are arranged and corresponding methods are adopted. Patent Document 1 describes a technique for estimating the distance between a heavy machine and a person based on an image captured by a camera arranged on the heavy machine and ensuring safety.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique described in Patent Document 1 requires a camera to be mounted on the heavy machine, and a plurality of cameras are required to ensure the safety around the heavy machine. Therefore, it lacks versatility and simplicity.

[0005] Against such a background, an object of the present invention is to obtain a technique for performing safety management of heavy machinery having versatility and simplicity.

Means for Solving the Problems

[0006] The present invention includes an image data acquisition unit that acquires image data of a captured image of a heavy machine and other objects, an image recognition unit that performs image recognition of the heavy machine and the other objects shown in the captured image, a positioning unit that performs positioning using laser light between the image-recognized heavy machine and the other objects, a separation distance calculation unit that calculates the separation distance between the heavy machine and the other objects measured using the laser light, and a notification unit that gives a warning when the separation distance is equal to or less than a threshold value. Based on the captured image, the separation distance between the heavy machine and the other object is estimated, and based on the estimated separation distance, positioning of the heavy machine and the other object using the laser light is performed It is a survey data processing device.

[0007] In the present invention, it is preferable to include a storage unit that stores the threshold value set according to the type or model number of the heavy machine, and a specifying unit that specifies the type or model number of the heavy machine, and the threshold value is selected based on the specified content.

[0008] In the present invention, it is preferable that the separation distance between the heavy machine and the other objects is estimated based on the captured image, and the positioning of the heavy machine and the other objects using the laser light is performed based on the estimated separation distance.

[0009] In the present invention, it is preferable that when the estimated separation distance becomes equal to or less than a specific value, the positioning of the heavy machine and the other objects using the laser light is performed. In the present invention, the estimation of the separation distance is based on the distance from the shooting position to the heavy machine and the direction of the heavy machine from the shooting position based on the size of the image of the heavy machine in the captured image, and the distance from the shooting position to the other object and the direction of the other object from the shooting position based on the size of the image of the other object in the captured image.

[0010] In the present invention, it is preferable that the conditions in the process of estimating the separation distance between the heavy machine and the other objects based on the captured image are corrected based on the result of the positioning of the heavy machine and the other objects using the laser light. In the present invention, an example of the condition is the dimension of the predetermined heavy machine and / or the other object.

[0012] The present invention executes acquisition of image data of a captured image of a heavy machine and other objects, image recognition of the heavy machine and the other objects shown in the captured image, positioning using laser light between the heavy machine and the other objects that have been image recognized, calculation of the separation distance between the heavy machine and the other objects that have been positioned using the laser light, and notification of a warning when the separation distance is equal to or less than a threshold value, Based on the captured image, the separation distance between the heavy machine and the other object is estimated, and based on the estimated separation distance, positioning of the heavy machine and the other object using the laser light is performed and can also be understood as a survey data processing method.

[0013] The present invention is a program that can be read and executed by a computer, and causes the computer to execute acquisition of image data of a captured image of a heavy machine and other objects, image recognition of the heavy machine and the other objects shown in the captured image, positioning using laser light between the heavy machine and the other objects that have been image recognized, calculation of the separation distance between the heavy machine and the other objects that have been positioned using the laser light, and notification of a warning when the separation distance is equal to or less than a threshold value, Based on the captured image, the separation distance between the heavy machine and the other object is estimated, and based on the estimated separation distance, positioning of the heavy machine and the other object using the laser light is performed and can also be understood as a program for survey data processing.

Advantages of the Invention

[0014] According to the present invention, a technique for performing safety management of a heavy machine having versatility and simplicity can be obtained.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0016] 1. First Embodiment (Overview) The outline of the embodiment is shown in FIG. 1. FIG. 1 shows a surveying device 100. The surveying device 100 is a total station having a camera, a laser positioning function, and an automatic tracking function for the survey target.

[0017] FIG. 1 shows construction machines 201 to 204 and workers 301 to 304 performing civil engineering work. The surveying device 100 performs safety management of the construction machines 201 to 204 and the workers 301 to 304. Specifically, the surveying device 100 performs monitoring to prevent interference between the construction machines 201 to 204 and the workers 301 to 304, and monitoring to prevent interference between the construction machines. In this example, the risk of the above interference is evaluated using the surveying function provided in the surveying device 100, and notification is performed when it is determined that there is a concern about the risk.

[0018] The above processing can be executed by arranging the surveying device 100 and is performed automatically. Therefore, it is possible to perform safety management of construction machines with versatility and simplicity.

[0019] (Surveying device) FIG. 2 is a perspective view (A) and (B) of the surveying device 100. (A) is a perspective view seen from the front side, and (B) is a perspective view seen from the back side. The surveying device 100 includes a base portion 122 fixed on a tripod 121, a horizontal rotation portion 123 capable of horizontal rotation on the base portion 122, and a vertical rotation portion 124 held in a state capable of vertical rotation (elevation angle control and depression angle control) on the horizontal rotation portion 123.

[0020] The horizontal rotation and the vertical rotation are performed by a motor. The horizontal angle of the horizontal rotation portion 123 (the pointing direction in the horizontal direction of the optical axis of the telescope 105) and the vertical angle of the vertical rotation portion 124 (the elevation angle or the depression angle of the optical axis of the telescope 125 (telescope camera 102)) are precisely measured by an encoder.

[0021] On the front surface of the vertical rotation unit 124, a telescope 125 and a wide-angle camera 101 are arranged. On the back surface, a shore approach portion 126 of the telescope 125 and a touch panel display 128 are arranged. The telescope 125 also serves as the optical system of the telephoto camera 102 shown in FIG. 3. Further, laser light for distance measurement is irradiated outward through the objective lens of the telescope 125, and the reflected light is received.

[0022] The touch panel display 128 is an operation panel and display of the surveying device 100. Various types of information related to the operation of the surveying device 100 and information related to the surveying results are displayed on the touch panel display 128.

[0023] (Block diagram of the surveying device) FIG. 3 is a functional block diagram of the surveying device 100. The surveying device 100 includes a wide-angle camera 101, a telephoto camera 102, a camera control unit 103, a drive control unit 104, an image data acquisition unit 105, an image recognition unit 106, a heavy machine identification unit 107, a warning distance acquisition unit 108, a positioning unit 109, a rough separation distance calculation unit 110, a warning distance determination unit 111, a separation distance calculation unit 112, a warning distance determination unit 113, a notification unit 114, a data storage unit 115, and a communication device 116.

[0024] Some of the functional units such as the camera control unit 103, the drive control unit 104, the image data acquisition unit 105, the image recognition unit 106, the heavy machine identification unit 107, the warning distance acquisition unit 108, the positioning unit 109, the rough separation distance calculation unit 110, the warning distance determination unit 111, the separation distance calculation unit 112, the warning distance determination unit 113, the notification unit 114, and the data storage unit 115 are realized by a computer.

[0025] The computer includes a CPU, a storage device, and an interface. An operation program for executing the functions of the above functional units is read and executed by the computer, thereby realizing the above functional units. A form in which some or all of the functional units are realized by dedicated hardware is also possible.

[0026] In addition, it is also possible to implement some or all of the above functional units on an external computer. In this case, the surveying device 100 and the computer are connected by a communication line, the surveying device 100 is remotely operated by the computer, surveying data is transmitted from the surveying device 100 to the computer, and some or all of the processes described later are performed on the computer.

[0027] The wide-angle camera 101 is a digital still camera that captures images over a relatively wide range. The telephoto camera 102 captures relatively narrow-angle telephoto images via the telescope 125. Both can capture still images and moving images.

[0028] The relationship of the external calibration elements (position and orientation) of the optical systems of the wide-angle camera 101, the telephoto camera 102, and the positioning unit 109 (to be described later) in the surveying device 100 is known. The optical axes of the optical systems of the telephoto camera 102 and the positioning unit 109 are on the same axis (on the optical axis of the telescope 125). The optical axis of the wide-angle camera 106 has a positional relationship parallel to the optical axes of the optical systems of the narrow-angle camera 111 and the positioning unit 118 (the optical axis of the telescope 105).

[0029] The camera control unit 103 controls the operations of the wide-angle camera 101 and the telephoto camera 102. Specifically, the camera control unit 102 controls the timing of shooting, magnification, shutter speed, and other shooting conditions.

[0030] The drive control unit 104 sends a control signal to a drive circuit that drives a motor for rotating the horizontal rotation unit 123, and controls the horizontal rotation of the horizontal rotation unit 123. In addition, the drive control unit 104 sends a control signal to a drive circuit that drives a motor for rotating the vertical rotation unit 124, and controls the vertical rotation of the vertical rotation unit 124.

[0031] The image data acquisition unit 105 acquires the image data of the images captured by the wide-angle camera 101 and the telephoto camera 102. The image recognition unit 106 performs the process of step S101 in FIG. 4. The heavy equipment identification unit 107 performs the process of step S102 in FIG. 4. The warning distance acquisition unit 108 performs the process of step S103 in FIG. 4.

[0032] The positioning unit 109 performs positioning using laser light (distance measuring light). The positioning unit 109 includes a light emitting unit for distance measuring light, a light receiving unit thereof, their optical systems, a circuit related to light emission, and a circuit related to light reception. The positioning unit 109 also includes a circuit and an arithmetic unit for performing calculations for distance measurement, and an arithmetic unit for calculating the position of a reflection point (measurement point) based on the measured distance value and the direction of the optical axis of the distance measuring light.

[0033] The positioning of the measurement point is calculated from the distance to the measurement point measured by the distance measuring light and the direction of the optical axis of the distance measuring light. The distance is calculated using the principle of light wave distance measurement. There are a method using the phase difference of the received distance measuring light and a method using the propagation time for calculating the distance. In this example, distance measurement is performed using the method of using the phase difference.

[0034] In the method using the phase difference, a reference optical path is provided in the positioning device (in this case, the surveying device 100), and the distance from the surveying device 100 to the measurement point is calculated from the difference (phase difference) between the light reception timing of the distance measuring light propagating through this reference optical path and the light reception timing of the distance measuring light reflected from the measurement point. In the method using the propagation time, the distance from the surveying device 100 to the measurement point is calculated based on the time until the distance measuring light hits the measurement point and is reflected and returned.

[0035] The direction of the measurement point as seen from the surveying device 100 (the direction of the optical axis of the distance measuring light) is obtained by measuring the rotation angles of the horizontal rotation unit 123 and the vertical rotation unit 124. The rotation angles of the horizontal rotation unit 123 and the vertical rotation unit 124 are precisely measured by an encoder.

[0036] By knowing the distance from the surveying device 100 to the measurement point and its direction, the position (coordinates) of the measurement point in the coordinate system with the surveying device 100 as the origin is calculated. Here, if the external calibration elements (position and orientation) of the surveying device 100 in the absolute coordinate system are known, the position in the absolute coordinate system can be obtained. The absolute coordinate system is the coordinate system used in maps and GNSS, and its position is described by, for example, latitude, longitude, and altitude.

[0037] The approximate distance calculation unit 110 performs the process of step S105 in FIG. 4. The warning distance determination unit 111 performs the process of step S106 in FIG. 4. The distance calculation unit 112 performs the process of step S109 in FIG. 4. The warning distance determination unit 113 performs the process of step S110 in FIG. 4. The notification unit 114 performs the process of step S111 in FIG. 4.

[0038] The data storage unit 115 stores data necessary for the operation of the surveying device 100, an operation program, and data obtained as a result of surveying. The communication device 116 communicates with external devices. The communication is performed using a wireless LAN standard or a telephone line.

[0039] (An example of the processing procedure) FIG. 4 shows an example of the processing procedure performed by the surveying device 100. Hereinafter, an example of performing safety management of heavy equipment using the surveying device 100 will be described. Here, a process for preventing interference between the heavy equipment and the worker will be described.

[0040] The program for executing the process in FIG. 4 is stored in a suitable storage medium and executed by the CPU of the computer built in the surveying device 100. It is also possible to perform the process in FIG. 4 on a PC or a server. It is also possible to store the program for executing the process in FIG. 4 in a server and download and use it.

[0041] The following process starts with the surveying device 100 installed at the construction site as shown in FIG. 1. Prior to the process, it is assumed that the external calibration elements (position and orientation) of the surveying device 100 in the absolute coordinate system are obtained and are known.

[0042] The process in FIG. 4 starts while the heavy equipment is operating and the construction site where the worker is working is continuously photographed by the wide-angle camera 101 and / or the telephoto camera 102 of the surveying device 100. The continuous photographing is performed at intervals of about 0.5 seconds to 5 seconds. It is also possible to take a video and use its frame images as continuous photographed images.

[0043] The processing in FIG. 4 is performed on each of the continuously captured images. If the calculation cannot keep up with the interval at which the captured images are obtained, the processing is performed on each image obtained at intervals such as the 1st, 3rd, 5th, etc.

[0044] When the processing is started, first, recognition of images of a person and a heavy machine is performed from the captured images (step S101). This processing is performed using known image recognition software. This processing is performed by the image recognition unit 106.

[0045] Here, a worker involved in construction is assumed as the person. Detection of a person's image is performed using a known image detection algorithm for detecting a person's image. Recognition of a heavy machine's image is performed by preparing a reference image of the heavy machine assumed in advance and comparing it with that.

[0046] Also, in this example, each worker wears a vest and a helmet with an identification display, and each heavy machine has an identification display. In the above image recognition, this identification information is also acquired and associated with the image information.

[0047] Here, when a plurality of people are image-recognized and a plurality of heavy machines are image-recognized, one set among them is selected and the processing in FIG. 4 is executed. For other sets, the processing in FIG. 4 is executed in parallel for each set.

[0048] For example, it is assumed that workers 301 to 303 and a heavy machine 201 are shown in the captured image and they are image-recognized. In this case, the processing in FIG. 4 is performed in parallel for each of the sets of worker 301 and heavy machine 201, worker 302 and heavy machine 201, and worker 303 and heavy machine 201.

[0049] Next, the heavy machine is specified (step S102). Specifying the heavy machine has forms of specifying the type of the heavy machine and specifying the model number of the heavy machine. The type of the heavy machine refers to types classified by structure or use such as a hydraulic excavator, a bulldozer, and a self-propelled crane. The model number of the heavy machine is the model of the heavy machine identified by an identification number or name given by a manufacturer or a sales maker.

[0050] When a heavy machine is identified, the warning distance of the identified heavy machine is obtained (step S103). The warning distance is the minimum value of the distance between the heavy machine and a person that can ensure safety. The warning distance varies depending on the type and model of the heavy machine. The warning distance is preset and stored in an appropriate storage area.

[0051] In this example, the warning distance corresponding to the type of the heavy machine and the warning distance corresponding to the model number of the heavy machine are preset. And when the model number can be recognized, the warning distance corresponding to the model number is obtained. Even if the model number cannot be recognized, when the type of the heavy machine can be recognized, the warning distance corresponding to the type of the heavy machine is obtained.

[0052] Next, the positioning of the person and the heavy machine recognized by image recognition in step S101 is performed (step S104). This process is performed using the laser positioning function of the surveying device 100. The positioning of the person is performed by aiming at the waist part. The positioning of the heavy machine is performed by aiming at the image center or the part of the vehicle body.

[0053] Next, the approximate separation distance is calculated (step S105). The approximate separation distance is the separation distance between the person and the heavy machine estimated from the captured image. The approximate separation distance is an estimated value and contains errors.

[0054] There are two methods to obtain the approximate separation distance. First, the first method will be described. In the first method, the approximate separation distance is obtained from the separation distance between the person and the heavy machine on the captured screen.

[0055] First, the on-screen separation distance, which is the separation distance between the person and the heavy machine on the captured screen, is obtained. The on-screen separation distance is grasped by the number of pixels. Next, based on the three-dimensional positions of both obtained in step S104, the actual separation distance between both is calculated. And the relationship between the on-screen separation distance and the actual separation distance is obtained.

[0056] Based on this relationship, the approximate separation distance is obtained from the on-screen separation distance. In this case, if the on-screen separation distance becomes half from the initial stage, it is estimated that the approximate separation distance also becomes half. This method is simple, but it cannot evaluate the distance changes in the depth direction of the photographed screen, that is, the direction away from the camera and the direction approaching the camera.

[0057] Next, a second method for obtaining the approximate separation distance will be described. In the second method, it is possible to capture the distance changes in the depth direction of the photographed screen, that is, the direction away from the camera and the direction approaching the camera.

[0058] Hereinafter, the details of the second method for obtaining the approximate separation distance will be described. First, previously, the positioning data of the heavy machine and the person respectively recognized in the photographed image have been obtained (step 104). This positioning is performed with high accuracy by the positioning function using the laser light provided in the surveying device 100.

[0059] In this state, the size of the object recognized in the image is associated with the distance information obtained by the positioning. Here, the objects are the heavy machine and the person recognized by image recognition.

[0060] The size of the object in the image is obtained by counting the number of pixels. Also, pay attention to the part where the dimension of the object is known or can be inferred, and set the information of the actual dimension as an estimated value. For example, in the case of a person, assume that the dimension in the height direction is 170 cm. Also, in the case of a heavy machine, adopt the average ground height in the type of the heavy machine or the value of the ground height on the catalog data based on the obtained model number.

[0061] Also, in the image, when a part with a known length can be recognized, it is also possible to obtain the dimension of that part and associate it with the recognized image.

[0062] By obtaining the actual dimensions of the parts recognized by image recognition, the approximate relationship between the number of pixels in the photographed image and the actual dimensions of the objects shown is obtained. This is done for both the person and the heavy machine.

[0063] Here, in the camera to be used, the relationship between the distance to the object to be photographed and the size (number of pixels) of its image is obtained in advance. This can be obtained as data that quantitatively grasps the fact that objects closer appear larger and those farther away appear smaller.

[0064] Specifically, for a gauge with a reference length (for example, 1 m), a calibration curve is obtained in advance with the distance from the camera to the gauge on the vertical axis and the number of pixels in the photographed image obtained when viewing the gauge from the front on the horizontal axis. This calibration curve is described, for example, in Japanese Patent Laid-Open No. 2018-13343.

[0065] Using this calibration curve, the distance from the camera to the recognition target part is obtained based on the photographed image. For example, focus on the images of a person and a heavy machine recognized in the photographed image. First, consider the time when the calculation of the approximate separation distance in step S105 starts.

[0066] At this time, the number of pixels N in the height direction of the person in the photographed image can be known. Here, assume that the height of the person being focused on is set to h = 170 cm. Also, assume that the above calibration curve for a 1 m gauge has been obtained. In this case, by applying the number of pixels N×1.7 to this calibration curve, the distance D corresponding to the number of pixels N is obtained.

[0067] On the other hand, in step S104, a high-precision distance D0 to this person has been obtained. Here, if D = D0, it means that the setting of h = 170 cm was appropriate.

[0068] If D≠D0, the set value of h is changed so that D = D0. That is, it is determined that it was not appropriate to set the height (height) of the person shown in the photographed image to 170 cm, and the set value of the person's height is corrected so that D = D0. In this way, the calibration of the calculated value (estimated value) of the distance from the camera (surveying device 100) to the object obtained from the photographed image is performed.

[0069] The same process is also carried out for construction machinery, and the distance from the camera (surveying device 100) based on the captured image to the construction machinery is calculated. Thus, based on the captured image of a person and construction machinery, the approximate distance from the camera (surveying device 100) to the person and construction machinery shown in the captured image is calculated.

[0070] On the other hand, the positions (screen positions) of the images of the person and construction machinery on the screen of the captured image, that is, the coordinates in the screen, can be known by analyzing the image. If the position on the captured screen of the image being focused on is determined, the direction of the image as seen from the camera (shooting viewpoint) can be determined by setting a direction line connecting that point and the projection origin of the camera.

[0071] Once the direction is known, based on the direction information and the distance information obtained from the above image analysis, the positions of the person and construction machinery with the camera (surveying device 100) as the origin can be calculated. If the positions of the person and construction machinery are known, the distance (separation distance) between them can be calculated. Thus, the approximate separation distance, which is the separation distance between the person and construction machinery, is calculated (estimated) from the captured image.

[0072] The accuracy of the positions of the person and construction machinery obtained from the captured image by the above-described method is not high, but in the initial stage, it is calibrated by the positioning data obtained by the laser positioning function of the surveying device 100, so high accuracy can be obtained.

[0073] The shooting is continuously repeated, and for each of a plurality of images that are discretely distributed on the time axis, the process of obtaining the above approximate separation distance is repeatedly performed. At this time, initially, high calculation accuracy (estimation accuracy) can be obtained, but as the process is repeated, the calculation accuracy (estimation accuracy) decreases. That is, when repeatedly calculating the approximate separation distance for continuously captured images, the accuracy decreases over time.

[0074] This is because the resolution of the calibration curve showing the relationship between the number of pixels of the image and the distance is low, and as the distance becomes farther, the degree becomes more prominent. Also, as time passes, changes occur in the orientation and posture of people and heavy machinery, resulting in errors due to such influences. This problem is reduced by performing the process of step S108. Note that if the object is stationary, there is no reduction in accuracy.

[0075] After step S105, it is determined whether or not the calculated approximate separation distance is less than or equal to the warning distance obtained in step S103 (step S106). The warning distance is a distance set to a value larger than the warning distance, and is a standard for making a preliminary determination before giving a warning. The warning distance may be set for each type of heavy machinery or for each heavy machinery, or may be set to a specific value.

[0076] As an example of setting the warning distance for each type of heavy machinery or for each heavy machinery, for example, it may be set to the warning distance of the corresponding heavy machinery + 3 m. As an example of setting a uniform warning distance, an example is given where the warning distance is set to a value exceeding the maximum value of the warning distances among the assumed heavy machinery.

[0077] The warning distance may be variably set according to the situation. When the distance to the object becomes farther, the change in the number of pixels with respect to the change in distance (the change in the apparent image size in the captured image) becomes smaller, and the resolution of the estimated distance decreases. That is, as the distance becomes farther, the accuracy of the approximate separation distance decreases. Therefore, for an object farther than a specified distance, a larger value of the warning distance is taken to ensure a margin regarding ensuring safety.

[0078] In step S106, if the approximate separation distance is less than or equal to the warning distance, the process proceeds to step S107, and if not, the process proceeds to step S108. In step S107, the positioning of the person and the heavy machinery that are the objects here is performed. This positioning is performed using the laser positioning function of the surveying device 100.

[0079] In step S108, it is determined whether or not a prescribed time has elapsed since the start time when the process of step S105 after step S104 was first started at this stage. Here, if the prescribed time has elapsed, the processes from step S104 and below are repeated, and if the prescribed time has not elapsed, the processes from step S105 and below are repeated. The prescribed time is selected, for example, from around 2 seconds to 60 seconds. By performing the determination in step S108, step S104 is periodically executed, and an increase in the error of the distance calculation based on the captured image is suppressed.

[0080] Also, by performing the process of step S108, the frequencies of the laser positioning in steps S104 and S107 can be suppressed. If the frequency of the laser positioning is high, an operation involving fine adjustment of the optical axis of the surveying device 100 is required each time, and it becomes necessary to finely move the movable part of the surveying device 100. This leads to an increase in power consumption and a decrease in practicality. By performing the process of step S108, this problem can be suppressed.

[0081] After step S107, the separation distance between the person and the heavy machine, which is the target here, is calculated based on the positioning data obtained in step S107 (step S109). The calculation of the separation distance here is performed by calculating the distance between the position of the person and the position of the heavy machine obtained in step S107. Next, it is determined whether or not the separation distance calculated in step S109 is equal to or less than the warning distance (step S110).

[0082] In step S110, when the separation distance calculated in step S109 is equal to or less than the warning distance, a warning is issued here indicating that the separation distance between the person and the heavy machine, which is the target, is equal to or less than the warning distance (step S111). In step S110, when the separation distance calculated in step S109 is not equal to or less than the warning distance, the process proceeds to step S108.

[0083] The notification in step S111 is performed wirelessly. In this example, the operator carries a wireless device, and the heavy equipment is equipped with a wireless device. The above notification is performed on this wireless device by wireless communication. When this notification is made, a warning sound is output from the corresponding wireless device, which is recognized by the operator or the heavy equipment operator. It is also possible to use a form in which a warning lamp blinks, a form in which notification is made by vibration, etc. There is also a method of generating a warning sound in the target heavy equipment.

[0084] Here, the heavy equipment and the wireless device mounted on it are individually identified, and the wireless device carried by the operator can also be individually identified. For example, assume that the heavy equipment 201 in FIG. 1 and the operators 301 to 303 are the objects of monitoring, and the process in FIG. 4 is being performed. Here, the heavy equipment 201 is identified from other heavy equipment in the captured image, and the wireless device mounted on it is also identified. Also, the operators 301 to 303 can be identified in the captured image, and furthermore, the wireless devices they carry are also identified.

[0085] Here, assume that the separation distance between the heavy equipment 201 and the operator 301 has become equal to or less than the warning distance. In this case, notification is made to the wireless device of the heavy equipment 201 and the wireless device carried by the operator 301. Of course, it is also possible to make all the objects being monitored the objects of notification.

[0086] (Others) The objects of monitoring are not limited to combinations of people and heavy equipment, and combinations such as heavy equipment and heavy equipment, heavy equipment and buildings, heavy equipment and standing trees, heavy equipment and natural structures such as cliffs, heavy equipment and other machines (generators, etc.), and heavy equipment and materials are also possible.

Description of Reference Numerals

[0087] 100… Measuring device, 101… Wide-angle camera, 102… Telephoto camera, 121… Tripod, 122… Base part, 123… Horizontal rotation part, 124… Vertical rotation part, 125… Telescope, 126… Eyepiece part of the telescope, 128… Touch panel display

Claims

1. An image data acquisition unit that acquires image data of a captured image of a heavy machine and other objects; An image recognition unit that performs image recognition of the heavy machine and the other objects shown in the captured image; A positioning unit that performs positioning of the heavy machine and the other objects using laser light after the image recognition; A separation distance calculation unit that calculates the separation distance between the heavy machine and the other objects measured using the laser light; A notification unit that notifies a warning when the separation distance is equal to or less than a threshold value and based on the captured image, the separation distance between the heavy machine and the other objects is estimated, A survey data processing device that performs positioning of the heavy machine and the other objects using the laser light based on the estimated separation distance.

2. A storage unit that stores the threshold value set according to the type or model number of the heavy machine; An identification unit that identifies the type or model number of the heavy machine and The survey data processing device according to claim 1, wherein the threshold value is selected based on the identified content.

3. The survey data processing device according to claim 1, wherein when the estimated separation distance becomes equal to or less than a specific value, positioning of the heavy machine and the other objects using the laser light is performed.

4. The estimation of the separation distance is based on the size of the image of the heavy machine in the captured image, the distance from the shooting position to the heavy machine, the direction of the heavy machine from the shooting position, based on the size of the image of the other object in the captured image, the distance from the shooting position to the other object, and the direction of the other object from the shooting position and is performed based on claim 1 or 3. The survey data processing device described.

5. The survey data processing device according to claim 1, wherein the conditions in the process of estimating the separation distance between the heavy machine and the other objects based on the captured image are corrected based on the result of positioning the heavy machine and the other objects using the laser light.

6. The survey data processing device according to claim 5, wherein the condition is the dimension of the heavy machine and / or the other object determined in advance.

7. Acquisition of image data of a captured image of a heavy machine and other objects, Image recognition of the heavy machine and the other objects shown in the captured image, Positioning of the heavy machine and the other objects using laser light after the image recognition, Calculation of the separation distance between the heavy machine and the other objects measured using the laser light, Notification of a warning when the separation distance is equal to or less than a threshold value Execute, Based on the captured image, the separation distance between the heavy machine and the other object is estimated, A surveying data processing method for performing positioning of the heavy machine and the other object using the laser light based on the estimated separation distance.

8. A program that can be read and executed by a computer, Causes the computer to Acquire image data of a captured image of a heavy machine and another object, Perform image recognition of the heavy machine and the other object shown in the captured image, Perform positioning of the heavy machine and the other object using laser light based on the image recognition, Calculate the separation distance between the heavy machine and the other object positioned using the laser light, Notify a warning when the separation distance is equal to or less than a threshold value And execute, Based on the captured image, the separation distance between the heavy machine and the other object is estimated, A program for processing surveying data for performing positioning of the heavy machine and the other object using the laser light based on the estimated separation distance.

Images