Construction machinery motion control system

Abstract

To provide an operation control system of construction machine capable of designating a construction machine of an operation object more surely, and capable of confirming, more easily in a shorter time, whether or not a command to the correct operation object is issued.SOLUTION: Based on a predetermined position in a peripheral image designated by an operation device by an operator, the direction of a construction management terminal detected by a terminal posture detection device and the position of the construction management terminal detected by the terminal position detection device, a position of a designated region corresponding to the predetermined position in a construction site is calculated. Based on the position of the designated region and the positional information of a construction machine detected by a machine position detection device, it is determined whether or not the construction machine exists in the designated region. In the case where it is determined that the construction machine exists in the designated region, stop control information is created for stopping the construction machine as the control information, and the stop control information is outputted to the machine control device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a construction machine operation control system that is applied to construction machines such as hydraulic excavators. [Background technology]

[0002] As a technology for automatically operating multiple construction machines that perform different tasks at a construction site, for example, Patent Document 1 discloses a construction machine construction method in which multiple construction machines that perform different tasks have automatic driving functions and the multiple construction machines are managed by a construction management unit, the construction machine construction method comprising a construction information transmission process in which the construction management unit transmits construction position information for each construction machine to the construction machine, a construction information reception process in which the construction machine receives the construction position information transmitted in the construction information transmission process for each construction machine, and an automatic driving work process in which the construction machine performs work in automatic driving using the construction position information received in the construction information reception process for each construction machine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2016-132912 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, the construction management unit transmits construction position information to each of the multiple construction machines at the construction site. The multiple construction machines at the construction site perform work by automatic driving based on the construction position information they receive. This allows the multiple construction machines to operate by automatic driving under the management of the construction management unit.

[0005] Furthermore, if an operator operating the construction system visually inspects the area where the construction machinery is working and recognizes that an unusual situation has occurred, the operator issues a command to the construction machinery to stop working according to the situation, i.e., a work stop command. The work stop command is sent from the construction management unit to each construction machinery via a transmitter / receiver device, causing the construction machinery to stop working. This makes it possible to avoid a decline in construction efficiency due to the occurrence of an unusual situation when working with construction machinery.

[0006] However, the above-mentioned conventional technology does not explicitly indicate the correspondence between the target of a command issued in the system and the actual construction machine to which the command is to be applied. In particular, when multiple construction machines of the same type are operating in parallel, it takes time to identify the target of a command, which may reduce construction efficiency. Furthermore, because the target of a command is determined based on the operator's visual inspection, there is a risk of misjudging the target and issuing a command to the wrong construction machine, resulting in unintended results. Furthermore, to confirm whether a command has been issued to the correct target, it is necessary to wait until the target construction machine actually begins operating as instructed, which may reduce construction efficiency.

[0007] The present invention has been made in consideration of the above, and aims to provide an operation control system for construction machinery that can more reliably specify the construction machinery to be operated, and that can more easily and in a shorter time confirm whether a command has been issued to the correct operation target. [Means for solving the problem]

[0008] The present application includes a plurality of means for solving the above-mentioned problems, and one example thereof is a construction machine operation control system including a construction management terminal that creates control information for operating a construction machine, and a machine control device that controls the operation of the construction machine based on the control information, further including a terminal attitude detection device that detects the orientation of the construction management terminal, a terminal position detection device that detects the position of the construction management terminal, and a machine position detection device that detects position information of the construction machine, wherein the construction management terminal includes a camera that photographs a construction site, a display device that displays the construction site photographed by the camera as a surrounding image, an operation device that allows an operator to specify a predetermined position within the surrounding image displayed on the display device, and and a terminal control device, wherein the terminal control device calculates the position of a designated area corresponding to the designated position at the construction site based on the predetermined position in the surrounding image designated by the operator using the operating device, the orientation of the construction management terminal detected by the terminal attitude detection device, and the position of the construction management terminal detected by the terminal position detection device, determines whether the construction machine is present within the designated area based on the position of the designated area and the position information of the construction machine detected by the machine position detection device, and if it is determined that the construction machine is present within the designated area, creates stop control information to stop the construction machine as the control information and outputs the stop control information to the machine control device. [Effects of the Invention]

[0009] According to the present invention, the construction machine to be operated can be more reliably specified, and it can be more easily and in a shorter time confirmed whether a command has been issued to the correct operation target. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a functional block diagram showing a schematic configuration of a construction machine operation control system according to a first embodiment. [Figure 2] 3 is a flowchart showing the processing of the operation control system for a construction machine according to the first embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a construction machine operating at a construction site. [Figure 4] 1A and 1B are diagrams illustrating the basic principle of camera attitude detection. [Figure 5] FIG. 3 is a diagram showing how a construction machine is designated by the operation device according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing a construction machine displayed on a display device of a construction management terminal. [Figure 7] FIG. 10 is a diagram for explaining the principle of determining whether or not a construction machine can be stopped, illustrating a case where the zero moment point is inside the indication polygon. [Figure 8] FIG. 10 is a diagram showing an example of information displayed on a display device of the construction management terminal. [Figure 9] FIG. 10 is a diagram for explaining the principle of determining whether or not a construction machine can be stopped, illustrating a case where the zero moment point is outside the instruction polygon. [Figure 10] FIG. 10 is a diagram showing an example of information displayed on a display device of the construction management terminal. [Figure 11] FIG. 10 is a diagram showing how a construction machine is designated by an operation device according to a second embodiment. [Figure 12] 10 is a flowchart showing the processing of a construction machine operation control system according to a second embodiment. [Figure 13] FIG. 10 is a diagram showing how a construction machine is designated by an operation device according to a third embodiment. [Figure 14] 10 is a flowchart showing the processing of a construction machine operation control system according to a third embodiment. [Figure 15] FIG. 10 is a diagram showing how a construction machine is designated by an operation device according to a fourth embodiment. [Figure 16] 10 is a flowchart showing the processing of a construction machine operation control system according to a fourth embodiment. [Figure 17] FIG. 13 is a diagram showing an example of a display of an estimated vertical error on a display device according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, a hydraulic excavator will be described as an example of a construction machine, but the present invention is not limited to this and can also be applied to other construction machines operating at construction sites.

[0012] First Embodiment A first embodiment of the present invention will be described with reference to FIGS.

[0013] FIG. 1 is a functional block diagram showing the outline of the configuration of a construction machine operation control system according to this embodiment.

[0014] In Figure 1, the construction machinery operation control system allows a user to operate a specified construction machinery by specifying the construction machinery on the camera image, and has functional parts provided in a construction machinery 300 (e.g., a hydraulic excavator) operating at a construction site and functional parts provided in a construction management terminal 100 operated by a user who operates the construction machinery 300.

[0015] The construction management terminal 100 is a device (e.g., a mobile terminal such as a tablet or smartphone) carried and operated by a user, such as a site manager who manages multiple construction machines (in Figure 1, construction machine 300 is shown as a representative) operating at a construction site, and is roughly composed of a control device 101, a display device 102, an operation device 103, a transmission / reception device 104, an attitude detection device 109, a position detection device 110, a camera 111, and a status calculation device 112.

[0016] The control device 101 and the state calculation device 112 are configured by an arithmetic device having, for example, an input interface provided in the construction management terminal 100, a central processing unit (CPU) which is a processor, a memory device such as a read only memory (ROM) or random access memory (RAM), an output interface, etc. In other words, each function of the control device 101 and the state calculation device 112 is configured by performing predetermined arithmetic processing on signals input via the input interface and information stored in the ROM and RAM, etc., according to a control program stored in the ROM, and outputting the results via the output interface.

[0017] The control device 101 (terminal control device) performs calculations based on user input, the status of the construction management terminal, camera footage, information input from the construction machinery, etc., and creates control information for operating the construction machinery 300, for example, and outputs the results.

[0018] The display device 102 (terminal display device) displays on a screen to show to a user (operator) information created and output by the control device 101. The display device 102 is, for example, a liquid crystal monitor.

[0019] The operation device 103 (terminal operation device) detects input from the user and inputs the detected information to the control device 101. The operation device 103 is an input device such as a touch panel, and is placed on the surface of the display device 102, thereby constituting a GUI (Graphical User Interface) together with the display device 102.

[0020] The attitude detection device 109 (terminal attitude detection device) detects the physical direction in which the construction management terminal is facing in a coordinate system set in the construction site space, and outputs the detection result as terminal attitude information. As the attitude detection device 109, for example, sensors such as a magnetic sensor that detects geomagnetism, an acceleration sensor, an angular velocity sensor (gyro sensor), an inertial measurement unit (IMU), etc. can be used alone or in combination.

[0021] The position detection device 110 (terminal position detection device) detects the position of the construction management terminal within the construction site space and outputs the detection result as terminal position information. The position detection device 110 is, for example, a GNSS (Global Navigation Satellite System).

[0022] The camera 111 (terminal camera) is attached to the construction management terminal, captures images within the construction site space, and outputs the captured images to the control device 101. The camera 111 is, for example, a photographing device using a CMOS image sensor or the like.

[0023] The state calculation device 112 calculates the position and attitude of the construction management terminal within the construction site space based on the terminal attitude information input from the attitude detection device 109 and the terminal position information input from the position detection device 110, and outputs the calculation result to the control device 101 as terminal state information.

[0024] The transmitting / receiving device 104 transmits information (such as control information described later) output by the control device 101 to the transmitting / receiving device 105 of the construction machine 300 via the network 200, and also receives information (such as unstoppable information described later) output from the transmitting / receiving device 105 of the construction machine 300 and inputs it to the control device 101. The transmitting / receiving device 104 is, for example, a communication device that performs wireless communication.

[0025] The construction machine 300 is, for example, a work machine such as a hydraulic excavator operating at a construction site, and is roughly composed of a transmitter / receiver device 105, a position detection device 106, a control device 107, a control target 108, an operating status detection device 113, and a stop possibility determination device 114.

[0026] The control device 107, the movable state detection device 113, and the stoppability determination device 114 are configured by, for example, an arithmetic device having an input interface provided on the construction machine 300, a central processing unit (CPU) which is a processor, a memory device such as a read only memory (ROM) or random access memory (RAM), an output interface, etc. In other words, each function of the control device 107, the movable state detection device 113, and the stoppability determination device 114 is configured by performing predetermined arithmetic processing on signals input via the input interface and information stored in the ROM and RAM, etc., in accordance with a control program stored in the ROM, and outputting the results via the output interface.

[0027] The transmitting / receiving device 105 (machine transmitting / receiving device) receives information (such as control information described later) output from the transmitting / receiving device 104 of the construction management terminal 100 via the network 200 and inputs it to the control device 107, and also transmits information generated by the construction machine 300 (such as unstoppable information described later) to the transmitting / receiving device 104 of the construction management terminal 100. The transmitting / receiving device 105 is, for example, a communication device that performs wireless communication.

[0028] The position detection device 106 (machine position detection device) detects the position of the construction machine within the construction site space and outputs the resulting position information. The position detection device 106 is, for example, a GNSS (Global Navigation Satellite System).

[0029] The control device 107 (machine control device) generates a control amount according to the input control information in order to operate the construction machine 300, and outputs the control amount as a signal including the control amount to each device (control target) of the vehicle body that can be operated.

[0030] The controlled object 108 is a control mechanism of the construction machine that operates based on the control amount input from the control device 107. In the hydraulic excavator exemplified as the construction machine 300 in this embodiment, the controlled object includes, for example, the engine and various hydraulic cylinders.

[0031] The operating state detection device 113 (machine operating state detection device) detects the state of each part of the construction machine 300 and the attitude of the construction machine 300, and outputs machine operating state information as the detection result, which is made up of machine attitude information, which is information indicating the attitude of the construction machine 300, and the control amount currently being applied.

[0032] The stop possibility determination device 114 determines, based on the machine operation status information output by the operation status detection device 113, whether the construction machine 300 is in a state where stopping its operation would result in unstable behavior that could lead to tipping or slipping, and if the determination result shows that stopping its operation would result in unstable behavior, it outputs stop impossibility information to the transmission / reception device 105, or if stopping its operation would not result in unstable behavior, it outputs stop control information to the control device 107.

[0033] In the construction machinery operation control system configured as described above, the operator specifies the desired construction machinery 300 on the construction management terminal 100, and the system determines whether the operating state of the specified construction machinery 300 can be stopped, and stops the operation according to the determination result.

[0034] FIG. 2 is a flowchart showing the processing of the construction machine operation control system.

[0035] In FIG. 2, when the construction management terminal 100 starts operation, it first detects an input from the operation device 103 (step S201).

[0036] FIG. 3 is a diagram illustrating an example of a construction machine operating at a construction site.

[0037] FIG. 3 illustrates a case where a total of three shovels, a first shovel 301, a second shovel 302, and a third shovel 303, are present within the construction site space.

[0038] In this state, the construction management terminal is equipped with a liquid crystal display as the display device 102, which displays the image of the construction site space input from the camera 111. In addition, a touch panel is provided as the operation device 103, overlaid on the surface of the display device 102. By operating the operation device 103, the user can specify a point on the image of the construction site space captured by the camera 111. As a result of the specification, the operation device 103 detects this input.

[0039] Next, the attitude of the camera 111 is calculated (step S202).

[0040] FIG. 4 is a diagram illustrating the basic principle of camera attitude detection.

[0041] As shown in Fig. 4, the position (xc, yc, zc) of the camera 111 on the construction management terminal in the coordinate system that indicates the construction site space is called the camera position 401. The camera position 401 is determined using the position detection device 110. For example, the position detection device 110 can calculate the position within the construction site space from the latitude and longitude on the earth obtained using a GPS (Global Positioning System). As another example, a total station or the like can be used to obtain the position within the construction site space.

[0042] Furthermore, the direction vector (xe, ye, ze) indicating the direction of the camera 111 is called a camera direction vector 402. The camera direction vector 402 is obtained using the orientation detection device 109. The orientation detection device 109 may calculate the direction from sensors such as a geomagnetic sensor or an angle sensor.

[0043] Next, the position designated by the user (operator of the construction management terminal 100) by operating the operation device 103 on the display device 102 is calculated within the construction site space (step S203).

[0044] FIG. 5 is a diagram showing how a construction machine is designated by the operating device.

[0045] In Figure 5, the camera 111 is attached to the back of the construction management terminal. The construction machine candidate is assumed to be the construction machine photographed at the position of a point on the display device 102 that the user specifies by operating the operation device 103. This point is called the specified point 501. The coordinates of the specified point 501 on the display device 102 are called Ps. The construction machine candidate indicated by the specified point 501 exists on a half line that is extended from the focal position of the camera 111 in the direction corresponding to the specified point 501. This half line is called the specified line 502. The construction machine that has an intersection with the specified line 502 is assumed to be the construction machine to be operated (the construction machine to be operated 503). In subsequent processing, this construction machine to be operated 503 will be acquired, and the specified point 501 used for this purpose will be processed as the position specified by the user.

[0046] Next, it is determined whether or not the verification process has been carried out for all construction machines connected to the network 200 (step S204), and if the determination result is YES, the process proceeds to step S212.

[0047] If the determination result in step S204 is NO, that is, if it is determined that there is a construction machine for which verification processing has not been performed, the position of one construction machine to be subjected to verification processing is acquired (step S205).

[0048] The position information of the construction machine detected by the position detection device 106 of one of the construction machines connected to the network 200 is transmitted by the transmission / reception device 105 of the construction machine, received by the transmission / reception device 104 of the construction management terminal 100 via the network 200, and input to the control device 101. The control device 101 converts the received position of the construction machine from the three-dimensional coordinate system of the construction site space to the coordinate system of the camera 111 using the camera position 401, the camera direction vector 402, and the geometric characteristics of the lens and image sensor held by the camera 111. The coordinate system of the camera 111 is equivalent to the coordinate system indicated by the planar image of the construction site space displayed on the display device 102.

[0049] FIG. 6 is a diagram showing the construction machine displayed on the display device of the construction management terminal.

[0050] As shown in FIG. 6, the position of the construction machine detected by the position detection device 106 is expressed as a position 601 of the construction machine on the image in the coordinate system of the camera 111.

[0051] Next, it is determined whether the position 601 of the construction machine on the image of the display device 102 overlaps with the specified point 501 (step S206).

[0052] Construction machinery occupies a certain amount of space within the construction site. This is called the occupied space V. In the example of Figure 6, the occupied space V is schematically represented by a rectangular parallelepiped, but in reality it represents the space occupied by the current shape of the construction machinery 503 to be operated, obtained from information output by the operating status detection device 113 of the construction machinery 503 to be operated, when it is placed at a position obtained from information output by the position detection device 106 of the construction machinery 503 to be operated. When this occupied space V is projected onto the coordinate system of the camera 111, it occupies a certain area on the two-dimensional image displayed on the display device 102, as shown in Figure 6. This is called the occupied area 602 of the construction machinery on the image. It is symbolically represented as A. The relationship between A and the occupied space V can be expressed as A = P(V). Here, P is the projection from the occupied space V, which is a three-dimensional space, to the two-dimensional area A. If the specified point 501 is included in the occupation area 602 of the construction machine on the image, that is, if Ps ⊂ A, it is determined that the two overlap.

[0053] If the determination result in step S206 is NO, that is, if it is determined that the position of the construction machine on the image does not overlap with the specified point, the process returns to step S204.

[0054] Furthermore, if the determination result in step S206 is YES, information to be used for operation control of the construction machine 503 to be operated that has been determined to overlap with the specified point 501, such as the current position of the construction machine 503 to be operated obtained from the position detection device 106, is obtained (step S207).

[0055] Next, the operating status of the construction machine 503 to be operated is acquired (step S208). The operating status detection device 113 acquires information such as the positions of the boom, arm, and bucket of the excavator, the rotation angle of the upper rotating body, and the inclination of the lower traveling body.

[0056] Next, it is determined whether or not the construction machine 503 to be operated can be stopped stably in the current processing (step S209).

[0057] FIG. 7 is a diagram for explaining the principle of determining whether or not a construction machine can be stopped, and illustrates a case where the zero moment point is inside the indication polygon.

[0058] FIG. 7 illustrates an example in which the construction machine 503 to be operated is descending a slope. The ground contact position 701 of the construction machine 503 to be operated is the position where a pair of parallel tracks of the construction machine 503 to be operated contacts the ground, as obtained from the operating state detection device 113. In this state, the support polygon 702 of the construction machine 503 to be operated indicates a rectangular area including the space between the parallel tracks. Next, the center of gravity 703 of the construction machine 503 to be operated is determined from the state of the boom, arm, bucket, upper swing body, and lower running body of the construction machine 503 to be operated, which are input from the operating state detection device 113. The ZMP (Zero Moment Point) 704, which is the center of pressure of the ground reaction force of this center of gravity 703, is determined. If the ZMP 704 is within the area of ​​the support polygon 702, the construction machine 503 to be operated is said to be in a stable state. In the example of FIG. 7, the ZMP 704 is within the area of ​​the support polygon 702. When the construction machine 503 to be operated is in this state, it is determined that the construction machine 503 to be operated can be stopped stably.

[0059] If the determination result in step S209 is YES, that is, if it is determined that the construction machine can be stopped, the control device 107 sends a stop signal (a control amount equivalent to stopping) to the controlled object 108, and stops the construction machine 503 to be operated (step S210).

[0060] FIG. 8 is a diagram showing an example of information displayed on the display device of the construction management terminal.

[0061] As shown in Fig. 8, when stop control information is sent from the construction management terminal 100 to the construction machine 503 to be operated, a display content (message) 901 indicating that the transmission of the specified operation signal (i.e., stop control signal) has been completed is displayed near the position 601 of the construction machine on the image of the display device 102. This allows the operator of the construction management terminal to more easily and in a shorter time confirm whether or not a command has been issued to the correct operation target.

[0062] Furthermore, if the judgment result in step S209 is NO, that is, if it is judged that the construction machine cannot be stopped, the reason why operation of the construction machine 503 to be operated is not possible is sent to the control device 101 via the transmission / reception device 105 of the construction machine, the network 200, and the transmission / reception device 104 of the construction management terminal 100 (step S211).

[0063] FIG. 9 is a diagram for explaining the principle of determining whether or not a construction machine can be stopped, and illustrates a case where the zero moment point is outside the instruction polygon.

[0064] In Figure 9, the construction machine 503 to be operated is shown descending a slope with a steeper angle than the example shown in Figure 7. Based on the center of gravity 801 in the different states, a ZMP 802 in the different states is calculated. The ZMP 802 in the different states exists outside the area of ​​the support polygon 702. This result is sent to the control device 101. The sent message 803 contains the name identifying the construction machine that sent the message 803, operation feasibility information indicating whether a stop operation can be performed, and, if a stop operation cannot be performed, the reason for this. When the control device 101 receives this message 803, it displays the contents on the display device 102.

[0065] FIG. 10 is a diagram showing an example of information displayed on the display device of the construction management terminal.

[0066] In Fig. 10, display content (message) 902 created based on the telegram 803 is displayed near the position 601 of the construction machine on the image of the display device 102. This allows the user to understand the reason why the construction machine 503 being operated is not in a stopped state, and encourages them to take other measures.

[0067] When the processing of step S210 or step S211 is completed, it is then determined whether or not to terminate the operation of the construction machine operation control system (step S212).

[0068] For example, it is possible to determine that the operation is to be terminated by the user turning off the power of the construction management terminal. If the determination result in step S212 is YES, that is, if it is determined that the operation is to be terminated, the system is terminated. Also, if the determination result in step S212 is NO, the process returns to step S201.

[0069] In this embodiment, the case where the ZMP 704 is used to determine whether the construction machine 503 to be operated can be stopped is exemplified, but other methods can also be used for the determination. This type of processing stops the construction machine 503 to be operated specified by the user, or if stopping the construction machine 503 would make the operation of the construction machine 503 to be operated unstable, it does not stop the machine and displays the reason to the user, making it possible to operate the construction machine stably.

[0070] In the present embodiment configured as described above, in the operation control system for a construction machine, the system includes a terminal control device (control device 101) that creates control information for operating a construction machine, a terminal display device (display device 102) that displays the control information created by the terminal control device, a terminal operation device (operation device 103) that, when an operator designates at least a part of the control information displayed on the terminal display device, detects the coordinates of the designated control information as input coordinate values ​​and outputs the detected input coordinate values, and a terminal transmission / reception device (transmission / reception device 104) that transmits and receives the control information, a machine transmission / reception device (transmission / reception device 105) that transmits and receives the control information, a machine position detection device (position detection device 106) that acquires the position of the construction machine within the construction site space and outputs it as machine position information, a machine control device (control device 107) that outputs a control signal according to the input control information, and a construction machine having a controlled object that is a control mechanism of the construction machine that operates in accordance with the control signal input from the machine control device, and the construction management terminal detects the orientation of the construction management terminal within the construction site space and outputs it as terminal orientation information. The construction management terminal further comprises a detection device (attitude detection device 109), a terminal position detection device (position detection device 110) that acquires the position of the construction management terminal within the construction site space and outputs it as terminal position information, a terminal camera (camera 111) that is provided integrally with the construction management terminal and captures the situation around the construction management terminal and outputs it as a surrounding image, and a terminal state calculation device (state calculation device 112) that calculates and outputs terminal state information that indicates the state of the construction management terminal based on the terminal attitude information detected by the terminal attitude detection device and the terminal position information detected by the terminal position detection device, and is configured so that when the surrounding image output by the terminal camera is displayed on the terminal display device, the construction machine present within the construction site space that corresponds to the input coordinate value specified on the terminal operation device is identified as the operation target based on the input coordinate value specified on the terminal display device and the terminal state information calculated by the terminal state calculation device, and stop control information is generated, which is control information for stopping the construction machine identified as the operation target, so that the construction machine to be operated can be more reliably specified, and it can be confirmed more easily and in a shorter time whether a command has been issued to the correct operation target.

[0071] <Second embodiment> A second embodiment of the present invention will be described with reference to FIGS.

[0072] This embodiment shows a case where, when there are multiple construction machines to be operated at a location specified by a user (operator of the construction management terminal), one machine is extracted from the multiple machines and processed.

[0073] Fig. 11 is a diagram showing how a construction machine is designated by an operating device. Fig. 12 is a flowchart showing the processing of the construction machine operation control system. In Figs. 11 and 12, the same reference numerals are used to designate the same components as in the first embodiment, and their explanations will be omitted.

[0074] In FIG. 11 , the point on the display device 102 that the user specifies by operating the operation device 103 is called the specified point 1001 in the second example. The coordinates of the specified point 1001 in the second example on the display device 102 are called P's. The construction machine candidate indicated by the position of the specified point 1001 in the second example is located on a half-line extending from the focal position of the camera 111 in the direction corresponding to the specified point 1001 in the second example. This half-line is called the specified line 1002 in the second example. In this example, there are three construction machines that intersect with the specified line 1002 in the second example. These have a spatial anteroposterior relationship on the specified line 1002 in the second example. This is because multiple physical objects such as construction machines cannot exist in the exact same position in space at the same time. In the image captured by the camera 111, the construction machine located closest to the camera 111 in the construction site space is shown. In other words, in this case, the user is specifying the construction machine closest to the construction management terminal. This construction machine to be operated is called the construction machine to be operated 1003 in the second example.

[0075] In FIG. 12, when the construction management terminal 100 starts operation, it first detects an input from the operation device 103 (step S201).

[0076] Next, the attitude of the camera 111 is calculated (step S202).

[0077] Next, the position designated by the user (operator of the construction management terminal 100) by operating the operation device 103 on the display device 102 is calculated within the construction site space (step S203).

[0078] Next, it is determined whether or not the verification process has been carried out for all construction machines connected to the network 200 (step S204), and if the determination result is YES, the process proceeds to step S212.

[0079] If the determination result in step S204 is NO, that is, if it is determined that there is a construction machine for which verification processing has not been performed, the position of one construction machine to be subjected to verification processing is acquired (step S205).

[0080] Next, it is determined whether the position 601 of the construction machine on the image of the display device 102 overlaps with the specified point 501 (step S206). If the specified point 1001 in the second example is included in the occupation area 602 of the construction machine on the image, that is, if P's ⊂ A, it is determined that the two overlap.

[0081] If the determination result in step S206 is YES, the position in the construction site space where the construction machine exists, where P's ⊂ A, is stored in the control device 101 (step S1101), and the process returns to step S204.

[0082] If the determination result in step S206 is NO, the process returns to step S204.

[0083] If the determination result in step S204 is YES, the construction machine closest to the position of the camera 111 is extracted from the information on the construction machines saved in step S1101 as the construction machine 1003 to be operated in the second example (step S207).

[0084] Next, the operating status of the construction machine to be operated is acquired (step S208).

[0085] Next, it is determined whether or not the construction machine to be operated can be stopped stably in the current processing (step S209).

[0086] If the determination result in step S209 is YES, that is, if it is determined that the construction machine can be stopped, the control device 107 sends a stop signal (a control amount equivalent to stopping) to the controlled object 108, and stops the construction machine being operated (step S210).

[0087] Furthermore, if the judgment result in step S209 is NO, that is, if it is judged that the construction machine cannot be stopped, the reason why operation of the construction machine to be operated is not possible is sent to the control device 101 via the transmission / reception device 105 of the construction machine, the network 200, and the transmission / reception device 104 of the construction management terminal 100 (step S211).

[0088] When the processing of step S210 or step S211 is completed, it is then determined whether or not to terminate the operation of the construction machine operation control system (step S212).

[0089] If the determination result in step S212 is YES, that is, if it is determined that the process should be terminated, the system is terminated. If the determination result in step S212 is NO, the process returns to step S201.

[0090] The other configurations are the same as those in the first embodiment.

[0091] The present embodiment configured as above can also achieve the same effects as the first embodiment.

[0092] Furthermore, when there are multiple construction machines on the specified line 1002 of the second example obtained by specifying the specified point 1001 of the second example, it is possible to obtain the construction machine displayed at the position of the specified point 1001 of the second example on the construction management terminal as the construction machine 1003 to be operated of the second example, making it possible to operate the construction machine specified from the user's perspective without error.

[0093] <Third embodiment> A third embodiment of the present invention will be described with reference to FIGS.

[0094] This embodiment shows a case where, when there are a plurality of construction machines to be operated in an area designated by a user (operator of a construction management terminal), these are extracted and processed.

[0095] Fig. 13 is a diagram showing how a construction machine is designated by an operating device. Fig. 14 is a flowchart showing the processing of the construction machine operation control system. In Figs. 13 and 14, the same reference numerals are used to designate the same components as those in the first and second embodiments, and their explanations will be omitted.

[0096] In FIG. 13 , the area on the display device 102 that is specified by operating the operation device 103 is called a specified area 1201 in the third example. The specified area 1201 in the third example is symbolically represented as Rs. The specified area 1201 in the third example can be represented, for example, as a rectangular area, and can be represented by specifying two points on the display device 102 and using these points as the upper left and lower right vertices of a rectangle. The construction machine candidate indicated by the position of the specified area 1201 in the third example is located inside a quadrangular pyramid that extends from the focal position of the camera 111 in a direction corresponding to the specified area 1201 in the third example. This quadrangular pyramid is called a specified quadrangular pyramid 1202 in the third example. In this example, there are two construction machines that intersect with the specified quadrangular pyramid 1202 in the third example. These construction machines to be operated are called a first operation target construction machine 1203 in the third example and a second operation target construction machine 1204 in the third example, respectively.

[0097] The first operation target construction machine 1203 of the third example and the second operation target construction machine 1204 of the third example occupy a certain area on the two-dimensional image displayed on the display device 102, similar to the explanation of Figure 6 in the first embodiment. The occupied area of ​​the first operation target construction machine 1203 of the third example is represented by the symbol A1, and the occupied area of ​​the second operation target construction machine 1204 of the third example is represented by the symbol A2. Here, neither A1∩Rs nor A2∩Rs is an empty set.

[0098] In FIG. 14, when the construction management terminal 100 starts operation, it first detects an input from the operation device 103 (step S201).

[0099] Next, the attitude of the camera 111 is calculated (step S202).

[0100] Next, the specified area 1201 of the third example is calculated (step S1301). As described above, two points are specified on the display device 102, and the area Rs is represented by a rectangle with these points as the upper left and lower right corners.

[0101] Next, it is determined whether or not the verification process has been carried out for all construction machines connected to the network 200 (step S204), and if the determination result is YES, the process proceeds to step S212.

[0102] If the determination result in step S204 is NO, that is, if it is determined that there is a construction machine for which verification processing has not been performed, the position of one construction machine to be subjected to verification processing is acquired (step S205).

[0103] Next, it is determined whether the designated area 1201 of the third example overlaps with the occupied area (step S1302). For the construction machine acquired in step S205, the occupied area on the two-dimensional image displayed on the display device 102 is set as A, and A∩Rs is calculated. If A∩Rs is not an empty set, it is determined that the designated area 1201 of the third example overlaps with the occupied area.

[0104] If the determination result in step S1302 is YES, the position in the construction site space where the construction machine exists, where P's ⊂ A, is stored in the control device 101 (step S1101), and the process returns to step S204.

[0105] If the determination result in step S1302 is NO, the process returns to step S204.

[0106] If the determination result in step S204 is YES, the information on the construction machines stored in step S1101 is extracted one by one (step S207).

[0107] Next, the operating status of the construction machine to be operated is acquired (step S208).

[0108] Next, it is determined whether or not the construction machine to be operated can be stopped stably in the current processing (step S209).

[0109] If the determination result in step S209 is YES, that is, if it is determined that the construction machine can be stopped, the control device 107 sends a stop signal (a control amount equivalent to stopping) to the controlled object 108, and stops the construction machine being operated (step S210).

[0110] Furthermore, if the judgment result in step S209 is NO, that is, if it is judged that the construction machine cannot be stopped, the reason why operation of the construction machine to be operated is not possible is sent to the control device 101 via the transmission / reception device 105 of the construction machine, the network 200, and the transmission / reception device 104 of the construction management terminal 100 (step S211).

[0111] After the processing of step S210 or step S211 is completed, it is next determined whether all information has been extracted in step S207 from the information on the construction machines saved in step S1101, i.e., the first construction machine to be operated 1203 in the third example and the second construction machine to be operated 1204 in the third example in Figure 13, i.e., whether all targets have been verified (step S1303).

[0112] If the determination result in step S1303 is NO, the process returns to step S207. If the determination result in step S1303 is YES, it is determined whether or not to terminate the operation of the construction machine operation control system (step S212).

[0113] If the determination result in step S212 is YES, that is, if it is determined that the process should be terminated, the system is terminated. If the determination result in step S212 is NO, the process returns to step S201.

[0114] The other configurations are the same as those of the first and second embodiments.

[0115] The present embodiment configured as above can also achieve the same effects as the first embodiment.

[0116] Furthermore, when multiple construction machines are present on the specified pyramid 1202 of the third example obtained by specifying the specified area 1201 of the third example, it becomes possible to operate all of the relevant construction machines, making it possible to quickly avoid events that may occur due to the related operations of multiple construction machines.

[0117] <Fourth embodiment> A fourth embodiment of the present invention will be described with reference to FIGS.

[0118] This embodiment shows a case where there are multiple construction machines to be operated in an area specified by a user (operator of a construction management terminal), and the user selects the object to be operated from among these.

[0119] Fig. 15 is a diagram showing how a construction machine is designated by an operating device. Fig. 16 is a flowchart showing the processing of the construction machine operation control system. In Fig. 15 and Fig. 16, the same reference numerals are used to designate the same components as those in the first to third embodiments, and their explanations will be omitted.

[0120] In FIG. 15 , the area on the display device 102 that is specified by operating the operation device 103 is called the specified area 1401 of the fourth example. The specified area 1401 of the fourth example is symbolically represented as R's. The specified area 1401 of the fourth example can be expressed, for example, as a rectangular area, and can be expressed as a rectangle by specifying two points on the display device 102 and using these points as the upper left and lower right corners. The construction machine candidate indicated by the position where the specified area 1401 of the fourth example exists is located inside a quadrangular pyramid that extends from the focal position of the camera 111 in the direction corresponding to the specified area 1401 of the fourth example. This quadrangular pyramid is called the specified quadrangular pyramid 1402 of the fourth example. In this example, there are three construction machines that intersect with the specified quadrangular pyramid 1402 of the fourth example. These construction machines to be operated are referred to as a first construction machine to be operated 1403 in the fourth example, a second construction machine to be operated 1404 in the fourth example, and a third construction machine to be operated 1405 in the fourth example, respectively.

[0121] The first operation target construction machine 1203 of the third example and the second operation target construction machine 1204 of the third example occupy a certain area on the two-dimensional image displayed on the display device 102, as in the explanation of FIG. 6 in the first embodiment. The occupation area of ​​the first operation target construction machine 1403 of the fourth example is denoted by the symbol A'1, is named M1, and is located at a distance of 5 m from the camera 111. Similarly, the occupation area of ​​the second operation target construction machine 1404 of the fourth example is denoted by the symbol A'2, is named M2, and is located at a distance of 12 m from the camera 111. Furthermore, the occupation area of ​​the third operation target construction machine 1405 of the fourth example is denoted by the symbol A'3, is named M3, and is located at a distance of 6 m from the camera 111. Here, A'1∩R's, A'2∩R's, and A'3∩R's are not empty sets. The information about each of the above-mentioned construction machines is displayed as a construction machine list 1406 on the display device 102. In this example, the name of each construction machine and the distance from the camera 111 are shown, and a check mark indicates whether each construction machine has been selected. There is also an execute button for executing an operation.

[0122] In FIG. 16, when the construction management terminal 100 starts operation, it first detects an input from the operation device 103 (step S201).

[0123] Next, the attitude of the camera 111 is calculated (step S202).

[0124] Next, the specified area 1201 of the third example is calculated (step S1301). As described above, two points are specified on the display device 102, and the area Rs is represented by a rectangle with these points as the upper left and lower right corners.

[0125] Next, it is determined whether or not the verification process has been carried out for all construction machines connected to the network 200 (step S204), and if the determination result is YES, the process proceeds to step S212.

[0126] If the determination result in step S204 is NO, that is, if it is determined that there is a construction machine for which verification processing has not been performed, the position of one construction machine to be subjected to verification processing is acquired (step S205).

[0127] Next, it is determined whether the designated area 1201 of the third example overlaps with the occupied area (step S1302). For the construction machine acquired in step S205, the occupied area on the two-dimensional image displayed on the display device 102 is set as A, and A∩Rs is calculated. If A∩Rs is not an empty set, it is determined that the designated area 1201 of the third example overlaps with the occupied area.

[0128] If the determination result in step S1302 is YES, the position in the construction site space where the construction machine exists, where P's ⊂ A, is stored in the control device 101 (step S1101), and the process returns to step S204.

[0129] If the determination result in step S1302 is NO, the process returns to step S204.

[0130] Also, if the determination result in step S204 is YES, the information about the construction machines saved in the control device 101 in step S1101 is displayed in list form on the display device 102 (step S501). This results in a display content such as construction machine list 1406. The user operates the operation device 103 to switch check marks in the construction machine list 1406 on or off. Here, construction machines with check marks on are designated as operation targets. In this example, the check marks on the rows for the first operation target construction machine 1403 in the fourth example and the third operation target construction machine 1405 in the fourth example are on, and these construction machines are designated as operation targets. Next, the execute button is pressed to proceed to the next step in the processing.

[0131] Next, the information about the construction machines stored in step S1101 is extracted one by one (step S207).

[0132] Next, the operating status of the construction machine to be operated is acquired (step S208).

[0133] Next, it is determined whether or not the construction machine to be operated can be stopped stably in the current processing (step S209).

[0134] If the determination result in step S209 is YES, that is, if it is determined that the construction machine can be stopped, the control device 107 sends a stop signal (a control amount equivalent to stopping) to the controlled object 108, and stops the construction machine being operated (step S210).

[0135] Furthermore, if the judgment result in step S209 is NO, that is, if it is judged that the construction machine cannot be stopped, the reason why operation of the construction machine to be operated is not possible is sent to the control device 101 via the transmission / reception device 105 of the construction machine, the network 200, and the transmission / reception device 104 of the construction management terminal 100 (step S211).

[0136] After the processing of step S210 or step S211 is completed, it is next determined whether all information has been extracted in step S207 from the information on the construction machines saved in step S1101, i.e., the first construction machine to be operated 1203 in the third example and the second construction machine to be operated 1204 in the third example in Figure 13, i.e., whether all targets have been verified (step S1303).

[0137] If the determination result in step S1303 is NO, the process returns to step S207. If the determination result in step S1303 is YES, it is determined whether or not to terminate the operation of the construction machine operation control system (step S212).

[0138] If the determination result in step S212 is YES, that is, if it is determined that the process should be terminated, the system is terminated. If the determination result in step S212 is NO, the process returns to step S201.

[0139] The other configurations are the same as those of the first to third embodiments.

[0140] The present embodiment configured as above can also achieve the same effects as the first embodiment.

[0141] Furthermore, it is possible to specify only the construction machine selected by the user from among the construction machines that were selected as candidates before executing step S1501, and to operate it. This means that when operating multiple construction machines, construction machines that are presumed to be out of range, such as construction machines that are significantly far away from the camera 111, can be excluded from the operation targets, enabling efficient operation of construction machines.

[0142] <Fifth embodiment> A fifth embodiment of the present invention will be described with reference to FIG.

[0143] This embodiment shows a case where a construction machine is operated using equipment other than the camera 111 mounted on the construction management terminal.

[0144] 17 is a functional block diagram showing the general configuration of a construction machine operation control system according to this embodiment. In the figure, the same members as those in the first embodiment are given the same reference numerals, and their explanations will be omitted.

[0145] In this embodiment, a case where imaging equipment is added to the configuration of the first embodiment (see FIG. 1) is shown.

[0146] In FIG. 17, the photographing equipment 400 is roughly composed of a transmitting / receiving device 1601 (equipment transmitting / receiving device), a camera 1602 (equipment camera), an attitude detection device 1603 (equipment attitude detection device), and a position detection device 1604 (equipment position detection device).

[0147] A transmitting / receiving device 1601 provided in the imaging equipment transmits information acquired by the imaging equipment to the construction management terminal via the network 200.

[0148] The camera 1602 captures images of the construction site space and outputs the results to the transmitting / receiving device 1601 .

[0149] The orientation detection device 1603 detects the physical direction in which the imaging equipment is facing and outputs information on the orientation.

[0150] The position detection device 1604 detects the position of the imaging equipment within the construction site space and outputs the resulting position information.

[0151] The control device 101 of the construction management terminal 100 acquires the position and posture of the imaging equipment and the video captured by the camera 1602 via the transmitting / receiving device 104 , the network 200 , and the transmitting / receiving device 1601 .

[0152] The processing performed by the construction management terminal 100 using the information received uses, for example, the image captured by the camera 111 and the information output by the attitude detection device 109 and the position detection device 110, as explained in the first embodiment using Figure 2, but these are replaced with the image captured by the camera 1602 and the information output by the attitude detection device 1603 and the position detection device 1604 and processing is performed.

[0153] The other configurations are the same as those in the first embodiment.

[0154] The present embodiment configured as above can also achieve the same effects as the first embodiment.

[0155] In addition, it becomes possible to operate operating construction machinery in places where there is no construction management terminal, for example, in remote locations where there is photography equipment but the user cannot see it, which saves the user the trouble of having to carry the construction management terminal around and allows operations to be carried out efficiently.

[0156] <Additional Notes> The present invention is not limited to the above-described embodiments, and includes various modifications and combinations within the scope of the gist of the present invention. Furthermore, the present invention is not limited to those having all of the configurations described in the above-described embodiments, and includes those in which some of the configurations are omitted.

[0157] For example, in the fifth embodiment, an example is given of an operator operating a designated construction machine at a construction site by operating the construction management terminal 100, but this is not limited to this. A terminal having some of the functions of the construction management terminal 100 (control device 101, display device 102, operation device 103, transmission / reception device 104, status calculation device 112) may be installed in a management office at the construction site, and the construction machine may be operated from the management office using information from the photography equipment 400.

[0158] Furthermore, the above-described configurations, functions, etc. may be realized in part or in whole by designing them as, for example, integrated circuits, etc. Furthermore, the above-described configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. [Explanation of symbols]

[0159] 100...construction management terminal, 101...control device, 102...display device, 103...operation device, 104...transmitter / receiver, 105...transmitter / receiver, 106...position detection device, 107...control device, 108...control object, 109...posture detection device, 110...position detection device, 111...camera, 112...status calculation device, 113...operation status detection device, 114...stop possibility determination device, 200...network, 300...construction machinery, 1601...transmitter / receiver, 1602...camera, 1603...posture detection device, 1604...position detection device

Claims

1. A construction machine operation control system including a construction management terminal that creates control information for operating a construction machine, and a machine control device that controls the operation of the construction machine based on the control information, a terminal attitude detection device for detecting the orientation of the construction management terminal; A terminal position detection device that detects the position of the construction management terminal; a machine position detection device that detects the position information of the construction machine, The construction management terminal includes a camera that photographs the construction site, a display device that displays the construction site photographed by the camera as a surrounding image, an operation device that allows an operator to specify a predetermined position within the surrounding image displayed on the display device, and a terminal control device that creates the control information, The terminal control device calculating a position of a designated area corresponding to the predetermined position at the construction site based on the predetermined position in the surrounding image specified by an operator using the operation device, the orientation of the construction management terminal detected by the terminal attitude detection device, and the position of the construction management terminal detected by the terminal position detection device; determining whether the construction machine is present within the designated area based on the position of the designated area and the position information of the construction machine detected by the machine position detection device; A construction machinery operation control system characterized in that, when it is determined that the construction machinery is present within the designated area, stop control information for stopping the construction machinery is created as the control information, and the stop control information is output to the machinery control device.

2. 2. The construction machine operation control system according to claim 1, a machine operation state detection device that detects the posture of the construction machine and machine operation information consisting of the control information for operating the construction machine; Furthermore, the machine control device, the machine position detection device, and the machine operating state detection device are attached to the construction machine, The terminal control device calculating an outer shape of the construction machine from the machine operation information detected by the machine operation state detection device; determining whether the outer shape is within the designated area based on the position of the designated area, the position information of the construction machine, and the outer shape; A construction machine operation control system characterized in that, when the outer shape is present within the designated area, it is determined that the construction machine is present within the designated area.

3. 2. The construction machine operation control system according to claim 1, a machine operation state detection device that detects the posture of the construction machine and machine operation information consisting of the control information for operating the construction machine; a stop possibility determination device that determines whether the construction machine can be stopped stably based on the machine operation information detected by the machine operation state detection device; Furthermore, the machine control device, the machine position detection device, the machine operating state detection device, and the stop possibility determination device are attached to the construction machine, The stop possibility determination device When the machine control device receives the stop control information, it determines whether the construction machine can be stopped stably, When it is determined that the construction machine can be stopped stably, the machine control device is caused to stop the construction machine; A construction machinery operation control system characterized in that, when it is determined that the construction machinery is stable and cannot be stopped, unstoppable information indicating that the construction machinery is in an unstoppable state is sent to the construction management terminal.

4. 2. The construction machine operation control system according to claim 1, The terminal control device A construction machinery operation control system characterized by outputting stop control information to stop the construction machine among the multiple construction machines that is located closest to the construction management terminal based on the position of the construction management terminal detected by the terminal position detection device and the position information of the multiple construction machines detected by the machine position detection device when it is determined that there are multiple construction machines within the specified area.

5. 2. The construction machine operation control system according to claim 1, The terminal control device A construction machinery operation control system characterized by outputting stop control information to stop all of the construction machines determined to be present within the designated area when it is determined that multiple construction machines are present within the designated area.

6. 2. The construction machine operation control system according to claim 1, The terminal control device When it is determined that a plurality of construction machines exist within the designated area, information on all of the construction machines determined to exist within the designated area is output to the display device, and the information on the construction machines is displayed in a list; A construction machine operation control system characterized by outputting stop control information for stopping one of the plurality of construction machines selected by the operator from the list display using the operation device.

7. A construction machine operation control system including a control device that creates control information for operating a construction machine, and a machine control device that controls the operation of the construction machine based on the control information, A camera to photograph the construction site, a camera attitude detection device for detecting the orientation of the camera; a camera position detection device for detecting the position of the camera; a machine position detection device that detects the position information of the construction machine; a display device that displays the construction site photographed by the camera as a surrounding image; an operation device for an operator to specify a predetermined position within the surrounding image displayed on the display device, The control device calculating a position of a designated area corresponding to the predetermined position at the construction site based on the predetermined position in the surrounding image designated by an operator using the operation device, the orientation of the camera detected by the camera attitude detection device, and the position of the camera detected by the camera position detection device; determining whether the construction machine is present within the designated area based on the position of the designated area and the position information of the construction machine detected by the machine position detection device; A construction machinery operation control system characterized in that, when it is determined that the construction machinery is present within the designated area, stop control information for stopping the construction machinery is created as the control information, and the stop control information is output to the machinery control device.

Images