Remote control system

Abstract

The operator can be notified according to the estimated work content. [Solution] The remote control system comprises a construction machine, a server, and an input device. The construction machine includes a construction machine communication unit that transmits body data including state quantities related to the front device and attitude data of the front device to the server. The server includes a database that stores 3D design data and current terrain data which is the current terrain, a work content estimation unit that estimates the current work content of the construction machine using the body data, 3D design data, and current terrain data, a notification operation amount calculation unit that determines notification items to be transmitted from the work content estimated by the work content estimation unit and a correspondence table between the work content and information to be transmitted to the operator, and calculates notification operation amounts for notification items using attitude data, and a server communication unit. The input device includes a notification device that operates according to the notification operation amounts transmitted from the server, and the construction machine communication unit transmits the position data of the construction machine to the server.

Description

Technical Field

[0001] The present invention relates to a remote control system.

Background Art

[0002] For the purpose of labor saving due to labor shortages and use at disaster sites, remotely controlled construction machines that can be operated from a remote location are also used in the field of construction machinery. In remote operation, an operator operates the remote control device while checking the video of the work site on a monitor in a remote control room. When operating by boarding the construction machine, the operator can directly visually check the situation at the work site, and can directly experience the sound during work and the reaction force generated in the machine, so that the operation amount by the operation device can be adjusted according to the situation, thereby enabling efficient work. However, in remote operation, the situation of the work site and the construction machine can only be confirmed through monitor video, which makes it difficult to grasp the work status, tends to lead to cautious operations, and there is a demand for improved workability. Patent Document 1 discloses a remotely controlled robot configured to transmit the reaction force received by the robot to a control lever used by an operator, in addition to camera images in a remote control room, in order to give the operator a sense of presence at the site.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] Since construction machines are used for various types of work at work sites, the physical sensations that an operator can receive differ depending on the work content. Therefore, the invention described in Patent Document 1 still has room for improvement in order to allow the operator to experience a sensation closer to the actual operational feeling. This invention provides a remote control system for construction machinery that can improve work efficiency by giving operators a realistic feel for operating the machinery. [Means for solving the problem]

[0005] A remote control system according to a first aspect of the present invention comprises a construction machine, a server, and an input device, wherein the construction machine includes a construction machine communication unit that transmits body data including state quantities relating to the front device and attitude data of the front device to the server, the server includes a database storing 3D design data and current terrain data which is the current terrain, a work content estimation unit that estimates the current work content of the construction machine using the body data, the 3D design data, and the current terrain data, a notification operation amount calculation unit that determines the notification items to be transmitted from the work content estimated by the work content estimation unit and a correspondence table between the work content and information to be transmitted to the operator, and calculates the notification operation amount for the notification items using at least the attitude data, and a server communication unit, the input device includes a notification device, the notification device operates according to the notification operation amount transmitted from the server, and the construction machine communication unit transmits at least the position data of the construction machine to the server. [Effects of the Invention]

[0006] According to the present invention, the operator can be notified according to the estimated work content. [Brief explanation of the drawing]

[0007] [Figure 1] Overall configuration diagram of the remote control system [Figure 2] External view of a hydraulic excavator [Figure 3] Server hardware configuration diagram [Figure 4] A diagram showing an example of a notification item correspondence table. [Figure 5] A diagram showing an example of a notification expression table. [Figure 6]A diagram showing an example of actuator operation patterns and output amounts. [Figure 7] Configuration diagram of the control unit [Figure 8] Diagram explaining 3D design drawings and existing terrain data. [Figure 9] Flowchart showing the process of a hydraulic excavator [Figure 10] Flowchart showing server processing [Figure 11] Flowchart showing the processing of the input device [Figure 12] Functional configuration diagram of the remote control system in modified example 3 [Modes for carrying out the invention]

[0008] —First Embodiment— A first embodiment of the remote control system will be described below with reference to Figures 1 to 11.

[0009] Figure 1 is an overall diagram of the remote control system 1. The remote control system 1 comprises a hydraulic excavator 100 to be remotely controlled, a server 200, and an input device 300 as a remote control device. In this embodiment, the operator of the input device 300 is called the "operator." The operator is not on board the hydraulic excavator 100 and operates the hydraulic excavator 100 remotely. However, the distance between the hydraulic excavator 100 and the operator is not particularly limited; for example, it may be a short distance of about 10 meters, or a long distance of several kilometers to several hundred kilometers.

[0010] The hydraulic excavator 100 is a construction machine and includes a vehicle data acquisition unit 11, a load calculation unit 12, a construction history acquisition unit 13, a 3D machine guidance unit 14, a position data acquisition unit 15, a vehicle operation unit 16, and a construction machine communication unit 17.

[0011] The vehicle body data acquisition unit 11 acquires data related to the hydraulic excavator 100 as vehicle body data. The vehicle body data includes various data of the hydraulic control apparatus mounted on the hydraulic excavator 100, data related to the posture of each movable part of the hydraulic excavator 100, and position data calculated by the position data acquisition unit 15. The front device 1005 of the hydraulic excavator 100, which will be described later, has a large number of joints, and the angle of each joint can be acquired as vehicle body data. Therefore, the vehicle body data includes the three-dimensional position data of the bucket tip as data related to the posture of the front device 1005, or the three-dimensional position data of the tip of the front device 1005 can be easily calculated using the vehicle body data.

[0012] The load calculation unit 12 uses the output value of a hydraulic sensor that detects the pressure acting on each hydraulic actuator of the hydraulic excavator 100, and calculates vehicle body load data, which is the load acting on the vehicle body, using a known calculation formula. The vehicle body data includes both the output value of the hydraulic sensor and the vehicle body load data.

[0013] The construction history acquisition unit 13 acquires the three-dimensional position of the tip of the front device 1005 (the tip of the bucket 1013) included in the vehicle body data, and creates a construction history that is a history of the three-dimensional positions of the tip. The three-dimensional machine guidance unit 14 displays the positional relationship between the front device 1005 and the target construction surface on a monitor. The current posture of the front device 1005 can be acquired from the vehicle body data, and the target construction surface is obtained from three-dimensional design data. The position data acquisition unit 15 calculates the position data of the hydraulic excavator 100 using a Global Navigation Satellite System (GNSS), a Total Station (TS), or the like. The position data includes data not only in the planar direction but also in the height direction. The position data is, for example, a combination of latitude, longitude, and altitude.

[0014] The vehicle control unit 16 controls the hydraulic excavator 100 based on operation data received from the server 200. The construction machine communication unit 17 is a communication module that enables communication with other devices via the input device 300. The construction machine communication unit 17 may communicate directly with the server 200 and the input device 300, or it may communicate via at least one of a mobile phone base station, a satellite, and the internet. The construction machine communication unit 17 transmits vehicle data, including state quantities related to the front device 1005, and attitude data of the front device 1005 to the server 200. As mentioned above, the vehicle data includes position data of the hydraulic excavator 100.

[0015] The server 200 includes a notification operation amount calculation unit 21, a work content estimation unit 22, a database 23, a current status data creation unit 24, and a server communication unit 25. The server 200 also stores the notification item correspondence table 81 and notification expression table 82 (see Figures 4 and 5), which are created in advance and described later. The database 23 stores three-dimensional design data 93 and current terrain data 94, which is the current terrain. (See Figure 8) The three-dimensional design data 93 is the design data of the target terrain to be formed by construction, and is data that shows the terrain when construction by the hydraulic excavator 100 is completed. At the start of construction work by the hydraulic excavator 100, the three-dimensional design data and the current terrain data 94 do not match. At the end of construction work by the hydraulic excavator 100, the three-dimensional design data 93 and the current terrain data 94 match. The three-dimensional design data 93 is created in advance and is not updated in this embodiment. The current terrain data 94 is created or updated by the current status data creation unit 24.

[0016] The work content estimation unit 22 estimates the current work content of the hydraulic excavator 100 using the vehicle body data, the three-dimensional design data 93, and the current terrain data 94. The notification operation amount calculation unit 21 determines the information item to be transmitted from the work content estimated by the work content estimation unit 22 and the notification item correspondence table 81 which is a correspondence table between work contents and information to be transmitted to the operator, and calculates the notification operation amount using at least the posture data. The notification operation amount calculated by the notification operation amount calculation unit 21 is transmitted to the input device 300 by the server communication unit 25. The server communication unit 25 is a communication module that realizes communication of the server 200 with other devices. The server communication unit 25 may communicate directly with the hydraulic excavator 100 and the input device 300, or may communicate via at least one of a mobile phone base station, an artificial satellite, and the Internet. The server communication unit 25 transmits the notification operation amount 95 calculated by the notification operation amount calculation unit 21 to the input device 300.

[0017] The input device 300 includes an operation unit 31, an operation data collection unit 32, a notification control unit 33, a notification device 34, and an input device communication unit 35. The operation unit 31 is an input device operated by an operator, and includes an analog stick 31A and an analog trigger 31B. The numbers of the analog sticks 31A and the analog triggers 31B are not limited, and it is sufficient that each includes at least one. The operation data collection unit 32 collects operations on the operation unit 31 by the operator as operation data. The operation data is, for example, a combination of the direction and tilt angle of the analog stick 31A that is tilted, and the depression amount of the analog trigger 31B. Each of the direction, the angle, and the depression amount is represented by a value of 0 to 255, for example.

[0018] The notification device 34 is a device for notifying the operator. The form of notification by the notification device 34 is not particularly limited. In the present embodiment, vibration is used for notification, but voice or video may be used. The notification device 34 is, for example, a vibration motor, a piezoelectric element, a speaker, a liquid crystal display, or the like. The notification device 34 may be a combination of a plurality of devices, for example, the notification device 34 may be composed of a plurality of motors.

[0019] The notification control unit 33 controls the notification device 34 based on the notification operation amount transmitted from the server 200. For example, the notification control unit 33 changes the amplitude and frequency of the vibration motor, which is the notification device 34, based on the notification operation amount. If the notification device 34 is composed of multiple devices, the notification operation amount may include data that identifies the device, such as a motor identifier. The input device communication unit 35 is a communication module that enables communication between the input device 300 and other devices. The input device communication unit 35 may communicate directly with the server 200 and the hydraulic excavator 100, or it may communicate via at least one of a mobile phone base station, a satellite, and the internet. The input device communication unit 35 transmits operation data to the server 200. This operation data is then transmitted from the server 200 to the hydraulic excavator 100.

[0020] Figure 2 is an external view of the hydraulic excavator 100. The hydraulic excavator 100 comprises a lower traveling body 1003, an upper slewing body 1004, and a front device 1005. The lower traveling body 1003 comprises a pair of traveling sections 1006 and a traveling hydraulic motor 1007. The traveling sections 1006 are driven by the traveling hydraulic motor 1007. The upper slewing body 1004 is connected to the lower traveling body 1003 in the vertical direction shown, with a slewing hydraulic motor 1008 as the axis. By operating the slewing hydraulic motor 1008, the upper slewing body 1004 rotates relative to the lower traveling body 1003.

[0021] The upper slewing body 1004 is equipped with an engine 1001, which is the prime mover, and a main pump 1002 driven by the engine 1001. The hydraulic fluid supplied by the main pump 1002 allows the lower traveling body 1003, the upper slewing body 1004, and the front device 1005 to operate independently. The front device 1005 is mounted on the upper slewing body 1004. The front device 1005 is equipped with a boom 1009, a boom cylinder 1010 for driving the boom 1009, an arm 1011, an arm cylinder 1012 for driving the arm 1011, a bucket 1013, and a bucket cylinder 1014 for driving the bucket 1013. Each cylinder extends and retracts due to the hydraulic fluid supplied from the main pump 1002. The hydraulic excavator 100 performs excavation and leveling work by driving the upper rotating body 1004, the rotation axis 1015 of the boom 1009, the boom 1009, the rotation axis 1016 of the arm 1011, the arm 1011, and the rotation axis 1017 of the bucket 1013.

[0022] Figure 3 is a hardware configuration diagram of server 200. Server 200 comprises a central processing unit (CPU) 41, a read-only memory device (ROM) 42, a read-write memory device (RAM) 43, and a server communication unit 25. The CPU 41 performs the various calculations mentioned above by loading the program stored in ROM 42 into RAM 43 and executing it. Server 200 may be implemented using an FPGA (Field Programmable Gate Array), which is a rewritable logic circuit, or an ASIC (Application Specific Integrated Circuit), which is an application-specific integrated circuit, instead of the combination of CPU 41, ROM 42, and RAM 43. Furthermore, server 200 may be implemented using a different configuration combination, such as a combination of CPU 41, ROM 42, RAM 43 and FPGA, instead of the combination of CPU 41, ROM 42, and RAM 43.

[0023] In Figure 3, for convenience, the server 200 is shown as being composed of a single hardware device, but the server 200 may be composed of multiple hardware devices. In this case, the hardware devices may be installed adjacent to each other, or they may be connected via a local area network or the internet. The functions for performing calculations for the hydraulic excavator 100 and the input device 300 can also be realized by the combination of CPU 41, ROM 42, and RAM 43 shown in Figure 3.

[0024] Figure 4 shows an example of a notification item correspondence table 81. The notification item correspondence table 81 contains multiple records, each record having fields for estimation condition 4100, work group 4200, work content 4300, and notification item 4400. The notification item correspondence table 81 is referenced to identify the work content and the notification item corresponding to the identified work content. Estimation condition 4100 is the condition for identifying the work content 4300. For example, the first record states that the work content 4300 is estimated to be "heavy equipment delivery" if the following three conditions are met: "location with a slope of 3% or less", "intermittent forward and reverse operation", and "vehicle body tilt not present in the current terrain data or 3D design drawings".

[0025] Work group 4200 is a classification of work content 4300. For example, the work content 4300 listed in records 1 to 3, namely "Heavy equipment loading," "Moving," and "Climbing a slope," all belong to work group 4200 "Moving System." Work content 4300 describes the work being performed by hydraulic excavator 100. Notification items 4400 are items that should be reported in response to work content 4300. For example, if "Heavy equipment loading," the work content 4300 listed in the first record, is estimated, then "High center of gravity" and "Tilting of construction equipment" will be selected as items to be reported.

[0026] The work content estimation unit 22 may estimate multiple work contents 4300 simultaneously. For example, if the estimation conditions 4100 for the third and sixth records, namely "locations with a large slope in the current terrain data" and "locations where the current terrain is lower than the three-dimensional design data," are simultaneously met, the work content estimation unit 22 may estimate that two work contents 4300, namely "climbing" and "filling in holes," are being performed simultaneously.

[0027] Figure 5 shows an example of a notification representation table 82. The notification representation table 82 contains multiple records, each record having fields for notification item 5100, actuator 5200, operation pattern 5300, and presentation amount 5400. Notification item 5100 corresponds to and is identical to notification item 4400 in Figure 4. Actuator 5200 indicates a notification device 34 used for notification. In actuator 5200, the "vibration actuator of the grip portion" refers to the first vibration motor 3021 or the second vibration motor 3022 (see Figure 7) provided on the operation section 31 of the input device 300 shown in Figure 1, and the "analog stick reaction motor" refers to the second reaction motor 3012 (see Figure 7) provided on the operation section 31 of the input device 300 shown in Figure 1. Although not shown in Figure 5, the left and right of notification item 5100 and the left and right of actuator 5200 may be linked. For example, if notification item 5100 is "high risk of falling," the left vibration actuator on the grip may be vibrated if there is a high risk of falling to the left, and the right vibration actuator on the grip may be vibrated if there is a high risk of falling to the right.

[0028] The operation pattern 5300 indicates the pattern of operation used for notification. Even when using the same actuator 5200, multiple items can be notified to the operator by changing the operation pattern 5300. The presentation quantity 5400 indicates the parameter used for notification. For example, it indicates whether to change the amplitude or the frequency. For example, as shown in the first record, if "high center of gravity" is selected as the notification item 5100, the "vibration actuator of the grip part" is used for notification. In this case, the actuator operates in a sine wave pattern over time, and is controlled so that the amplitude increases as the center of gravity is higher.

[0029] Figure 6 shows an example of actuator operation patterns and output quantities. The first example waveform 5310 shows the case where the amplitude of a sine wave increases over time. Referring to the example in Figure 5, this figure shows that the center of gravity rises over time. The second example waveform 5330 shows the case where the amplitude of a triangular wave increases over time. Referring to the example in Figure 5, this figure shows that the risk of tipping over increases over time. The third example waveform 5340 shows a superposition of a sine wave and a triangular wave. Referring to the example in Figure 5, this figure shows that both the high center of gravity and the high risk of tipping over are left unaddressed, and that both the center of gravity and the risk of tipping over increase over time.

[0030] Figure 7 is a diagram showing the configuration of the control unit 31. The control unit 31 is a control device used, for example, in video games or radio-controlled toys. An operator operates the control unit 31, and this operation input is collected as operation data by the operation data acquisition unit 32. The control unit 31 includes a first analog trigger 3001, a second analog trigger 3002, an analog stick 3003, a first reaction motor 3011, a second reaction motor 3012, a first vibration motor 3021, a second vibration motor 3022, and a push button 3801.

[0031] The first analog trigger 3001 and the second analog trigger 3002 are used to input the forward and reverse movement of the lower vehicle 1003. The analog stick 3003 is used to input the operation of the front device 1005. The push button 3801 is set to an ON or OFF state by the operator. The operator's operations on the first analog trigger 3001, the second analog trigger 3002, the analog stick 3003, and the push button 3801 are collected as operation data by the operation data acquisition unit 32.

[0032] The first reaction motor 3011 acts on the first analog trigger 3001 and the second analog trigger 3002. The first reaction motor 3011 and the second reaction motor 3012 express information related to movement, such as load information applied to the lower traveling body 1003, as a reaction force to the user's operation. The second reaction motor 3012 acts on the analog stick 3003 and expresses information related to work, such as load information applied to the front device 1005, as a reaction force to the user's operation. The first vibration motor 3021 and the second vibration motor 3022 provide information such as the risk of tipping over and information related to the 3D machine guidance system.

[0033] Figure 8 illustrates the three-dimensional design data 93 and the current terrain data 94. In Figure 8, the hatching indicates the current terrain data 94, and the solid lines indicate the three-dimensional design data 93. As mentioned above, the same three-dimensional design data 93 is stored in the 3D machine guidance unit 2103 and the database 23. The three-dimensional design data 93 is a drawing showing the terrain after the work by the hydraulic excavator 100 has been completed, in other words, a drawing showing the terrain that the operator is targeting.

[0034] The existing terrain data 94 shown in Figure 8 is a flat plane, while the three-dimensional design data 93 has an embankment 7012 and a hole 7011. Therefore, the difference between the three-dimensional design data 93 and the existing terrain data 94 shown in Figure 7 is the absence of the embankment 7012 and the hole 7011. Thus, at locations where the hole 7001 exists in the three-dimensional design data 93, i.e., where the elevation in the existing terrain data 94 is higher than that in the three-dimensional design data 93, it is clear that the operator will need to use the hydraulic excavator 100 to excavate. Also, at locations where the embankment 7012 exists in the three-dimensional design data 93, i.e., where the elevation in the existing terrain data 94 is lower than that in the three-dimensional design data 93, it is clear that the operator will need to use the hydraulic excavator 100 to spread and compact the embankment. In this way, the relationship between the three-dimensional design data 93 and the existing terrain data 94 limits the types of work that can be performed.

[0035] Figure 9 is a flowchart showing the processing of the hydraulic excavator 100. First, in step S311, the construction machine communication unit 17 establishes communication with the server 200 and the input device 300 and proceeds to step S312. In step S312, the vehicle operation unit 16 receives operation data from the server 200. In the following step S313, the vehicle operation unit 16 controls the hydraulic excavator 100 based on the operation data received in step S312.

[0036] In the following step S314, the position data acquisition unit 15 and the vehicle body data acquisition unit 11 measure the position and orientation of the hydraulic excavator 100. The position data acquisition unit 15 calculates the position data of the hydraulic excavator 100. The vehicle body data acquisition unit 11 acquires data on the movable parts of the hydraulic excavator 100, such as the current stroke position of each cylinder in the front device 1005 and the slewing angle of the upper slewing body 1004, and calculates the orientation of the hydraulic excavator 100.

[0037] In the following step S315, the construction history acquisition unit 13 calculates the construction history. Specifically, the construction history acquisition unit 13 acquires the three-dimensional position of the claw of the front device 1005 included in the vehicle body data and creates a construction history, which is a history of the three-dimensional position of the claw. The three-dimensional position of the claw can be calculated from the position data, which is the current position of the hydraulic excavator 100, and the posture of the hydraulic excavator 100 calculated in step S314. The construction history may be a history of the three-dimensional position of the claw since the hydraulic excavator 100 was last started, or a history of the three-dimensional position of the claw since the last time the construction history was sent to the server 200.

[0038] In the following step S316, the load calculation unit 12 measures the hydraulic pressure and calculates the vehicle load data. As mentioned above, the vehicle data acquired by the vehicle data acquisition unit 11 also includes data on the current hydraulic pressure of the hydraulic excavator 100. Therefore, the load calculation unit 12 acquires the hydraulic pressure data from the vehicle data and calculates the load using a known calculation formula. In the following step S317, the construction machinery communication unit 17 transmits the measurement results and calculation results to the server 200. Specifically, the construction machinery communication unit 17 transmits the measurement results of the vehicle's position and attitude in step S314 and the calculation results in steps S315 to S316 to the server 200.

[0039] In the following step S318, the hydraulic excavator 100 determines whether or not it has received a termination command. If the hydraulic excavator 100 determines that it has received a termination instruction, it terminates the process shown in Figure 9; if it determines that it has not received a termination command, it returns to step S312. The termination command may be transmitted from the input device 300 to the server 200 and the hydraulic excavator 100, for example, by an operator operating the controller 3000. The hydraulic excavator 100 and the server 200 may also have a mechanism that transmits a termination command to themselves and other devices when predetermined conditions are met. For example, when the remote termination button provided on the hydraulic excavator 100 is pressed, a termination command may be transmitted to each device.

[0040] Although this flowchart shows steps S312 to S317 being executed sequentially, the order of execution may be changed, and the execution frequencies may differ. For example, steps S314 to S317 may be executed before steps S312 to S313, or steps S314 to S316 may be executed any number of times before step S317 is executed.

[0041] Figure 10 is a flowchart showing the processing of the server 200. In step S321, the server communication unit 25 establishes communication with the hydraulic excavator 100 and the input device 300 and proceeds to step S322. In step S322, the server communication unit 25 transmits the operation data received from the input device 300 to the hydraulic excavator 100. In the following step S323, the server communication unit 25 receives the construction history and vehicle data from the hydraulic excavator 100 and updates the database 23. The server communication unit 25 may add the received construction history and vehicle data to the database 23 as is, or it may add only the difference from the previous version to the database 23. Furthermore, the server communication unit 25 may add the update date and time when updating the database 23.

[0042] In the following step S324, the current condition data creation unit 24 creates or updates current terrain data 94 using the construction history stored in the database 23 and saves it to the database 23. The construction history is a history of the three-dimensional position data of the tip of the front device 1005, so it includes data in the height direction as well as the horizontal direction. Therefore, the current condition data creation unit 24 can use the construction history to calculate the current ground height, i.e., the terrain, at each position in the horizontal direction.

[0043] In the following step S325, the work content estimation unit 22 estimates the work content by referring to the database 23. Specifically, the work content estimation unit 22 refers to the vehicle body data and determines whether each of the estimation conditions 4100 in the notification item correspondence table 81 is met. The work content estimation unit 22 then estimates that the work content 4300 for a given record has been executed if all of the estimation conditions 4100 for that record are met. The work content estimation unit 22 may also refer to the three-dimensional design data 93 as needed.

[0044] In the following step S326, the notification operation amount calculation unit 21 calculates the notification operation amount based on the work content estimated in step S325 and transmits it to the input device 300. First, the notification operation amount calculation unit 21 refers to the notification item correspondence table 81 to identify the notification item 4400 corresponding to the work content 4300 estimated in step S325. Next, the notification operation amount calculation unit 21 uses the vehicle data stored in the database 23 to determine the degree of notification for notification item 4400. For example, if notification item 4400 is "high center of gravity", the height of the center of gravity of the hydraulic excavator 100 is calculated using the vehicle data and the height of the center of gravity is evaluated. The relationship between the height of the center of gravity and the degree of notification, for example, the amplitude, may be a monotonically increasing linear function relationship or a higher-dimensional function relationship.

[0045] Furthermore, if the notification item 4400 is "high risk of tipping over," the notification unit 21 calculates the center of gravity of the hydraulic excavator 100 and evaluates that the risk of tipping over is higher the further the center of gravity of the hydraulic excavator 100 extends outward from the lower traveling body 1003. Also, if the notification item 4400 is "crawler engagement with the ground," the notification unit 21 evaluates that slippage has occurred, i.e., the crawler is not engaged with the ground, if the position of the hydraulic excavator 100 has not changed despite detecting the rotation of the crawler from the hydraulics. Furthermore, the notification unit 21 refers to the notification expression table 82 to identify the actuator 5200, operation pattern 5300, and presentation amount 5400 for the same notification item 5100 as notification item 4400. Finally, the notification unit 21 converts the calculated height into a presentation amount 5400 and calculates the notification amount.

[0046] In the following step S327, the input device communication unit 35 determines whether or not it has received a termination command. If the input device communication unit 35 determines that it has received a termination instruction, it terminates the process shown in Figure 10; otherwise, it returns to step S322. In this flowchart, steps S322 to S326 are shown to be executed consecutively, but the order of execution may be changed, or the execution frequency may differ. For example, step S322 may be executed before or after at least one of steps S323 to S326.

[0047] Figure 11 is a flowchart showing the processing of the input device 300. In step S331, the input device 300 establishes communication with the hydraulic excavator 100 and the server 200 and proceeds to step S332. In step S332, the operation data collection unit 32 creates operation data based on the operator's operation and transmits it to the server 200 via the input device communication unit 35. In the following step S333, the notification control unit 33 receives the notification operation amount from the server 200. In the following step S334, the notification control unit 33 operates the notification device 34 based on the notification operation amount received in step S333. In the following step S335, the input device 300 determines whether or not it has received a termination command. If the input device communication unit 35 determines that it has received a termination instruction, it terminates the processing shown in Figure 11; if it determines that it has not received a termination command, it returns to step S332.

[0048] The correlation between Figures 9 and 11 is as follows: The operation data received by the hydraulic excavator 100 in step S312 of Figure 9 is transmitted from the input device 300 in step S322 of Figure 11. The data transmitted from the hydraulic excavator 100 to the server 200 in step S317 of Figure 9 is received in step S323 of Figure 10. The operation data received in step S322 of Figure 10 is transmitted from the input device 300 in step S332 of Figure 11. Similarly, the operation data transmitted to the hydraulic excavator 100 in step S322 of Figure 10 is received in step S312 of Figure 9.

[0049] The notification operation amount transmitted to the input device 300 in step S326 of Figure 10 is received in step S333 of Figure 11. The operation data transmitted to the server 200 in step S332 of Figure 11 is received in step S322 of Figure 10. The notification operation amount 95 received in step S333 of Figure 11 is transmitted from the server 200 in step S326 of Figure 10.

[0050] According to the first embodiment described above, the following effects and advantages can be obtained. (1) The remote control system 1 comprises a construction machine, a hydraulic excavator 100, a server 200, and an input device 300. The hydraulic excavator 100 includes a construction machine communication unit 17 that transmits body data including state quantities related to the front device 1005, and attitude data of the front device 1005 to the server 200. The server 200 includes a database 23 that stores three-dimensional design data 93 and current terrain data 94 which is the current terrain, a work content estimation unit 22 that estimates the current work content of the hydraulic excavator 100 using the body data, three-dimensional design data 93, and current terrain data 94, a notification operation amount calculation unit 21 that determines the notification items to be transmitted from the work content estimated by the work content estimation unit 22 and a notification item correspondence table 81 which is a correspondence table between the work content and the information to be transmitted to the operator, and calculates a notification operation amount 95 using at least the attitude data, and a server communication unit 25. The input device 300 is equipped with a notification device 34, which operates according to a notification operation amount 95 transmitted from the server, and the construction machine communication unit 17 transmits at least the position data of the hydraulic excavator 100 to the server 200. As a result, the remote control system 1 can notify the operator according to the estimated work content. That is, the remote control system 1 estimates the work content of the hydraulic excavator 100, calculates a notification operation amount 95 for notification items corresponding to the estimated work content, and controls the notification device 34 based on the notification operation amount, thereby enabling notification to the operator according to the estimated work content. For example, if the work content is estimated to be heavy equipment loading, the system can notify the operator of the height of the center of gravity and the tilt of the construction machine that should be paid attention to when loading heavy equipment, as shown in the first record of the example in Figure 4.

[0051] (2) The notification operation amount calculation unit 21 uses the three-dimensional design data 93 and the current terrain data 94 to calculate the notification operation amount.

[0052] (3) The vehicle data includes at least one of the hydraulic pressure of the front device 1005, position data, and tilt of the hydraulic excavator 100.

[0053] (4) The hydraulic excavator 100 is equipped with a construction history acquisition unit 13 that acquires the three-dimensional position of the claw of the front device 1005 included in the vehicle body data and creates a construction history which is a history of the three-dimensional position of the claw. The construction machine communication unit 17 transmits the construction history to the server 200, and the server 200 is equipped with a current data creation unit 24 that creates current terrain data 94 using the construction history and outputs it to the work content estimation unit 22. The work content estimation unit 22 uses the current terrain data 94 created by the current data creation unit 24.

[0054] (5) The hydraulic excavator 100 is equipped with a hydraulic sensor and a load calculation unit 12 that calculates the load data of the construction machine using the output value of the hydraulic sensor. The construction machine communication unit 17 transmits the load data to the server 200, and the work content estimation unit 22 estimates the work content using the load data, as shown in the fourth record from the top in the example shown in Figure 4.

[0055] (6) The notification operation amount calculation unit 21 refers to the notification item correspondence table 81 to identify the notification item 4400 corresponding to the work content 4300 output by the work content estimation unit 22. Furthermore, the notification operation amount calculation unit 21 refers to the notification expression table 82 to identify the type of notification device 34 corresponding to the notification item 5100 with the same name, namely the actuator 5200 and the operation pattern 5300 of the notification device 34.

[0056] (Variation 1) In the first embodiment described above, the hydraulic excavator 100, server 200, and input device 300 had different configurations. However, the server 200 may be integrated with either the hydraulic excavator 100 or the input device 300. The server 200 has no hardware features and no particular restrictions on its installation location. Therefore, from the viewpoint of ease and speed of implementation, there is an advantage to deploying the server 200 on the cloud. However, from the viewpoint of reducing the total number of hardware components constituting the remote control system 1 and simplifying management, there is an advantage to integrating the server 200 with either the hydraulic excavator 100 or the input device 300.

[0057] According to this modified example, in addition to the effects and advantages of the first embodiment, the following effects and advantages can be obtained. (7) The server 200 is configured as an integral part of the hydraulic excavator 100 or the input device 300. Therefore, the total number of hardware components constituting the remote control system 1 is reduced, making it easier to manage.

[0058] (Modification 2) In the first embodiment described above, the work content estimation unit 22 and the notification operation amount calculation unit 21 used vehicle data obtained after the hydraulic excavator 100 actually operated for calculation. However, the work content estimation unit 22 and the notification operation amount calculation unit 21 may also perform calculations using operation data before the hydraulic excavator 100 actually operates. As explained in the first embodiment, operation data is transmitted from the input device 300 to the hydraulic excavator 100 via the server 200. Therefore, the server 200 can grasp the operation data before the hydraulic excavator 100 actually operates based on the operation data. By further introducing simulation software for the hydraulic excavator 100 into the server 200, the operation of the hydraulic excavator 100 based on the operation data can be calculated in advance.

[0059] According to this modified example, in addition to the effects and advantages of the first embodiment, the following effects and advantages can be obtained. (8) The input device 300 includes a notification device 34, an operating unit 31 operated by the operator, and an input device communication unit 35 that transmits the operator's operation of the operating unit 31 as operation data to the server 200. The work content estimation unit 22 estimates the work content using the posture data and operation data. Therefore, before the hydraulic excavator 100 actually operates, the operation of the hydraulic excavator 100 is calculated from the operation data and vehicle data is generated, allowing the operator to be notified at an early stage.

[0060] (Variation 3) Figure 12 is a functional configuration diagram of the remote control system 1 in this modified example. In the first embodiment described above, the load calculation unit 12 was mounted on the hydraulic excavator 100. However, the load calculation unit 12 may be mounted on the server 200. In this case, as shown in Figure 12, the hydraulic excavator 100 does not have the load calculation unit 12, and the server 200 has the load calculation unit 12. The output values ​​of the hydraulic sensors used by the load calculation unit 12 for calculations are included in the vehicle data transmitted from the hydraulic excavator 100. The work content estimation unit 22 estimates the work content using the vehicle load data calculated by the load calculation unit 12, which is the same as in the first embodiment.

[0061] According to this modified example, in addition to the effects and advantages of the first embodiment, the following effects and advantages can be obtained. (9) The hydraulic excavator 100 is equipped with a hydraulic sensor. The construction machine communication unit 17 transmits the output value of the hydraulic sensor to the server 200. The server 200 is equipped with a load calculation unit 12 that calculates the vehicle load data of the hydraulic excavator 100 using the output value of the hydraulic sensor. The work content estimation unit 22 estimates the work content using the vehicle load data. This reduces the computational load on the hydraulic excavator 100.

[0062] (Modification 4) In the first embodiment described above, the work content estimation unit 22 and the notification operation amount calculation unit 21 used vehicle data obtained after the hydraulic excavator 100 was actually operating for calculation. However, the work content estimation unit 22 and the notification operation amount calculation unit 21 may also assume that the current operation data input is maintained for a predetermined period, for example, 2 seconds, estimate the position and posture of the hydraulic excavator 100 after a predetermined time, and calculate the work content estimation and notification operation amount based on the estimation result.

[0063] (Other variations) The first embodiment described above may be modified as follows.

[0064] (Modification 1) The control unit 31 may adopt the shape of an input device typically found in the driver's seat of a hydraulic excavator, rather than the shape of a video game or radio control controller.

[0065] (Modification 2) The control unit 31 may be equipped with an operating pedal instead of the pair of analog triggers 340. In this case, the operating pedal may be hardware separate from the control unit 31, or the operating pedal and the control unit 31 may be integrated into one unit.

[0066] (Modification 3) The first vibration motor 3021 and the second vibration motor 3022 may be built into the operator's seat instead of being built into the control unit 31.

[0067] (Amendment 4) The construction machinery comprising the remote control system 1 is not limited to the hydraulic excavator 100, but may be applied to various construction machinery equipped with a front device.

[0068] (Change 5) The vehicle body data does not need to include the position data of the hydraulic excavator 100. However, in this case, the construction machinery communication unit 17 transmits the position data to the server 200 independently of the vehicle body data.

[0069] In the embodiments and modifications described above, the configuration of the functional blocks is merely an example. Several functional configurations shown as separate functional blocks may be integrated, or a configuration represented in one functional block diagram may be divided into two or more functions. Furthermore, some of the functions of one functional block may be provided by other functional blocks.

[0070] In the embodiments and modifications described above, the program is stored in the ROM 42, but the program may also be stored in a non-volatile storage device (not shown). Furthermore, the server 200 may have an input / output interface (not shown), and the program may be read from other devices via the input / output interface and a medium available to the server 200 when needed. Here, "medium" refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through such a network. Also, some or all of the functions realized by the program may be realized by hardware circuits or FPGAs.

[0071] The embodiments and modifications described above may be combined in any way. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments that can be conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0072] 1: Remote control system 11: Vehicle body data acquisition unit 12:Load calculation section 13: Construction History Acquisition Department 15: Location data acquisition unit 16: Vehicle body control unit 17: Construction Machinery Communications Department 21: Notification operation amount calculation unit 22: Work Content Estimation Department 23: Database 24: Current Status Data Creation Department 25: Server Communication Unit 31:Operation unit 32: Operation Data Collection Unit 33: Notification Control Unit 34: Notification device 35: Input device communication unit 81: Notification Item Correspondence Table 82: Reporting Expression Table 93: Three-dimensional design data 94: Current topographic data 95: Notification activity 100: Hydraulic excavator 1005: Front device

Claims

1. In a remote control system comprising construction machinery, a server, and an input device, The aforementioned construction machine, The construction machinery communication unit includes a unit that transmits vehicle body data, including state quantities related to the front device, and attitude data of the front device to the server. The aforementioned server, A database that stores 3D design data and current terrain data, A work content estimation unit that estimates the current work content of the construction machine using the vehicle body data, the three-dimensional design data, and the current terrain data, A notification action amount calculation unit determines the notification items to be transmitted from the work content estimated by the work content estimation unit and the correspondence table between the work content and the information to be transmitted to the operator, and calculates the notification action amount for the notification items using at least the posture data, It includes a server communication unit, The input device is equipped with a notification device, The notification device operates according to the notification operation amount transmitted from the server, The construction machinery communication unit is a remote control system that transmits at least the location data of the construction machinery to the server.

2. In the remote control system according to claim 1, The notification operation amount calculation unit is a remote control system that uses the three-dimensional design data and the current terrain data to calculate the notification operation amount.

3. In the remote control system according to claim 1, The construction machine further includes a construction history acquisition unit that acquires the three-dimensional position of the toe of the front device included in the vehicle body data and creates a construction history which is a history of the three-dimensional position of the toe. The construction machinery communication unit transmits the construction history to the server. The server further comprises a current conditions data creation unit that creates the current conditions terrain data using the construction history and outputs it to the work content estimation unit. The aforementioned work content estimation unit is a remote operation system that uses the current terrain data created by the aforementioned current data creation unit.

4. In the remote control system according to claim 1, The construction machine further comprises a hydraulic sensor and a load calculation unit that calculates vehicle load data of the construction machine using the output value of the hydraulic sensor. The construction machinery communication unit transmits the vehicle load data to the server. The aforementioned work content estimation unit further estimates the work content using the vehicle load data, and is a remote control system.

5. In the remote control system according to claim 1, The aforementioned construction machine is equipped with a hydraulic sensor, The construction machinery communication unit transmits the output value of the hydraulic sensor to the server. The server further comprises a load calculation unit that calculates vehicle load data of the construction machine using the output value of the hydraulic sensor. The aforementioned work content estimation unit further estimates the work content using the vehicle load data, and is a remote control system.

6. In the remote control system according to claim 1, The aforementioned input device is The operating unit operated by the aforementioned operator, The system further comprises an input device communication unit that transmits the operation of the control unit by the operator to the server as operation data, The work content estimation unit is a remote operation system that estimates the work content using the posture data and the operation data.

7. In the remote control system according to claim 1, The notification operation amount calculation unit identifies a presentation item corresponding to the work content output by the work content estimation unit, and identifies the type of notification device and the operation pattern of the notification device corresponding to the presentation item, in a remote control system.

Images