WORK SUPPORT SYSTEM FOR EXCAVATORS

Abstract

A work support system for an excavator, capable of further improving the excavator's work efficiency, is provided. The work support system comprises an excavator; an environmental detection device configured to detect environmental information from the excavator's construction site; and a simulation device configured to capture the environmental information detected by the environmental detection device during excavator movement and generate a three-dimensional virtual spatial model of the construction site. Thus, the excavator's work system updates the environmental information according to the excavator's work and adjusts the movement content based on the environmental information, thereby further improving the excavator's work efficiency.

Description

TECHNICAL AREA

[0001] The present disclosure relates to a work support system for an excavator. STATE OF THE ART

[0002] Conventionally, a work support system for assisting excavation movements and the like of an excavator is known in order to increase the excavator's work efficiency. Patent document 1, for example, discloses a work support system for evaluating the work quality of multiple operators, for learning the parameters of the excavator's operating model based on the operating data of the best operator, and for supporting the operator based on the learned operating model.

[0003] With this type of excavator work support system, by examining and recognizing in advance earth and sand information such as the soil quality of the site to be excavated by the excavator (excavation object), the movement content of the excavator, which corresponds to the earth and sand information, can be adjusted in the excavation movement. QUOTE LIST PATENT DOCUMENT Patent document 1: Japanese unexamined Patent application Publication no. 2016-156193 SUMMARY OF INVENTIONAL PROBLEM

[0004] Furthermore, even if the soil and sand information is recorded in advance, the soil and sand conditions on the actual construction site can change slightly depending on the excavation location, excavation depth, etc. Therefore, even if the work support system adjusts the excavator's movement based on the soil and sand information, the actual excavator may not be able to perform the excavation as planned.

[0005] The present disclosure provides a work support system for an excavator that is able to further improve the work efficiency of an excavator by updating environmental information in conjunction with the movement of the excavator and setting movement content based on the environmental information. SOLUTION TO PROBLEM

[0006] According to one aspect of the present disclosure, a work support system for an excavator is provided, wherein the work support system comprises the excavator; an environment detection device configured to detect environmental information of a construction site of the excavator; and a simulation device configured to capture the environmental information detected by the environment detection device during a movement of the excavator and to generate a three-dimensional virtual spatial model of the construction site. ADVANTAGEOUS EFFECTS OF INVENTION

[0007] According to one aspect, the work efficiency of the excavator can be further improved by updating environmental information according to the movement of the excavator and adjusting operating content based on the environmental information. BRIEF DESCRIPTION OF THE DRAWINGS [ Fig. 1] Fig. Figure 1 is a diagram showing an example of a work support system according to the first embodiment. [ Fig. 2] Fig. Figure 2 is a functional block diagram showing a configuration example of the work support system. [ Fig. 3] Fig. Figure 3 is a diagram showing an example of a three-dimensional virtual space model generated by the virtual space generation part. [ Fig. 4] Fig. Figure 4 is an explanatory diagram showing the excavation movement of a virtual excavator. [ Fig. 5] Fig. Figure 5 is an explanatory diagram showing the earth and sand discharge movement of a virtual excavator. [ Fig. 6] Fig. Figure 6 is an explanatory diagram showing the control of an excavation movement of an excavator by the work support system according to the first embodiment. [ Fig. 7] Fig. Figure 7 is a diagram showing an example of a work support system according to the second embodiment. [ Fig. 8] Fig. Figure 8 is a functional block diagram showing a configuration example of a work support system. [ Fig. 9] Fig. Figure 9 is an explanatory diagram showing the control of an excavation movement of an excavator by the work support system according to the second embodiment. [ Fig. 10] Fig. Figure 10 is an explanatory diagram showing the control of an excavation movement of an excavator by the work support system according to the modification example. DESCRIPTION OF EXECUTION FORMS

[0008] An embodiment of the present disclosure is described below with reference to the drawings. In each of the drawings, the same components are designated by the same reference numerals, and duplicate descriptions may be omitted. [First embodiment]

[0009] First, the work support system SYS1 according to the first embodiment of the present invention is described with reference to Fig. 1 described. Fig. Figure 1 is a diagram showing a work support system SYS1 according to the first embodiment.

[0010] An excavator 100, which is applied to the work support system SYS1, comprises a lower running body 1, an upper rotating body 3 which is mounted on the lower running body 1 in such a way that it is able to rotate by means of a rotary mechanism 2, an excavation attachment AT and a cabin 10.

[0011] The lower chassis 1 of the excavator 100 of the present embodiment has a pair of right and left tracks 1C. The tracks 1C are driven by a drive hydraulic motor 2M, which is a drive actuator mounted on the lower chassis 1.

[0012] The upper rotating body 3 is rotatably mounted on the lower carriage 1 via a rotary mechanism 2. The rotary mechanism 2 is driven by a rotary hydraulic motor 2A, which is a rotary actuator mounted on the upper rotating body 3. The rotary actuator can be an electric actuator (an electric rotary generator).

[0013] A boom 4 is mounted on the upper rotating body 3. An arm 5 is mounted at the tip of the boom 4, and a bucket 6, serving as an end attachment, is mounted at the tip of the arm 5. The boom 4, the arm 5, and the bucket 6 form a digging attachment AT, which serves as an example of an attachment. The boom 4 is driven by a boom cylinder 7, the arm 5 is driven by a boom cylinder 8, and the bucket 6 is driven by a bucket cylinder 9. The boom cylinder 7, the boom cylinder 8, and the bucket cylinder 9 form an attachment actuator. The end attachment can be a slope bucket.

[0014] The boom 4 is mounted to be vertically rotatable relative to the upper rotating body 3. A boom angle sensor S1 is attached to the boom 4. The boom angle sensor S1 detects a boom angle α, which is a rotation angle of the boom 4. The boom angle α is, for example, the lifting angle from a state in which the boom 4 is fully lowered. Therefore, the boom angle α is at its maximum when the boom 4 is fully raised.

[0015] The arm 5 is rotatably mounted relative to the boom 4. An arm angle sensor S2 is attached to the arm 5. The arm angle sensor S2 detects an arm angle β, which is a rotation angle of the arm 5. The arm angle β is, for example, the opening angle from the fully closed state of the arm 5. Therefore, the arm angle β is at its maximum when the arm 5 is fully open.

[0016] The blade 6 is rotatably mounted relative to the arm 5. A blade angle sensor S3 is attached to the blade 6. The blade angle sensor S3 detects the blade angle γ, which is the rotation angle of the blade 6. The blade angle γ is the opening angle from the state in which the blade 6 is most closed. Therefore, the blade angle γ is at its maximum when the blade 6 is fully open.

[0017] Each of the boom angle sensor S1, arm angle sensor S2, and blade angle sensor S3 can use either a single accelerometer or a combination of an accelerometer and a gyroscope. Alternatively, the boom angle sensor S1 can be a stroke sensor attached to the boom cylinder 7, a rotary encoder, a potentiometer, or an inertial measuring device. The same applies to the arm angle sensor S2 and the blade angle sensor S3.

[0018] The upper rotating body 3 is equipped with the cabin 10, which serves as an operator's cabin, and a power source such as a motor 11 is mounted on it. The upper rotating body 3 is equipped with a space detection device 70, an orientation detection device 71, and a positioning device 72, and is also equipped with various sensors of the excavator 100, such as a machine body tilt sensor S4 and a rotation angular velocity sensor S5. Furthermore, an operating device 21, a dispensing device 22, a control unit 30, and the like are provided inside the cabin 10. For the sake of simplicity, in this description, the side of the upper rotating body 3 to which the excavation attachment AT is attached is referred to as the front, and the side to which the counterweight is attached is referred to as the rear.

[0019] The spatial detection device 70 is a device for detecting a three-dimensional real space (environmental information) around the excavator 100. The spatial detection device 70 is configured to measure the direction and distance to a detected object from either the spatial detection device 70 or the excavator 100. The spatial detection device 70 includes, for example, an ultrasonic sensor, a millimeter-wave radar, a monocular camera, a stereo camera, a LiDAR, a distance imaging sensor, an infrared sensor, or any combination thereof.In the present embodiment, the space detection device 70 comprises a front sensor 70F, which is attached to the front end of the upper surface of the cabin 10, a rear sensor 70B, which is attached to the rear end of the upper surface of the upper rotating body 3, a left sensor 70L, which is attached to the left end of the upper surface of the upper rotating body 3, and a right sensor 70R, which is attached to the right end of the upper surface of the upper rotating body 3. The space detection device 70 may include an upper sensor (not shown) attached to the excavator 100, which detects an object located in the space above the upper rotating body 3.

[0020] The orientation detection device 71 detects information relating to the relative relationship between the orientation of the upper rotating body 3 and the orientation of the lower driving body 1. The orientation detection device 71 can, for example, be formed by a combination of a geomagnetic sensor attached to the lower driving body 1 and a geomagnetic sensor attached to the upper rotating body 3. Alternatively, the orientation detection device 71 can be formed by a combination of a GNSS receiver attached to the lower driving body 1 and a GNSS receiver attached to the upper rotating body 3. The orientation detection device 71 can be a rotary encoder, a rotary position sensor, or any combination thereof.In a configuration where the upper rotating body 3 is driven by a three-phase generator to rotate, the orientation detection device 71 can be formed by a resolver. The orientation detection device 71 can, for example, be attached to a central joint provided with respect to the rotation mechanism 2 to implement relative rotation between the lower drive body 1 and the upper rotating body 3.

[0021] The orientation detection device 71 can be formed by a camera mounted on the upper rotating body 3. In this case, the orientation detection device 71 applies known image processing to the image information captured by the camera to extract an image of the lower driving body 1 that is included in the image information. Subsequently, the orientation detection device 71 detects the longitudinal direction of the lower driving body 1 from the image of the lower driving body 1 and derives an angle formed between the longitudinal direction of the upper rotating body 3 and the longitudinal direction of the lower driving body 1. The longitudinal direction of the upper rotating body 3 is derived from the mounting position of the camera. Since, in particular, the track 1C protrudes from the upper rotating body 3, the orientation detection device 71 can identify the longitudinal direction of the lower driving body 1 by detecting the image of the track 1C.The orientation detection device 71 can be integrated into the control unit 30. The camera can also use a room detection device 70.

[0022] The positioning device 72 is configured to measure the position of the upper rotating body 3. In the present embodiment, the positioning device 72 is a GNSS receiver that detects the position of the upper rotating body 3 and outputs the detected value to the controller 30. The positioning device 72 can also be a GNSS compass. Since the positioning device 72 can detect both the position and orientation of the upper rotating body 3 in this case, it also functions as an orientation detection device 71.

[0023] The machine body tilt sensor S4 detects the tilt of the upper rotating body 3 with respect to a predetermined plane. In the present embodiment, the machine body tilt sensor S4 is an accelerometer that detects the tilt angle of the upper rotating body 3 about the longitudinal axis and about the transverse axis with respect to the horizontal plane. The longitudinal axis and the transverse axis of the upper rotating body 3 are, for example, orthogonal to each other and pass through an excavator center point, which is a point on the rotation axis of the excavator 100.

[0024] The rotational acceleration sensor S5 detects the angular velocity of the upper rotating body 3. In the present embodiment, the rotational velocity sensor S5 is a gyroscope, but can also be a resolver, an encoder, or a combination thereof. The rotational velocity sensor S5 can detect the rotational speed. The rotational speed can be calculated from the angular velocity.

[0025] In the following, at least one of the boom angle sensor S1, arm angle sensor S2, bucket angle sensor S3, machine body tilt sensor S4, or rotary angle sensor S5 will also be referred to as a position detection device. The position of the excavation attachment AT is detected, for example, based on the respective outputs of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3.

[0026] The control device 21 is a device provided in the cab 10 for an operator to operate the excavator 100. The control device 21 has, for example, a control lever and a control pedal for controlling the drive of the excavator 100's actuator. The actuator comprises at least one hydraulic or electric actuator.

[0027] Furthermore, the operating device 21 includes an information input device (for example, a right console box, a left console box) for an operator of the excavator 100 to input information into the control unit 30. The information input device can, for example, be a control panel installed near the display device of the output device 22. Alternatively, the information input device can be a touch panel used as a display device, or a voice input device such as a microphone located in the cab 10. The information input device can also be a communication device for receiving information from an external source.

[0028] The output device 22 comprises at least one display device and one sound output device. The display device is a liquid crystal display installed in cabin 10. The display device may be a display of a portable device, such as a smartphone. The sound output device comprises at least one device for outputting sound to an operator in cabin 10 or a device for outputting sound to an operator outside cabin 10. The sound output device may be a loudspeaker of a portable device.

[0029] The controller 30 is a control device for controlling the excavator 100. In the present embodiment, the controller 30 is formed by a computer comprising one or more processors, memory (volatile memory, non-volatile memory), and the like. The one or more processors read a program corresponding to each function from the memory and execute it. Each function includes, for example, a machine guidance function for guiding manual operation of the excavator 100 by an operator, or a machine control function for automatic (or autonomous) operation of the excavator 100. The controller 30 may include a contact avoidance function to automatically operate or brake the excavator 100 to prevent contact between the excavator 100 and an object located around the excavator 100.

[0030] This means that work support is a term that includes the automatic operation of the excavator 100 on behalf of the operator, support for the operation of the excavator 100 by the operator, and the provision of operating information to the operator of the excavator 100. For the work support of the excavator 100, the work support system SYS1 uses the control unit 30, which is mounted on the excavator 100 and performs each function, and the environmental detection device 79 (see Fig. 2) which supplies information to the control unit 30. In addition to the work support system SYS1, which is formed solely by the excavator 100, an administrative device 200, capable of communicating with the excavator 100, may be mounted outside the excavator 100. By using the administrative device 200, the work support system SYS1 can support the movements of several excavators 100 (or other work machines) in such a way that they are coordinated. Furthermore, the work support system SYS1 may include an external spatial detection device 300, which has the same function as the spatial detection device 70, located outside the excavator 100.

[0031] Next, an example of the work support system SYS1 according to the first embodiment will be described with reference to Fig. 2 described. Fig. Figure 2 is a functional block diagram showing a configuration example of the work support system SYS1.

[0032] The excavator 100 comprises, as a configuration of the work support system SYS1, a control unit 30, a space detection device 70, an orientation detection device 71, a positioning device 72, various solenoid valves 41, various actuators 42, and a communication device T1. On the other hand, the management device 200 comprises a computer unit 210 for carrying out various processes of the work support system SYS1 and a communication device T2.

[0033] The external spatial detection device 300 detects the condition of the construction site where the excavator 100 is located. In addition to mapping the construction site, the detection of the site's condition includes, for example, measuring distance, shape, and direction. The external spatial detection device 300 comprises, for example, an ultrasonic sensor, a millimeter-wave radar, a monocular camera, a stereo camera, a LiDAR, a distance imaging sensor, an infrared sensor, or any combination thereof, installed at the construction site. The external spatial detection device 300 communicates either wirelessly or via a wired connection with at least one of the communication devices T1 of the excavator 100 or T2 of the management device 200 and successively transmits detection information about the detected condition of the construction site.

[0034] The control unit 30 of the excavator 100 comprises a virtual space generation part 31, an excavator state identification part 32, an excavator state estimation part 33, an actuator drive part 34, a determination part 36, an operational prediction part 37, an operational intervention part 38, and a motion simulator 39 as functional blocks. Although the virtual space generation part 31, the excavator state identification part 32, the excavator state estimation part 33, the actuator drive part 34, the determination part 36, the operational prediction part 37, the operational intervention part 38, and the motion simulator 39 are distinguished for the sake of simplicity, they need not be physically distinguished, and they can be wholly or partially comprised of common software or hardware components.

[0035] The computer body 210 of the management device 200 comprises, as functional blocks, a determination part 211, an operational prediction part 212, an operational intervention part 213, and a motion simulator 214. Although the determination part 211, the operational prediction part 212, the operational intervention part 213, and the motion simulator 214 are distinguished for the sake of simplicity, they need not be physically differentiated and may be wholly or partially comprised of common software or hardware components. The determination part 211, the operational prediction part 212, the operational intervention part 213, and the motion simulator 214 have the same functions as the determination part 36, the operational prediction part 37, the operational intervention part 38, and the motion simulator 39 of the controller 30. The management device 200 may include a virtual space generation part 215 (see the dotted line in Fig. 2), which has the same function as the virtual space generation part 31 of the controller 30. In the following, each functional block of the controller 30 is described as a representative example, and descriptions of each individual functional block of the computer body 210 are omitted. The work support system SYS1 is sufficient to have the functions of virtual space generation parts 31, 215, determination parts 36, 211, operational prediction parts 37, 212, operational intervention parts 38, 213, and motion simulators 39, 214 in at least one of the excavator 100 or the management device 200.

[0036] The virtual space generation unit 31 generates a three-dimensional virtual space model on the virtual three-dimensional coordinates within the virtual space generation unit 31, based on the detection information from the space detection device 70, the orientation detection device 71, and the positioning device 72, as well as the information from the operating device 21. The three-dimensional virtual space model is formed into a virtual rectangular parallelepiped, cube, sphere, or hemisphere according to the imaging area of ​​the space detection device 70. The three-dimensional virtual space model can be image information displayed on the display device of the output device 22. In this case, the three-dimensional virtual space model is a three-dimensional topographic image and is generated by computer graphics.

[0037] The three-dimensional virtual space model is typically multi-layered information in which object information is superimposed with topographic information, representing the topography around the excavator 100 as seen by the operator seated in the cab 10. Hereinafter, changeable information (parameters) applied to the three-dimensional virtual space model is referred to as environmental information. The environmental information includes topographic information and object information. The virtual space generation part 31 performs known image processing on the detection information acquired by at least one of the front sensor 70F, the rear sensor 70B, the left sensor 70L, or the right sensor 70R of the space detection device 70, and extracts the topographic information and object information contained within the detection information.At this point, for example, the object included in the current image information can be extracted by comparing several parts of past image information with the current image information. Then, the virtual space generation part 31 arranges the extracted topographic and object information in a three-dimensional virtual space model and reproduces information about the virtual environment surrounding cabin 10 (operator) in the three-dimensional virtual space model.

[0038] The object information extracted to render the three-dimensional virtual space model includes, for example, excavated earth (including mounds, holes, walls, trenches, etc.), static objects distinct from excavated objects, the excavator 100 itself, construction equipment such as another excavator, vehicles, animals including humans, plants, and the like. When arranging the object information in relation to the three-dimensional virtual space model, the virtual space generation unit 31 can use information such as the distance and direction between the excavator 100 and the object information measured by the ultrasonic sensor, millimeter-wave radar, LiDAR, and the like. Thus, various objects at the construction site where the excavator 100 is located, and the coordinates of the objects, are appropriately rendered in the three-dimensional virtual space model.

[0039] Furthermore, the virtual space generation unit 31 can receive the detection information from the external space detection device 300 and generate a three-dimensional virtual space model that is mapped by the external space detection device 300. Alternatively, the virtual space generation unit 31 can be configured to generate a three-dimensional virtual space model by integrating the detection information from the space detection device 70 of the excavator 100 and the detection information from the external space detection device 300.

[0040] Fig. Figure 3 is a diagram showing an example of a three-dimensional virtual space model 50 generated by the virtual space generation part 31. For example, a virtual excavator 51, obtained by reproducing the excavator 100 in a real space, is positioned in the three-dimensional virtual space model 50 of the virtual space generation part 31. The virtual excavator 51 corresponds to the shape, position, orientation, etc., of the excavator 100 at an actual construction site. The position, orientation, and the like of the virtual excavator 51 in the three-dimensional virtual space model 50 are identified by an excavator state identification part 32, which is described below, and are determined, for example, based on the detection information from at least one of the space detection device 70 or the orientation detection device 71.The position, orientation, and the like of the virtual excavator 51 can be determined or adjusted using the detection information of a position positioning device, the positioning device 72, and the like.

[0041] In the three-dimensional virtual space model 50, the excavated object 52 (hill in Fig. 3), which are the topographic information present around the virtual excavator 51, and a virtual dump truck 53 (unloading object), which is the object information, arranged, encompassing the shape itself. Although the representation of other environmental information in Fig. If step 3 is omitted, the object information extracted from the detection information is appropriately arranged in the three-dimensional virtual space model 50. Furthermore, the virtual space generation part 31 can generate the three-dimensional virtual space model 50 by leaving an image that is difficult to extract from the detection information as the background of the three-dimensional virtual space model 50.

[0042] Furthermore, according to the present embodiment, the virtual space generation part 31 links (adds) various types of additional information with the topographic information and object information represented in the three-dimensional virtual space model 50.

[0043] For example, in addition to the shape, position, or orientation of the virtual excavator 51, further information about the excavator is added, such as an identification number, type, operating time, type and orientation of the attachment AT, as well as soil and sand information (additional information) such as the weight, volume, and density of the soil and sand placed in the bucket 6. This additional excavator information can be estimated based on the detection information from the environmental detection device 79, or information previously stored in the controller 30 can be used.Furthermore, the earth and sand information stored in the excavator 100 can be estimated on the basis of information obtained by detecting the load applied when the excavator 100 is carrying out excavation, which is detected by a pressure sensor, a load sensor or the like, which is part of the environmental detection device 79.

[0044] That is, the environmental detection device 79 is a device that detects information that influences the environmental information forming the three-dimensional virtual space model 50 inside or outside the excavator 100. The environmental detection device 79 can include various sensors, such as a space detection device 70, an orientation detection device 71, a positioning device 72, a position detection device (the boom angle sensor S1, the arm angle sensor S2, the bucket angle sensor S3, the machine body tilt sensor S4, and the angular velocity sensor S5), a pressure sensor (not shown), a load sensor, and an operating sensor for the control device 21.The environmental detection device 79 can also include detection information from a device installed outside the excavator 100 (the external space detection device 300, other work machines and vehicles (for example, a dump truck)).

[0045] Furthermore, additional information about the excavated object 52 of the three-dimensional virtual space model 50 is added, for example, the amount of sediment accumulation, weight, density, hardness, and soil quality of the earth and sand. This additional information about the excavated object is estimated by the excavation condition estimator 33 described below, based on the detection information from the space detection device 70. Alternatively, the additional information about the excavated object can be estimated from the pressure and load exerted on the bucket 6 of the excavator 100 when the excavator 100 is in operation, the weight of earth and sand detected by the dump truck when the earth and sand are being unloaded into the dump truck, an image, or the like.Alternatively, some of the additional information about the excavation object can be taken from the construction site planning data, which was stored in advance in the control unit 30.

[0046] On the other hand, in addition to the shape, position, and orientation, the virtual dump truck 53 of the three-dimensional virtual space model 50 is given additional information about the unloading object, such as identification number, type, loading area size, loading area condition, and operating time. Furthermore, soil and sand information (additional information of the unloading object), such as weight, volume, and density of the soil and sand loaded on the loading area of ​​the virtual dump truck 53, can be added to the loading area of ​​the virtual dump truck 53 (the unloading object). The additional information of the unloading object can be estimated based on object information extracted from the detection information of the space recognition device 70, or information previously stored in the controller 30 can be used.The earth and sand information is added to the virtual dump truck 53 of the three-dimensional virtual space model 50 by detecting the weight, image, and other properties of the earth and sand on the actual dump truck, which change when the earth and sand are unloaded by the excavator 100, and receiving the information.

[0047] The additional information to be added to the environmental information can include various types of information different from those mentioned above. For example, if a person or other work machine exists around excavator 100, its object information is arranged in the three-dimensional virtual space model 50. It is preferred that additional information about the relative distance to excavator 100 be added to the object information. Furthermore, it is preferred that additional information such as identification number, type, and operating time be added to the information of other work machines.

[0048] The virtual space generation unit 31 can integrate the construction site planning data stored in the controller 30 into the three-dimensional virtual space model 50. The planning data contains the completed shape of the construction site excavation, and the virtual space generation unit 31 overlays and displays graphics, such as computer graphics, on the three-dimensional virtual space model 50, showing the position of the completed shape. Furthermore, the construction data can include the condition of the soil and sand (position, shape, soil quality, hardness) of the previously investigated excavation site, and this information can be added to the topographic information of the three-dimensional virtual space model 50.

[0049] The virtual space generation unit 31 then sequentially updates the environmental information of the three-dimensional virtual space model 50 based on the detection information from the space detection device 70, the orientation detection device 71, and the positioning device 72, the information from the control device 21, and the communication information from the external space detection device 300 and the dump truck. For example, when the excavator 100 excavates the excavation object at the construction site, the environmental information of the three-dimensional virtual space model 50 can be changed at that time based on the detection information. Even if the soil and sand information (additional information) of the excavation object 52 is added in advance, the soil and sand information may differ when the excavator 100 actually performs the excavation.In such a case, the virtual space generation unit 31 modifies or corrects the earth and sand information based on the detection information obtained when the excavator 100 actually carried out the excavation. Thus, the virtual space generation unit 31 can generate the three-dimensional virtual space model 50, which is even closer to the environment of the construction site.

[0050] Referring to Fig. 2. The excavator state identification part 32 is configured to identify the state of the excavator 100 (position of the excavation attachment AT, etc.), encompassing the position and orientation of the excavator 100. The position of the excavator 100 is, for example, the latitude, longitude, and altitude of the excavator 100's reference point. The excavator state identification part 32 identifies the position of the excavator 100 based on the output of the positioning device 72 and identifies the orientation of the excavator 100 based on the output of the orientation detection device 71. The position and orientation of the excavator 100, identified by the excavator state identification part 32, are displayed on the virtual excavator 51 of the three-dimensional virtual space model 50.

[0051] The excavation condition estimator 33 estimates additional information about the excavated object required for the operation of the excavator 100, based on the detection information from the environmental detection device 79. The term "operation" of the excavator 100 in this specification encompasses a series of movements, such as an excavation movement with respect to the earth and sand that constitute the excavated object, the transport of the earth and sand associated with the lifting and rotating motion of the excavator 100, an unloading movement of the earth and sand with respect to the loading platform of the skip, and the return of the lowering and rotating motion of the excavator 100. Examples of additional information about the excavated object include the hardness of the soil to be excavated, the density of the soil, the soil quality, and the like.During excavation, a reaction force is exerted on the excavation attachment AT by the excavation object in real space. Therefore, the excavation condition estimator 33 can estimate additional information about the excavation object based on the pressure detected by the pressure sensor located in the hydraulic path of each cylinder of the excavation attachment AT, or the load detected by the load sensor located at a suitable position on the excavation attachment AT. In addition to the excavation process relative to the excavation object, the excavation condition estimator 33 can estimate soil and sand information based on detection data from various sensors while the soil and sand are held by the excavation attachment AT.The additional information of the excavated object, estimated by the excavation condition estimator 33, is added to the earth and sand information of the virtual excavator 51 of the three-dimensional virtual space model 50 as described above and is appropriately reflected in the additional information of the excavated object 52, which is excavated by the virtual excavator 51.

[0052] The actuator drive unit 34 is configured to drive various solenoid valves 41 and various actuators 42 mounted on the excavator 100. The actuator drive unit 34 outputs an operating signal to the corresponding solenoid valve 41 or actuator 42, based on the control signal processed by the controller 30 itself, in addition to the operating signal from the operating device 21. By driving the solenoid valve 41 and the actuator 42 that have received the operating signal, the excavator 100 performs various movements (digging, lifting and rotating, earth and sand unloading, lowering and rotating, etc.) during operation.

[0053] On the other hand, the determining element 36 of the control unit 30 is configured to determine whether there is any information to be reported to the operator of the excavator 100 regarding the situation around the excavator 100. For example, the determining element 36 determines whether there is information to be reported to the operator of the excavator 100 based on at least one of the detection information from the environmental detection device 79 provided in the excavator 100, or the set movement content of the excavator 100. The determining element 36 can also be configured to determine, based on the detection information from the external spatial detection device 300, whether information needs to be reported to the operator of the excavator 100. The external spatial detection device 300 can, for example, be a sensor (camera, LiDAR, etc.) attached to another work machine, or a sensor (camera, LiDAR, etc.).), which is attached to a flying object, such as a multicopter (drone) flying over the construction site. For the determination, the determination part 36 can use the three-dimensional virtual space model 50 of the virtual space generation part 31 or the simulation result of the motion simulator 39 described below.

[0054] For example, the determining element 36, based on the detection information from the spatial detection device 70 or the external spatial detection device 300, determines at least one aspect of the position, orientation, or movement of another machine or vehicle. Whether or not a reportable event has occurred can be determined based on the presence or absence of the same or a similar situation compared to previous instances. For example, if the determining element 36 detects the presence of a person near the excavator 100 (within the movement range of the excavation attachment AT), the determining element 36 can determine that a reportable event has occurred.

[0055] If the determining element 36 detects the presence of a slope around the excavator 100, it can further determine that a situation must be reported to the operator. At this point, the determining element 36 can determine the gradient based on the topographic information from the planning data previously stored in the controller 30. Alternatively, if the determining element 36 detects the presence of an object (for example, an electrical cable) that could interfere with work outside the area covered by the three-dimensional virtual space model 50, the determining element 56 can determine that there is a situation that must be reported to the operator.

[0056] If it is determined that there is a situation that needs to be reported to the operator of excavator 100, the determining unit 36 ​​performs processing to alert the operator. For example, the determining unit 36 ​​transmits information about the situation to the output device 22. The output device 22 can then communicate the information received from the determining unit 36 ​​about the situation to the operator.

[0057] The operating prediction unit 37 is configured to predict the operating signal after a predetermined time based on the operating signal from the operating device 21 or the signal from the management device 200. This is to prevent the deterioration of operating responsiveness caused by delays due to processing overload and communication delays. The predetermined time is, for example, a few milliseconds up to a few tens of milliseconds. For instance, the operating prediction unit 37 predicts the operating signal after a predetermined time based on the transition of the operating signal (tilt angle of the operating lever of the operating device 21) during a predetermined time in the past.For example, if the operational prediction unit 37 detects that the tilt angle of the control lever has tended to increase over a predetermined time in the past, the operational prediction unit 37 predicts that the tilt angle will be greater than the current tilt angle after a predetermined time. Thus, the excavator can move 100 while the delay of the operating signal is reduced.

[0058] The operational intervention part 38 determines, based on the detection information from the environmental detection device 79, whether or not the operator should intervene in the operation of the excavator 100 and intervenes in the operation as necessary. For example, the operational intervention part 38 acts as a contact avoidance function to prevent contact between the excavator 100 and other tasks by intervening in the operator's operation. The operational intervention part 38 can use the three-dimensional virtual space model 50 of the virtual space generation part 31 or can use the simulation result of the motion simulator 39 described below to determine the operational intervention.

[0059] If the operational intervention part 38 detects that there is a risk of contact between the excavator 100 and an object around the excavator 100, the operational intervention part 38 determines that intervention in the operation is mandatory. For example, if the operational intervention part 38 detects the presence of a person on the left side of the excavator 100 and the start of the counterclockwise rotation process (the process of tilting the left control lever to the left), the operational intervention part 38 determines that intervention in the operation is mandatory. In this case, the operational intervention part 38 overrides the operating signal that was generated based on the counterclockwise rotation process in order to prevent the upper rotating body 3 from rotating counterclockwise.The operational intervention part 38 can intervene in the operation when, based on the detection information from the external spatial detection device 300, it determines whether the excavator 100 and the object are in contact with each other or not.

[0060] Furthermore, the operating intervention part 38 can release the braking process (stop, deceleration, etc.) at the time of intervention based on the release condition that is fulfilled at the time of intervention (for example, the operating lever is temporarily returned to the neutral position or the release button is pressed).

[0061] The motion simulator 39 is configured to simulate the movement of the excavator 100 in the three-dimensional virtual space model 50. The term "movement of the excavator 100" refers to various classified operations performed by the excavator 100. For example, excavation work consists of several movements, such as digging, lifting and rotating, unloading earth and sand, and lowering and rotating. The motion simulator 39 simulates the movement of the excavator 100 in response to the ON operation of the automatic control button 23 (see Fig. 6) to support the work of the excavator 100. The automatic control button 23 is provided, for example, in the control device 21 in the cabin 10.

[0062] In particular, the motion simulator 39 simulates the work (excavation, lifting and rotating, earth and sand unloading, lowering and rotating, etc.) of the virtual excavator 51 using the three-dimensional virtual space model 50 generated by the virtual space generation unit 31. That is, the excavator 100's control unit 30 functions as a simulation device according to the present invention. The information simulated by the motion simulator 39 can be displayed on the display device of the output device 22. The topographic information of the three-dimensional virtual space model 50 can change according to the simulated movement of the virtual excavator 51.

[0063] Fig. Figure 4 is an explanatory diagram illustrating the excavation movement of the virtual excavator 51. As shown in Fig. As shown in Figure 4, when the excavation movement is simulated, the motion simulator 39 generates, for example, several excavation trajectories (simulation motion information) for moving the excavation attachment 51a of the virtual excavator 51 relative to the excavation object 52 in the three-dimensional virtual space model 50. The multiple excavation trajectories are generated based on the area in which the excavation attachment AT of the excavator 100 can actually move in real space and the additional information about the excavation object 52 using the detection information of the position detection device of the excavator 100 in real space.

[0064] For example, the motion simulator 39 sets the movement range (including the direction of movement, movement distance, position, and the like of the excavation attachment 51a) of the excavator attachment 51a of the virtual excavator 51 to the position of the excavation object 52. Furthermore, the motion simulator 39 assumes several excavation patterns in which the sequence of the excavation points of the excavator attachment 52 within the movement range of the excavator attachment 51a and the excavation quantity (or excavation depth) are appropriately modified. When assuming the multiple patterns, the density, hardness, soil properties, and the like of the soil and sand in the soil and sand information of the excavation object 52 are used.

[0065] The motion simulator 39 then generates several excavation trajectories of the virtual excavator 51, corresponding to the movement range of the excavation attachment 51a and the multiple excavation patterns. Example (a) is shown in the upper right figure. Fig. 4 is a pattern where the first excavation point is the upper part of the excavation object 52, and then the surface of the excavation object 52 is excavated downwards in sequence, with the inner part of the excavation object 52 being excavated after the surface. On the other hand, example (b) is shown in the upper right figure of Fig. 4. A pattern where the first excavation point is the central point of the excavation object 52, and the surrounding area is excavated in detail. For example, pattern (a) is used when the soil and sand of the excavation object 52 are soft, and pattern (b) is used when the soil and sand of the excavation object 52 are hard. However, as described above, the density, hardness, and soil of the excavation object in real space may differ from the pre-movement information during the excavation movement of the excavator 100. Therefore, it is preferred that the motion simulator 39 generate multiple excavation trajectories when the density, hardness, and soil of the excavation object 52 are changed, even if the additional information about the excavation object is linked.

[0066] After generating multiple excavation trajectories, the motion simulator 39 evaluates (simulates) these trajectories to select an optimal one. For example, an objective function for efficient excavation of the object 52 can be applied when evaluating the optimal trajectory, and constraints such as the movement speed, working time, and safety of the excavation section 51a can be used to assess the optimal trajectory. Thus, the motion simulator 39 can obtain an optimal excavation trajectory that corresponds to the shape, position, and additional information of the currently detected object.

[0067] When the work support system SYS1 automatically controls the excavator 100, the controller 30 provides this optimal excavation trajectory to the actuator drive unit 34 as simulation motion information. Thus, the actuator drive unit 34 can control the various solenoid valves 41 and the various actuators 42 to move the excavation attachment 51a in the real space along the excavation trajectory (see the diagram below). Fig. 4).

[0068] When the excavation attachment AT performs excavation in the real space along the excavation trajectory, the controller 30 acquires detection information from the environmental detection device 79 (comprising a pressure sensor and a load sensor) and communication information from the dump truck. Based on this information, the virtual space generation unit 31 updates the shape, position, and orientation of the topographic and object information of the three-dimensional virtual space model 50 and the additional information added to it.

[0069] Furthermore, the motion simulator 39 corrects the excavation trajectory based on updated topographic information, object information, and additional information from the three-dimensional virtual space model 50. The motion simulator 39 can reselect several previously calculated excavation trajectories or recalculate excavation trajectories. When the optimal excavation trajectory is re-provided by the motion simulator 39, the actuator drive unit 34 switches to this excavation trajectory to move the excavation attachment AT in real space.

[0070] Fig. Figure 5 is an explanatory diagram showing the earth and sand unloading movement of the virtual excavator 51. As in Fig. As shown in Figure 5, when the earth and sand unloading movement is simulated, the motion simulator 39 generates several earth and sand unloading trajectories (simulation motion information) for moving the attachment 51a of the virtual excavator 51 in relation to the virtual dump truck 53 of the three-dimensional virtual space model 50. The multiple earth and sand unloading trajectories are generated based on the area in which the excavation attachment AT of the excavator 100 can actually move in real space, based on various sensors of the excavator 100 in real space and the unloading information of the virtual dump truck 53.

[0071] For example, the motion simulator 39 sets the range of motion (including the direction of movement, distance of movement, position, and the like of the excavation attachment 51a) of the excavation attachment 51a of the virtual excavator 51 to the position above the loading platform of the virtual dump truck 53 during earth and sand unloading. Furthermore, the motion simulator 39 assumes several earth and sand unloading patterns in which the position of the load on the loading platform of the virtual dump truck 53 within the range of motion of the excavator attachment 51a and the load quantity are appropriately modified. When assuming the multiple patterns, the sediment quantity, density, hardness, and soil quality of the earth and sand in the earth and sand information of the excavator 100 and / or the dump truck are used.

[0072] The motion simulator 39 then generates several earth and sand discharge trajectories of the virtual excavator 51, corresponding to the movement range of the excavation attachment 51a and the multiple earth and sand discharge patterns. In the upper right diagram of Fig. 5, for example, is the location of the earth and sand unloading at the rear end of the loading platform of the virtual dump truck 53, and the trajectory in which the earth and sand on the loading platform is leveled forward by the excavation attachment 51a after the earth and sand unloading has been carried out is shown.

[0073] After generating several soil and sand discharge trajectories, the motion simulator 39 evaluates (simulates) these trajectories to select the optimal one. The optimal trajectory can be evaluated using constraints such as the movement speed of the excavation attachment 51a, the working time, and occupational safety, while applying an objective function to load the soil and sand evenly onto the loading platform of the virtual dump truck 53. Thus, the motion simulator 39 can obtain the optimal trajectory based on the currently acquired soil and sand information.

[0074] When the excavator 100 is automatically controlled by the work support system SYS1, the controller 30 provides the actuator drive unit 34 with this optimal earth and sand discharge trajectory. Thus, the actuator drive unit 34 can control various solenoid valves 41 and various actuators 42 to move the excavation attachment AT in the actual space along the earth and sand discharge trajectory (see the diagram below). Fig. 5).

[0075] When the excavation attachment AT performs an earth and sand unloading movement along the earth and sand unloading trajectory in real space, the controller 30 acquires detection information from the environment detection device 79. Based on this information, the virtual space generation unit 31 updates the shape, position, and orientation of the topographic and object information of the three-dimensional virtual space model 50, as well as the additional information added to it.

[0076] The motion simulator 39 corrects the optimal earth and sand discharge trajectory based on updated topographic information, object information, and additional information from the three-dimensional virtual space model 50. The motion simulator 39 can reselect several previously calculated earth and sand discharge trajectories or recalculate the earth and sand discharge trajectory. Once the optimal earth and sand discharge trajectory is available, the actuator drive unit 34 switches to this earth and sand discharge trajectory to move the excavation attachment AT of the real space.

[0077] The work support system SYS1 of the excavator 100 according to the first embodiment is essentially constructed as described above, and its operation is described below with reference to Fig. 6 described. Fig. Figure 6 is an explanatory diagram illustrating the control of the excavator 100's operation by the work support system SYS1 according to the first embodiment. An example of automatic control (machine control function) of the excavator 100 by the controller 30 is described below. Furthermore, the work support system SYS1 is not limited to this and can also operate as a machine guidance function to direct the operator along the trajectory generated by the controller 30. Additionally, the work support system SYS1 can automatically control and guide the excavator 100 through the management device 200.

[0078] The control unit 30 automatically starts the operation of the excavator 100 based on the operator activating the automatic control button 23 of the excavator 100. After starting, the virtual space generation unit 31 creates the three-dimensional virtual space model 50 based on the stored environmental information from before the operation of the excavator 100 (step S1). The environmental information stored before the operation is saved in the control unit 30 and can adopt the three-dimensional virtual space model 50 from the previous operation (for example, from the previous day).As described above, various types of information are added to the topographic information and object information of the three-dimensional virtual space model 50, such as additional information of the virtual excavator 51, earth and sand information (additional information) of the virtual excavator 51, additional information of the excavation object 52, additional information of the unloading location (unloading object) and earth and sand information (additional information) of the virtual dump truck 53.

[0079] Then, the motion simulator 39 simulates the work of the virtual excavator 51 in the three-dimensional virtual space model 50 (step S2). That is, the motion simulator 39 generates each optimal trajectory of the work (excavator movement, lifting and rotating movement, earth and sand unloading movement, lowering and rotating movement) based on the environmental information. Thus, the controller 30 can control the excavation trajectory, the lifting and rotating trajectory, the earth and sand unloading trajectory, and the lowering and rotating trajectory, which are continuous on the time axis of the work.

[0080] The actuator drive unit 34 receives information on the excavation trajectory, the lifting and rotating trajectory, the earth and sand unloading trajectory, and the lowering and rotating trajectory generated by the motion simulator 39 (step S3). Thus, the actuator drive unit 34 controls the various solenoid valves 41 and the various actuators 42 according to these trajectories on the excavator 100 in the real space (step S4). That is, the excavator 100 in the real space automatically performs the excavation movement, the lifting and rotating movement, the earth and sand unloading movement, and the lowering and rotating movement as the actual work.

[0081] The actuator drive unit 34 can correct the movement (position, speed, acceleration, etc.) of the excavation attachment AT by feeding back the detection information from the various sensors of the excavator 100 during the excavation movement to the actuator drive unit 34. Furthermore, the actuator drive unit 34 can perform feedback control of each movement based on the detection information from various sensors of the excavator 100 itself during the lifting and rotating operation, the earth and sand unloading operation, and the lowering and rotating operation.

[0082] During actual operation, the controller 30 acquires information from the environment detection device 79 (excavation movement information encompassing the operation of the space detection device 70, the orientation detection device 71, the positioning device 72, and various sensors or the operating device 21; communication information from the external space detection device 300 and the dump truck) at a predetermined time. The predetermined time can be set sequentially (at each predetermined interval) during the operation of the excavator 100, or it can be set to capture the log stored by a movement in conjunction with the end of a movement. The virtual space generation unit 31 updates the topographic information, object information, and additional information of the three-dimensional virtual space model 50 based on this information (step S5).For example, the virtual space generation part 31 updates the additional information (soil hardness, soil density, soil quality, etc.) of the excavation object 52 of the three-dimensional virtual space model 50 to the information estimated by the excavation condition estimation part 33. Furthermore, the additional information and the soil and sand information of the excavator of the virtual excavator 51, or the additional information and soil and sand information of the virtual dump truck 53 (unloading object), are also updated appropriately.

[0083] Thus, the motion simulator corrects 39 different trajectories (excavation trajectory, lifting and rotating trajectory, earth and sand unloading trajectory, and lowering and rotating trajectory) of the work based on the updated information (step S6). The corrected trajectories are transmitted to the actuator drive unit 34 and reflected during control by the actuator drive unit 34 (step S7).

[0084] The SYS1 work support system can appropriately move the excavator 100 according to the real-time changes on the construction site by repeating the aforementioned movements during operation. As a result, the SYS1 work support system enables the excavator 100 to perform its work efficiently and accurately.

[0085] It should be noted that the work support system SYS1 according to the present invention is not limited to the embodiments described above, and various modifications can be applied. For example, if the controller 30 of the work support system SYS1 detects that a person is around the excavator 100, based on environmental information, it is preferred to generate a trajectory (motion information) of the virtual excavator 51 to avoid the person in the three-dimensional virtual space model 50. For example, if a person is to the left of the virtual excavator 51, the trajectory with respect to the lifting and rotating trajectory and the lowering and rotating trajectory of the virtual excavator 51 can be switched from a counterclockwise trajectory to a clockwise trajectory. [Second embodiment]

[0086] Next, a work support system SYS2 of the excavator 100 according to the second embodiment will be described with reference to Fig. 7 and Fig. 8 described. Fig. Figure 7 is a diagram that illustrates an example of the SYS2 work support system according to the second embodiment. Fig. Figure 8 is a function block diagram that represents a configuration example of the work support system SYS2.

[0087] The work support system SYS2 according to the second embodiment differs from the work support system SYS1 according to the first embodiment in that the excavator 100 is remotely controlled by the remote control room RC, which is located at a position remote from the excavator 100. The configuration, with the exception of the remote control of the excavator 100, is essentially the same as that of the first embodiment, and a detailed description thereof is omitted.

[0088] The remote control room RC includes a remote control 30R, a sound output device A2, an interior imaging device C2, a display device RD, and a communication device T3. The remote control room RC also includes a driver's seat DS, in which the operator sits who remotely controls the excavator 100.

[0089] The 30R remote control is a computing device that performs various arithmetic operations. Like the 30 control unit of the 100 excavator, the 30R remote control is a computer comprising one or more processors and memory. The various functions of the 30R remote control are implemented by the processor, which executes programs stored in the memory.

[0090] The sound output device A2 outputs sound and is configured so that a sound collection device (not shown) attached to the excavator 100 plays back the collected sound.

[0091] The interior imaging device C2 records the interior of the remote control room RC. For example, the interior imaging device C2 is a camera installed inside the remote control room RC that records the operator OP sitting in the driver's seat DS.

[0092] The RD display device shows information about the situation around the excavator 100. For example, the RD display device is a multi-display consisting of a total of 9 monitors with 3 vertical and 3 horizontal rows, configured to show the conditions of the space in front of, to the left of, and to the right of the excavator 100. Alternatively, the RD display device can also be a head-mounted display that an operator can wear.

[0093] The communication device T3 is configured to be able to communicate with the communication device T1 of the excavator 100, the communication device T2 of the management device 200, the external space detection device 300 and the like.

[0094] The remote control room (RC), positioned around the operator's seat (DS), has a similar structure to the operator's seat installed in the cab (10) of the excavator (100). Specifically, a left console box is located to the left of the operator's seat (DS), and a right console box is located to the right of the operator's seat (DS). A left control lever is located at the upper front end of the left console box, and a right control lever is located at the upper front end of the right console box. A drive lever and a drive pedal are located in front of the operator's seat (DS). The left control lever, the right control lever, the drive lever, and the drive pedal form an operating device (21R) of the remote control room.

[0095] The control device 21R is equipped with an operating sensor 29R for detecting the operating input of the control device 21R. The operating sensor 29R includes, for example, a tilt sensor for detecting the tilt angle of the control lever, an angle sensor for detecting the deflection angle of the control lever about a deflection axis, etc. The operating sensor 29R can also be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operating sensor 29R outputs information about the detected operating input of the control device 21R to the remote control 30R. Based on the received information, the remote control 30R generates an operating signal and transmits this signal to the excavator 100.

[0096] As in Fig. As shown in Figure 8, the 30R remote control comprises, as functional blocks, an operator state identification part 61, an image combination part 62, and an operating signal generation part 63. Although the operator state identification part 61, the image combination part 62, and the operating signal generation part 63 are distinguished for the sake of simplicity, they need not be physically distinct and may consist wholly or partly of common software components or hardware components.

[0097] The operator state identification part 61 is configured to identify the state of an operator in the remote control room RC. For example, the operator state identification part 61 identifies the position of the operator's eye (operator's gaze point) and the operator's gaze direction, based on the imaging information from the interior imaging device C2. Specifically, the operator state identification part 61 performs suitable image processing on the image captured by the interior imaging device C2 to identify the position of the operator's gaze point in the coordinate system of the operator room and the coordinates of the operator's gaze direction.

[0098] The operator state identification unit 61 can derive the operator's viewpoint position and direction of view (OP) based on output from devices other than the interior imaging device C2, such as a LiDAR installed in the remote control room RC or an inertial measuring device attached to the head-mounted display serving as the display device RD. The inertial measuring device may include a positioning device. The operator state identification unit 61 transmits information regarding the operator's viewpoint position (E1) and direction of view (OP) to the excavator 100 via the communication device T3.

[0099] The image combination unit 62 generates a composite image by combining the detection information (imaging information) received from the excavator 100 or the three-dimensional virtual space model 50 by the space detection device 70 with another image. The space detection device 70 of the excavator 100 converts the operator's viewpoint position and viewing direction in the coordinate system of the operating room, transmitted by the remote control 30R, into coordinates in the excavator's coordinate system, thereby acquiring the imaging information of each sensor and transmitting it to the remote control 30R.

[0100] The other image can be a planning surface image, which is an image generated based on planning data. In the present embodiment, the image combination part 62 overlays a figure, such as a computer graphic, onto the environmental information as a planning surface image. This figure represents the position of the planning surface based on planning data previously stored in the memory of the remote control 30R. The image combination part 62 determines the position for overlaying the planning surface image based on the position and orientation of the excavator 100, which were identified by the excavator state identification part 32 of the controller 30.

[0101] The operating signal generation unit 63 is configured to generate an operating signal to be transmitted to the excavator 100. The operating signal generation unit 63 generates an operating signal based on the output of the operating sensor 29R of the remote control room RC. Essentially, the excavator 100 receives the operating signal generated when the operator of the remote control room RC performs an operation while viewing the image on the display device RD, and the excavator 100 executes a movement corresponding to the operating signal.

[0102] It should be noted that the remote control of the excavator 100 is not limited to operation by the operator in the remote control room RC and can also be carried out, for example, by using a portable terminal 400, as described in Fig. Figure 7 shows how this can be carried out. In this case, most of the movements of the excavator 100 can be set to be automatically controlled in advance to simplify operation by the portable terminal 400.

[0103] In the SYS2 work support system of the excavator 100, to which the remote control room RC is applied, the work support trajectory can be generated in the excavator 100's control unit 30 or in the remote control unit 30R of the remote control room RC. Therefore, the remote control unit 30R can include the virtual space generation part 65, the determination part 66, the operational prediction part 67, the operational intervention part 68, and the motion simulator 69, which have the same functions as the virtual space generation part 31, the determination part 36, the operational prediction part 37, the operational intervention part 38, and the motion simulator 39. Alternatively, the SYS2 work support system can generate work support trajectories through the management device 200.

[0104] When work support is performed by remote control, the remote control 30R generates multiple trajectories for each shared movement of the work and selects the optimal trajectory by evaluating each trajectory, similar to the controller 30. The remote control 30R then transmits information about the trajectories of each individual movement to the excavator 100, and the actuator drive part 34 of the controller 30 controls the movement of the excavator 100 along the trajectories of each movement.

[0105] The work support system SYS2 according to the second embodiment is essentially constructed as described above, and its operation is described below with reference to Fig. 9 described. Fig. Figure 9 is an explanatory diagram showing the control of the excavator 100's operation by the SYS2 work support system according to the second embodiment. An example of automatic control (machine control function) of the excavator 100 by the 30R remote control is described below. The SYS2 work support system is not limited to this, and it can also operate as a machine guidance function to direct the operator along the (simulated) trajectory generated by the 30R remote control.

[0106] The remote control 30R automatically controls the work of the excavator 100 when the operator operates the control device 21R while viewing the imaging information acquired by the excavator 100's space detection device 70. The virtual space generation unit 65 generates a three-dimensional virtual space model 50 based on the environment detection device 79 (detection information from the space detection device 70, the orientation detection device 71, the positioning device 72 and various sensors or information from the control device 21 and communication information from the external space detection device 300 and the dump truck) (step S11).

[0107] Subsequently, the motion simulator 69 simulates the work of the virtual excavator 51 in the three-dimensional virtual space model 50 (step S12). Thus, the remote control 30R can receive the excavation trajectory, the lifting and rotating trajectory, the earth and sand unloading trajectory, and the lowering and rotating trajectory (simulation motion information), which are continuous on the time axis of operation.

[0108] The remote control 30R transmits the operating instructions for the excavation trajectory, the lifting and rotating trajectory, the earth and sand unloading trajectory, and the lowering and rotating trajectory, generated by the motion simulator 69, to the control unit 30 (step S13). The actuator drive unit 34 of the control unit 30 then controls the various solenoid valves 41 and the various actuators 42 according to these trajectories in the excavator 100 in the real space (step S14). The actuator drive unit 34 can correct the movement (position, speed, acceleration, etc.) of the excavation attachment AT by feeding back the detection information from the various sensors of the excavator 100 to the actuator drive unit 34 during each working movement.

[0109] When the controller 30 acquires the detection information from the environmental detection device 79 and the communication information from the dump truck during actual operation, the controller 30 transmits the acquired information to the remote control 30R. The virtual space generation unit 65 of the remote control 30R updates the topographic information, object information, and additional information of the three-dimensional virtual space model 50 based on this information (step S15). Thus, the motion simulator 69 corrects various trajectories (excavation trajectory, lifting and rotating trajectory, earth and sand unloading trajectory, lowering and rotating trajectory) of the work based on the updated information (step S16). The corrected trajectories are transmitted from the remote control 30R to the controller 30 and reflected in the controller by the actuator drive unit 34 of the controller 30 (step S17).

[0110] As described above, the SYS2 work support system, even when used with the 30R remote control, can appropriately move the 100 excavator according to the real-time changes on the construction site, by repeating the movements described above during operation. As a result, the SYS2 work support system allows the 100 excavator to perform the work efficiently and accurately.

[0111] The SYS2 work support system according to the second embodiment is not limited to the configuration described above, and various modifications can be applied. The operation of the SYS2 work support system according to one modification example is described below with reference to Fig. 10 described. Fig.Figure 10 is an explanatory diagram illustrating the control of the excavator 100's operation by the SYS2 work support system according to the modification example. In the SYS2 work support system according to the modification example, the operator operates the control device 21R while viewing the three-dimensional virtual space model 50 displayed on the display device RD in the remote control room RC, and transmits the operating signal from the operating sensor 29R from the remote control 30R to the controller 30.

[0112] In this case, the virtual space generation unit 65 of the remote control 30R displays the environmental information (topographic and object information) in the three-dimensional virtual space model 50 based on the detection information from the environmental detection device 79 transmitted by the control 30. Based on the topographic and object information of the three-dimensional virtual space model 50, the operator can control the excavation, lifting and rotating, earth and sand unloading, and lowering and rotating movements of the work. At this point, the motion simulator 69 can generate various work trajectories and guide the operator along these trajectories. Furthermore, the operational intervention unit 68 can intervene in the operation and correct the operating signal from the operational sensor 29R if the operator's actions deviate significantly from the generated trajectory.

[0113] Thus, the controller 30 can control the operation of the excavator 100 in the real space based on the operating signal received from the remote control 30R. At this point, the controller 30 performs feedback control of each movement based on detection information detected by various sensors of the excavator 100.

[0114] Furthermore, the controller 30 transmits detection information from the environmental detection device 79 and communication information from the dump truck to the remote control 30R during operation. Thus, the virtual space generation unit 65 of the remote control 30R updates the environmental information of the three-dimensional virtual space model 50 during operation and can be used to generate the trajectory of the motion simulator 69 and for operation by the operator.

[0115] The work support systems SYS1 and SYS2 of the excavator 100, according to the embodiments disclosed herein, are exemplary in every respect and not limiting. The embodiments can be modified and improved in various ways without deviating from the scope and substance of the appended claims. The items described in the above several embodiments can have other configurations without contradiction and can be combined without contradiction.

[0116] This international application is based on Japanese patent application No. 2022-203642, which was filed with the Japanese Patent Office on December 20, 2022, and claims priority therefrom, with all its contents incorporated by reference. REFERENCE MARK LIST 21, 21R Operating device 30 Control 30R remote control 50 three-dimensional virtual space model 51 virtual excavators 70 Room detection device 79 Environmental detection device 100 excavators 200 administrative device 300 external room detection devices RC remote control room SYS1, SYS2 Work Support System QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2016-156193

[0003] JP 2022-203642

[0116]

Claims

[1] Work support system for an excavator, comprising: the excavator; an environmental detection device configured to detect environmental information at the excavator's construction site; and a simulation device configured to capture environmental information detected by the environment detection device during movement of the excavator and to generate a three-dimensional virtual spatial model of the construction site. [2] Work support system for the excavator according to claim 1, wherein the environment detection device acquires the environment information at a predetermined time during the movement of the excavator. [3] Work support system for the excavator according to claim 1, wherein the environmental information includes motion information that is captured during the movement of the excavator. [4] Work support system for the excavator according to claim 3, wherein the motion information includes excavation motion information that is captured in conjunction with an excavation movement of the excavator. [5] Working support system for the excavator according to claim 4, wherein the environmental information comprises soil and sand information estimated from the excavation movement information. [6] Work support system for the excavator according to any one of claims 1 to 5, wherein the simulation device provides the excavator with simulation motion information which is simulated by a virtual excavator in the three-dimensional virtual space model reproduced by the simulation device. [7] Work support system for the excavator according to claim 6, wherein the excavator is controlled at the construction site based on the simulation movement information. [8] Work support system for the excavator according to claim 6, wherein the simulation device generates several trajectories of the virtual excavator with different conditions relating to an excavation movement or an earth and sand unloading movement and provides as the simulation movement information an optimal trajectory among the several trajectories based on a result of simulation of the several trajectories. [9] Work support system for the excavator according to claim 6, wherein the simulation device generates the simulation motion information for avoiding a person when it detects that the person is around the excavator, based on the environmental information. [10] Working support system for the excavator according to any one of claims 1 to 5, further comprising a remote control room located at a place remote from the excavator, wherein The simulation device simulates the movement of a virtual excavator in the three-dimensional virtual space model reproduced by the simulation device, based on an operating instruction from a remote control room operating device. [11] Work support system for the excavator according to any one of claims 1 to 5, wherein the simulation device is provided in a controller configured to control the excavator or in a management device capable of communicating with the excavator, the management device being located outside the excavator. [12] Working support system for the excavator according to any one of claims 1 to 5, wherein the environmental information is stored in the simulation device before the excavator movement is carried out, and The simulation device generates the three-dimensional virtual spatial model based on the stored environmental information before the excavator movement is carried out. [13] Work support system for the excavator according to one of claims 1 to 5, wherein the simulation device extracts as object information an object included in the environment information and a position of the object and arranges the extracted object information in the three-dimensional virtual space model. [14] Work support system for the excavator according to claim 13, wherein the simulation device adds additional information corresponding to the object information arranged in the three-dimensional virtual space model. [15] Work support system for the excavator according to claim 14, wherein the object information comprises at least one of the information of a shape or position of the excavator, information of a shape or position of an object to be excavated by the excavator, or information of a shape or position of an unloading object into which the excavated object is to be unloaded. [16] Work support system for the excavator according to claim 15, wherein the additional information corresponding to the information of the shape or position of the excavator includes at least one of a position, an identification number, a type, a working time or a component condition of the excavator. [17] Working support system for the excavator according to claim 15, wherein the additional information corresponding to the information of the shape or position of the excavated object comprises at least one of a sediment accumulation quantity, a weight, a density, a hardness or soil quality of the excavated object. [18] Work support system for the excavator according to claim 15, wherein the additional information corresponding to the information of the shape or position of the unloading object comprises at least one of a position of the unloading object; an identification number; a type; a size of a loading area; a condition of the loading area; a working time; or a weight, volume or density of the unloading object. [19] Work support system for the excavator according to any one of claims 1 to 5, wherein the environment detection device comprises a space detection device configured to detect topographic information of the construction site and object information of an object at the construction site, and the space detection device is arranged in the excavator or at a location remote from the excavator.

Images