Work machine, work machine management system, work machine management device

Abstract

The present application relates to a work machine, a management system of a work machine, and a management device of a work machine. The work machine includes: an operation information acquisition section that acquires operation information including environment information indicating an environment around the machine and state information indicating a state of the machine; an inference section that infers, based on the operation information, a work step in which a digging surface formed by a digging operation satisfies a predetermined condition; and a display control section that displays information indicating the work step inferred by the inference section.

Description

Technical Field

[0001] This invention relates to a work machinery, a management system for the work machinery, and a management device for the work machinery. Background Technology

[0002] Previously, there was a type of hydraulic excavator that automatically adjusted the position of the bucket tip relative to the design surface during operations that excavate a slope along the design surface to form a ramp.

[0003] Previous technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2013-217137 Summary of the Invention

[0004] The technical problem to be solved by the invention In the aforementioned prior art, the impact of the slope tilt and shape changes caused by excavation on the environment around the excavator was not considered, thus requiring further improvements in safety.

[0005] The purpose of this invention is to improve safety.

[0006] means for solving technical problems The operating machinery according to the embodiments of the present invention includes: an operation information acquisition unit that acquires operation information including environmental information indicating the environment around the machine and state information indicating the state of the machine; an inference unit that infers, based on the operation information, an operation step in which the excavation face formed by the excavation action satisfies predetermined conditions; and a display control unit that displays information indicating the operation step inferred by the inference unit.

[0007] The management system for the work machinery according to the embodiments of the present invention includes the work machinery and a management device that communicates with the work machinery. In the management system for the work machinery, the management device has: an information acquisition unit that acquires operation information from the work machinery, including environmental information representing the environment around the work machinery and status information representing the status of the work machinery; an inference unit that infers, based on the operation information, the operation steps in which the excavation face formed by the excavation action of the work machinery satisfies predetermined conditions; and an output unit that outputs information representing the operation steps inferred by the inference unit to the work machinery.

[0008] The management device for operating machinery according to the embodiments of the present invention is a management device for operating machinery that communicates with the operating machinery. The management device for operating machinery includes: an information acquisition unit that acquires operation information from the operating machinery, including environmental information representing the environment around the operating machinery and status information representing the state of the operating machinery; an inference unit that infers, based on the operation information, an operation step in which the excavation face formed by the excavation action of the operating machinery satisfies predetermined conditions; and an output unit that outputs information representing the operation step inferred by the inference unit to the operating machinery.

[0009] Invention Effects This invention improves safety. Attached Figure Description

[0010] Figure 1 This is a diagram illustrating an example of the system structure of an excavator's management system.

[0011] Figure 2 It is a diagram illustrating the hardware structure of the various devices in the excavator's management system.

[0012] Figure 3 It is a diagram illustrating the functional structure of the various devices in the excavator's management system.

[0013] Figure 4A This is the first diagram illustrating the operating procedures of an excavator.

[0014] Figure 4B This is the second diagram illustrating the operating procedures of an excavator.

[0015] Figure 5 This is a flowchart illustrating the processing of the learning section of the management device.

[0016] Figure 6 It is a sequence diagram illustrating the actions of the excavator's management system.

[0017] Figure 7 This is a diagram showing an example of information representing the steps of a task. Detailed Implementation

[0018] The management system of the excavator according to this embodiment will be described below with reference to the accompanying drawings. Figure 1 This diagram illustrates an example of the system structure of an excavator's management system. In this embodiment, the excavator 100 will be described as an example of a working machine.

[0019] The excavator management system SYS of this embodiment includes an excavator 100 and a management device 200. In the following description, the excavator 100 management system SYS will be simply referred to as management system SYS.

[0020] In the management system SYS of this embodiment, the excavator 100 and the management device 200 are connected via a network or the like.

[0021] First, the structure of the excavator 100 in this embodiment will be described. Figure 1 The image shows a side view of the excavator 100.

[0022] The excavator 100 has a lower traveling body 1, a slewing mechanism 2, and an upper slewing body 3. The upper slewing body 3 is rotatably mounted on the lower traveling body 1 via the slewing mechanism 2. A boom 4 is installed in the upper slewing body 3. A stick 5 is installed at the front end of the boom 4, and a bucket 6, serving as an end attachment, is installed at the front end of the stick 5.

[0023] The boom 4, stick 5, and bucket 6 constitute an example of an excavation attachment. Furthermore, the boom 4 is driven by the boom cylinder 7, the stick 5 by the stick cylinder 8, and the bucket 6 by the bucket cylinder 9. A boom angle sensor S1 is installed in the boom 4, a stick angle sensor S2 is installed in the stick 5, and a bucket angle sensor S3 is installed in the bucket 6.

[0024] The boom angle sensor S1 is configured to detect the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is an acceleration sensor capable of detecting the rotation angle (hereinafter referred to as "boom angle") of the boom 4 relative to the upper rotating body 3. The boom angle is, for example, the minimum angle when the boom 4 is lowered to its maximum extent, and increases as the boom 4 is raised.

[0025] The stick angle sensor S2 is configured to detect the rotation angle of the stick 5. In this embodiment, the stick angle sensor S2 is an acceleration sensor capable of detecting the rotation angle of the stick 5 relative to the boom 4 (hereinafter referred to as the "stick angle"). The stick angle is, for example, the smallest angle when the stick 5 is closed to its maximum extent, and increases as the stick 5 is opened.

[0026] The bucket angle sensor S3 is configured to detect the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is an acceleration sensor capable of detecting the rotation angle of the bucket 6 relative to the stick 5 (hereinafter referred to as "bucket angle"). The bucket angle is, for example, the smallest angle when the bucket 6 is closed to its maximum extent, and increases as the bucket 6 is opened.

[0027] The boom angle sensor S1, stick angle sensor S2, and bucket angle sensor S3 can be a potentiometer using a variable resistor, a stroke sensor that detects the stroke of the corresponding hydraulic cylinder, a rotary encoder that detects the rotation angle around the connecting pin, a gyroscope sensor, or a combination of an accelerometer and a gyroscope sensor, respectively.

[0028] A boom rod pressure sensor S7R and a boom underpressure sensor S7B are installed in boom cylinder 7. A stick rod pressure sensor S8R and a stick underpressure sensor S8B are installed in stick cylinder 8.

[0029] A bucket rod pressure sensor S9R and a bucket bottom pressure sensor S9B are installed in the bucket cylinder 9. The boom rod pressure sensor S7R, boom bottom pressure sensor S7B, stick pressure sensor S8R, stick bottom pressure sensor S8B, bucket rod pressure sensor S9R, and bucket bottom pressure sensor S9B are collectively referred to as "cylinder pressure sensors".

[0030] The boom rod pressure sensor S7R detects the pressure in the rod-side oil chamber of the boom cylinder 7 (hereinafter referred to as "boom rod pressure"). The boom bottom pressure sensor S7B detects the pressure in the bottom-side oil chamber of the boom cylinder 7 (hereinafter referred to as "boom bottom pressure"). The stick pressure sensor S8R detects the pressure in the rod-side oil chamber of the stick cylinder 8 (hereinafter referred to as "stick pressure"). The stick bottom pressure sensor S8B detects the pressure in the bottom-side oil chamber of the stick cylinder 8 (hereinafter referred to as "stick bottom pressure").

[0031] The bucket rod pressure sensor S9R detects the pressure in the rod-side oil chamber of the bucket cylinder 9 (hereinafter referred to as "bucket rod pressure"), and the bucket bottom pressure sensor S9B detects the pressure in the bottom-side oil chamber of the bucket cylinder 9 (hereinafter referred to as "bucket bottom pressure").

[0032] The upper rotating body 3 houses a driver's cabin 10, which serves as the driver's compartment, and is equipped with a power source such as an engine 11. Furthermore, a sensor for detecting CO2 emissions can be installed near the emission mechanism of the engine 11. A counterweight 29 can be installed at the rear of the upper rotating body 3.

[0033] Furthermore, the upper rotating body 3 is equipped with a controller 30, a display device 40, an input device 42, a sound output device 43, a storage device 47, a positioning device P1, a fuselage tilt sensor S4, a rotational angular velocity sensor S5, a camera device S6, and a communication terminal T1.

[0034] The upper rotating body 3 can be equipped with an energy storage unit that supplies electricity and an electric generator that generates electricity using the rotational drive force of the engine 11. The energy storage unit may be, for example, a capacitor or a lithium-ion battery. The electric generator can function as an electric motor to drive mechanical loads, or it can function as a generator to supply electricity to electrical loads.

[0035] The controller 30 functions as the main control unit for driving the excavator 100. In this embodiment, the controller 30 is composed of a computer including a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). Various functions of the controller 30 are implemented, for example, by the CPU executing programs stored in the ROM. These functions may include, for example, at least one of equipment guidance functions that guide the operator in manually operating the excavator 100, and equipment control functions that automatically support the operator's manual operation of the excavator 100.

[0036] The display device 40 is configured to display various information. The display device 40 can be connected to the controller 30 via a communication network such as CAN, or it can be connected to the controller 30 via a dedicated line.

[0037] The input device 42 is configured to allow the operator to input various information into the controller 30. The input device 42 includes at least one of a touch panel, rotary switch, and diaphragm switch disposed in the cockpit 10.

[0038] The sound output device 43 is configured to output sound. The sound output device 43 may be, for example, a vehicle speaker connected to the controller 30, or an alarm such as a buzzer. In this embodiment, the sound output device 43 is configured to output various information as sound according to a sound output command from the controller 30.

[0039] Storage device 47 is configured to store various types of information. Storage device 47 may be a non-volatile storage medium such as a semiconductor memory. Storage device 47 can store information output by various devices during the operation of excavator 100, or information acquired by various devices before the operation of excavator 100 begins. Furthermore, storage device 47 can store operating information of excavator 100. This operating information is periodically transmitted to management device 200 via communication terminal T1. Details of the operating information will be described later.

[0040] The positioning device P1 is configured to determine the position of the upper rotating body 3. The positioning device P1 is also configured to determine the orientation of the upper rotating body 3. In this embodiment, the positioning device P1 is, for example, a GNSS compass, which detects the position and orientation of the upper rotating body 3 and outputs the detected values ​​to the controller 30. Therefore, the positioning device P1 can also function as an orientation detection device for detecting the orientation of the upper rotating body 3. The orientation detection device can be an azimuth sensor mounted on the upper rotating body 3.

[0041] The chassis tilt sensor S4 is configured to detect the tilt of the upper rotating body 3. In this embodiment, the chassis tilt sensor S4 is an acceleration sensor that detects the forward and backward tilt angle of the upper rotating body 3 relative to an imaginary horizontal plane around the front and rear axes and the left and right tilt angle around the left and right axes. The front and rear axes and the left and right axes of the upper rotating body 3 are orthogonal to each other, for example, at a point on the rotation axis of the excavator 100, i.e., the center point of the excavator.

[0042] The rotational angular velocity sensor S5 is configured to detect the rotational angular velocity of the upper rotating body 3. The rotational angular velocity sensor S5 can be configured to detect or calculate the rotation angle of the upper rotating body 3. In this embodiment, the rotational angular velocity sensor S5 is a gyroscope sensor. The rotational angular velocity sensor S5 can also be a rotary transformer, rotary encoder, etc.

[0043] The camera device S6 is an example of a spatial recognition device, configured to acquire image data representing the surroundings of the excavator 100. In this embodiment, the camera device S6 includes a front camera S6F that captures the space in front of the excavator 100, a left camera S6L that captures the space to the left of the excavator 100, a right camera S6R that captures the space to the right of the excavator 100, and a rear camera S6B that captures the space behind the excavator 100.

[0044] The camera device S6 is, for example, a single-lens camera with an imaging element such as a CCD or CMOS sensor, which outputs the captured images to the display device 40. The camera device S6 can be a stereo camera, a distance imaging camera, etc. Furthermore, the camera device S6 can be replaced by other spatial recognition devices such as a three-dimensional distance imaging sensor, an ultrasonic sensor, a millimeter-wave radar, a LiDAR, or an infrared sensor, or it can be replaced by a combination of other spatial recognition devices and a camera.

[0045] The front camera S6F is installed, for example, in the ceiling of the cockpit 10, i.e., inside the cockpit 10. However, the front camera S6F can also be installed on the roof of the cockpit 10, the side of the boom 4, or on the exterior of the cockpit 10. The left camera S6L is installed on the left end of the upper surface of the upper rotating body 3, the right camera S6R is installed on the right end of the upper surface of the upper rotating body 3, and the rear camera S6B is installed on the rear end of the upper surface of the upper rotating body 3.

[0046] The communication terminal T1 is configured to control communication with external devices located outside the excavator 100. In this embodiment, the communication terminal T1 controls communication with external devices via satellite communication networks, mobile phone communication networks, or the Internet. External devices include, for example, management devices 200 such as servers installed in external facilities, and support devices 300 such as smartphones carried by workers around the excavator 100.

[0047] Furthermore, in the excavator 100 of this embodiment, the various sensors, controllers 30, and engine control unit 50 described below (see reference) are all included. Figure 2 These devices can communicate with each other via CAN (Controller Area Network) and each transmits and receives data via CAN. That is, in this embodiment, the controller 30 receives output values ​​from various sensors or signals from the engine control unit 50 via CAN.

[0048] If the management device 200 of this embodiment receives operation information from the excavator 100, it infers the operation steps that take into account the impact on the environment around the excavator 100 based on the received operation information, and notifies the operator of the excavator 100.

[0049] More specifically, for example, if the excavator 100 begins to form a slope, the management device 200, based on the operating information received from the excavator 100, infers that the slope of the slope (excavation face) formed by the excavator 100's digging action meets predetermined conditions, and notifies the operator of the excavator 100. The digging action refers to the action of inserting the bucket 6 into the sand and closing the boom 5 (pulling it forward) while simultaneously closing the bucket 6.

[0050] In this embodiment, the effects of the slope tilt and shape changes caused by excavation on the soil pile can be taken into account, and the effects of the slope tilt and shape changes caused by excavation on the environment around the excavator 100 can be suppressed.

[0051] Specifically, for example, in this embodiment, it is possible to prevent the collapse or toppling of the soil pile caused by excavation, and the machine falling onto the slope caused by excavation, thereby improving safety. Details of the management device 200 will be described later.

[0052] In addition, Figure 1 In this example, the management device 200 is implemented through a single information processing device, but it is not limited to this. The management device 200 can be implemented through multiple information processing devices. In other words, the functions implemented by the management device 200 can be implemented through multiple information processing devices.

[0053] Next, refer to Figure 2 The hardware structure of each device in the management system SYS is described. Figure 2 It is a diagram illustrating the hardware structure of the various devices in the excavator's management system.

[0054] First, the hardware structure of the excavator 100 will be described. The cab 10 contains a controller 30, a display device 40, a communication terminal T1, etc.

[0055] Engine 11 is a diesel engine that uses isochronous control to maintain a constant engine speed regardless of changes in engine load. The fuel injection quantity and timing related to engine 11 are controlled by engine control unit 50.

[0056] The rotating shaft of the engine 11 is connected to the rotating shafts of the main pump 14 and the pilot pump 15, which are hydraulic pumps.

[0057] The regulator 13 is configured to control the displacement of the main pump 14. In this embodiment, the regulator 13 controls the displacement of the main pump 14 by adjusting the swashplate deflection angle of the main pump 14 according to a control command from the controller 30. For example, the controller 30 changes the displacement of the main pump 14 by outputting a control command to the regulator 13 based on the output of the discharge pressure sensor 28, etc. The regulator 13 is configured to send data representing the swashplate deflection angle to the controller 30.

[0058] The main pump 14 is configured to supply working oil to the control valve 17 via a working oil line. In this embodiment, the main pump 14 is a swashplate variable capacity hydraulic pump.

[0059] The oil temperature sensor 14c is configured to detect the temperature of the working oil flowing through the working oil pipeline between the working oil tank and the main pump 14. Furthermore, the oil temperature sensor 14c is configured to send the detected data to the controller 30.

[0060] The pilot pump 15 is configured to supply pilot pressure to various hydraulic control devices such as operating devices via pilot lines. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pump 15 may be omitted. In this case, the function performed by the pilot pump 15 can be achieved by the main pump 14. In this case, the main pump 14, in addition to supplying working oil to the control valve 17, can also supply working oil to the operating device after reducing the pressure of the working oil through a throttle or the like.

[0061] Control valve 17 is a hydraulic control device for controlling the hydraulic system of excavator 100. Control valve 17 is connected to hydraulic actuators such as the right-side travel hydraulic motor, the left-side travel hydraulic motor, the boom cylinder 7, the stick cylinder 8, the bucket cylinder 9, and the swing hydraulic motor.

[0062] The controller 30 is a control device for controlling the excavator 100. In this embodiment, the controller 30 is composed of a microcomputer including a CPU, volatile memory, and non-volatile memory. Furthermore, the various functions of the controller 30 are implemented by the CPU executing programs stored in the non-volatile memory.

[0063] The display device 40 is mounted between the right pillar and the driver's seat, and is configured so that the operator sitting in the driver's seat can view the screen. Furthermore, the display device 40 is configured to display various information according to instructions from the controller 30. In this embodiment, the display device 40 is a liquid crystal display connected to the controller 30.

[0064] The image display unit 41 displays an image. The switch panel 45 is a panel that includes various hardware switches. The switch panel 45 may be a touch panel disposed on the image display unit 41.

[0065] In this embodiment, the display device 40 is configured to operate by receiving power from the storage battery 70. The storage battery 70 is charged using power generated by the alternator 11a (generator) of the engine 11. Power from the storage battery 70 is also supplied to the controller 30, communication terminal T1, etc. For example, the starting device 11b of the engine 11 is configured to be driven by power from the storage battery 70 to start the engine 11.

[0066] The communication terminal T1 is configured to control communication between the excavator 100 and external devices. In this embodiment, the communication terminal T1 controls wireless communication between at least one of the excavator 100 and the management device 200 via at least one of satellite communication lines, mobile phone communication lines, and short-range wireless communication lines.

[0067] The engine control unit 50 is configured to control the engine 11. Furthermore, the engine control unit 50 is configured to send data indicating the state of the engine 11, such as coolant temperature, to the controller 30.

[0068] The engine speed adjustment control panel 75 is used to adjust the engine speed of the engine 11. The engine speed adjustment control panel 75 sends data indicating the set state of the engine speed to the controller 30. The engine speed adjustment control panel 75 is configured to switch between four modes: SP mode, H mode, A mode, and idle mode.

[0069] SP mode is the speed mode selected when prioritizing workload, utilizing the highest engine speed. H mode is the speed mode selected when balancing workload and fuel efficiency, utilizing the second highest engine speed. A mode is the speed mode selected when prioritizing fuel efficiency while operating the excavator 100 with low noise, utilizing the third highest engine speed. Idle mode is the speed mode selected when setting the engine 11 to idle, utilizing the lowest engine speed. The engine 11 is controlled to rotate at a constant engine speed corresponding to the speed mode set by the engine speed adjustment control panel 75.

[0070] Next, the hardware structure of the management device 200 of this embodiment will be described. The management device 200 of this embodiment is a computer having a CPU 201, a storage device 202, a communication device 203, an input device 204 and a display device 205 that are connected to each other via a bus.

[0071] CPU 201 controls the overall operation of management device 200. Storage device 202 stores programs executed by CPU 201, various information related to excavator 100, etc. Communication device 203 communicates with excavator 100 and support device 300 via network.

[0072] The input device 204 is used to input information into the management device 200, for example, via a keyboard, a pointing device, etc. The display device 205 displays various information output from the management device 200, such as via a monitor.

[0073] Next, refer to Figure 3 The functional structure of the excavator 100 and the management device 200 is explained. Figure 3 It is a diagram illustrating the functional structure of the various devices in the excavator's management system.

[0074] First, the functions of the excavator 100 will be explained. The excavator 100 has an operation information acquisition unit 31, an operation information output unit 32, and a display control unit 33. Each of the excavator 100's units can be implemented by the controller 30 reading and executing programs stored in the storage device of the controller 30.

[0075] In this embodiment, the operation information acquisition unit 31 periodically acquires the operation information of the excavator 100. Here, the operation information of this embodiment will be explained.

[0076] Specifically, the operational information in this embodiment includes position information indicating the current position of the machine, orientation information indicating the orientation of the machine, and work content information indicating the work being performed. Furthermore, the operational information in this embodiment includes posture information indicating the machine's attitude and information indicating the digging reaction force. This information can be represented based on sensor values ​​output from the accelerometer and gyroscope sensors installed on the auxiliary device, sensor values ​​output from the cylinder pressure sensor, and pilot pressure. This information can be an example of status information indicating the state of the excavator 100.

[0077] Furthermore, the operational information of the embodiment includes image data captured by the camera device S6. Additionally, the image data includes still image data and animation data. Moreover, the camera device S6 may be an example of a spatial recognition device, and the operational information may include three-dimensional data representing the terrain surrounding the excavator 100 acquired by the camera device S6, which is a spatial recognition device.

[0078] In this embodiment, the image data acquired by the camera device S6 is an example of environmental information representing the environment surrounding the excavator 100. Furthermore, the environmental information in this embodiment may include information indicating the state of the excavated soil mound. In other words, the image represented by the image data acquired by the camera device S6 may include an image of the soil mound. The information indicating the state of the soil mound includes information such as its shape, soil type, presence of cracks, and water leakage.

[0079] In this embodiment, the operation information output unit 32 sends the operation information acquired by the operation information acquisition unit 31 to the management device 200 via the communication terminal T1.

[0080] The display control unit 33 controls the display in the excavator 100. Specifically, the display control unit 33 can display the work steps deduced by the management device 200 on the display device 40. Furthermore, a display device 40A, which is different from the display device 40, is provided in the cab 10 of the excavator 100, and the display control unit 33 can display information indicating the work steps on the display device 40A.

[0081] Furthermore, the display control unit 33 of this embodiment can overlay information representing the work steps deduced by the management device 200 onto the field of view around the excavator 100 (the machine) as observed by the operator inside the cab 10. At this time, an image projection device with the front window of the cab 10 as the projection surface can be installed inside the cab 10, and the display control unit 33 can control the image projection device to display information representing the work steps on the front window.

[0082] Furthermore, in this embodiment, information indicating the work steps can be displayed on a head-mounted display worn by the operator. At this time, the display control unit 33 can send information indicating the work steps to the head-mounted display.

[0083] Thus, in this embodiment, by displaying information indicating the work steps, the operator can confirm the work steps based on the information superimposed on the field of vision while seated in the driver's seat and observing the area around the excavator 100. Therefore, the operator can grasp the work steps along with the immediate field of vision. That is, the operator can intuitively understand the work steps.

[0084] Furthermore, some of the functions of controller 30 can also be implemented by other controllers (control devices). That is, the functions of controller 30 can be implemented by multiple controllers distributed among them. For example, the controller that implements the device guidance function and the device control function can be different from the controller that implements the view overlap display function.

[0085] Next, the functions of the management device 200 will be described. The management device 200 of this embodiment includes an information acquisition unit 210, a learning unit 220, an inference unit 230, and an output unit 240. Each unit of the management device 200 is implemented by the CPU 201 of the management device 200 reading and executing a program stored in the storage device 202.

[0086] The information acquisition unit 210 acquires various types of information. Specifically, the information acquisition unit 210 periodically acquires operational information from the excavator 100. The learning unit 220 uses training data to generate or update a machine learning model 231 that has been trained to analyze the relationship between operational information and work steps. Details regarding the training data and the learning process performed by the learning unit 220 will be described later.

[0087] The inference unit 230 stores the learned model 231. In other words, the function of the inference unit 230 is realized through the learned model 231. The learned model 231 is a learned model that takes operational information as input and outputs information representing the operation steps.

[0088] In other words, after learning, model 231 takes operational information, including state information representing the state of excavator 100 and environmental information representing the surrounding environment of excavator 100, as input, and outputs information representing the operation steps of excavator 100 at the work site.

[0089] In this embodiment, environmental information is assumed to be included in the operational information acquired by the excavator 100, but it is not limited to this. Environmental information can be acquired, for example, by a camera device (spatial recognition device) mounted on a flying object that flies around the excavator 100. In this case, the management device 200 can acquire environmental information from the flying object and operational information including status information from the excavator 100, and input the environmental information received from the flying object and the operational information acquired from the excavator 100 into the learned model 231. Alternatively, environmental information can be acquired by support devices carried by workers performing operations at the work site, camera devices installed at the work site, etc., and sent to the management device 200.

[0090] If the information acquisition unit 210 acquires operation information, the inference unit 230 inputs the acquired operation information into the learning completion model 231 and outputs information representing the operation steps output from the learning completion model 231.

[0091] The output unit 240 sends the information of the operation steps inferred by the inference unit 230 to the excavator 100.

[0092] Next, refer to Figure 4A , Figure 4B The operating steps of the excavator 100 in this embodiment will be further explained. Figure 4A This is the first diagram illustrating the operating procedures of an excavator. Figure 4B This is the second diagram illustrating the operating procedures of an excavator.

[0093] exist Figure 4A , Figure 4B The example schematically illustrates the situation where the excavator 100 moves the tip of the bucket 6 along the target construction surface to excavate a pile of soil and perform the operation of forming a slope.

[0094] exist Figure 4A The diagram shows a scenario where the current ground surface 402A is excavated from above to form a target construction surface 401A. At this time, the excavator 100 automatically controls the operation of auxiliary devices based on the operator's manual operation via the equipment guidance function, ensuring that the working parts of the bucket 6, such as the tip and back, align with the target construction surface 401A. Furthermore, the target construction surface 401A is preset in the excavator 100.

[0095] At this time, the management device 200 of this embodiment notifies the operator of the work steps so that the slope of the excavated surface after excavation of the current ground surface 402A meets predetermined conditions. The work steps here may include, for example, information such as the location to be excavated next and the depth of excavation (height of the excavated surface).

[0096] Furthermore, the predetermined conditions in this embodiment include a reference for the slope of the excavation face as stipulated in the Occupational Safety and Health Law and its Enforcement Order. The slope of the excavation face refers to the angle of the excavation face relative to the horizontal plane.

[0097] Furthermore, under predetermined conditions, the height and slope of the excavation face can be determined based on the type of soil mound. Specific examples of these predetermined conditions are shown below.

[0098] In situations where the collapse of the earth mound or the falling of mud and rocks could pose a danger to the operator of the machinery, When the height of the excavation face is less than 20m, the slope of the excavation face is less than 90 degrees.

[0099] When the height of the excavation face is more than 20m, the slope of the excavation face should be less than 75 degrees.

[0100] <Mounds of earth composed of bedrock other than the aforementioned bedrock> When the height of the excavation face is less than 5m, the slope of the excavation face is less than 90 degrees.

[0101] When the height of the excavation face is more than 5m, the slope of the excavation face should be less than 60 degrees.

[0102] <Mounds other than those mentioned in the previous items> When the height of the excavation face is less than 2m, the slope of the excavation face is less than 90 degrees.

[0103] When the height of the excavation face is more than 2m, the slope of the excavation face should be less than 45 degrees.

[0104] In this embodiment, if the operator of the excavator 100 begins to form a slope, the management device 200 infers the operation steps that meet the predetermined conditions of the slope and height of the excavation face based on the operation information received from the excavator 100 and the learned model 231, and notifies the operator of the excavator 100 of the operation steps.

[0105] Thus, in this embodiment, by specifying predetermined conditions, it is possible to perform work that complies with the occupational safety and health regulations based on the Occupational Safety and Health Act and its Enforcement Order.

[0106] exist Figure 4A In the example, excavator 100 begins excavation from the upper end Ta of the current ground surface 402A. At this time, management device 200, through learning model 231, determines the state of the soil pile based on the image data contained in the operation information. Specifically, learning model 231 determines whether the soil pile has cracks, leaks, etc.

[0107] Furthermore, if it is determined that the soil mound does not have cracks, leaks, or other defects, the management device 200, after learning the model 231, infers the operational steps for ensuring that the slope and height H11 of the excavation face 450 formed by excavation meet the predetermined conditions corresponding to the state of the soil mound, and notifies the operator of the excavator 100. The slope of the excavation face 450 refers to the angle θ11 between the excavation face 450 and the horizontal plane H.

[0108] The operating procedures notified to the operator from the management device 200 may include information such as the location and depth of the excavation during the excavation process.

[0109] More specifically, the operating procedures notified to the operator from the management device 200 can be drawn by the tip of the bucket 6. Figure 4A The operation method is the same as the curve shown by the dashed line 410 in the figure.

[0110] If an excavation face 450 is formed, the management device 200 infers that the slope and height H12 of the excavation face 451 formed by the subsequent excavation meet the predetermined conditions corresponding to the state of the soil mound, and notifies the operator of the excavator 100 of the operation steps. The slope of the excavation face 451 refers to the angle θ12 between the excavation face 451 and the horizontal plane H.

[0111] Furthermore, the management device 200 outlines the tip of the bucket 6. Figure 4A The operation method shown by the dashed line 411 in the figure is communicated to the operator as a work procedure.

[0112] Thus, in this embodiment, the operator of the excavator 100 is notified of the operation steps that the excavated face meets the predetermined conditions.

[0113] Therefore, according to this embodiment, it is possible to prevent the soil pile from collapsing or crumbling from above during excavation operations, and to prevent the excavator 100 from being buried by sand and soil due to the collapse or crumbling of the soil pile.

[0114] Furthermore, in cases where cracks or leaks exist in the soil pile, the management device 200 of this embodiment can send a notification to the excavator 100 urging it to evacuate from the work site. Upon receiving this notification, the excavator 100 can display the message urging evacuation from the work site on the display device 40. Additionally, the excavator 100 can output a sound urging evacuation from the work site to the workers at the work site via the sound output device 43. Thus, in this embodiment, workers at the work site can be evacuated, thereby improving safety.

[0115] exist Figure 4B The diagram shows a situation where the current ground surface 402B is excavated downwards to form a target construction surface 401B. At this time, the excavator 100 is set with the target construction surface 401B, and the operation of the auxiliary devices is automatically controlled according to the manual operation of the operator to make the working parts such as the tip and back of the bucket 6 aligned with the target construction surface 401B.

[0116] exist Figure 4B In the example, excavator 100 begins excavation from the end Tb of the current ground surface 402B. At this time, management device 200, having completed learning model 231, determines the state of the mound based on the image data contained in the operation information.

[0117] Next, assuming there are no cracks or leaks in the soil mound, the management device 200, through the learned model 231, infers that the slope and height H21 of the excavated face 460 formed by excavation meet the predetermined conditions corresponding to the state of the soil mound, and notifies the operator of the excavator 100 of the operational steps. The slope of the excavated face 460 refers to the angle θ21 between the excavated face 460 and the horizontal plane H.

[0118] Here, the operating steps notified to the operator from the management device 200 can be drawn by the tip of the bucket 6. Figure 4B The operation method is the same as the curve shown by the dashed line 420 in the figure.

[0119] If a digging face 460 is formed, the management device 200 deduces that the slope and height H22 of the digging face 461 formed by the subsequent digging meet the predetermined conditions corresponding to the state of the soil mound, and notifies the operator of the excavator 100 of the work steps. The slope of the digging face 461 refers to the angle θ22 between the digging face 461 and the horizontal plane H.

[0120] Then, the management device 200 outlines the tip of the bucket 6. Figure 4B The operation method shown by the dashed line 421 is communicated to the operator as a work procedure.

[0121] Thus, in this embodiment, by guiding the operator of the excavator 100 through operational steps to ensure the excavated face meets predetermined conditions, it is possible to prevent the excavator 100 from overturning or falling during operation. Furthermore, in this embodiment, it is possible to prevent excavation reaction forces that cannot be balanced by the counterweight 29 from being applied to the bucket 6.

[0122] Next, refer to Figure 5 The generation and updating of the learning completed model 231 of the management device 200 based on this embodiment will be explained.

[0123] Figure 5 This is a flowchart illustrating the processing of the learning unit of the management device. In this embodiment, the learning unit 220 of the management device 200 acquires training data (step S501).

[0124] Here, the training data for this embodiment will be explained. The training data for this embodiment is the operating information of the excavator 100 when the excavation face meets predetermined conditions.

[0125] In other words, the training data is the operational information when the excavation face formed by excavation meets the predetermined conditions corresponding to the type of soil mound.

[0126] The operational information of the excavator 100 used as training data could be, for example, operational information obtained when a highly skilled operator is operating the excavator 100.

[0127] The operational information includes environmental information (including the shape of the mound and the soil type) indicating the environment around the excavator 100 when a skilled operator is operating it, as well as status information (including various sensor values ​​and information indicating the machine's posture).

[0128] That is, the operational information of an excavator 100 operated by a skilled operator can be considered as a dataset that establishes a corresponding association between environmental information (environmental conditions) surrounding the excavator 100 and operational steps (correct data) that take into account the impact on the surrounding environment of the excavator 100. In this embodiment, operational information obtained when a skilled operator is operating the excavator 100 can be collected and used as training data.

[0129] In addition, in this embodiment, the operation information of a specific operator operating the excavator 100 can be used as training data, but it is not limited to this.

[0130] The management device 200 can pre-store the operation information of the excavator 100 when it has carried out excavation operations, and extract the operation information from the stored operation information to ensure that the excavation surface formed by excavation always meets the predetermined conditions corresponding to the shape of the soil mound, and use it as training data.

[0131] In this embodiment, by collecting training data and performing machine learning, the learned model 231 can infer the work steps like a skilled operator.

[0132] If training data is acquired, the learning unit 220 of the management device 200 performs machine learning on the training data (step S502) and generates or updates the learned model 231 (step S503).

[0133] The learning completion model 231 in this embodiment can be a learning completion model centered on a neural network.

[0134] A neural network is a so-called deep neural network that has one or more intermediate layers (hidden layers) between the input layer and the output layer. In a neural network, each of the multiple neurons constituting each intermediate layer is assigned a weighted parameter representing the connection strength with the lower layers. Moreover, the neurons in each layer form the neural network in such a way that the sum of the values ​​obtained by multiplying the input values ​​from the multiple neurons in the upper layer by the weighted parameters assigned to each neuron in the upper layer is passed through a threshold function and output to the neurons in the lower layers.

[0135] In the learning unit 220, machine learning is performed on the neural network, specifically deep learning, to optimize the aforementioned weighted parameters. Thus, the neural network can take the operational information acquired by the information acquisition unit 210 as input signals and output information indicating the work steps, showing that the excavation face formed by the excavation action meets predetermined conditions, as output signals.

[0136] In this embodiment, when training data is acquired, if a fully learned model 231 has not yet been generated, the management device 200 generates a fully learned model 231. Furthermore, when training data is acquired, if a fully learned model 231 has not yet been generated, the management device 200 can update the fully learned model 231 based on learning performed by the learning unit 220 each time operational information that becomes training data is acquired.

[0137] In this embodiment, by updating the learned model 231 in this way, the accuracy of the inference results based on the inference unit 230 can be improved.

[0138] Next, refer to Figure 6 The operation of the management system SYS in this embodiment will be explained. Figure 6 It is a sequence diagram illustrating the actions of the excavator's management system.

[0139] In the management system SYS of this embodiment, if the excavation operation is started, the excavator 100 acquires operation information through the operation information acquisition unit 31 (step S601), and sends the acquired operation information to the management device 200 through the operation information output unit 32 (step S602).

[0140] If operation information is acquired through the information acquisition unit 210, the management device 200 inputs the acquired operation information into the learned model 231 through the inference unit 230 (step S603). Next, the inference unit 230 acquires the information representing the work steps output from the learned model 231 (step S604). Then, the management device 200 sends the information representing the work steps to the excavator 100 through the output unit 240 (step S605).

[0141] If information indicating the work steps is received, the excavator 100 displays the information indicating the work steps (step S606).

[0142] The following is for reference. Figure 7 Here, a display example of the excavator 100 in this embodiment will be described. Figure 7 This is a diagram showing an example of information representing the steps of a task.

[0143] exist Figure 7The image shows the forward view 510 when an operator inside the cockpit 10 observes the front window.

[0144] like Figure 7 As shown, the operator's forward field of view 510 includes the working area in front of the upper slewing body 3, and the upper right part includes auxiliary devices (boom 4, stick 5, and bucket 6). Furthermore, an image projection device installed in the cab 10 displays (projects) information images 410A, 411A, and 412A in the operator's forward field of view 510 (specifically, in the front window of the cab 10, which serves as the projection surface).

[0145] Information images 410A, 411A, and 412A are examples of information representing the operational steps in an excavation operation. Specifically, information images 410A, 411A, and 412A represent the location and depth of excavation (the height of the resulting excavation face) through a single excavation action, respectively.

[0146] Information images 410A, 411A, and 412A are displayed at positions corresponding to the excavated location within the mound included in the forward field of view 510. Furthermore, the display method of information images 410A, 411A, and 412A can vary depending on the excavation depth. Specifically, information images 410A, 411A, and 412A can be displayed using colors or other methods corresponding to the excavation depth.

[0147] In addition, Figure 7 The example shown illustrates the simultaneous display of information images 410A, 411A, and 412A, but it is not limited to this. In this embodiment, a corresponding information image can be displayed for each digging action.

[0148] Specifically, for example, if the excavator 100 begins digging, only the information image 410A corresponding to the initially dug position is displayed. Furthermore, if the digging operation at the position corresponding to information image 410A ends and the next digging operation begins, the display control unit 33 can display only information image 411A as the position to be dug after the position corresponding to information image 410A. In this case, operation information can be sent from the excavator 100 to the management device 200 each time a digging operation is performed.

[0149] Furthermore, the management device 200 of this embodiment can update the displayed information images based on the operation information sent from the excavator 100 each time a digging operation is performed. Specifically, for example, if the position corresponding to information image 410A is excavated, the subsequently displayed information images 411A and 412A are updated to the results inferred from the operation information sent by the excavator 100 to the management device 200 after the digging operation at the position corresponding to information image 410A. Thus, in this embodiment, by inferring the work steps each time a digging operation is performed, the work steps based on the latest soil pile state can be communicated to the operator.

[0150] Furthermore, if the slope of the excavated face becomes greater than the value specified in the predetermined conditions, or if it approaches a predetermined range relative to the value specified in the predetermined conditions, the management device 200 of this embodiment can issue a warning to the operator. The warning can be issued... Figure 7 The forward field of view 510 shown is displayed in a position easily identifiable by the operator. Furthermore, the warning can be output as an audio signal.

[0151] Furthermore, if it is determined that there are cracks or leaks in the soil pile, the management device 200 of this embodiment can display information indicating the presence of cracks or leaks within the forward field of view 510. In this case, information images 410A, 411A, and 412A may not be displayed.

[0152] Furthermore, the management device 200 of this embodiment can be displayed on the display device 40 of the excavator 100, along with the display of information images relative to the forward field of view 510. Figure 4A , Figure 4B The image shown displays the side view of the excavator 100, the image representing the target construction surface, and the guide image corresponding to the dotted lines 410, 411, 420, and 421.

[0153] Furthermore, the management device 200 of this embodiment can replace projecting information images onto the front window, and instead display images of the side of the excavator 100, images representing the target construction surface, and guide images as information representing the work steps on the display device 40.

[0154] Furthermore, in this embodiment, information images 410A, 411A, and 412A can be displayed on a head-mounted display worn by the operator.

[0155] Furthermore, in this embodiment, information images 410A, 411A, and 412A can be displayed on the display device 40 of the excavator 100. At this time, the display control unit 33 of the excavator 100 only needs to overlay the information images 410A, 411A, and 412A on the image in front of the excavator 100 captured by the front camera S6F and display it on the display device 40.

[0156] Furthermore, in this embodiment, the management device 200 notifies the operator of information indicating the work steps, but is not limited to this. The management device 200 can control the operation of the excavator 100 based on the information indicating the work steps.

[0157] At this time, if information indicating the operation steps is received from the management device 200, the controller 30 of the excavator 100 can generate instruction values ​​to make the auxiliary device operate according to the information indicating the operation steps, and make the auxiliary device operate according to the information indicated by the operation steps.

[0158] Furthermore, the management device 200 of this embodiment can send information indicating the work steps to a remote control room for remotely operating the excavator 100. At this time, the information indicating the work steps can be overlaid on the front image of the excavator 100 and displayed on a display device provided in the remote control room.

[0159] Thus, by applying this embodiment to the remote operation of the excavator 100, for example, even if the excavator 100 is operating in an environment inaccessible to humans, the excavator 100 can perform operations with high safety, taking into account the impact on the environment surrounding the excavator 100.

[0160] Furthermore, the function of the management device 200 in this embodiment can be set in the controller 30 of the excavator 100.

[0161] At this time, the controller 30 of the excavator 100 only needs to have the functions of the learning unit 220 and the inference unit 230, input the operation information obtained by the machine into the trained model 231 of the machine, and display the information representing the operation steps output from the trained model 231.

[0162] Furthermore, in this embodiment, the learning unit 220 can be configured in the management device 200, and the inference unit 230 can be configured in the excavator 100. When the learning unit 220 generates a trained model 231, the management device 200 sends the trained model 231 to the excavator 100. The excavator 100 implements the function of the inference unit 230 by storing the trained model 231 in the controller 30. Thus, each time the trained model 231 is updated via the learning unit 220, the management device 200 can update the trained model 231 stored in the excavator 100.

[0163] Thus, in this embodiment, by providing the function of the inference unit 230 in the excavator 100, for example, the excavator 100 can perform excavation operations with high safety even in the event of a communication interruption between the excavator 100 and the management device 200.

[0164] Furthermore, in the above embodiments, the excavator 100 is taken as an example of a working machine, but the working machine is only required to perform digging operations and is not limited to an excavator.

[0165] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the embodiments described above. Various modifications and substitutions can be applied to the above embodiments without departing from the scope of the present invention. Furthermore, the features described separately can be combined as long as they do not create technical contradictions.

[0166] Furthermore, this international application asserts priority based on Japanese Patent Application No. 2023-211306, filed on December 14, 2023, and incorporates the entire contents of Japanese Patent Application No. 2023-211306 into this international application.

[0167] Explanation of symbols 1-Lower traveling body, 2-Slewing mechanism, 3-Upper slewing body, 4-Boom, 5-Stick, 6-Bucket, 30-Controller, 31-Operation information acquisition unit, 32-Operation information output unit, 33-Display control unit, 100-Excavator, 200-Management device, 210-Information acquisition unit, 220-Learning unit, 230-Inference unit, 231-Learned model, 240-Output unit.

Claims

1. A type of operating machinery, comprising: The operation information acquisition unit acquires operation information, including environmental information indicating the environment surrounding the machine and status information indicating the status of the machine. The inference unit, based on the operational information, infers the operational steps in which the excavation face formed by the excavation action meets predetermined conditions; and The display control unit displays information indicating the operation steps inferred by the inference unit.

2. The operating machinery according to claim 1, wherein, The environmental information includes the shape of the mound excavated by the excavation action and the soil composition of the mound. The predetermined condition is that the slope of the excavation face is less than the slope specified according to the type of soil mound excavated by the excavation action.

3. The operating machinery according to claim 2, wherein, The predetermined condition is that the height of the excavation face is within a range specified according to the type of mound excavated by the excavation action.

4. The operating machinery according to claim 1, wherein, Each time the excavation action is performed, the inference unit acquires the operational information and infers the operation steps.

5. The operating machinery according to claim 2, wherein, The inference unit determines whether there are cracks or leaks in the soil pile based on the environmental information, and outputs a warning if it determines that there are cracks or leaks in the soil pile.

6. The operating machinery according to claim 1, wherein, The inference part is a learned model generated through machine learning. The machine learning method uses a dataset obtained by establishing a corresponding association between environmental information representing the environment around the working machinery and information representing the work steps when the excavation face formed in the excavation operation meets the predetermined conditions corresponding to the type of soil mound represented by the environmental information, as training data.

7. The operating machinery according to claim 1, comprising: Upper rotating body; Lower walking body; and An auxiliary device is provided on the upper rotating body. The machine has a control unit that controls the operation of auxiliary devices according to the operation steps deduced by the inference unit.

8. A management system for operating machinery, comprising operating machinery and a management device communicating with said operating machinery, wherein in the management system for operating machinery, The management device has: The information acquisition unit acquires operational information from the operating machinery, including environmental information representing the environment surrounding the operating machinery and status information representing the status of the operating machinery. The inference unit, based on the operational information, infers the operational steps in which the excavation face formed by the digging action of the working machinery meets predetermined conditions; and The output unit outputs information representing the work steps inferred by the inference unit to the work machine.

9. A management device for operating machinery, which communicates with the operating machinery, the management device for operating machinery comprising: The information acquisition unit acquires operational information from the operating machinery, including environmental information representing the environment surrounding the operating machinery and status information representing the status of the operating machinery. The inference unit, based on the operational information, infers the operational steps in which the excavation face formed by the digging action of the working machinery meets predetermined conditions; and The output unit outputs information representing the work steps inferred by the inference unit to the work machine.

Images