Construction machine and support system for construction machine

Abstract

This construction machine is provided with: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; an operation cabin mounted on the upper revolving body; and the space recognition device is arranged on a stand column in the cab.

Description

Technical Field

[0001] The present invention relates to a construction machine and a support system for the construction machine. Background Art

[0002] Conventionally, there is known a construction machine having a camera disposed at a position capable of photographing an operator sitting on a driver's seat and an operating lever operated by the operator, and a camera disposed at a position capable of photographing a travel operating lever and a travel operating pedal. It is also known that the images photographed by these cameras are used in the construction machine to prevent unintended erroneous operations by the operator.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-111895 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The operator sitting in the driver's seat does not always face forward. Therefore, in the prior art, it is impossible to capture the image of the operator's facial parts such as eyes, nose, mouth, etc.

[0008] Therefore, in view of the above circumstances, an object of the present invention is to capture an image of a facial portion of an operator.

[0009] Means for solving problems

[0010] The construction machine of the present invention comprises: a lower traveling body; an upper rotating body rotatably mounted on the lower traveling body; a cab mounted on the upper rotating body; and a space recognition device arranged on a column in the cab.

[0011] The support system for construction machinery of the present invention includes a construction machinery and a management device, wherein the construction machinery has: a lower traveling body; an upper rotating body, which is rotatably mounted on the lower traveling body; a cab, which is mounted on the upper rotating body; a space recognition device, which is arranged on a column in the cab; and an output unit, which outputs a plurality of image data taken by the space recognition device to the management device, and the management device has a storage unit for storing the plurality of image data.

[0012] Effects of the Invention

[0013] The present invention aims to capture an image of a facial portion of an operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram showing an example of a system configuration of a support system for a shovel.

[0015] Figure 2 This is a diagram for explaining the arrangement of the camera device inside the cockpit.

[0016] Figure 3 This is a diagram showing a state where the front window is halfway opened.

[0017] Figure 4 This is a block diagram showing a configuration example of a drive system of a shovel.

[0018] Figure 5 This is a diagram for explaining an example of a state detection method of the state detection unit.

[0019] Figure 6 This is a flowchart for explaining the processing of the shovel according to the first embodiment.

[0020] Fig. 7A The first figure shows an example of output image data.

[0021] Figure 7B The second figure shows an example of output image data.

[0022] Figure 8 This is a flowchart for explaining the operation of the shovel according to the second embodiment.

[0023] Fig. 9 This is a diagram for explaining the system configuration of a support system for a construction machine according to a third embodiment. DETAILED DESCRIPTION

[0024] (First Embodiment)

[0025] Hereinafter, the embodiment will be described with reference to the accompanying drawings. In addition, in the accompanying drawings, the X-axis, the Y-axis, and the Z-axis are axes that are orthogonal to each other. Specifically, the X-axis extends along the front-rear axis of the excavator, the Y-axis extends along the left-right axis of the excavator, and the Z-axis extends along the rotation axis of the excavator. In this embodiment, the X-axis and the Y-axis extend in the horizontal direction, and the Z-axis extends in the vertical direction.

[0026] Figure 1 1 is a diagram showing an example of a system configuration of a shovel support system. The shovel support system SYS of the present embodiment includes a shovel 100 and a management device 200. In the following description, the shovel support system SYS is simply referred to as a support system SYS.

[0027] In the support system SYS of the present embodiment, the shovel 100 and the management device 200 are connected via a network etc. The shovel 100 is an example of a working machine.

[0028] In the shovel 100 of the present embodiment, a plurality of camera devices are arranged at positions that always take a facial image of an operator sitting on a driver's seat in a cab 10 described later. In addition, in the present embodiment, the state of the operator is detected based on the image data taken by the plurality of camera devices arranged in the cab 10, and the detection result and the image data are associated and stored.

[0029] Furthermore, the shovel 100 according to the present embodiment can transmit information including image data and detection results captured by a plurality of imaging devices to the management device 200 , and manage the information in the management device 200 .

[0030] Furthermore, in the present embodiment, the detection result and image data of the operator's state can be displayed in the management device 200. The details of the operator's state in the present embodiment will be described later.

[0031] The image data of this embodiment includes animation data and still image data. Figure 1 In the example of FIG. 1 , the support system SYS includes the shovel 100 and the management device 200, but is not limited thereto. The support system SYS may also include a support device for supporting an operator operating the shovel 100. The support device may be a portable terminal device such as a smartphone, a tablet terminal, or a wearable terminal.

[0032] And, in Figure 1 In the example of FIG. 2 , the management device 200 is implemented by one information processing device, but the present invention is not limited thereto. The management device 200 may also be implemented by a plurality of information processing devices. In other words, the functions implemented by the management device 200 may be implemented by a plurality of information processing devices.

[0033] Furthermore, in the present embodiment, a plurality of imaging devices are arranged in the cockpit 10, but the present invention is not limited thereto. The imaging device may be arranged at a position where it can always capture the facial image of the operator sitting in the driver's seat.

[0034] Hereinafter, the shovel 100 according to the present embodiment will be described. Figure 1 A side view of the excavator 100 is shown in FIG.

[0035] The excavator 100 is an example of a construction machine, and includes a lower traveling body 1, a slewing mechanism 2, and an upper slewing body 3. In the excavator 100, the upper slewing body 3 is rotatably mounted on the lower traveling body 1 via the slewing mechanism 2. A boom 4 is mounted on the upper slewing body 3. An arm 5 is mounted at the front end of the boom 4, and a bucket 6 as an end attachment is mounted at the front end of the arm 5.

[0036] The boom 4, the arm 5, and the bucket 6 constitute an excavation attachment as an example of an attachment. The boom 4 is driven by a boom cylinder 7, the arm 5 is driven by an arm cylinder 8, and the bucket 6 is driven by a bucket cylinder 9. A boom angle sensor S1 is mounted on the boom 4, an arm angle sensor S2 is mounted on the arm 5, and a bucket angle sensor S3 is mounted on the bucket 6.

[0037] The boom angle sensor S1 is configured to detect the rotation angle of the boom 4. In the present embodiment, the boom angle sensor S1 is an acceleration sensor that can detect the rotation angle of the boom 4 relative to the upper swing body 3 (hereinafter referred to as the "boom angle"). The boom angle is, for example, the minimum angle when the boom 4 is lowered to the maximum limit, and increases as the boom 4 is raised.

[0038] The arm angle sensor S2 is configured to detect the rotation angle of the arm 5. In the present embodiment, the arm angle sensor S2 is an acceleration sensor that can detect the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the "arm angle"). The arm angle is, for example, the minimum angle when the arm 5 is closed to the maximum extent, and increases as the arm 5 is opened.

[0039] The bucket angle sensor S3 is configured to detect the rotation angle of the bucket 6. In the present embodiment, the bucket angle sensor S3 is an acceleration sensor that can detect the rotation angle of the bucket 6 relative to the boom 5 (hereinafter referred to as the "bucket angle"). The bucket angle is, for example, the smallest angle when the bucket 6 is closed to the maximum extent, and increases as the bucket 6 is opened.

[0040] The boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be a potentiometer utilizing a variable resistor, a stroke sensor for detecting the stroke amount of the corresponding hydraulic cylinder, a rotary encoder for detecting the rotation angle around a connecting pin, a gyro sensor, or a combination of an acceleration sensor and a gyro sensor.

[0041] A boom rod pressure sensor S7R and a boom cylinder bottom pressure sensor S7B are mounted on the boom cylinder 7. An arm rod pressure sensor S8R and an arm cylinder bottom pressure sensor S8B are mounted on the arm cylinder 8.

[0042] A bucket rod pressure sensor S9R and a bucket cylinder bottom pressure sensor S9B are mounted on the bucket cylinder 9. The boom rod pressure sensor S7R, the boom cylinder bottom pressure sensor S7B, the arm rod pressure sensor S8R, the arm cylinder bottom pressure sensor S8B, the bucket rod pressure sensor S9R, and the bucket cylinder bottom pressure sensor S9B are also collectively referred to as "cylinder pressure sensors".

[0043] The boom rod pressure sensor S7R detects the pressure of the rod side oil chamber of the boom cylinder 7 (hereinafter referred to as the "boom rod pressure"), and the boom cylinder bottom pressure sensor S7B detects the pressure of the bottom side oil chamber of the boom cylinder 7 (hereinafter referred to as the "boom cylinder bottom pressure"). The arm rod pressure sensor S8R detects the pressure of the rod side oil chamber of the arm cylinder 8 (hereinafter referred to as the "arm rod pressure"), and the arm cylinder bottom pressure sensor S8B detects the pressure of the bottom side oil chamber of the arm cylinder 8 (hereinafter referred to as the "arm cylinder bottom pressure").

[0044] The bucket rod pressure sensor S9R detects the pressure of the rod side oil chamber of the bucket cylinder 9 (hereinafter referred to as “bucket rod pressure”), and the bucket cylinder bottom pressure sensor S9B detects the pressure of the bottom side oil chamber of the bucket cylinder 9 (hereinafter referred to as “bucket cylinder bottom pressure”).

[0045] The upper swing body 3 is provided with a cab 10 as a driver's cabin and is equipped with a power source such as an engine 11. In addition, a sensor for detecting the amount of CO2 emission may be provided near the emission mechanism of the engine 11.

[0046] Furthermore, the upper rotating body 3 is provided with a controller 30, a display device 40, an input device 42, a sound output device 43, a storage device 47, a positioning device P1, a body tilt sensor S4, a rotation angular velocity sensor S5, a camera device S6 and a communication device T1.

[0047] The upper rotating body 3 may be equipped with a power storage unit for supplying electric power and an electric generator for generating electric power using the rotational driving force of the engine 11. The power storage unit is, for example, a capacitor or a lithium-ion battery. The electric generator may function as a motor to drive a mechanical load, or may function as a generator to supply electric power to an electric load.

[0048] The controller 30 functions as a main control unit that performs drive control of the excavator 100. In the present embodiment, the controller 30 is composed of a computer including a CPU, a RAM, and a ROM. Various functions of the controller 30 are realized, for example, by the CPU executing a program stored in the ROM. The various functions may include, for example, at least one of a device guidance function that guides the operator's manual operation of the excavator 100 and a device control function that automatically supports the operator's manual operation of the excavator 100.

[0049] The display device 40 is configured to display various information. The display device 40 may be connected to the controller 30 via a communication network such as CAN, or may be connected to the controller 30 via a dedicated line.

[0050] The input device 42 is configured to allow the operator to input various information to the controller 30. The input device 42 includes at least one of a touch panel, a knob switch, a membrane switch, and the like provided in the cockpit 10.

[0051] The sound output device 43 is configured to output sound. The sound output device 43 may be, for example, a vehicle-mounted speaker connected to the controller 30 or an alarm such as a buzzer. In the present embodiment, the sound output device 43 is configured to output various information by sound according to a sound output command from the controller 30 .

[0052] The storage device 47 is configured to store various information. The storage device 47 is, for example, a nonvolatile storage medium such as a semiconductor memory. The storage device 47 may store information outputted by various devices during the operation of the shovel 100 or information acquired via various devices before the operation of the shovel 100 starts.

[0053] Specifically, the storage device 47 may store information including image data captured by a plurality of imaging devices (cameras) disposed in the cockpit 10 and detection results of the operator's state detected by the controller 30 .

[0054] Furthermore, the storage device 47 may store data on the target construction surface acquired via the communication device T1, etc. The target construction surface may be set by the operator of the shovel 100 or by a construction manager or the like.

[0055] The positioning device P1 is configured to measure the position of the upper rotating body 3. The positioning device P1 is configured to be able to measure the orientation of the upper rotating body 3. In the present embodiment, the positioning device P1 is, for example, a GNSS compass, which detects the position and orientation of the upper rotating body 3 and outputs the detection value to the controller 30. Therefore, the positioning device P1 can also function as an orientation detection device for detecting the orientation of the upper rotating body 3. The orientation detection device may be an azimuth sensor mounted on the upper rotating body 3.

[0056] The body tilt sensor S4 is configured to detect the tilt of the upper rotating body 3. In the present embodiment, the body tilt sensor S4 is an acceleration sensor that detects the front-rear tilt angle about the front-rear axis and the left-right tilt angle about the left-right axis of the upper rotating body 3 relative to the imaginary horizontal plane. The front-rear axis and the left-right axis of the upper rotating body 3 are orthogonal to each other at, for example, a point on the rotating axis of the excavator 100, that is, the center point of the excavator.

[0057] The rotation angular velocity sensor S5 is configured to detect the rotation angular velocity of the upper rotating body 3. The rotation angular velocity sensor S5 may also be configured to detect or calculate the rotation angle of the upper rotating body 3. In the present embodiment, the rotation angular velocity sensor S5 is a gyro sensor. The rotation angular velocity sensor S5 may also be a resolver, a rotary encoder, or the like.

[0058] The camera device S6 is an example of a space recognition device, and is configured to acquire images of the vicinity of the shovel 100. In the present embodiment, the camera device S6 includes a front camera S6F for photographing the space in front of the shovel 100, a left camera S6L for photographing the space to the left of the shovel 100, a right camera S6R for photographing the space to the right of the shovel 100, and a rear camera S6B for photographing the space behind the shovel 100.

[0059] Furthermore, the imaging device S6 of the present embodiment may include a plurality of imaging devices installed in the cockpit 10. The details of the arrangement of the plurality of imaging devices installed in the cockpit 10 will be described later.

[0060] The camera device S6 is, for example, a monocular camera having an imaging element such as a CCD or a CMOS, and outputs the captured image to the display device 40. The camera device S6 may also be a stereo camera, a distance image camera, etc. Furthermore, the camera device S6 may be replaced by other space recognition devices such as a three-dimensional distance image sensor, an ultrasonic sensor, a millimeter wave radar, a LIDAR, or an infrared sensor, or may be replaced by a combination of other space recognition devices and a camera.

[0061] The front camera S6F is installed, for example, on the ceiling of the cockpit 10, that is, inside the cockpit 10. However, the front camera S6F may be installed outside the cockpit 10, such as on the roof of the cockpit 10 or on the side of the boom 4. The left camera S6L is installed on the left end of the upper surface of the upper rotating body 3, the right camera S6R is installed on the right end of the upper surface of the upper rotating body 3, and the rear camera S6B is installed on the rear end of the upper surface of the upper rotating body 3.

[0062] The communication device T1 controls communication with external devices outside the shovel 100. In the present embodiment, the communication device T1 controls communication with external devices via a satellite communication network, a mobile phone communication network, or the Internet. The external device may be, for example, a management device 200 such as a server installed in an external facility, or a support device such as a smartphone carried by a worker around the shovel 100.

[0063] The shovel 100 can transmit the image data captured by the camera S6 to an external device such as the management device 200 via the communication device T1. With this structure, a worker or manager outside the shovel 100 can visually recognize the state of the shovel 100 and the state of the operator through a display device such as a monitor connected to the management device 200 or a support device.

[0064] Next, refer to Figure 2 The arrangement of the imaging device S6 inside the cockpit 10 according to the present embodiment will be described. Figure 2 This is a diagram for explaining the arrangement of the camera device inside the cockpit.

[0065] Figure 2 It is a perspective view of the interior of the cockpit 10 , and shows a state when looking forward from the driver's seat in the cockpit 10 .

[0066] like Figure 2 As shown, a driver's seat 90 is provided in the cockpit 10. A left control panel 90L is provided on the left side of the driver's seat 90, and a right control panel 90R is provided on the right side of the driver's seat 90. A left operating lever 26L is installed on the upper front end of the left control panel 90L, and a right operating lever 26R is installed on the right control panel 90R at a position corresponding to the left operating lever 26L. A main monitor 40M as one of the display devices 40 is installed on the upper front end of the right control panel 90R.

[0067] In the cockpit 10, a camera S6PL is mounted on the left column 111L, and a camera S6PR is mounted on the right column 111R. The left column 111L and the right column 111R are Figure 3 A portion of the frame body 110 is shown. The camera S6PL and the camera S6PR are imaging devices and are examples of a three-dimensional sensor.

[0068] Furthermore, in this embodiment, as described later Figure 3 As shown, the cameras S6PL and S6PR are mounted on the left and right pillars at a position between the height H1 of the seat surface 90m of the driver's seat 90 and the height H2 of the upper end of the headrest 90c of the driver's seat 90. In addition, the "height" in this embodiment can be, for example, the vertical distance from the floor surface 10f of the cockpit 10.

[0069] In the present embodiment, by disposing the cameras S6PL and S6PR in this manner, it is possible to capture an operator sitting on the driver's seat 90 and looking forward from both the left and right sides.

[0070] Specifically, for example, when the operator sitting on the driver's seat 90 faces right, the camera S6PR mounted on the right pillar 111R can be used to capture an image including the operator's face. Also, when the operator sitting on the driver's seat 90 faces left, the camera S6PL mounted on the left pillar 111L can be used to capture an image including the operator's face.

[0071] Furthermore, the cameras S6PL and S6PR of the present embodiment are installed so as not to obstruct the field of vision of the operator through the front window 62. Therefore, the cameras S6PL and S6PR of the present embodiment are set to have a size that fits within the width of the left pillar 111L and the right pillar 111R.

[0072] In this embodiment, by setting the cameras S6PL and S6PR to such sizes, it is possible to prevent the cameras S6PL and S6PR from interfering with the opening and closing of the window front 60. In other words, in this embodiment, it is possible to prevent the cameras S6PL and S6PR from interfering with the opening and closing operation of the front window 62.

[0073] In addition, Figure 2 In the example of FIG. 1 , one camera is installed on each of the left and right pillars, but the number of pillars installed on the left pillar 111L and the right pillar 111R is not limited thereto. In the present embodiment, for example, a plurality of camera devices may be installed on each of the left and right pillars.

[0074] Furthermore, the cameras S6PL and S6PR of this embodiment may be movable. Specifically, for example, guide rails having grooves may be installed on the left column 111L and the right column 111R, and the cameras S6PL and S6PR may be embedded in the guide rails installed on the left column 111L and the right column 111R.

[0075] At this time, the operator can move the installation positions of the cameras S6PL and S6PR by sliding the cameras S6PL and S6PR on the guide rails.

[0076] Furthermore, in this embodiment, for example, a plurality of mounting portions for mounting the cameras S6PL and S6PR may be provided on the left column 111L and the right column 111R, respectively. Specifically, the mounting portion may be, for example, a USB hub including a plurality of USB (Universal Serial Bus) connectors.

[0077] At this time, by connecting the cameras S6PL and S6PR to any USB connector included in the USB hub, the installation positions of the cameras S6PL and S6PR can be moved.

[0078] Furthermore, the cameras S6PL and S6PR of this embodiment may be provided with fixing members for fixing to the left column 111L and the right column 111R, respectively, on the back of the housing, etc. For example, when the left column 111L and the right column 111R are formed of metal such as iron, the fixing members may be magnets or screws.

[0079] At this time, the installation positions of the cameras S6PL and S6PR can be moved by fixing the cameras S6PL and S6PR to arbitrary positions of the left column 111L and the right column 111R.

[0080] Thus, in this embodiment, by making the cameras S6PL and S6PR movable, the cameras S6PL and S6PR can be moved to a position suitable for capturing the facial image of the operator according to the seat height of the operator when the operator sits on the driver's seat 90, etc.

[0081] In the present embodiment, although not shown in the drawings, a left monitor and a right monitor may be provided on the left column 111L and the right column 111R at positions that do not interfere with the cameras S6PL and S6PR.

[0082] At this time, the left monitor is preferably installed at a height higher than the main monitor 40M and lower than the height of the left rear reflector 10c. The "height" is, for example, a vertical distance from the ground. Furthermore, it is preferably installed at a height substantially the same as the height of the left camera S6L. Similarly, the right monitor is preferably installed at a height higher than the main monitor 40M and lower than the height of the left rear reflector 10c. Furthermore, it is preferably installed at a height substantially the same as the height of the right camera S6R.

[0083] The rear monitor 40B is mounted on the upper portion of the right pillar 111R so as to be arranged along the front ceiling frame 113. Figure 3 A portion of the frame body 110 is shown.

[0084] The rear monitor 40B may be attached to the upper portion of the left pillar 111L so as to be arranged along the front ceiling frame 113 .

[0085] In this way, the left monitor can be arranged at the left part of the visual field of the operator who is sitting on the driver's seat and observing the front excavator like a left rear reflector. And, the right monitor can be arranged at the right part of the visual field like a right rear reflector.

[0086] Furthermore, in the present embodiment, the rear monitor 40B is arranged at the upper part of the visual field like a rear reflector. Therefore, the operator of the excavator can intuitively recognize that the image projected on the left monitor is a mirror image of the left rear of the excavator. Similarly, the operator can intuitively recognize that the image projected on the right monitor is a mirror image of the right rear of the excavator, and the image projected on the rear monitor 40B is a mirror image of the rear of the excavator.

[0087] The images displayed on the left monitor, the right monitor, and the rear monitor 40B correspond to the images captured by the left camera S6L, the right camera S6R, and the rear camera S6B, respectively. That is, the left monitor, the right monitor, and the rear monitor 40B each independently project different directions. Furthermore, the display of the left monitor, the right monitor, and the rear monitor 40B starts simultaneously with the start of the main monitor 40M when the operator turns on the key. However, it may also start simultaneously with the start of the engine 11.

[0088] Furthermore, the left monitor, the right monitor, and the rear monitor 40B are installed so as not to obstruct the field of vision of the operator through the front window 62. Therefore, in the present embodiment, the left monitor and the right monitor have a size that fits within the width of the left pillar 111L and the right pillar 111R, and the rear monitor 40B is installed at the upper right corner of the front window 62. However, the left monitor and the right monitor may have a width wider than the width of the left pillar 111L and the right pillar 111R, and the rear monitor 40B may have a size that fits within the width of the front ceiling frame 113. Furthermore, the left monitor, the right monitor, and the rear monitor 40B are installed at a position that does not obstruct the opening and closing of the front window 62.

[0089] The screen size and resolution of each of the left monitor, the right monitor, and the rear monitor 40B are preferably selected so that the image of a person located within a predetermined distance (e.g., 12 m) from the excavator is displayed on the screen larger than a predetermined size (e.g., 7 mm×7 mm). For example, a monitor having a screen size of 7-inch (7 inches) or larger is used as the left monitor and the right monitor, and a monitor having a screen size of 7-inch (7 inches) or 8-inch (8 inches) is preferably used.

[0090] Furthermore, the left monitor and the right monitor are installed at the same height relative to a reference horizontal plane. The reference horizontal plane is, for example, the ground where the excavator is located. Figure 2 The center line of the cab indicated by the single-dot chain line becomes symmetrical on the left and right.

[0091] Furthermore, the left monitor, the right monitor, and the rear monitor 40B may be configured so that the installation angles can be adjusted according to the body shape, working posture, etc. of the operator sitting on the driver's seat 90 .

[0092] like Figure 2 As shown, a driver's seat 90 is provided in the center of the cab 10, and a left operating lever 26L and a right operating lever 26R are provided on both sides thereof. Therefore, the operator sits on the driver's seat 90, operates the left operating lever 26L with his left hand, and operates the right operating lever 26R with his right hand, thereby moving the bucket 6 to a desired position to perform excavation work.

[0093] An image display unit 41M and a switch panel 42M of the main monitor 40M are provided in the front right of the operator's seat 90. The operator of the shovel can understand the operation status of the shovel by observing the image display unit 41M. Figure 2 In the example of FIG. 4 , the image display unit 41M displays a bird's-eye view image. The bird's-eye view image is an example of a composite image generated based on images captured by the rear camera S6B and the left and right side cameras. Specifically, the bird's-eye view image is a viewpoint conversion image showing the state of the excavator's surroundings when viewed from a virtual viewpoint directly above.

[0094] A left monitor is mounted on the left column 111L, and a right monitor is mounted on the right column 111R. The left monitor and the right monitor may be mounted at positions such that when the operator captures the bucket 6 in the central field of vision through the front window 62 of the cab 10, the left monitor and the right monitor can be captured in the peripheral field of vision. Therefore, when the operator performs excavation work while capturing the bucket 6 in the central field of vision, the operator can capture the left rear and right rear states of the shovel projected on the left monitor and the right monitor in the peripheral field of vision without moving the line of sight.

[0095] The left console 90L is provided with a rotary knob 52 and an operating device 53. When the operator wishes to change the range of the left monitor that displays the image of the left rear of the shovel, the operator operates the rotary knob 52 to select the left camera S6L. Then, the operator operates the operating device 53 to change the direction of the left camera S6L, thereby changing the range of the left monitor. The same applies to changing the range of the left rear reflector 10c.

[0096] Next, refer to Figure 3 The opening and closing of the front window 62 will be described. Figure 3 This is a diagram showing the front window being opened. Figure 3 (A) is the main view. Figure 3 Middle (C) is the left side view.

[0097] The cockpit 10 includes an operating device 26 as a driving control unit, a driver's seat 90, a display device 40, a door lock lever 45, cameras S6PL and S6PR, etc. Although not shown in the figure, the cockpit 10 is equipped with Figure 1 The controller 30 shown etc.

[0098] The operating device 26 has a left operating lever 26L, a right operating lever 26R, a travel lever 26B, a pedal 26C, etc. A door lock rod 45 and a door lock valve 19 are provided at the lower left side of the driver's seat 90. When the door lock rod 45 is lifted so that the operator cannot exit from the cockpit 10, the door lock valve 19 is switched to a connected state (open state), so that the pilot line is connected, and various operating devices become operable. On the other hand, when the door lock rod 45 is pressed down so that the operator can exit from the cockpit 10, the door lock valve 19 is switched to a non-connected state (closed state), so that the pilot line is cut off, and various operating devices become inoperable.

[0099] The driver's seat 90 has an armrest 90a, a backrest 90b and a headrest 90c. In this embodiment, the height from the floor surface 10f of the cockpit 10 to the seat surface 90m of the driver's seat 90 is H1, and the height from the floor surface 10f of the cockpit 10 to the upper end of the headrest 90c is H2.

[0100] Here, the cameras S6PL and S6PR of the present embodiment are arranged at positions where the height from the floor surface 10f of the cockpit 10 is between H1 and H2 on the left pillar 111L and the right pillar 111R, respectively. In other words, the cameras S6PL and S6PR of the present embodiment are arranged so that the height from the floor surface 10f of the cockpit 10 is within the height range H3.

[0101] In the present embodiment, by arranging the cameras S6PL and S6PR in this manner, it is possible to capture the facial image of the operator regardless of the sitting height of the operator sitting on the seat surface 90m.

[0102] The cockpit 10 has a frame body 110. The frame body 110 is formed by combining a longitudinal frame, a transverse frame and a connecting frame. The longitudinal frame has a pair of left and right columns 111 (111L, 111R) located on the front side (travel direction side, Z1 side) and a pair of left and right longitudinal frames (columns) 112 located on the rear side (Z2 side). The transverse frame has a front ceiling frame 113 spanning between the left column 111L and the right column 111R on the front side and a rear ceiling frame 114 spanning between the left and right columns 112 on the rear side. The left and right pair of columns 111 on the front side and the left and right pair of columns 112 on the rear side are connected by a pair of left and right connecting frames 115, respectively.

[0103] In this embodiment, Figure 2 In the example of FIG. 1 , the camera devices can be respectively installed on the left and right pillars 112 at the rear side. In this case, at least four camera devices are set in the cockpit 10.

[0104] In the present embodiment, by installing the imaging devices on the left and right pillars 112 at the rear, the facial image of the operator can be captured even when the operator faces rearward.

[0105] The cockpit 10 has a window front 60 disposed on a front frame formed by the frame body 110. Also, the cockpit 10 has side windows 65 disposed on left and right frames formed by the frame body 110, respectively. Also, the cockpit 10 has a skylight disposed on a top frame formed by the frame body 110.

[0106] The window front portion 60 includes a lower front window 61 , a front window 62 , and an upper front window 63 .

[0107] The front window 62 has a sliding mechanism that can slide the front window 62. In the present embodiment, the front window 62 is separated from the lower front window 61 so that the front window 62 can be accommodated inside the cockpit 10 when the front window 62 is slid.

[0108] The sliding mechanism of this embodiment includes a pair of left and right slide rails 111a respectively provided on the inner side surfaces of a pair of left and right columns 111 and a pair of left and right slide rails 115a respectively provided on the inner side surfaces of left and right connecting frames 115. The front window 62 is arranged between the left and right slide rails 111a and between the left and right slide rails 115a. The slide rails 111a and the slide rails 115a may be formed so that the track grooves are continuous, and a sliding portion (not shown) provided at the end of the front window 62 or the like can move from the slide rails 111a to the slide rails 115a. The sliding portion may be a roller or the like.

[0109] The front window 62 is arranged between the lower front window 61 and the upper front window 63 when closed. The front window 62 is separated from the lower front window 61 and the upper front window 63 and slides in the opening direction (Y1 side) when opened. The front window 62 slides in the closing direction (Y2 side) and is arranged between the lower front window 61 and the upper front window 63 when closed.

[0110] The front window 62 is provided with a handle portion 62a at the upper left position and the upper right position, respectively. The handle portion 62a may also be provided only on either side. The operator grasps the handle portion 62a to slide the front window 62 in the opening direction or the closing direction. The handle portion 62a may have a locking mechanism for fixing the front window 62 between the lower front window 61 and the upper front window 63.

[0111] The upper front window 63 and the front window 62 may be provided with a sealing member not shown therein. The sealing member may be provided at the lower edge of the upper front window 63 and the upper edge of the front window 62. The sealing member provided at the lower edge of the upper front window 63 may have an eaves on the front side (Z1 side). The eaves prevent rainwater and the like from entering from between the two sealing members. The two sealing members may be structures that can be separated from each other.

[0112] The lower front window 61 is directly fixed to the frame body 110 etc. A sealing member may be interposed between the lower front window 61 and the front window 62 .

[0113] When the front window 62 is closed, it is arranged in the same plane with the upper front window 63 through the sealing member, thereby constituting the window front part 60 of the cockpit 10. When the operator opens the front window 62 by opening the handle portion 62a, the front window 62 moves to the rear side (Z2 side) and detaches from the upper front window 63. At this time, the front window 62 also detaches from the lower front window 61. The opening operation refers to the unlocking operation of releasing the fixation of the front window 62 and the operation of sliding the front window 62 upward.

[0114] Next, if the operator grasps the handle portion 62a of the front window 62 to open it, the front window 62 opens as shown in FIG. Figure 3 At this time, since the cameras S6PL and S6PR are within the width of the left pillar 111L and the right pillar 111R, the front window 62 and the cameras S6PL and S6PR do not contact each other. In other words, the cameras S6PL and S6PR do not interfere with the opening and closing of the front window 62.

[0115] When the front window 62 is slid, it is arranged at a position parallel to the top surface of the cockpit 10. At this time, the portion of the front surface of the cockpit 10 where the front window 62 is arranged is in an open state.

[0116] Next, refer to Figure 4 The configuration of the drive system of the shovel 100 will be described. Figure 4 FIG. 1 is a block diagram showing a configuration example of a drive system of an excavator. Figure 4 In the figure, the mechanical power system, the high-pressure hydraulic pipeline, the pilot pipeline and the electrical control system are respectively represented by double lines, thick solid lines, dashed lines and dotted lines.

[0117] The driving system of the excavator 100 mainly includes an engine 11, a regulator 13, a main pump 14, a pilot pump 15, a control valve 17, an operating device 26, a discharge pressure sensor 28, an operating pressure sensor 29, a controller 30, a proportional valve 31, a working mode rotary knob 32, etc.

[0118] The engine 11 is a driving source of the shovel. In the present embodiment, the engine 11 is, for example, a diesel engine that operates to maintain a predetermined rotation speed. The output shaft of the engine 11 is connected to the input shafts of the main pump 14 and the pilot pump 15 .

[0119] The main pump 14 supplies hydraulic oil to the control valve 17 via a high-pressure hydraulic line. In the present embodiment, the main pump 14 is a swash plate type variable displacement hydraulic pump.

[0120] The regulator 13 controls the discharge amount of the main pump 14. In the present embodiment, the regulator 13 controls the discharge amount of the main pump 14 by adjusting the tilt angle of the swash plate of the main pump 14 according to a control command from the controller 30.

[0121] The pilot pump 15 supplies hydraulic oil to various hydraulic control devices including the operating device 26 and the proportional valve 31 via a pilot line. In the present embodiment, the pilot pump 15 is a fixed displacement type hydraulic pump.

[0122] The control valve 17 is a hydraulic control device for controlling the hydraulic system in the shovel. The control valve 17 includes control valves 171 to 176 and a relief valve 177. The control valve 17 can selectively supply the hydraulic oil discharged from the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176.

[0123] The control valves 171 to 176 control the flow rate of the hydraulic oil from the main pump 14 to the hydraulic actuator and the flow rate of the hydraulic oil from the hydraulic actuator to the hydraulic oil tank. The hydraulic actuator includes a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left travel hydraulic motor 1A, a right travel hydraulic motor 1B, and a swing hydraulic motor 2A.

[0124] The relief valve 177 controls the flow rate of the hydraulic oil discharged from the main pump 14 that flows to the hydraulic oil tank without passing through the hydraulic actuator (hereinafter referred to as “the relief flow rate”). The relief valve 177 may be provided outside the control valve 17 .

[0125] The operating device 26 is a device for the operator to operate the hydraulic actuator. In the present embodiment, the operating device 26 supplies the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the control valves corresponding to the hydraulic actuators respectively via the pilot lines. The pressure (pilot pressure) of the hydraulic oil supplied to the pilot ports is a pressure corresponding to the operation direction and operation amount of the lever or pedal (not shown) of the operating device 26 corresponding to the hydraulic actuators respectively.

[0126] The discharge pressure sensor 28 detects the discharge pressure of the main pump 14. In the present embodiment, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0127] The operating pressure sensor 29 detects the operation content of the operator using the operating device 26. In the present embodiment, the operating pressure sensor 29 detects the operation direction and operation amount of the lever or pedal of the operating device 26 corresponding to the hydraulic actuator in the form of pressure (operating pressure), and outputs the detected value to the controller 30. The operation content of the operating device 26 may also be detected using other sensors besides the operating pressure sensor.

[0128] The controller 30 is a control unit that controls the entire shovel 100. The function of the controller 30 of this embodiment will be described in detail later.

[0129] The proportional valve 31 operates according to the control command output by the controller 30. In the present embodiment, the proportional valve 31 is a solenoid valve that adjusts the secondary pressure of the pilot port of the relief valve 177 introduced from the pilot pump 15 into the control valve 17 according to the current command output by the controller 30. The proportional valve 31 operates, for example, in the following manner: the larger the current command, the larger the secondary pressure introduced into the pilot port of the relief valve 177.

[0130] The operation mode rotary knob 32 is a control panel for the operator to select the operation mode, and can switch between a plurality of different operation modes. In addition, the operation mode rotary knob 32 always sends data indicating the setting state of the engine speed and the setting state of the acceleration / deceleration characteristics corresponding to the operation mode to the controller 30.

[0131] The operation mode rotation knob 32 can switch the operation mode between a plurality of stages including the SP mode, the H mode, the A mode, and the IDLE mode. That is, the operation mode rotation knob 32 of the present embodiment can switch the setting conditions of the shovel 100.

[0132] In addition, the SP mode is an example of the first mode, and the H mode is an example of the second mode. Figure 4 The state in which the SP mode is selected by the operation mode rotary knob 32 is shown.

[0133] SP mode is the operating mode selected when workload is given priority, using the highest engine speed and the highest acceleration and deceleration characteristics. H mode is the operating mode selected when workload and fuel efficiency are both taken into consideration, using the second highest engine speed and the second highest acceleration and deceleration characteristics.

[0134] The A mode is a working mode selected when the acceleration and deceleration characteristics of the hydraulic actuator corresponding to the lever operation are to be relaxed, accurate operability and safety are to be improved, and the excavator is to be operated with low noise, and the third highest engine speed and the third highest acceleration and deceleration characteristics are to be used. The IDLE mode is a working mode selected when the engine 11 is to be set to an idle state, and the lowest engine speed and the lowest acceleration and deceleration characteristics are to be used.

[0135] Here, when the action of each actuator stops during engine driving in each operating mode (high idle state), the controller 30 maintains the engine 11 at the speed set for each operating mode. If the high idle state continues for a predetermined time, the controller 30 can switch the engine speed to a low idle state. The idle state includes a high idle state and a low idle state.

[0136] In addition, in the above description, the names of the stages of the operating mode are set as SP mode, H mode, A mode, and IDLE mode, but the names of the stages are not limited to this. For example, the names of the SP mode, H mode, and A mode may be set as POWER mode, STD mode, ECO mode, and IDLE mode (low idle state), respectively. The operating mode is not limited to the present embodiment, and may be set to five or more stages.

[0137] The rotation speed of the engine 11 is constantly controlled to the engine speed of the operation mode set by the operation mode rotary knob 32. And the opening of the discharge valve 177 is controlled according to the discharge valve opening characteristic of the operation mode set by the operation mode rotary knob 32.

[0138] In the present embodiment, the above-mentioned working modes are sometimes expressed as setting conditions of the excavator 100, and the information indicating the setting conditions is expressed as setting condition information. Setting condition information refers to information in which a corresponding association is established between a designated item and the value of the item. The designated item refers to, for example, an item indicating the state of the engine speed corresponding to each working mode, and an item indicating the state of the acceleration / deceleration characteristics. Therefore, the setting condition information of the present embodiment includes an item indicating the state of the engine speed corresponding to each working mode and the value of the item, and an item indicating the state of the acceleration / deceleration characteristics and the value of the item.

[0139] exist Figure 4 In the structural diagram of , the ECO mode is set as one of the modes selected by the working mode rotary knob 32, but the ECO mode switch may be provided separately from the working mode rotary knob 32. In this case, the engine speed can be adjusted corresponding to each mode selected by the working mode rotary knob 32, and when the ECO mode switch is turned on, the acceleration and deceleration characteristics corresponding to each mode of the working mode rotary knob 32 are smoothly changed.

[0140] Furthermore, the operation mode can be changed by voice input. In this case, the shovel is provided with a voice input device for inputting the voice uttered by the operator into the controller 30. Furthermore, the controller 30 is provided with a voice recognition unit for recognizing the voice input by the voice input device.

[0141] In this way, the operation mode is selected by the mode selection unit such as the operation mode rotation knob 32, the ECO mode switch, and the voice recognition unit.

[0142] Next, the functions of the controller 30 of the present embodiment will be described. The controller 30 of the present embodiment includes an image data acquisition unit 301 , a state detection unit 302 , and an output unit 303 .

[0143] The image data acquisition unit 301 acquires image data captured by the camera device S6. More specifically, the image data acquisition unit 301 acquires image data (animation data) captured by the camera S6PL and the camera S6PR. In addition, when cameras are also arranged on the left and right pillars 112 on the rear side, the image data acquisition unit 301 also acquires image data captured by these cameras.

[0144] The state detection unit 302 detects the state of the operator based on the image data acquired by the image data acquisition unit 301. At this time, in this embodiment, image data suitable for state detection is selected from the image data respectively captured by the camera S6PL and the camera S6PR according to the state of the operator to be detected. Then, the state detection unit 302 detects the state of the operator based on the selected image data. The details of the processing of the state detection unit 302 will be described later.

[0145] Furthermore, the controller 30 stores information in which the image data acquired by the image data acquisition unit 301 and the detection result based on the state detection unit 302 are associated with each other in the storage device 47. In the following description, the image data (animation data) acquired by the image data acquisition unit 301 and the information in which the detection result based on the state detection unit 302 is associated with each other are sometimes described as state history information.

[0146] The output unit 303 outputs the state history information stored in the storage device 47 to the communication device T1. In other words, the output unit 303 outputs the state history information to the management device 200 via the communication device T1.

[0147] In addition, the controller 30 of the present embodiment may, for example, stop capturing image data using the cameras S6PL and S6PR when the state in which the operator does not sit in the driver's seat 90 continues for a certain period of time. The state in which the operator does not sit in the driver's seat 90 refers to a state in which the image represented by the image data acquired by the image data acquisition unit 301 does not include the image of the operator. The certain period of time may be, for example, about 1 hour.

[0148] Furthermore, the controller 30 of the present embodiment may turn off the engine 11 as the driving source when the state in which the operator does not sit on the driver's seat 90 continues for a certain period of time or longer.

[0149] Hereinafter, the state of the operator and the processing of the state detection unit 302 in this embodiment will be described.

[0150] The state of the operator in this embodiment includes the operator's posture when sitting on the driver's seat 90, whether the operator is wearing equipment (helmet, seat belt, etc.), expression, movement, etc.

[0151] Specifically, for example, the operator's state includes a state of sitting with legs crossed, a state of not wearing a helmet, etc. The crossed-legs posture may induce erroneous operation, and is therefore an inappropriate posture for operating the shovel 100. Furthermore, the state of not wearing a helmet means that the equipment for improving the safety of the operator during work is not worn, which is an undesirable state.

[0152] In addition, the operator's state includes, for example, a yawning state, a closed eye state, a continuous blinking state, etc. The yawning state is inferred as a state where the operator's concentration on work is reduced. In addition, a state where the eyes are closed for more than a certain period of time is inferred as a state where the operator feels sleepy. In addition, a continuous blinking state is inferred as a state where the operator feels fatigued.

[0153] Furthermore, the state of the operator includes, for example, a state in which the operator performs an action unrelated to work, etc. A state in which the operator performs an action unrelated to work is, for example, estimated as a state in which the operator is not performing work.

[0154] In this embodiment, the state detection unit 302 detects the above-mentioned multiple states as the state of the operator. In other words, the state detection unit 302 can detect posture, presence or absence of equipment, drowsiness, decreased concentration, fatigue level, work attitude, etc. as the state of the operator. In addition, the state of the operator detected by the state detection unit 302 can be predetermined.

[0155] Furthermore, when detecting a plurality of states of the operator, the state detection unit 302 of this embodiment selects image data corresponding to the state to be detected from the image data captured by the camera S6PL and the camera S6PR, and detects the state using the selected image data.

[0156] Specifically, for example, when detecting drowsiness, decreased concentration, fatigue, etc. as the state of the operator, the expression of the operator becomes important. Therefore, the state detection unit 302 selects image data (animation data) including a facial image from the image data captured by the camera S6PL and the camera S6PR, respectively. Then, the state detection unit 302 detects whether the operator feels drowsy, whether the concentration is decreased, the fatigue, etc. based on the selected image data, and stores the state history information including the detection result in the storage device 47.

[0157] Furthermore, when detecting the posture, the presence or absence of equipment, the working attitude, etc. as the state of the operator, the full-body image of the operator becomes important. Therefore, the state detection unit 302 selects image data (animation data) including the full body of the operator from the image data captured by the camera S6PL and the camera S6PR, respectively. Then, the state detection unit 302 detects whether the posture of the operator is appropriate, whether the equipment is worn appropriately, and whether the operator is working, based on the selected image data, and stores the state history information including the detection results in the storage device 47.

[0158] The state history information of this embodiment is sent to the management device 200 via the output unit 303 and managed by the management device 200. The state history information may include identification information for specifying an operator and a machine number for specifying the shovel 100, for example.

[0159] Therefore, in this embodiment, for example, when the manager wants to understand the state of a certain operator's concentration being reduced during work, he only needs to input the identification information that specifies the operator and the information specifying the state "concentration being reduced" to the management device 200. When the management device 200 receives the input, the management device 200 displays the image data that is associated with the detection result "concentration being reduced" in the state history information including the identification information of the operator.

[0160] The state detection unit 302 of the present embodiment may be, for example, a learned model generated by learning based on machine learning or the like.

[0161] Below, reference Figure 5 A state detection method by the state detection unit 302 according to the present embodiment will be described.

[0162] Figure 5 302 is a diagram for explaining an example of a state detection method based on a state detection unit. The state detection unit 302 of this embodiment may be a learned model constructed around a neural network (DNN). In other words, the controller 30 may have a learned model for realizing the state detection unit 302.

[0163] The neural network DNN is a so-called deep neural network that has one or more intermediate layers (hidden layers) between the input layer and the output layer. In the neural network DNN, a weight parameter indicating the connection strength with the lower layer is specified for each of the multiple neurons constituting each intermediate layer. Moreover, the neurons of each layer constitute the neural network DNN in the following manner: the sum of the values ​​obtained by multiplying the input values ​​of the multiple neurons from the upper layer by the weight parameters specified for each neuron in the upper layer is output to the neurons in the lower layer through a threshold function.

[0164] The neural network DNN is used as an object to perform machine learning, specifically deep learning, to achieve the optimization of the above-mentioned weight parameters. Thus, the neural network DNN can input the image data acquired by the image data acquisition unit 301 as an input signal x, and output the probability that the operator's state is a predetermined state as an output signal y.

[0165] In this embodiment, the output signal y1 outputted from the neural network DNN indicates that the predicted probability of the operator's state being in an inappropriate posture when the input signal x is input is 10%, and the output signal y2 indicates that the probability of the operator's state being in a state of decreased concentration when the input signal x is input is 50%.

[0166] The state detection unit 302 of this embodiment can, for example, obtain a group of multiple states included in the output signal y and the probability that the operator's state is each of the multiple states as a detection result of the operator's state, and accumulate it as part of the state history information.

[0167] In the present embodiment, by accumulating the status history information in this way, even after the operator's work is completed, it is possible for a manager or the like to grasp the details of the status history of the operator during the work.

[0168] Furthermore, the state detection unit 302 may determine the state with the highest probability among all the states included in the output signal y as the state (detection result) of the operator when the input signal x is input.

[0169] Specifically, for example, assuming that the state with the highest probability among the multiple states included in the output signal y is the state of decreased concentration, the state detection unit 302 can select the "state of decreased concentration" from the multiple states as the state of the operator at that moment and use it as the detection result.

[0170] The neural network DNN is, for example, a convolutional neural network (CNN). CNN is a neural network to which existing image processing techniques (convolution processing and pooling processing) are applied.

[0171] Next, refer to Figure 6 The operation of the shovel 100 according to the present embodiment will be described. Figure 6 This is a flowchart for explaining the processing of the shovel according to the first embodiment.

[0172] The controller 30 of the shovel 100 according to the present embodiment acquires image data from the camera S6PL and the camera S6PR via the image data acquisition unit 301 (step S601 ).

[0173] Next, the state detection unit 302 selects image data to be used for detection according to the type of state in the acquired plurality of image data, and detects the state of the operator using the selected image data (step S602 ).

[0174] Next, the controller 30 stores the state history information including the detection result and the image data for detecting the state in the storage device 47 via the output unit 303 (step S603 ).

[0175] In the shovel 100 of the present embodiment, image data suitable for detecting the state of the operator is selected from a plurality of image data (moving image data) captured by a plurality of imaging devices disposed in the cab 10, and the state of the operator is detected using the selected image data.

[0176] In other words, in the present embodiment, for example, when the state of an operator that a manager or the like wants to detect is set as the "purpose", image data suitable for detection of the purpose is automatically selected from a plurality of image data captured in the cockpit 10. Furthermore, in the present embodiment, the state of the operator as the purpose is detected using the selected image data.

[0177] As described above, in the present embodiment, the state of the operator can be detected according to the desire of the operator or the construction manager who manages the shovel 100 or the like.

[0178] Below, reference Fig. 7A , Figure 7B An example of image data capturing the state of the operator, which is stored in the storage device 47 as the state history information, will be described. Fig. 7A is a first diagram showing an example of output image data, Figure 7B The second figure shows an example of output image data. Fig. 7A The image 71 shown is an example of an image output as a result of detecting that the operator is in an inappropriate posture.

[0179] In the image 71, it can be seen that the operator is holding the lever. However, the operator is operating the lever while crossing his legs, and this posture is inappropriate as a posture during operation.

[0180] Figure 7B The image 72 shown shows a state where the operator is not working, that is, the operator is not holding the lever and the operator is not wearing equipment (safety helmet). In this case, it is an example of a screen output as a result of detecting that the operator is in an inappropriate posture in a state where the excavator 100 can be operated (the engine 11 is turned on and the door lock valve 19 is open).

[0181] In the image 72, it can be seen that the operator is not wearing a helmet and is crossing his legs, and is not operating the shovel 100. In particular, by combining with various detection information output from various sensors installed in the shovel 100, it is possible to confirm whether the operator's state is appropriate based on the state of the shovel 100 (whether it is in an operable state, etc.).

[0182] In the present embodiment, by displaying such an image as a detection result on the management device 200, the manager can understand the state of the operator when the state is detected.

[0183] (Second Embodiment)

[0184] The second embodiment is described below with reference to the accompanying drawings. The difference between the second embodiment and the first embodiment is that a three-dimensional model of the operator at work is created based on a plurality of image data acquired by a plurality of camera devices mounted on a column, and the state of the operator is detected based on the three-dimensional model. Therefore, in the following description of the second embodiment, the difference from the first embodiment is described.

[0185] Figure 8 This is a flowchart for explaining the operation of the shovel according to the second embodiment. The shovel 100 according to the present embodiment acquires image data (moving image data) captured by the cameras S6PL and S6PR through the image data acquisition unit 301 of the controller 30 (step S801 ).

[0186] Next, the controller 30 of this embodiment creates a three-dimensional model of the operator based on the multiple image data acquired by the image data acquisition unit 301 through the state detection unit 302, and detects the state of the operator based on the created three-dimensional model (step S802). The three-dimensional model of the operator refers to a stereoscopic model represented by a point group in a three-dimensional coordinate space.

[0187] Next, the controller 30 stores the state history information including the detection result and the image data for detecting the state in the storage device 47 via the output unit 303 (step S803 ).

[0188] Thus, in this embodiment, the three-dimensional model of the operator is created based on the multiple image data acquired by the image data acquisition unit 301, so there is no need to select image data suitable for the multiple states of the operator to be detected. In other words, in this embodiment, the three-dimensional model created based on the multiple image data of the operator photographed from different directions is used to detect the state of the operator, so the camera's shooting direction of the operator does not affect the detection of the state.

[0189] In the present embodiment, as the number of cameras disposed on the pillars in the cockpit 10 increases, a more accurate three-dimensional model can be created, and the accuracy of detecting the operator's state can be improved.

[0190] (Third Embodiment)

[0191] The third embodiment is described below. The third embodiment is different from the first and second embodiments in that the management device 200 is provided with a state determination function of the shovel 100 .

[0192] Fig. 9 This is a diagram for explaining the system configuration of a support system for a construction machine according to a third embodiment.

[0193] In the shovel support system SYS of the present embodiment, the management device 200 includes an image data acquisition unit 301 , a state detection unit 302 , and an output unit 303A.

[0194] In addition, the management device 200 can be a conventional computer including a processing unit and a storage device, and the functions of the image data acquisition unit 301, the status detection unit 302, and the output unit 303A can be realized by the processing unit reading and executing the status detection program stored in the storage device of the management device 200.

[0195] The shovel 100 according to the present embodiment transmits the image data captured by the camera S6PL and the camera S6PR to the management device 200 .

[0196] The management device 200 acquires a plurality of image data transmitted from the shovel 100 by the plurality of image data acquisition unit 301, detects the state of the operator by using the plurality of image data by the state detection unit 302, and stores state history information including the image data and the detection result.

[0197] The output unit 303A displays the status history information on a display device included in the management device 200 , a display connected to the management device 200 , or the like.

[0198] Specifically, for example, when information specifying a state that the manager wants to detect is input to the management device 200 together with the identification information of the operator and the body number of the shovel 100, the output unit 303A reads the state history information corresponding to the identification information of the operator and the body number of the shovel 100. Then, the management device 200 can display the specified detection result among the detection results included in the read state history information and the image data corresponding to the specified detection result.

[0199] Furthermore, when only the identification information of the operator and the body number of the shovel 100 are input, the output unit 303A can read the state history information corresponding to the identification information of the operator and the body number of the shovel 100, and display a list of detected states. Then, when a state is selected from the list of detected states, the output unit 303A can display image data associated with the selected state. Furthermore, for example, when the identification information of the operator is input, the output unit 303A can refer to the state history information associated with the identification information to display the number of times an inappropriate state is detected.

[0200] In the present embodiment, by outputting the detection result of the operator's state in this way, it is possible for the manager to understand the operator's state.

[0201] Furthermore, in the present embodiment, since the management device 200 detects the states of a plurality of image data, it is possible to reduce the processing load of the controller 30 of the shovel 100 .

[0202] In addition, in this embodiment, the cab 10 as the cab is set as the cab 10 of the excavator 100, but it is not limited to this. This embodiment can also be applied to working machines other than the excavator 100. For example, the working machine may include a gantry crane, a crawler crane, a traveling crane, a bridge crane, a jib crane, etc.

[0203] The gantry crane comprises: a pair of legs arranged in the left-right direction and capable of moving in the front-rear direction; a main beam erected between the legs; a main trolley capable of moving laterally along the main beam; a cab for operating the crane; and a passage portion connected to the main trolley and connecting the main trolley and the cab. This embodiment can be applied to the driver's seat of the gantry crane.

[0204] In a crawler crane, an upper slewing body is rotatably mounted on a lower traveling body having crawlers, and a tower is raisably mounted on the upper slewing body. In addition, a driver's seat is provided on the upper slewing body. In a crawler crane, a non-telescopic boom is pivotally supported on the upper end side of the tower so as to be able to rise and fall relative to the tower, and a tower strut that becomes an auxiliary component when the boom is raisably supported is pivotally supported on the other part of the upper end side of the tower, and the tower strut is clamped between the boom and the boom guy wire. This embodiment can be applied to the driver's seat of a crawler crane.

[0205] A traveling crane, for example, includes: a traveling part that travels on a guide rail; a fixed part that is disposed on the traveling part; a slewing part that is rotatably disposed on the fixed part; a cab that is disposed on the slewing part; and a cantilever that is disposed on the slewing part. In addition, the traveling crane includes: a sling that is used to sling cargo; a wire rope that is used to hoist and lower the sling; and a drum that is used to wind up and send out the wire rope. The traveling crane device can move cargo by traveling on the guide rail while the cargo is hoisted by the sling by driving the traveling part. This embodiment can be applied to the driver's seat of a traveling crane.

[0206] A bridge crane is installed in a building, and uses a hoist to lift an object (hereinafter referred to as "object") and moves the hoist in a horizontal direction, thereby transporting the object. The bridge crane has a main beam, which is installed in a manner of being erected between a pair of guide rails installed in the building, and the main beam can travel on the guide rails. In addition, a trolley that can travel laterally along the extension direction of the main beam is provided on the main beam. The hoist is installed on the trolley. That is, the hoist moves in the travel direction by the movement of the main beam, and the hoist moves in the lateral travel direction by the lateral movement of the trolley. In this way, the object is transported by the movement of the hoist. In addition, a cab for operating the bridge crane is provided on the main beam. This embodiment can be applied to the driver's seat of a bridge crane.

[0207] In a jib crane, a jib is installed at the end of a slewing body with a back stay erected so as to rise and fall freely around a rise and fall central axis. The jib rises and falls by winding and releasing a rising and falling rope, and the suspended object is raised and lowered from the top of the jib by hoisting and lowering a hoisting rope. This embodiment can be applied to the driver's seat of a jib crane.

[0208] As mentioned above, although the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the above embodiment, and various modifications and substitutions can be added to the above embodiment without departing from the scope of the present invention.

[0209] Furthermore, the present international application claims the priority based on Japanese Patent Application No. 2022-084645 filed on May 24, 2022, and the entire contents of Japanese Patent Application No. 2022-084645 are incorporated herein by reference.

[0210] Explanation of symbols

[0211] 10: cockpit, 11: engine, 30: controller, 100: excavator, 110: frame body, 111, 112: columns, 200: management device, 301: image data acquisition unit, 302: state detection unit, 303: output unit.

Claims

1. A construction machine, comprising: Lower walking body; An upper rotating body, mounted on the lower walking body so as to be rotatable; a cockpit mounted on the upper rotating body; and The space recognition device is arranged on a column in the cockpit.

2. The construction machine according to claim 1, wherein: The space recognition device is arranged between the height of a seat surface of a driver's seat provided in the cockpit and the height of an upper end portion of a headrest provided in the driver's seat.

3. The construction machine according to claim 1 or 2, wherein: The space recognition device is arranged at a position that does not interfere when a front window provided in the cockpit is opened or closed.

4. The construction machine according to claim 3, wherein: The space recognition device is movable.

5. The construction machine according to claim 4, wherein: The space recognition device is configured with multiple, The column has a mounting component for mounting a plurality of the space recognition devices. The plurality of space recognition devices include a fixing component that is detachable from the mounting component and is used to fix the column to each space recognition device.

6. The construction machine according to claim 5, wherein: The fixing component is a magnet.

7. The construction machine according to claim 1, comprising: An image data acquisition unit that acquires a plurality of image data acquired by the plurality of space recognition devices respectively; and The state detection unit detects the state of an operator sitting on a driver's seat provided in the cockpit using the plurality of image data.

8. The construction machine according to claim 7, wherein: The state detection unit selects image data corresponding to the state of the operator to be detected from among the plurality of image data, and detects the state of the operator based on the selected image data.

9. The construction machine according to claim 7 or 8, comprising: The output unit displays the image data when the state of the operator is detected by the state detection unit on the display device.

10. A construction machinery support system, comprising a construction machinery and a management device, wherein: The construction machine has: Lower walking body; An upper rotating body, mounted on the lower walking body so as to be rotatable; A cockpit, mounted on the upper rotating body; A space recognition device, arranged on a column in the cockpit; and an output unit, outputting the plurality of image data photographed by the space recognition device to the management device, The management device includes a storage unit that stores the plurality of image data.

Images