Control system of working machine, working machine, and control method of working machine

Abstract

To provide a control system of a working machine capable of calculating the position and azimuth of the working machine in the event of GNSS positioning failure.SOLUTION: A control system of a working machine includes: a first position and azimuth calculation unit that calculates the position and the azimuth of the working machine on the basis of GNSS radio waves; a second position and azimuth calculation unit that calculates the position and the azimuth of the working machine on the basis of multiple target images placed outside of the working machine; and a switching unit that switches between a first calculation mode to calculate the position and the azimuth of the working machine by a first position and azimuth calculation unit and a second calculation mode to calculate the position and the azimuth of the working machine by a second position and azimuth calculation unit.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a control system for a work machine, a work machine, and a control method for a work machine.

Background Art

[0002] In the technical field related to work machines, a technique of excavating an excavation target based on a target construction surface, as disclosed in Patent Document 1, is known. As a technique of excavating an excavation target based on a target construction surface, a machine guidance technique of presenting a guidance image showing the relative position between the target construction surface and the work machine to an operator of the work machine, and a machine control technique of assisting and controlling the operator's operation so that the work machine operates according to the target construction surface are known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When excavating an excavation target based on a target construction surface, it is necessary to calculate the position and azimuth angle of the work machine. The position and azimuth angle of the work machine are calculated using the Global Navigation Satellite System (GNSS). When a positioning failure of GNSS occurs, it becomes difficult to calculate the position and azimuth angle of the work machine.

[0005] An object of the present disclosure is to calculate the position and azimuth angle of a work machine when a positioning failure of GNSS occurs.

Means for Solving the Problems

[0006] According to the present disclosure, there is provided a control system for a working machine, comprising: a first position and azimuth calculation unit that calculates the position and azimuth of the working machine based on GNSS radio waves; a second position and azimuth calculation unit that calculates the position and azimuth of the working machine based on images of a plurality of targets installed outside the working machine; a switching unit that switches between a first calculation mode in which the first position and azimuth calculation unit calculates the position and azimuth of the working machine and a second calculation mode in which the second position and azimuth calculation unit calculates the position and azimuth of the working machine.

Advantages of the Invention

[0007] According to the present disclosure, the position and azimuth of the working machine are calculated when poor GNSS positioning occurs.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. Also, some components may not be used.

[0010] [Work Machine] FIG. 1 is a perspective view showing a work machine 1 according to an embodiment. FIG. 2 is a schematic diagram showing the work machine 1 according to the embodiment. FIG. 3 is a view showing the cab 2 of the work machine 1 according to the embodiment.

[0011] The work machine 1 operates at a work site. In the embodiment, the work machine 1 is a hydraulic excavator. In the following description, the work machine 1 is appropriately referred to as the hydraulic excavator 1.

[0012] The hydraulic excavator 1 includes a traveling body 3, a revolving body 4, a work implement 5, a hydraulic cylinder 6, an operation device 7, an in-vehicle monitor 8, a position sensor 9, an inclination sensor 10, an imaging device 11, and a control device 12.

[0013] As shown in FIG. 2, a three-dimensional site coordinate system (Xg, Yg, Zg) is defined at the work site. A three-dimensional vehicle body coordinate system (Xm, Ym, Zm) is defined for the revolving body 4. A three-dimensional camera coordinate system (Xc, Yc, Zc) is defined for the imaging device 11.

[0014] The on-site coordinate system is composed of an Xg axis extending north-south from the on-site reference point Og defined at the work site, a Yg axis extending east-west from the on-site reference point Og, and a Zg axis extending vertically from the on-site reference point Og.

[0015] The vehicle body coordinate system is composed of an Xm axis extending in the front-rear direction of the slewing body 4 from the representative point Om defined on the slewing body 4, a Ym axis extending in the left-right direction of the slewing body 4 from the representative point Om, and a Zm axis extending in the up-down direction of the slewing body 4 from the representative point Om. With the representative point Om of the slewing body 4 as a reference, the +Xm direction is the front of the slewing body 4, the -Xm direction is the rear of the slewing body 4, the +Ym direction is the left of the slewing body 4, the -Ym direction is the right of the slewing body 4, the +Zm direction is the upper part of the slewing body 4, and the -Zm direction is the lower part of the slewing body 4.

[0016] The camera coordinate system is composed of an Xc axis extending in the width direction of the camera 13 from the optical center Oc of one camera 13 constituting the imaging device 11, a Yc axis extending in the up-down direction of the camera 13 from the optical center Oc, and a Zc axis extending in a direction parallel to the optical axis of the optical system of the camera 13 from the optical center Oc.

[0017] The traveling body 3 travels while supporting the slewing body 4. The traveling body 3 has a pair of crawlers 3A. By the rotation of the crawlers 3A, the traveling body 3 performs a traveling operation. The traveling operation of the traveling body 3 includes a forward operation and a backward operation. The hydraulic excavator 1 can be moved to the work site by the traveling body 3.

[0018] The slewing body 4 is supported by the traveling body 3. The slewing body 4 is disposed above the traveling body 3. The slewing body 4 performs a slewing operation about the slewing axis RX while being supported by the traveling body 3. The slewing axis RX is parallel to the Zm axis. The slewing operation of the slewing body 4 includes a left slewing operation and a right slewing operation. The operator's cab 2 is provided on the slewing body 4.

[0019] The working machine 5 is supported by the slewing body 4. The working machine 5 performs work. In the embodiment, the work performed by the working machine 5 includes an excavation work of excavating an excavation target and a loading work of loading the excavated material onto a loading target.

[0020] The working machine 5 includes a boom 5A, an arm 5B, and a bucket 5C. The base end portion of the boom 5A is rotatably connected to the front portion of the revolving body 4. The base end portion of the arm 5B is rotatably connected to the tip end portion of the boom 5A. The base end portion of the bucket 5C is rotatably connected to the tip end portion of the arm 5B.

[0021] The hydraulic cylinder 6 operates the working machine 5. The hydraulic cylinder 6 includes a boom cylinder 6A, an arm cylinder 6B, and a bucket cylinder 6C. The boom cylinder 6A raises and lowers the boom 5A. The arm cylinder 6B performs a digging operation and a dumping operation on the arm 5B. The bucket cylinder 6C performs a digging operation and a dumping operation on the bucket 5C. The base end portion of the boom cylinder 6A is connected to the revolving body 4. The tip end portion of the boom cylinder 6A is connected to the boom 5A. The base end portion of the arm cylinder 6B is connected to the boom 5A. The tip end portion of the arm cylinder 6B is connected to the arm 5B. The base end portion of the bucket cylinder 6C is connected to the arm 5B. The tip end portion of the bucket cylinder 6C is connected to the bucket 5C.

[0022] As shown in FIG. 3, the operating device 7 is disposed in the cab 2. The operating device 7 is operated to operate at least one of the traveling body 3, the revolving body 4, and the working machine 5. The operating device 7 is operated by an operator who has boarded the cab 2. The operator can operate the operating device 7 while seated on the driver's seat 14 disposed in the cab 2.

[0023] The operating device 7 includes a left working lever 7A and a right working lever 7B that are operated for the operations of the revolving body 4 and the working machine 5, a left traveling lever 7C and a right traveling lever 7D that are operated for the operation of the traveling body 3, and a left foot pedal 7E and a right foot pedal 7F.

[0024] When the left operation lever 7A is operated in the front-rear direction, the arm 5B performs a dumping operation or an excavation operation. When the left operation lever 7A is operated in the left-right direction, the revolving body 4 performs a left revolving operation or a right revolving operation. When the right operation lever 7B is operated in the left-right direction, the bucket 5C performs an excavation operation or a dumping operation. When the right operation lever 7B is operated in the front-rear direction, the boom 5A performs a lowering operation or a raising operation. Note that when the left operation lever 7A is operated in the front-rear direction, the revolving body 4 may perform a right revolving operation or a left revolving operation, and when the left operation lever 7A is operated in the left-right direction, the arm 5B may perform a dumping operation or an excavation operation.

[0025] When the left travel lever 7C is operated in the front-rear direction, the left crawler 3A of the traveling body 3 performs a forward movement or a backward movement. When the right travel lever 7D is operated in the front-rear direction, the right crawler 3A of the traveling body 3 performs a forward movement or a backward movement.

[0026] The left foot pedal 7E is interlocked with the left travel lever 7C. The right foot pedal 7F is interlocked with the right travel lever 7D. The traveling body 3 may be moved forward or backward by operating the left foot pedal 7E and the right foot pedal 7F.

[0027] The in-vehicle monitor 8 is disposed in the driver's cab 2. The in-vehicle monitor 8 is disposed at the right front of the driver's seat 14. The in-vehicle monitor 8 has a display device 8A and an input device 8B.

[0028] The display device 8A displays prescribed display data. Examples of the display device 8A include flat panel displays such as a liquid crystal display (LCD) or an organic EL display (OELD).

[0029] The input device 8B generates input data when operated by an operator. Examples of the input device 8B include a button switch, a computer keyboard, and a touch panel.

[0030] The position sensor 9 detects the position in the on-site coordinate system. The position sensor 9 detects the position in the on-site coordinate system by using the Global Navigation Satellite System (GNSS). The Global Navigation Satellite System includes the Global Positioning System (GPS). The Global Navigation Satellite System detects the position defined by the coordinate data of latitude, longitude, and altitude. The position sensor 9 includes a GNSS receiver that receives GNSS radio waves from GNSS satellites. The position sensor 9 is disposed on the rotating body 4. In an embodiment, the position sensor 9 is disposed on the counterweight of the rotating body 4.

[0031] The position sensor 9 includes a first position sensor 9A and a second position sensor 9B. The first position sensor 9A and the second position sensor 9B are disposed at different positions on the rotating body 4. In an embodiment, the first position sensor 9A and the second position sensor 9B are disposed at intervals in the left-right direction on the rotating body 4. The first position sensor 9A detects a first positioning position indicating the position where the first position sensor 9A is disposed. The second position sensor 9B detects a second positioning position indicating the position where the second position sensor 9B is disposed.

[0032] The tilt sensor 10 detects the acceleration and angular velocity of the rotating body 4. The tilt sensor 10 includes an Inertial Measurement Unit (IMU). The tilt sensor 10 is disposed on the rotating body 4. In an embodiment, the tilt sensor 10 is installed below the driver's cab 2.

[0033] The imaging device 11 images the front of the rotating body 4. The imaging device 11 is disposed on the rotating body 4. In an embodiment, the imaging device 11 is disposed on the upper part of the driver's cab 2. The imaging device 11 includes a plurality of cameras 13. The camera 13 includes an optical system and an image sensor that receives light through the optical system. As the image sensor, a Charge Coupled Device (CCD) sensor or a Complementary Metal Oxide Semiconductor (CMOS) sensor is exemplified.

[0034] In the embodiment, four cameras 13 are provided. The cameras 13 include a camera 13A, a camera 13B, a camera 13C, and a camera 13D. A stereo camera 15 is constituted by a set of cameras 13. In the embodiment, a first stereo camera 15A is constituted by a set of cameras 13A and 13C. A second stereo camera 15B is constituted by a set of cameras 13B and 13D.

[0035] The camera 13A and the camera 13C of the stereo camera 15A are arranged at intervals in the left - right direction of the swivel body 4. The camera 13B and the camera 13D of the stereo camera 15B are arranged at intervals in the left - right direction of the swivel body 4. The optical axes of the optical systems of the cameras 13A and 13C are substantially parallel to the Xg axis. The optical axes of the optical systems of the cameras 13B and 13D are inclined downward toward the front of the swivel body 4.

[0036] [Control System] FIG. 4 is a block diagram showing a control system 30 of the working machine 1 according to the embodiment. The hydraulic excavator 1 includes a control system 30. The control system 30 has an in - vehicle monitor 8, a position sensor 9, an inclination sensor 10, an imaging device 11, and a control device 12. The control device 12 controls the hydraulic excavator 1. The control device 12 includes a computer system.

[0037] The control device 12 has a storage unit 16, a first position and orientation calculation unit 17, a second position and orientation calculation unit 18, an inclination angle calculation unit 19, a switching unit 20, a three - dimensional data calculation unit 21, a display control unit 22, and a correction unit 23.

[0038] The storage unit 16 stores prescribed storage data. The storage unit 16 stores target data related to a target 24 described later. A plurality of targets 24 are installed outside the hydraulic excavator 1. The target data includes the three - dimensional positions of the respective plurality of targets 24. The target data includes correlation data indicating the relationship between the identification data defined by the identification mark 27 of the target 24 and the three - dimensional position of the target 24.

[0039] The first position and orientation calculation unit 17 calculates the position and orientation angle of the slewing body 4 in the field coordinate system based on the detection data of the position sensor 9. As described above, the position sensor 9 includes a GNSS receiver that receives GNSS radio waves. The first position and orientation calculation unit 17 calculates the position and orientation angle of the slewing body 4 based on the GNSS radio waves. The orientation angle of the slewing body 4 is, for example, the orientation angle of the slewing body 4 with respect to the Xg axis.

[0040] The first position and orientation calculation unit 17 calculates the position of the slewing body 4 based on at least one of the first positioning position detected by the first position sensor 9A and the second positioning position detected by the second position sensor 9B. The first position and orientation calculation unit 17 calculates the orientation angle of the slewing body 4 based on the relative position between the first positioning position detected by the first position sensor 9A and the second positioning position detected by the second position sensor 9B.

[0041] The second position and orientation calculation unit 18 calculates the position and orientation angle of the slewing body 4 in the field coordinate system based on the image acquired by the imaging device 11. As described above, a plurality of targets 24 are installed outside the hydraulic excavator 1. The imaging device 11 images the targets 24. The second position and orientation calculation unit 18 acquires images of the plurality of targets 24 from the imaging device 11. The second position and orientation calculation unit 18 calculates the position and orientation angle of the slewing body 4 based on the images of the plurality of targets 24 installed outside the hydraulic excavator 1.

[0042] The tilt angle calculation unit 19 calculates the tilt angle of the slewing body 4 based on the detection data of the tilt sensor 10. The tilt angle of the slewing body 4 includes the roll angle and pitch angle of the slewing body 4. The roll angle refers to the tilt angle of the slewing body 4 in the tilt direction centered on the Xg axis. The pitch angle refers to the tilt angle of the slewing body 4 in the tilt direction centered on the Yg axis. The tilt angle calculation unit 19 calculates the roll angle and pitch angle of the slewing body 4 based on the detection data of the tilt sensor 10.

[0043] The switching unit 20 switches between a first calculation mode in which the first position and orientation calculation unit 17 calculates the position and orientation angle of the revolving body 4 and a second calculation mode in which the second position and orientation calculation unit 18 calculates the position and orientation angle of the revolving body 4.

[0044] The three-dimensional data calculation unit 21 calculates the distance between the stereo camera 15 and the imaging target based on a set of images captured by the stereo camera 15. An example of the imaging target is an excavation target excavated by the working machine 5. The three-dimensional data calculation unit 21 calculates the three-dimensional data of the imaging target by performing stereo processing on the images of the same imaging target captured by a set of cameras 13 of the stereo camera 15. The three-dimensional data calculation unit 21 calculates the three-dimensional data in the camera coordinate system.

[0045] The display control unit 22 controls the display device 8A of the in-vehicle monitor 8. The display control unit 22 causes the display device 8A to display prescribed display data.

[0046] The correction unit 23 corrects the error of the tilt sensor 10.

[0047] [Calculation mode] FIG. 5 is a schematic diagram for explaining the calculation mode of the position and orientation angle of the revolving body 4 according to the embodiment. In the embodiment, the position and orientation angle of the revolving body 4 are calculated by at least one of the first calculation mode and the second calculation mode. The position of the revolving body 4 includes the position of the representative point Om of the revolving body 4 in the site coordinate system. The orientation angle of the revolving body 4 includes the orientation angle of the vehicle body coordinate system with respect to the representative point Om of the revolving body 4 in the site coordinate system.

[0048] The first calculation mode is a calculation mode for calculating the position and orientation angle of the revolving body 4 based on GNSS radio waves. In the first calculation mode, the first position and orientation calculation unit 17 calculates the position and orientation angle of the revolving body 4 based on the detection data of the position sensor 9.

[0049] The second calculation mode is a calculation mode for calculating the position and azimuth angle of the rotating body 4 based on images of a plurality of targets 24. In the second calculation mode, the second position and azimuth calculation unit 18 calculates the position and azimuth angle of the rotating body 4 based on the images of the targets 24 captured by the imaging device 11.

[0050] When GNSS positioning failure occurs, it may be difficult for the first position and azimuth calculation unit 17 to calculate the position and azimuth angle of the rotating body 4. GNSS positioning failure includes a decrease in GNSS positioning accuracy and inability to perform positioning. Examples of GNSS positioning failure include insufficient intensity of GNSS radio waves received by the position sensor 9 or multi-path of GNSS radio waves. The multi-path of GNSS radio waves refers to a phenomenon in which GNSS radio waves transmitted from GNSS satellites are reflected by the ground, buildings, etc., or reflected or refracted in the ionosphere, and the position sensor 9 receives GNSS radio waves from a plurality of transmission paths, resulting in an error in the detected position.

[0051] When GNSS positioning failure does not occur, the position and azimuth angle of the rotating body 4 are calculated in the first calculation mode. When GNSS positioning failure occurs, the position and azimuth angle of the rotating body 4 are calculated in the second calculation mode.

[0052] The switching unit 20 switches between a first calculation mode and a second calculation mode based on the reception status of GNSS signals. The first position and orientation calculation unit 17 can determine whether the reception status of GNSS signals is good or not. The first position and orientation calculation unit 17 can, for example, determine the intensity of GNSS signals. The switching unit 20 switches between the first calculation mode and the second calculation mode based on the reception status of GNSS signals by the position sensor 9. Further, the switching unit 20 switches between the first calculation mode and the second calculation mode based on whether the first position and orientation calculation unit 17 can calculate the position and azimuth angle of the rotating body 4. For example, when the intensity of GNSS signals is insufficient and the reception status of GNSS signals is poor, the first position and orientation calculation unit 17 is likely to be in a state where it cannot calculate the position and azimuth angle of the rotating body 4. On the other hand, when the intensity of GNSS signals is sufficient and the reception status of GNSS signals is good, the first position and orientation calculation unit 17 is likely to be in a state where it can calculate the position and azimuth angle of the rotating body 4.

[0053] When the reception status of GNSS signals changes from good to bad, the switching unit 20 switches from the first calculation mode to the second calculation mode. Further, when the first position and orientation calculation unit 17 changes from a state where it can calculate the position and azimuth angle of the rotating body 4 to a state where it cannot calculate, the switching unit 20 switches from the first calculation mode to the second calculation mode.

[0054] When the reception status of GNSS signals changes from bad to good, the switching unit 20 switches from the second calculation mode to the first calculation mode. Further, when the first position and orientation calculation unit 17 changes from a state where it cannot calculate the position and azimuth angle of the rotating body 4 to a state where it can calculate, the switching unit 20 switches from the second calculation mode to the first calculation mode.

[0055] In an embodiment, the display control unit 22 causes the display device 8A to display the reception status of GNSS signals. As shown in FIG. 5, when the reception status of GNSS signals changes from a good state to a bad state, the display control unit 22 may cause the display device 8A to display that the reception status of GNSS signals is bad. Based on the display data displayed on the display device 8A, an operator can recognize that the reception status of GNSS signals is bad. In the embodiment, the switching from the first calculation mode to the second calculation mode may be performed based on an operation of the input device 8B by the operator. An operator who recognizes that the reception status of GNSS signals is bad operates the input device 8B to generate input data for performing the switching from the first calculation mode to the second calculation mode. The switching unit 20 switches from the first calculation mode to the second calculation mode based on the input data from the input device 8B.

[0056] When switched from the first calculation mode to the second calculation mode, the display control unit 22 may cause the display device 8A to display that the switching from the first calculation mode to the second calculation mode has been performed. Thereby, the operator can recognize that the switching from the first calculation mode to the second calculation mode has been performed.

[0057] On the other hand, when the reception status of GNSS signals changes from a bad state to a good state, the display control unit 22 causes the display device 8A to display that the reception status of GNSS signals is good. Based on the display data displayed on the display device 8A, an operator can recognize that the reception status of GNSS signals is good. The switching from the second calculation mode to the first calculation mode may be performed based on an operation of the input device 8B by the operator. An operator who recognizes that the reception status of GNSS signals is good operates the input device 8B to generate input data for performing the switching from the second calculation mode to the first calculation mode. The switching unit 20 switches from the second calculation mode to the first calculation mode based on the input data from the input device 8B.

[0058] When switched from the second calculation mode to the first calculation mode, the display control unit 22 may cause the display device 8A to display that the mode has been switched from the second calculation mode to the first calculation mode. Thereby, the operator can recognize that the mode has been switched from the second calculation mode to the first calculation mode.

[0059] [Target] FIG. 6 is a view showing a target 24 installed at a work site according to the embodiment. As shown in FIG. 6, the target 24 is disposed outside the hydraulic excavator 1 at the work site. A plurality of targets 24 are arranged around the hydraulic excavator 1 at the work site. The target 24 includes a mark drawn on a display board 25. In the embodiment, a grounding plate 26 is fixed to the lower end portion of the display board 25. The display board 25 is placed on the ground of the work site via the grounding plate 26. Note that the display board 25 may be fixed to the work site. The target 24 may be attached to, for example, a structure at the work site. The target 24 may be erected at the work site using a member such as a pile.

[0060] FIG. 7 is a view showing the target 24 according to the embodiment. The target 24 includes an identification mark 27 and radiation marks 28 arranged around the identification mark 27. The identification mark 27 includes identification data for identifying the target 24. In the embodiment, the identification mark 27 includes a two-dimensional barcode for identifying the target 24. A reference point Ot is defined for the target 24. The radiation marks 28 extend in the radial direction from the reference point Ot of the target 24. The radiation marks 28 have a plurality of lines 28A extending in the radial direction from the reference point Ot of the target 24. The lines 28A include the edges of the radiation marks 28. The reference point Ot of the target 24 is defined at the intersection of the plurality of lines 28A.

[0061] After the target 24 is installed at the work site, the position of the target 24 is measured by a surveying instrument. The surveying instrument measures the three-dimensional position of the target 24 in the field coordinate system. The three-dimensional position of the target 24 includes the three-dimensional position of the reference point Ot. The surveying instrument measures the three-dimensional position of the reference point Ot. The three-dimensional position of each of the plurality of targets 24 measured by the surveying instrument is stored in the storage unit 16. The storage unit 16 stores correlation data indicating the relationship between the identification data of the target 24 defined by the identification mark 27 and the three-dimensional position of the target 24 measured by the surveying instrument. When the target 24 is identified based on the identification mark 27, the three-dimensional position of the identified target 24 is identified.

[0062] [Second calculation mode] Next, a method for calculating the position and azimuth angle of the rotating body 4 in the second calculation mode will be described. FIG. 8 is a flowchart showing a method for calculating the position and azimuth angle of the rotating body 4 according to the embodiment. FIG. 9 is a schematic diagram for explaining a method for calculating the position and azimuth angle of the rotating body 4 according to the embodiment.

[0063] When the position and azimuth angle of the rotating body 4 cannot be calculated in the first calculation mode, the position and azimuth angle of the rotating body 4 are calculated in the second calculation mode. In the embodiment, the second position and azimuth calculation unit 18 calculates the position and azimuth angle of the rotating body 4 based on the images of the plurality of targets 24 and the tilt angle of the rotating body 4. The second position and azimuth calculation unit 18 acquires images of the plurality of targets 24 from the imaging device 11. The second position and azimuth calculation unit 18 acquires the tilt angle of the rotating body 4 from the tilt angle calculation unit 19. As described above, the tilt angle of the rotating body 4 includes the roll angle and pitch angle of the rotating body 4.

[0064] The plurality of targets 24 are imaged by the imaging device 11. The imaging device 11 simultaneously images the plurality of targets 24. As shown in FIG. 9, three targets 24 are simultaneously imaged by the imaging device 11. The second position and azimuth calculation unit 18 acquires the images 29 of the three targets 24 imaged by the imaging device 11 (step SA1).

[0065] As shown in FIG. 9, three targets 24 are arranged in one image 29.

[0066] The second position and orientation calculation unit 18 identifies the target 24 based on the identification data defined by the identification mark 27 of the target 24 (step SA2).

[0067] The second position and orientation calculation unit 18 specifies the target 24 based on the identification mark 27 in the image 29. The second position and orientation calculation unit 18 acquires the three-dimensional position of the target 24 from the storage unit 16 based on the identification mark 27 in the image 29 and the correlation data stored in the storage unit 16 (step SA3).

[0068] As described above, the three-dimensional position of the target 24 has been measured in advance by a surveying instrument and stored in the storage unit 16. Further, the storage unit 16 stores in advance correlation data indicating the relationship between the identification data defined by the identification mark 27 of the target 24 and the three-dimensional position of the target 24. Therefore, the second position and orientation calculation unit 18 can acquire the three-dimensional position of the target 24 based on the identification mark 27 in the image 29 and the correlation data stored in the storage unit 16.

[0069] The second position and orientation calculation unit 18 acquires the two-dimensional position of the target 24 in the image 29 (step SA4).

[0070] The two-dimensional position of the target 24 in the image 29 includes the two-dimensional position of the reference point Ot defined for the target 24. As described above, the target 24 has a radiation mark 28 including a line 28A. The second position and orientation calculation unit 18 calculates the two-dimensional position of the reference point Ot in the image 29 based on the radiation mark 28 in the image 29. The second position and orientation calculation unit 18 can calculate the two-dimensional position of the reference point Ot in the image 29 with high accuracy based on the radiation mark 28 by performing image processing on the image 29 of the target 24. In the following description, the reference point Ot in the image 29 is appropriately referred to as the reference point Oti.

[0071] The tilt angle calculation unit 19 acquires the detection data of the tilt sensor 10 when the target 24 is being imaged, and calculates the pitch angle and roll angle of the revolving body 4 when the target 24 is being imaged. The second position and orientation calculation unit 18 acquires the roll angle and pitch angle of the revolving body 4 when the target 24 is being imaged from the tilt angle calculation unit 19 (step SA5).

[0072] Based on the three-dimensional positions of the three targets 24 acquired in step SA3, the two-dimensional position of the target 24 in the image 29 acquired in step SA4, and the roll angle and pitch angle of the revolving body 4 acquired in step SA5, the second position and orientation calculation unit 18 calculates the position and azimuth angle of the camera 13 in the field coordinate system (step SA6).

[0073] In the embodiment, the second position and orientation calculation unit 18 calculates the position and azimuth angle of the camera 13 in the field coordinate system based on the bundle method, which is a kind of block adjustment method in aerial triangulation. Aerial triangulation refers to a method of calculating the imaging position and imaging direction of each of a plurality of images 29 captured by a plurality of cameras 13 based on the coordinates of known reference points Ot, utilizing the collinearity condition indicating the rectilinear propagation of light and the geometric properties of aerial photographs.

[0074] To calculate the position and azimuth angle of the camera 13 based on the bundle method, the second position and orientation calculation unit 18 acquires the three-dimensional positions of the three reference points Ot, the two-dimensional positions of the reference points Oti in the image 29, and the roll angle and pitch angle of the revolving body 4. The three-dimensional positions of the reference points Ot are three-dimensional positions in the field coordinate system. The two-dimensional positions of the reference points Oti are two-dimensional positions in the image coordinate system defined in the image 29. The image coordinate system is represented by a uv coordinate system with the upper left corner of the image 29 as the origin, the horizontal direction as the u axis, and the vertical direction as the v axis. The two-dimensional positions of the reference points Oti function as pass points for combining the overlapping portions of the plurality of images 29.

[0075] For example, when the three-dimensional position of the reference point Ot in the field coordinate system is P(X, Y, Z), the three-dimensional position of the reference point Ot in the camera coordinate system is Pc(Xc, Yc, Zc), the two-dimensional position of the reference point Oti in the image coordinate system is p(x, y), the position of the optical center Oc in the field coordinate system is O(Xo, Yo, Zo), the rotation matrix indicating the orientation of the camera 13 in the field coordinate system is R, and the internal parameter matrix is k, the conditions of the following equations (1), (2), and (3) are satisfied.

[0076] p = k·Pc …(1) P = R·PC+O …(2) P = R·(k -1 ·p) …(3)

[0077] The second position and orientation calculation unit 18 can calculate the position and orientation angle of the camera 13 in the field coordinate system by performing a convergence calculation on the three-dimensional positions of the three reference points Ot, the two-dimensional positions of the reference points Oti in the image 29, and the roll angle and pitch angle of the rotating body 4 based on the bundle method.

[0078] The second position and orientation calculation unit 18 calculates the position and orientation angle of the rotating body 4 in the field coordinate system based on the position and orientation angle of the camera 13 calculated in step SA6 (step SA7).

[0079] The relative position between the optical center Oc of the camera 13 and the representative point Om of the rotating body 4 is known. Also, the transformation matrix for transforming the vehicle body coordinate system based on the representative point Om defined on the rotating body 4 and the camera coordinate system based on the optical center Oc of the camera 13 is known. Therefore, the second position and orientation calculation unit 18 calculates the position and orientation angle of the camera 13 in the field coordinate system based on the bundle method using the image 29 obtained by imaging the target 24, and calculates the position and orientation angle of the rotating body 4 in the field coordinate system by performing coordinate transformation on the position and orientation angle of the camera 13 based on the transformation matrix.

[0080] The processes from step SA1 to step SA7 described above are executed when the target 24 is imaged. After the traveling body 3 performs a traveling operation, when calculating the position and azimuth angle of the slewing body 4, the processes from step SA1 to step SA7 described above are executed again.

[0081] Note that in the processes from step SA1 to step SA7 described above, it is not necessary for three targets 24 to be imaged, and at least two targets 24 may be imaged.

[0082] [Calculation of Position and Azimuth after Slewing Operation] After the position and azimuth angle of the slewing body 4 are calculated, when the traveling body 3 performs a traveling operation, in order to calculate the position and azimuth angle of the slewing body 4, the target 24 is imaged. When the target 24 is imaged, the processes from step SA1 to step SA7 described above are executed again.

[0083] On the other hand, after calculating the position and azimuth angle of the slewing body 4 by the processes from step SA1 to step SA7 described above, when the slewing body 4 performs a slewing operation without the traveling body 3 performing a traveling operation, the second position and azimuth calculation unit 18 can calculate the position and azimuth angle of the slewing body 4 based on the image 29 of at least one target 24 without using at least two targets 24.

[0084] FIG. 10 is a schematic diagram for explaining a method of calculating the position and azimuth angle of the slewing body 4 according to the embodiment. In a state where the slewing body 4 is facing the first direction D1, by imaging at least two targets 24, the position and azimuth of the slewing body 4 are calculated according to the processes from step SA1 to step SA7 described above.

[0085] After the position and azimuth angle of the swivel body 4 are calculated, when the swivel body 4 rotates so as to face from the first direction D1 to the second direction D2 and at least one target 24 is imaged by the imaging device 11, the azimuth angle of the swivel body 4 when it faces the second direction D2 is calculated based on the image 29 of at least one target 24. After calculating the position and azimuth angle of the swivel body 4 using at least two targets 24, when the swivel body 4 rotates about the rotation axis RX so as to face from the first direction D1 to the second direction D2, the second position and azimuth calculation unit 18 can calculate the position and azimuth angle of the swivel body 4 based on the image 29 of at least one target 24 imaged by the imaging device 11.

[0086] That is, the second position and azimuth calculation unit 18 calculates the rotation angle θ based on the azimuth angle of the swivel body 4 before the rotation operation calculated using at least two targets 24 existing in the first direction D1, the image 29 of one target 24 existing in the second direction D2, the roll angle and pitch angle of the swivel body 4 before the rotation operation, and the roll angle and pitch angle of the swivel body 4 after the rotation operation. By calculating the rotation angle θ, the second position and azimuth calculation unit 18 can calculate the azimuth angle of the swivel body 4 after the rotation operation based on the azimuth angle of the swivel body 4 calculated using at least two targets 24 and the rotation angle θ. Further, when the traveling body 3 is not performing a traveling operation, since the position of the rotation axis RX does not change, the second position and azimuth calculation unit 18 can calculate the position of the swivel body 4 based on the calculated rotation angle θ.

[0087] Further, the second position and azimuth calculation unit 18 may simultaneously calculate the position of the rotation axis RX, the azimuth angle of the swivel body 4 before the swivel body 4 performs a rotation operation, and the azimuth angle of the swivel body 4 after the swivel body 4 performs a rotation operation, based on the image 29 of at least two targets 24 imaged by the imaging device 11 before the swivel body 4 performs a rotation operation, the roll angle and pitch angle of the swivel body 4 before the swivel body 4 performs a rotation operation, the image 29 of at least one target 24 imaged by the imaging device 11 after the swivel body 4 performs a rotation operation, and the roll angle and pitch angle of the swivel body 4 after the swivel body 4 performs a rotation operation.

[0088] Note that the second position and orientation calculation unit 18 can calculate the turning angle θ based on the detection data of the inclination sensor 10. As described above, the inclination sensor 10 includes an inertial measurement unit (IMU). The inertial measurement unit (IMU) functions as a turning sensor that detects the turning of the turning body 4. The second position and orientation calculation unit 18 can calculate the turning angle θ based on the detection data of the inertial measurement unit (IMU). Therefore, after the second position and orientation calculation unit 18 calculates the position and orientation angle of the turning body 4 using the three targets 24, when the turning body 4 turns without the traveling body 3 performing a traveling operation, based on the detection data of the inclination sensor 10 that detects the turning of the turning body 4, the position and orientation angle of the turning body 4 after the turning operation can be calculated.

[0089] FIG. 11 is a flowchart showing a method for calculating the position and orientation angle of the turning body 4 after the turning operation according to the embodiment. After the turning body 4 performs a turning operation, the second position and orientation calculation unit 18 determines whether the imaging device 11 can image the target 24. That is, the second position and orientation calculation unit 18 determines whether it can acquire an image of at least one target 24 after the turning body 4 performs a turning operation (step SB1).

[0090] In step SB1, when it is determined that an image of at least one target 24 can be acquired (step SB1: Yes), the second position and orientation calculation unit 18 calculates the orientation angle of the turning body 4 after the turning operation based on the image 29 of at least one target 24, the roll angle and pitch angle of the turning body 4 before the turning operation, and the roll angle and pitch angle of the turning body 4 after the turning operation (step SB2).

[0091] In step SB1, when it is determined that an image of at least one target 24 cannot be acquired (step SB1: No), the second position and orientation calculation unit 18 calculates the position and orientation angle of the turning body 4 after the turning operation based on the detection data of the inclination sensor 10 that detects the turning of the turning body 4 (step SB3).

[0092] As described above, when the first position and orientation calculation unit 17 is unable to calculate the position and orientation angle of the rotating body 4 and the imaging device 11 captures at least two targets 24 before the rotating body 4 performs a turning operation, the second position and orientation calculation unit 18 calculates the position and orientation angle of the rotating body 4 based on the images 29 of at least two targets 24 and the tilt angle of the rotating body 4. When the first position and orientation calculation unit 17 is unable to calculate the position and orientation angle of the rotating body 4 and the rotating body 4 performs a turning operation while the traveling body 3 is not performing a traveling operation, the second position and orientation calculation unit 18 calculates the position and orientation angle of the rotating body 4 based on the image 29 of at least one target 24 acquired by the imaging device 11 after the rotating body 4 performs a turning operation or the detection data of the tilt sensor 10 after the rotating body 4 performs a turning operation.

[0093] [Processing of the correction unit] Next, the processing of the correction unit 23 will be described. The correction unit 23 corrects the error of the tilt sensor 10. As described above, after the rotating body 4 performs a turning operation while the traveling body 3 is not performing a traveling operation, if the position and orientation angle of the rotating body 4 cannot be calculated based on the detection data of the position sensor 9 and the position and orientation angle of the rotating body 4 cannot be calculated based on the image 29 of at least one target 24, the second position and orientation calculation unit 18 can calculate the position and orientation angle of the rotating body 4 after the turning operation based on the detection data of the tilt sensor 10 including the IMU. When calculating the position and orientation angle of the rotating body 4 after the turning operation using the detection data of the tilt sensor 10, the position of the rotating body 4 is calculated by double integrating the acceleration detected by the tilt sensor 10 with respect to time, and the orientation angle of the rotating body 4 is calculated by integrating the angular velocity detected by the tilt sensor 10 with respect to time. When the detection data of the tilt sensor 10 is integrated, a cumulative error may occur in the calculation results of the position and orientation angle of the rotating body 4 due to the integration addition. That is, the error due to the integration of the acceleration or the angular velocity may accumulate, and the calculation accuracy of the position and orientation angle of the rotating body 4 may decrease.

[0094] When the reception status of GNSS signals is good and the first position and orientation calculation unit 17 can calculate the position and azimuth angle of the rotating body 4, the correction unit 23 can correct the errors in the position and azimuth angle of the rotating body 4 based on the calculation results of the first position and orientation calculation unit 17.

[0095] On the other hand, when the reception status of GNSS signals is poor and the first position and orientation calculation unit 17 cannot calculate the position and azimuth angle of the rotating body 4, the correction unit 23 can correct the errors in the position and azimuth angle of the rotating body 4 based on the calculation results of the second position and orientation calculation unit 18.

[0096] FIG. 12 is a flowchart showing a method for correcting the calculation results of the position and azimuth angle of the rotating body 4 according to the embodiment. The switching unit 20 determines whether the first position and orientation calculation unit 17 can calculate the azimuth angle of the rotating body 4 (step SC1).

[0097] In step SC1, if it is determined that the first position and orientation calculation unit 17 can calculate the azimuth angle of the rotating body 4 (step SC1: Yes), the correction unit 23 corrects the errors in the position and azimuth angle of the rotating body 4 based on the azimuth angle of the rotating body 4 calculated by the first position and orientation calculation unit 17 (step SC2).

[0098] In step SC1, if it is determined that the first position and orientation calculation unit 17 cannot calculate the azimuth angle of the rotating body 4 (step SC1: No), the correction unit 23 corrects the errors in the position and azimuth angle of the rotating body 4 based on the azimuth angle of the rotating body 4 calculated by the second position and orientation calculation unit 18 (step SC3).

[0099] [Computer System] FIG. 13 is a block diagram showing a computer system 1000 according to an embodiment. The above-described control device 12 includes the computer system 1000. The computer system 1000 includes a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the above-described control device 12 are stored in the storage 1003 as a computer program. The processor 1001 reads the computer program from the storage 1003, expands it in the main memory 1002, and executes the above-described processing according to the program. Note that the computer program may be distributed to the computer system 1000 via a network.

[0100] The computer program or the computer system 1000 can execute, according to the above-described embodiment, calculating the position and azimuth angle of the hydraulic excavator 1 in a first calculation mode based on GNSS radio waves, calculating the position and azimuth angle of the hydraulic excavator 1 in a second calculation mode based on images of a plurality of targets 24 installed outside the hydraulic excavator 1, and switching between the first calculation mode and the second calculation mode.

[0101] [Effect] As described above, according to the embodiment, based on the GNSS radio wave, the position and azimuth angle of the hydraulic excavator 1 are calculated in the first calculation mode. Based on the images 29 of the plurality of targets 24 installed outside the hydraulic excavator 1, the position and azimuth angle of the hydraulic excavator 1 are calculated in the second calculation mode. The first calculation mode and the second calculation mode are switched by the switching unit 20. Even if a GNSS positioning failure occurs and the position and azimuth angle of the hydraulic excavator 1 cannot be calculated in the first calculation mode, the position and azimuth angle of the hydraulic excavator 1 are calculated in the second calculation mode. Therefore, even when a GNSS positioning failure occurs, the hydraulic excavator 1 can perform work based on the machine guidance technology or the machine control technology.

[0102] The switching unit 20 switches between the first calculation mode and the second calculation mode based on the reception status of the GNSS radio wave. When the reception status of the GNSS radio wave changes from a good state to a bad state, the mode can be switched from the first calculation mode to the second calculation mode. When the reception status of the GNSS radio wave changes from a bad state to a good state, the mode can be switched from the second calculation mode to the first calculation mode. Thereby, the position and azimuth angle of the hydraulic excavator 1 are always calculated.

[0103] The switching unit 20 switches between the first calculation mode and the second calculation mode based on the possibility of calculating the position and azimuth angle of the hydraulic excavator 1 in the first calculation mode. When the state changes from a state where the position and azimuth angle of the hydraulic excavator 1 can be calculated in the first calculation mode to a state where they cannot be calculated, the mode can be switched from the first calculation mode to the second calculation mode. When the state changes from a state where the position and azimuth angle of the hydraulic excavator 1 cannot be calculated in the first calculation mode to a state where they can be calculated, the mode can be switched from the second calculation mode to the first calculation mode. Thereby, the position and azimuth angle of the hydraulic excavator 1 are always calculated.

[0104] The reception status of the GNSS radio wave is displayed on the display device 8A. Thereby, the operator of the hydraulic excavator 1 can recognize the reception status of the GNSS radio wave by checking the display device 8A.

[0105] Based on the input data from the input device 8B, the first calculation mode and the second calculation mode are switched. Thereby, based on the intention of the operator of the hydraulic excavator 1, the first calculation mode and the second calculation mode are switched.

[0106] That the first calculation mode and the second calculation mode are switched is displayed on the display device 8A. Thereby, the operator of the hydraulic excavator 1 can recognize that the first calculation mode and the second calculation mode are switched by checking the display device 8A.

[0107] The display device 8A is arranged in the driver's cab 2 of the hydraulic excavator 1. Thereby, the operator of the hydraulic excavator 1 can smoothly check the display device 8A.

[0108] Based on the detection data of the inclination sensor 10 arranged on the hydraulic excavator 1, the inclination angle of the hydraulic excavator 1 is calculated. Thereby, the second position and orientation calculation unit 18 can calculate the position and orientation angle of the hydraulic excavator 1 based on the images 29 of the plurality of targets 24 and the inclination angle of the hydraulic excavator 1.

[0109] When the first position and orientation calculation unit 17 is in a state where it is impossible to calculate the position and orientation angle of the revolving body 4 and the revolving body 4 is revolving while the traveling body 3 is not performing a traveling operation, the second position and orientation calculation unit 18 is based on at least one image 29 of the target 24 acquired after the revolving body 4 performs a revolving operation or the detection data of the inclination sensor 10 after the revolving body 4 performs a revolving operation. The position and orientation angle of the revolving body 4 can be calculated.

[0110] When the tilt sensor 10 includes an inertial measurement unit (IMU), the tilt sensor 10 can detect the azimuth angle of the hydraulic excavator 1. The correction unit 23 can correct the errors in the position and azimuth angle of the slewing body 4 based on the azimuth angle of the hydraulic excavator 1 calculated in the first calculation mode when the first position azimuth calculation unit 17 can calculate the position and azimuth angle of the hydraulic excavator 1. The correction unit 23 can correct the errors in the position and azimuth angle of the slewing body 4 based on the azimuth angle of the hydraulic excavator 1 calculated in the second calculation mode when the first position azimuth calculation unit 17 cannot calculate the position and azimuth angle of the hydraulic excavator 1.

[0111] [Other Embodiments] In the above-described embodiment, the second position azimuth calculation unit 18 calculates the position and azimuth angle of the camera 13 in the site coordinate system based on the three reference points Ot. The second position azimuth calculation unit 18 may calculate the position and azimuth angle of the camera 13 in the site coordinate system based on at least two reference points Ot. That is, the second position azimuth calculation unit 18 may calculate the position and azimuth angle of the camera 13 in the site coordinate system by performing a convergence calculation on the three-dimensional positions of at least two reference points Ot, the two-dimensional positions of the reference points Oti in the image 29, and the roll angle and pitch angle of the slewing body 4.

[0112] In the above-described embodiment, the second position azimuth calculation unit 18 calculates the position and azimuth angle of the camera 13 in the site coordinate system and calculates the position and azimuth angle of the slewing body 4 in the site coordinate system. The second position azimuth calculation unit 18 may calculate the position and azimuth angle of the camera 13 in the vehicle body coordinate system, or may calculate the position and azimuth angle of the camera 13 in the camera coordinate system. Also, the second position azimuth calculation unit 18 may calculate the position and azimuth angle of the slewing body 4 in the vehicle body coordinate system, or may calculate the position and azimuth angle of the slewing body 4 in the camera coordinate system.

[0113] In the above-described embodiment, the target 24 is imaged by the stereo camera 15. The target 24 may be imaged by a monocular camera.

[0114] In the above-described embodiment, it is assumed that the in-vehicle monitor 8 has the display device 8A and the input device 8B. For example, a tablet terminal may have the display device 8A and the input device 8B. That is, the display device 8A and the input device 8B may be separated from the hydraulic excavator 1. Further, in the above-described embodiment, it is assumed that the display device 8A and the input device 8B are arranged in the driver's cab 2. One or both of the display device 8A and the input device 8B may be arranged outside the driver's cab 2.

[0115] In the above-described embodiment, it is assumed that the reception status of the GNSS radio wave is displayed on the display device 8A. The display control unit 22 may cause the display device 8A to display, for example, recommendation display data for recommending switching between the first calculation mode and the second calculation mode. For example, when the reception status of the GNSS radio wave changes from a good state to a bad state, the display control unit 22 may cause the display device 8A to display character data such as "It is recommended to switch from the first calculation mode to the second calculation mode". When the reception status of the GNSS radio wave changes from a bad state to a good state, the display control unit 22 may cause the display device 8A to display character data such as "It is recommended to switch from the second calculation mode to the first calculation mode".

[0116] In the above-described embodiment, it was assumed that the switching between the first calculation mode and the second calculation mode is performed based on an operation of the input device 8B by the operator. The reception status of GNSS signals may not be displayed on the display device 8A. Also, the switching between the first calculation mode and the second calculation mode may be automatically performed by the control device 12. For example, when the reception status of GNSS signals changes from a good state to a bad state, the switching unit 20 may automatically switch from the first calculation mode to the second calculation mode regardless of the input data from the input device 8B. Further, when the reception status of GNSS signals changes from a bad state to a good state, the switching unit 20 may automatically switch from the second calculation mode to the first calculation mode regardless of the input data from the input device 8B. When the first calculation mode and the second calculation mode are automatically switched, the display control unit 22 may cause the display device 8A to display that the first calculation mode and the second calculation mode have been switched.

[0117] In the above-described embodiment, each of the storage unit 16, the first position and orientation calculation unit 17, the second position and orientation calculation unit 18, the tilt angle calculation unit 19, the switching unit 20, the three-dimensional data calculation unit 21, the display control unit 22, and the correction unit 23 may be configured by separate hardware.

[0118] In the above-described embodiment, it was assumed that the working machine 1 is a hydraulic excavator having a traveling body 3 and a slewing body 4. The working machine 1 does not necessarily have to have the traveling body 3 and the slewing body 4. The working machine 1 only needs to have a working implement, and may be, for example, a bulldozer or a wheel loader.

Description of Reference Numerals

[0119] 1…Hydraulic excavator (working machine), 2…Operator's cab, 3…Travel unit, 3A…Track, 4…Slewing unit, 5…Working equipment, 5A…Boom, 5B…Arm, 5C…Bucket, 6…Hydraulic cylinder, 6A…Boom cylinder, 6B…Arm cylinder, 6C…Bucket cylinder, 7…Operating device, 7A…Left working lever, 7B…Right working lever, 7C…Left travel lever, 7D…Right travel lever, 7E…Left foot pedal, 7F…Right foot pedal, 8…In-vehicle monitor, 8A…Display device, 8B…Input device, 9…Position sensor, 9A…First position sensor, 9B…Second position sensor, 10…Inclination sensor, 11…Imaging device, 12…Control device, 13…Camera, 13A…Camera, 13B…Camera, 13C…Camera, 13D…Camera, 14…Operator's seat, 15…Stereo camera, 15A…Stereo camera, 15B…Stereo camera, 16…Memory unit, 17…First position and orientation calculation unit, 18…Second position and orientation calculation unit, 19…Inclination angle calculation unit, 20…Switching unit, 21…Three-dimensional data calculation unit, 22…Display control unit, 23…Correction unit, 24…Target, 25…Display board, 26…Grounding plate, 27…Identification mark, 28…Emission mark, 28A…Line, 29…Image, 30…Control system, 1000…Computer system, 1001…Processor, 1002…Main memory, 1003…Storage, 1004…Interface, D1…First direction, D2…Second direction, Oc…Optical center, Ot…Reference point, Og…Field reference point, Om…Representative point, Oti…Reference point, RX…Slewing axis, θ…Slewing angle.

Claims

1. A control system for a working machine, comprising: a first position and orientation calculation unit that calculates the position and orientation angle of the working machine based on GNSS radio waves; a second position and orientation calculation unit that calculates the position and orientation angle of the working machine based on images of a plurality of targets installed outside the working machine; a switching unit that switches between a first calculation mode in which the first position and orientation calculation unit calculates the position and orientation angle of the working machine and a second calculation mode in which the second position and orientation calculation unit calculates the position and orientation angle of the working machine. A control system for a working machine.

2. The switching unit switches between the first calculation mode and the second calculation mode based on the reception status of the GNSS radio waves. The control system for a working machine according to Claim 1.

3. The switching unit switches between the first calculation mode and the second calculation mode based on whether the first position and orientation calculation unit can calculate the position and orientation angle of the working machine. The control system for a working machine according to Claim 1 or Claim 2.

4. Comprising a display control unit that causes a display device to display the reception status of the GNSS radio waves. The control system for a working machine according to any one of Claims 1 to 3.

5. The switching unit switches between the first calculation mode and the second calculation mode based on input data from an input device. The control system for a working machine according to Claim 4.

6. Comprising a display control unit that causes a display device to display that the first calculation mode and the second calculation mode have been switched. The control system for a working machine according to any one of Claims 1 to 3.

7. Comprising a display control unit that causes a display device to display a recommendation to switch between the first calculation mode and the second calculation mode. The control system for a working machine according to any one of Claims 1 to 3.

8. The display device is arranged in the cab of the working machine. The control system for a working machine according to any one of Claims 4 to 7.

9. An inclination sensor arranged on the working machine; an inclination angle calculation unit that calculates the inclination angle of the working machine based on the detection data of the inclination sensor. The second position and orientation calculation unit calculates the position and orientation angle of the working machine based on images of a plurality of targets and the inclination angle of the working machine. The control system for a working machine according to any one of Claims 1 to 8.

10. The work machine has a traveling body and a slewing body, The tilt sensor is disposed on the slewing body, The position and azimuth angle of the work machine are the position and azimuth angle of the slewing body, In a state where the first position and azimuth calculation unit cannot calculate the position and azimuth angle of the slewing body, and when the slewing body performs a slewing operation while the traveling body does not perform a traveling operation, the second position and azimuth calculation unit is based on at least one target image acquired after the slewing body performs a slewing operation. Calculate the position and azimuth angle of the slewing body, The control system of the work machine according to claim 9.

11. The work machine has a traveling body and a slewing body, The tilt sensor is disposed on the slewing body, The position and azimuth angle of the work machine are the position and azimuth angle of the slewing body, In a state where the first position and azimuth calculation unit cannot calculate the position and azimuth angle of the slewing body, and when the slewing body performs a slewing operation while the traveling body does not perform a traveling operation, the second position and azimuth calculation unit is based on the detection data of the tilt sensor after the slewing body performs a slewing operation. Calculate the position and azimuth angle of the slewing body, The control system of the work machine according to claim 9.

12. A correction unit for correcting an error of the tilt sensor is provided, In a state where the first position and azimuth calculation unit can calculate the position and azimuth angle of the work machine, the correction unit corrects the error of the position and azimuth angle of the slewing body based on the calculation result of the first position and azimuth calculation unit, and the first position and azimuth calculation unit cannot calculate the position and azimuth angle of the work machine. In this state, the error of the position and azimuth angle of the slewing body is corrected based on the calculation result of the second position and azimuth calculation unit, The control system of the work machine according to claim 10 or claim 11.

13. The work machine has a traveling body and a slewing body, The position and azimuth angle of the work machine are the position and azimuth angle of the slewing body, The control system of the work machine according to any one of claims 1 to 11.

14. A work machine comprising the control system of the work machine according to any one of claims 1 to 13. Work machine.

15. A control method for a work machine, Calculating the position and azimuth angle of the work machine in a first calculation mode based on GNSS radio waves, Calculating the position and azimuth angle of the work machine in a second calculation mode based on images of a plurality of targets installed outside the work machine, switching between the first calculation mode and the second calculation mode A control method for a working machine

16. Based on the reception status of the GNSS radio wave, the first calculation mode and the second calculation mode are switched The control method for a working machine according to claim 15

17. Based on the availability of calculating the position and azimuth of the working machine in the first calculation mode, the first calculation mode and the second calculation mode are switched The control method for a working machine according to claim 15 or claim 16

18. including displaying the reception status of the GNSS radio wave on a display device The control method for a working machine according to any one of claims 15 to 17

19. Based on input data from an input device, the first calculation mode and the second calculation mode are switched The control method for a working machine according to claim 18

20. including displaying on a display device that the first calculation mode and the second calculation mode have been switched The control method for a working machine according to any one of claims 15 to 17

Images