Working system

Abstract

Provided is a work system (1) capable of monitoring a detection value relating to the posture of a detection object. The object to be detected is selected from among a lower traveling body (21), an upper revolving body (22), a boom (31), an arm (32), and a bucket (33). A work system (1) is provided with: a posture detector that detects the posture of a detection object; an information acquisition device (5) that acquires three-dimensional information of the object to be detected; a posture calculation unit that calculates the posture of the detection target on the basis of the acquired three-dimensional information; and a deviation calculation unit that calculates a deviation between the posture detected by the posture detector and the posture calculated by the posture calculation unit.

Description

TECHNICAL FIELD

[0001] The present application relates to a work system. BACKGROUND

[0002] An automatic driving shovel is disclosed in Patent Literature 1, which determines a deviation of teaching position data from current position data based on current distance data.

[0003] On the other hand, for an automatic driving construction machine, if a detection value of a sensor that detects a posture of an attachment or the like has a deviation, an action of the construction machine deviates from a target action.

[0004] PRIOR ART DOCUMENT PATENT LITERATURE Patent Literature 1: Japanese Patent Laying-Open No. 10-183671 SUMMARY

[0005] An object of the present application is to provide a work system capable of monitoring and detecting a detection value related to a posture of a detection object.

[0006] Provided is a work system including a lower traveling body, an upper swing body, a boom, a stick, a distal end attachment, a posture detector, an information acquisition device, a posture calculation section, and a deviation calculation section. The upper swing body is swingably mounted on the lower traveling body. The boom is turnably connected to the upper swing body in a vertical direction. The stick is turnably connected to the boom in the vertical direction. The distal end attachment is turnably connected to the stick. The posture detector detects a posture of at least one detection object selected from the lower traveling body, the upper swing body, the boom, the stick, and the distal end attachment. The information acquisition device acquires three-dimensional information of the detection object. The posture calculation section calculates the posture of the detection object based on the three-dimensional information acquired by the information acquisition device. The deviation calculation section calculates a deviation of the posture detected by the posture detector from the posture calculated by the posture calculation section. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a side view showing a work system according to a first embodiment of the present application.

[0008] Figure 2 is a block diagram showing a part of constituent elements of the work system.

[0009] Figure 3 is a side view of an attachment according to the first embodiment of the present application, showing an example of arrangement of marks attached to the attachment.

[0010] Figure 4 is a side view of the attachment device involved in the first embodiment, which indicates a modification example of the arrangement of the marks.

[0011] Figure 5 is a flowchart indicating the processing performed in the first embodiment.

[0012] Figure 6 is a side view of the attachment device involved in the second embodiment.

[0013] Figure 7 is a flowchart indicating the processing performed in the second embodiment.

[0014] Figure 8 is a diagram indicating a three-dimensional model of the attachment device involved in the third embodiment of the present application.

[0015] Figure 9 is a flowchart indicating the processing performed in the third embodiment.

[0016] Figure 10 is a plan view indicating the posture of the upper swing body of the construction machine involved in the fourth embodiment of the present application.

[0017] Figure 11 is a flowchart indicating the processing performed in the fourth embodiment.

[0018] Figure 12 is a perspective view of the construction machine involved in the fifth embodiment of the present application.

[0019] Figure 13 is a flowchart indicating the processing performed in the fifth embodiment. DETAILED DESCRIPTION

[0020] Hereinafter, a suitable embodiment of the present application will be described with reference to the drawings.

[0021] Figure 1 A work system 1 involved in the first embodiment of the present application is indicated. The work system 1 includes a construction machine 20, a camera 5, and a plurality of sensors 50. Figure 1 The construction machine 20 exemplified is a hydraulic excavator. The construction machine 20 includes a machine body 24 including a lower traveling body 21 and an upper swing body 22, an attachment device 30, and a plurality of working cylinders 40.

[0022] The lower traveling body 21 is a portion that performs a traveling operation, and includes, for example, a pair of left and right tracks. The upper swing body 22 is swingably mounted to the lower traveling body 21 via a swing device 25. The upper swing body 22 includes a cab 23 that constitutes a front portion of the upper swing body 22.

[0023] The attachment device 30 is mounted to the upper swing body 22 so as to be turnable in the up-down direction. The attachment device 30 includes a boom 31, an arm 32, and a bucket 33. The boom 31 is turnably (can be raised and lowered) coupled to the upper swing body 22. The arm 32 is turnably coupled to the boom 31. The bucket 33 is an example of a distal end attachment device that constitutes a distal end portion of the attachment device 30. The bucket 33 is turnably coupled to the arm 32 in the front-rear direction of the upper swing body 22. The bucket 33 is a work member for performing a work such as excavation, grading, and scooping of sand.

[0024] The distal end attachment device is not limited to the bucket 33. The distal end attachment device can be a lifting magnet that holds scrap iron and the like, or can be another work member.

[0025] The plurality of work cylinders 40 are configured to actuate the attachment device 30 using hydraulic pressure. The plurality of work cylinders 40 are each a hydraulic work cylinder that is extendable and retractable. The plurality of work cylinders 40 can each be an electric work cylinder.

[0026] The plurality of work cylinders 40 include a boom work cylinder 41, an arm work cylinder 42, and a bucket work cylinder 43.

[0027] The boom work cylinder 41 turns the boom 31 relative to the upper swing body 22 by performing the extension and retraction operation. The boom work cylinder 41 has a base end portion turnably coupled to the upper swing body 22 and a distal end portion opposite the base end portion, the distal end portion being turnably coupled to the boom 31.

[0028] The arm work cylinder 42 turns the arm 32 relative to the boom 31 by performing the extension and retraction operation. The arm work cylinder 42 has a base end portion turnably coupled to the boom 31 and a distal end portion opposite the base end portion, the distal end portion being turnably coupled to the arm 32.

[0029] The bucket working cylinder 43 rotates the bucket 33 relative to the stick 32 by performing the telescopic movement. The bucket working cylinder 43 has a base end rotatably connected to the base end of the stick 32 and a distal end on the opposite side thereof. The distal end is rotatably connected to a connecting rod member 34, which is rotatably connected to the bucket 33.

[0030] The plurality of sensors 50 constitute a posture detector for detecting the posture of the object being detected. The object being detected is selected from the lower traveling body 21, the upper rotating body 22, the boom 31, the stick 32, and the bucket 33. In this embodiment, the object being detected is the boom 31, the stick 32, and the bucket 33.

[0031] The plurality of sensors 50 includes a boom tilt angle sensor 51, a stick tilt angle sensor 52, a bucket tilt angle sensor 53, a slewing angle sensor 54, and a tilt angle sensor 55.

[0032] The boom tilt angle sensor 51 is mounted on the boom 31 and detects the posture of the boom 31. Specifically, the boom tilt angle sensor 51 obtains the tilt angle of the boom 31 relative to a horizontal line (horizontal plane). The boom tilt angle sensor 51 is, for example, a tilt sensor such as an acceleration sensor. Alternatively, the boom tilt angle sensor 51 can be a rotation angle sensor that detects the rotation angle of the boom foot pin at the base end of the boom 31, or a stroke sensor that detects the stroke of the boom working cylinder 41.

[0033] The stick tilt angle sensor 52 is mounted on the stick 32 and detects the posture of the stick 32. Specifically, the stick tilt angle sensor 52 obtains the tilt angle of the stick 32 relative to the horizontal line (horizontal plane). The stick tilt angle sensor 52 is, for example, a tilt sensor such as an acceleration sensor. Alternatively, the stick tilt angle sensor 52 can be a rotation angle sensor that detects the rotation angle of the stick connecting pin at the base end of the stick 32, or a stroke sensor that detects the stroke of the stick working cylinder 42.

[0034] The bucket tilt angle sensor 53 is mounted on the connecting rod member 34 and detects the posture of the bucket 33. Specifically, the bucket tilt angle sensor 53 obtains the tilt angle of the bucket 33 relative to the horizontal line (horizontal plane). The bucket tilt angle sensor 53 is, for example, a tilt sensor such as an acceleration sensor. Alternatively, the bucket tilt angle sensor 53 can be a rotation angle sensor that detects the rotation angle of the bucket connecting pin at the base end of the bucket 33, or a stroke sensor that detects the stroke of the bucket working cylinder 43.

[0035] The rotation angle sensor 54 detects the rotation angle of the upper rotating body 22 relative to the lower traveling body 21. The rotation angle sensor 54 is, for example, an encoder, a resolver, or a gyroscope sensor. In this embodiment, the rotation angle of the upper rotating body 22 is 0° when the forward direction of the upper rotating body 22 is aligned with the forward direction of the lower traveling body 21.

[0036] The tilt angle sensor 55 is, for example, mounted on the upper rotating body 22 and detects the tilt angle of the lower traveling body 21 relative to the horizontal plane. The tilt angle sensor 55 can be, for example, a tilt sensor such as an acceleration sensor, or an IMU (Inertial Measurement Unit).

[0037] The camera device 5 functions as an information acquisition device for obtaining three-dimensional information of the object being detected. The camera device 5 is installed at the work site. For example, it may be mounted on a pillar erected at the work site. In this embodiment, the camera device 5 is a LiDAR (Light Detection and Ranging) or Laser Imaging Detection and Ranging device. The information acquisition device may also include multiple camera devices.

[0038] The camera device 5 captures images of the construction machinery 20. The camera device 5 acquires point cloud data (3D point cloud) representing the distances from the camera position to multiple points on the object being inspected. The point cloud data contains 3D information capable of determining the shape, etc., of the object being inspected. In this embodiment, the camera position is the location where the camera device 5 is installed.

[0039] The three-dimensional information is not limited to the point cloud data, and the camera device 5 is not limited to LiDAR. The camera device 5 can be a TOF (Time of Flight) sensor, a stereo camera, or the like.

[0040] like Figure 2 As shown, the engineering machinery 20 also includes a controller 11, a storage device 13, and a communication device 14.

[0041] The storage device 13 stores shape information. The shape information is information about the shape of the lower traveling body 21, the upper rotating body 22, the boom 31, the stick 32, and the bucket 33.

[0042] The communication device 14 communicates with the camera device 5. Additionally, the communication device 14 communicates with the portable terminal 7. The portable terminal 7 is a terminal carried by the operator managing the automatic driving of the construction machinery 20, such as a smartphone or tablet.

[0043] The detection values ​​obtained by the plurality of sensors 50 are input to the controller 11. Specifically, the information on the posture of the boom 31 detected by the boom tilt angle sensor 51 is input to the controller 11. The information on the posture of the stick 32 detected by the stick tilt angle sensor 52 is input to the controller 11. The information on the posture of the bucket 33 detected by the bucket tilt angle sensor 53 is input to the controller 11. The information on the slewing angle of the upper slewing body 22 relative to the lower traveling body 21, i.e., the slewing posture of the upper slewing body 22, detected by the slewing angle sensor 54, is input to the controller 11. The information on the tilt angle of the upper slewing body 22 relative to the horizontal plane, i.e., the posture of the lower traveling body 21, detected by the tilt angle sensor 55, is input to the controller 11.

[0044] The controller 11 enables the construction machinery 20 to perform predetermined automatic driving actions, thereby automatically activating the slewing device 25 and the auxiliary device 30. That is, the controller 11 performs automatic driving of the construction machinery 20. The predetermined automatic driving actions include, for example, repeatedly performing digging, lifting slewing, soil removal, and resetting slewing actions.

[0045] The controller 11 includes a posture calculation unit and a deviation calculation unit. The posture calculation unit calculates the posture of the detected object based on the three-dimensional information acquired by the camera device 5, independently of the plurality of sensors 50. The deviation calculation unit calculates the deviation between the posture of the detected object detected by the plurality of sensors 50 and the posture of the detected object calculated by the posture calculation unit.

[0046] The posture calculation unit of the controller 11 calculates the tilt angles of the boom 31, the stick 32, and the bucket 33 relative to the horizontal plane based on the three-dimensional information obtained by the camera device 5, as the postures of these components. The deviation calculation unit of the controller 11 calculates the deviation between the tilt angles of the boom 31, the stick 32, and the bucket 33 detected by the plurality of sensors 50 (i.e., the detected postures) and the tilt angles of the boom 31, the stick 32, and the bucket 33 calculated by the posture calculation unit (i.e., the calculated postures).

[0047] As shown in the side view of the accessory device 30 Figure 3As shown, the first angle θA between the stick centerline 71 (which serves as the centerline of the stick 32) and the vertical line 72 can be calculated based on the three-dimensional information of the boom 31 and the stick 32 obtained by the camera device 5. In this embodiment, in side view, the stick centerline 71 is a straight line connecting the rotation center axis of the stick 32 relative to the boom 31 (e.g., the center axis of the stick connecting pin) and the rotation center axis of the bucket 33 relative to the stick 32 (e.g., the bucket connecting pin). The first angle θA can be determined based on the tilt angle θ1 of the boom centerline 73 (which serves as the centerline of the boom 31) relative to the horizontal line, and the second angle θB between the boom centerline 73 and the stick centerline 71. The boom tilt angle θ1 is detected by the boom tilt angle sensor 51. In this embodiment, viewed from the side, the boom centerline 73 is a straight line connecting the rotation center axis of the boom 31 relative to the upper rotating body 22 (e.g., the center axis of the boom foot pin) and the rotation center axis of the stick 32 relative to the boom 31 (e.g., the stick connecting pin). On the other hand, the second angle θB can be determined based on the boom tilt angle θ1 and the tilt angle θ2 of the stick centerline 71 relative to the horizontal line, which is detected by the stick tilt angle sensor 52. Therefore, the reliability of the detection values ​​of the boom tilt angle sensor 51 and the stick tilt angle sensor 52 can be evaluated based on the first angle θA determined by the sensors 51 and 52 respectively (i.e., the angle corresponding to the tilt angles θ1 and θ2) and the deviation between this first angle θA and the first angle θA calculated by the posture calculation unit of the controller 11. That is, it is possible to assess whether the detection value of either the boom tilt angle sensor 51 or the stick tilt angle sensor 52 is deviated.

[0048] Based on the three-dimensional information of the boom 32 and the bucket 33 obtained by the camera device 5, the third angle θC formed by the bucket opening surface line 74 along the opening surface of the bucket 33 and the vertical line 75 in a side view can be calculated. In this embodiment, the bucket opening surface line 74 is a straight line connecting the rotation center axis of the bucket relative to the boom 32 (e.g., the bucket connecting pin) to the far end of the bucket 33. The third angle θC can be determined based on the boom tilt angle θ1, the second angle θB, and the fourth angle θD formed by the boom center line 71 and the opening surface line 74 of the bucket 33. The fourth angle θD can be determined based on the boom tilt angle θ2 and the tilt angle of the bucket opening surface line 74 relative to the horizontal line, i.e., the bucket tilt angle θ3, which is detected by the bucket tilt angle sensor 53. Therefore, based on the deviation between the third angle θC, which represents the posture of the bucket 33, determined by the plurality of sensors 50 (the boom tilt angle θ1, the stick tilt angle θ2, and the bucket tilt angle θ3), and the third angle θC calculated by the posture calculation unit of the controller 11, the reliability of the detection values ​​of the boom tilt angle sensor 51, the stick tilt angle sensor 52, and the bucket tilt angle sensor 53 can be evaluated. That is, it is possible to assess whether there is a deviation in the detection value of any one of the three sensors.

[0049] If the value calculated by the posture calculation unit for at least one of the first angle θA and the third angle θC deviates from the detection value of the plurality of sensors 50, the controller 11 sends an error signal to the portable terminal 7 carried by the operator. Upon receiving the error signal, the portable terminal 7 notifies the operator in accordance with the error signal, thereby enabling the operator to take actions such as stopping the automatic driving system.

[0050] In this way, it is possible to calculate the deviation between the pose of the detected object detected by the plurality of sensors 50 constituting the pose detector and the pose of the detected object calculated based on the three-dimensional information, and to evaluate the reliability of the detection values ​​of the plurality of sensors 50 based on the magnitude of the deviation. This makes it possible, for example, to monitor whether the detection values ​​of the plurality of sensors 50 have deviated from the allowable range set for the detection value.

[0051] The operating system 1 also has Figure 3 The multiple markings shown are boom marking 81, stick marking 82, and bucket marking 83. These markings 81, 82, and 83 are respectively affixed to the boom 31, stick 32, and bucket 33, which are the objects of the inspection.Figure 3 In the example shown, the boom mark 81 is set at the distal end of the boom 31, preferably at the rotation center of the stick 32 relative to the boom 31, the stick mark 82 is set at the distal end of the stick 32, preferably at the rotation center of the bucket 33 relative to the stick 32, and the bucket mark 83 is set at the distal end of the bucket 33.

[0052] The marks 81 to 83 each have high reflectivity. For example, the marks 81 to 83 are formed of materials with high reflectivity. This allows the three-dimensional information of each of the marks 81 to 83 to be obtained by the camera device 5 capturing images of the auxiliary device 30. The posture calculation unit can calculate the stick centerline 71 based on the three-dimensional information of the boom mark 81 and the stick mark 82. In addition, the posture calculation unit can calculate the bucket opening surface line 74 based on the three-dimensional information of the stick mark 82 and the bucket mark 83. Thus, the use of the three-dimensional information of each of the marks 81 to 83 makes the calculation of the posture of the object being detected easier. Furthermore, the base end of the boom 31 is in a position that cannot be... Figure 1 The position where the camera device 5 is shown is therefore excluded from the locations where the mark is placed. However, the mark can also be placed at the base end of the boom 31 as long as it is in a position where it can be photographed.

[0053] The number and arrangement of the plurality of marks are not limited. (See the side view of the auxiliary device 30.) Figure 4 In the variant shown, the boom 32 is marked with two boom markings 84 and 85, and the bucket 33 is marked with two bucket markings 86 and 87.

[0054] The two stick markings 84 and 85 are respectively positioned on the stick centerline 71, closer to the boom 31 than the center of the stick centerline 71, and closer to the bucket 33 than the center position. The three-dimensional information of the stick markings 84 and 85 allows for the calculation of the stick centerline 71, thereby enabling the calculation of the first angle θA formed by the stick centerline 71 and the vertical line 72.

[0055] The two bucket markings 86 and 87 are respectively positioned on the bucket opening surface 74 at a position closer to the boom 32 than the center position of the bucket opening surface 74, and at a position closer to the far end of the bucket 33 than the center position. The three-dimensional information of the bucket markings 86 and 87 enables the calculation of the bucket opening surface 74, thereby enabling the calculation of the third angle θC formed by the bucket opening surface 74 and the vertical line 75.

[0056] The plurality of markers can also be captured by a camera different from the imaging device 5. By overlapping the coordinate system of the point cloud data captured by the imaging device 5 with the coordinate system of the plurality of markers captured by the camera, the positions of the plurality of markers can be determined. Alternatively, each of the plurality of markers may also contain an AR (Augmented Reality) tag recording identification information that determines the position of the marker. This identification information allows the information acquisition device to easily determine the position of the marker by reading the identification information.

[0057] The use of the symbols is arbitrary. For example, the posture calculation unit can also determine the shapes of the boom 32 and the bucket 33 based on the point cloud data, and calculate the boom centerline 71 and the bucket opening surface line 74. The shape information stored in the storage device 13 is used to determine the shape.

[0058] Next, refer to Figure 5 The flowchart illustrates the processing performed by the controller 11.

[0059] The controller 11 instructs the camera device 5 to acquire reference point cloud data (step S1). The reference point cloud data is the point cloud data of the construction machinery 20 at a pre-set reference position. Next, the controller 11 initiates the operation of the auxiliary device 30 for automatic driving (step S2).

[0060] The controller 11 acquires multiple detection values ​​from the plurality of sensors 50 (step S3). The plurality of detection values ​​include the boom tilt angle θ1, the stick tilt angle θ2, and the bucket tilt angle θ3 in the current posture of the auxiliary device 30.

[0061] The controller 11 instructs the camera device 5 to acquire current point cloud data (step S4). The current point cloud data is the point cloud data of the engineering machinery 20 at the current location. The controller 11 calculates the difference between the reference point cloud data and the current point cloud data, thereby extracting the portion of the current point cloud data corresponding to the auxiliary device 30 (step S5). This extraction may also be accompanied by clustering or denoising.

[0062] The controller 11 projects the point cloud data extracted as described above onto a pre-set projection surface (step S6). The projection surface is, for example, a vertical surface.

[0063] The controller 11 determines the position of the mark attached to the auxiliary device 30, in Figure 3In the example shown, the positions of the marks 81 to 83 are determined (step S7). The controller 11 repeatedly performs the processing of steps S3 to S7 until all the positions of the marks 81 to 83 are determined (step S8 is "No").

[0064] After determining all the marks 81 to 83 (step S8 is "Yes"), the controller 11 calculates the stick centerline 71 and the bucket opening surface line 74 (step S9), and calculates the first angle θA and the third angle θC corresponding to the tilt angles of the stick centerline 71 and the bucket opening surface line 74 respectively (step S10).

[0065] The controller 11 determines whether the values ​​of the first angle θA and the third angle θC calculated as described above, i.e., the values ​​calculated for the posture of the boom 32 and the posture of the bucket 33 respectively, are consistent with the values ​​of the first angle θA and the third angle θC determined based on the values ​​detected by the plurality of sensors 50, i.e., the values ​​detected for the posture of the boom 32 and the posture of the bucket 33 respectively (step S11). If it is determined that the calculated values ​​of the first angle θA and the third angle θC are consistent with the detected values ​​(step S11 is "yes"), the controller 11 repeats the processing after step S3. On the other hand, if it is determined that the calculated value of at least one of the first angle θA and the third angle θC is not consistent with the detected value (step S11 is "no"), the controller 11 sends an error message to the portable terminal 7 (step S12).

[0066] When using the multiple markers (e.g., markers 81-83), the action of extracting the portion corresponding to the auxiliary device 30 from the point cloud data can be omitted. Figure 5 In this process, steps S1 and S4 to S6 can be omitted.

[0067] According to the operating system 1 described above, since the deviation between the posture detected by the plurality of sensors 50 constituting the posture detector and the posture calculated based on the three-dimensional information is calculated, the reliability of the detection values ​​of each of the plurality of sensors 50 can be evaluated based on the magnitude of the deviation. For example, it is possible to monitor whether the detection values ​​of each of the plurality of sensors 50 have deviated from the allowable range set for that detection value.

[0068] Specifically, in Figure 3In the example shown, the boom 31, the stick 32, and the bucket 33 are the objects being detected, and the plurality of sensors 50 detect the tilt angles θ1 to θ3 of these objects relative to the horizontal plane. On the other hand, the posture calculation unit of the controller 11 calculates a first angle θA and a third angle θC corresponding to the tilt angles based on the three-dimensional information. For the first angle θA and the third angle θC, the reliability of the detection values ​​of the plurality of sensors 50 can be evaluated based on the magnitude of the deviation between the values ​​determined according to the tilt angles θ1 to θ3 detected by the plurality of sensors 50 (i.e., the detected values) and the calculated values ​​based on the three-dimensional information.

[0069] In addition, with Figure 3 and Figure 4 The three-dimensional information associated with the symbols 81 to 87 shown makes it possible to easily calculate the pose of the detected object using this three-dimensional information.

[0070] Next, refer to Figure 6 and Figure 7 The operating system 101 of the second embodiment will be described below. The description of the second embodiment and subsequent embodiments will mainly focus on the differences from the first embodiment, and components that are common to the first embodiment will be given common reference numerals.

[0071] The detection objects involved in this second embodiment are the same as those in the first embodiment: the boom 31, stick 32, and bucket 33 of the construction machinery 20. The operating system 101 of this second embodiment includes the same controller 11 and storage device 13 as those in the first embodiment. The controller 11 includes a posture calculation unit and a deviation calculation unit. In addition to using the three-dimensional information obtained by the camera device 5, the posture calculation unit uses the shape information stored in the storage device 13—that is, information about the shape of each of the boom 31, stick 32, and bucket 33, which are the detection objects—to calculate an angle corresponding to the tilt angle of the detection object. The deviation calculation unit evaluates whether the detection values ​​of the boom tilt angle sensor 51, stick tilt angle sensor 52, and bucket tilt angle sensor 53 have deviated based on the deviation between the value calculated by the posture calculation unit and the angle value determined based on the tilt angle of the detection object detected by the plurality of sensors 50.

[0072] As shown in the side view of accessory 30 Figure 6As shown, in addition to setting boom markings 81, stick markings 82, and bucket markings 83 at the distal ends of the boom 31, the stick 32, and the bucket 33, respectively, a boom marking 80 is also set at the center of the boom 31. Therefore, in addition to calculating the stick centerline 71 based on the three-dimensional information of the boom markings 81 and 82, and calculating the bucket opening surface line 74 based on the three-dimensional information of the stick markings 82 and 83, the centerline of the distal portion of the boom 31, i.e., the distal boom centerline 76, can also be calculated based on the three-dimensional information of the boom markings 81 and 80. The posture calculation unit uses the shape information of the boom 31 and the distal boom centerline 76 stored in the storage device 13 to calculate the centerline of the entire boom 31, i.e., the boom centerline 73. Specifically, the shape information of the boom 31 provides the angle between the centerline 76 of the distal end of the boom and the centerline 73 of the boom, thereby enabling the centerline 73 of the boom to be appropriately calculated based on the centerline 76 of the distal end of the boom.

[0073] The marks 80-83 preferably have different shapes or different reflectivities to be distinguishable from each other. This allows the posture calculation unit to calculate the stick centerline 71, the bucket opening surface line 74, and the boom distal side centerline 76 more reliably. Furthermore, even if multiple marks are indistinguishable from each other, the necessary centerlines (e.g., the boom centerline 73 and the boom distal side centerline 76) can be calculated based on the lines connecting the marks, provided they do not intersect each other. Therefore, the posture calculation unit preferably determines whether the lines intersect each other, and only calculates the necessary centerlines based on the lines if it determines that the lines do not intersect each other.

[0074] Even without the boom marking 80, the shape of the boom 31 can be determined based on point cloud data, and the centerline 76 of the distal end of the boom can be calculated. The shape information stored in the storage device 13 is used to determine the shape.

[0075] The posture calculation unit uses the boom centerline 73 to calculate the tilt angle θ1 of the boom 31 relative to the horizontal plane. The deviation calculation unit calculates... Figure 1 The deviation between the boom tilt angle θ1 detected by the boom tilt angle sensor 51 and the boom tilt angle θ1 calculated by the posture calculation unit can be used to assess whether the detected value of the boom tilt angle sensor 51 has deviated.

[0076] The posture calculation unit calculates the second angle θB formed by the boom centerline 73 and the stick centerline 71, and based on this second angle θB and the boom tilt angle θ1 calculated as described above, calculates the angle θ2 of the stick centerline 71 relative to the horizontal plane. The deviation calculation unit calculates... Figure 1 The deviation between the stick tilt angle θ2 detected by the stick tilt angle sensor 52 and the stick tilt angle θ2 calculated by the posture calculation unit can be used to assess whether the detected value of the stick tilt angle sensor 52 has deviated.

[0077] The posture calculation unit calculates the fourth angle θD formed by the stick centerline 71 and the bucket opening surface line 74, and based on this fourth angle θD and the stick tilt angle θ2 calculated as described above, calculates the angle θ3 of the bucket opening surface line 74 relative to the horizontal plane. The deviation calculation unit calculates... Figure 1 The deviation between the bucket tilt angle θ3 detected by the bucket tilt angle sensor 53 and the bucket tilt angle θ3 calculated by the posture calculation unit can be used to assess whether the detected value of the bucket tilt angle sensor 53 has deviated.

[0078] Even without the symbols 80-83, the shapes of the boom 31, the stick 32, and the bucket 33 can be determined based on point cloud data, and the boom centerline 73, the stick centerline 71, and the bucket opening surface line 74 can be calculated respectively. The shape information stored in the storage device 13 is used to determine the shape.

[0079] Next, refer to Figure 7 The flowchart illustrates the processing performed in the second embodiment.

[0080] First, the controller 11 instructs the camera device 5 to acquire reference point cloud data (step S21). The reference point cloud data is the point cloud data of the construction machinery 20 at the reference position. Next, the controller 11 initiates the operation of the auxiliary device 30 utilizing automatic driving (step S22).

[0081] The controller 11 acquires multiple detection values ​​detected by the plurality of sensors 50 (step S23). The plurality of detection values ​​include the boom tilt angle θ1, the stick tilt angle θ2, and the bucket tilt angle θ3 in the current posture of the auxiliary device 30.

[0082] The controller 11 instructs the camera device 5 to acquire current point cloud data (step S24). The current point cloud data is the point cloud data of the engineering machinery 20 at the current location. The controller 11 calculates the difference between the reference point cloud data and the current point cloud data, thereby extracting the portion of the current point cloud data corresponding to the auxiliary device 30 (step S25). This extraction may also be accompanied by clustering or denoising.

[0083] The controller 11 projects the point cloud data extracted as described above onto a pre-set projection surface (step S26). The projection surface is, for example, a vertical surface. The controller 11 determines the positions of the marks 80 to 83 (step S27). The controller 11 repeats the processing of steps S23 to 27 until all positions of the marks 80 to 83 are determined (step S28 is "No").

[0084] After determining all the positions of the marks 80 to 83 (step S28 is "Yes"), the controller 11 calculates the center line 76 of the far end of the boom (step S29), and further uses the shape information of the boom 31 to calculate the center line 73 of the boom based on the center line 76 of the far end of the boom (step S30).

[0085] The posture calculation unit of the controller 11 calculates the boom tilt angle θ1, the stick tilt angle θ2, and the bucket tilt angle θ3 (step S31). The deviation calculation unit of the controller 11 determines whether the calculated values ​​of the tilt angles θ1 to θ3 are consistent with the values ​​detected by the plurality of sensors 50 (step S32). If it is determined that the calculated values ​​of the tilt angles θ1 to θ3 are consistent with the detected values ​​of the tilt angles θ1 to θ3 respectively (step S32 is "yes"), the controller 11 repeats the processing after step S23. On the other hand, if it is determined that the calculated value of any tilt angle θ1 to θ3 is not consistent with the detected value (step S32 is "no"), the controller 11 sends an error message to the portable terminal 7 (step S33).

[0086] When using the symbol 80, the action of extracting the portion corresponding to the auxiliary device 30 from the point cloud data can also be omitted. Figure 7 In this process, steps S21 and S24 to S26 can also be omitted.

[0087] According to the operating system 101 described above, in addition to using the three-dimensional information, the shape information stored in the storage device 13 is also used to calculate the pose of the detected object. The use of the shape information makes the calculation of the pose of the detected object easier.

[0088] Next, refer to Figure 8 and Figure 9 The operating system 201 according to the third embodiment will be explained.

[0089] The detection objects involved in this embodiment are the same as those in the second embodiment: the boom 31, stick 32, and bucket 33 of the construction machinery 20. The operating system 101 involved in this embodiment is equipped with... Figure 1 The controller 11 and storage device 13 shown are the same. The controller 11 includes a posture calculation unit and a deviation calculation unit. Similar to the second embodiment, the posture calculation unit uses not only the three-dimensional information obtained by the camera device 5, but also the shape information stored in the storage device 13—that is, information about the shapes of the boom 31, the stick 32, and the bucket 33—to calculate the tilt angle of the object being detected. On the other hand, in this third embodiment, the symbols 80 to 83 involved in the second embodiment are not required.

[0090] The controller 11 uses the detection values ​​from multiple sensors 50 to enable, for example... Figure 8 The three-dimensional model of the auxiliary device 30 shown is activated. The controller 11 calculates the current region of the boom 31 in the three-dimensional model, i.e., the current boom region, at a predetermined time. At the same time, the controller 11 instructs the camera device 5 to acquire the point cloud data of the actual construction machinery 20, i.e., the current point cloud data. The controller 11 extracts the point cloud data corresponding to the current boom region, i.e., the boom point cloud data, from the current point cloud data acquired as described above. The controller 11 uses the shape information of the boom 31 stored in the storage device 13 to perform this extraction.

[0091] The controller 11 calculates the proportion of the boom point cloud data in the current boom region of the 3D model, i.e., the occupancy rate of the boom point cloud data. The higher the occupancy rate, the smaller the deviation between the 3D model and the actual auxiliary device 30. That is, the deviation between the boom 31 posture determined by the detection value of the boom tilt angle sensor 51 (detected posture) and the boom 31 posture calculated by the posture calculation unit (calculated posture) is small. Therefore, the reliability of the detection value of the boom tilt angle sensor 51 can be evaluated based on the occupancy rate.

[0092] The same processing is performed on the boom 32 and the bucket 33 respectively, thereby enabling the reliability of the detection values ​​of the boom tilt angle sensor 52 and the bucket tilt angle sensor 53 to be evaluated respectively.

[0093] Next, refer to Figure 9 The flowchart is used to illustrate the processing performed in this third embodiment.

[0094] First, the controller 11 acquires the position of the construction machinery 20 and the orientation of the upper rotating body 22 within the construction machinery 20 (step S41). The controller 11 then positions the 3D model in virtual space at a location corresponding to the acquired position of the construction machinery 20, and aligns the orientation of the upper rotating body in the 3D model with the acquired orientation of the upper rotating body 22 (step S42). The controller 11 then uses the detection values ​​from the plurality of sensors 50 to initiate the movement of the 3D model (step S43).

[0095] The controller 11 calculates the current boom region of the boom 31 in the three-dimensional model at a predetermined time (step S44). At the same time, the controller 11 instructs the camera device 5 to acquire point cloud data (step S45), and extracts the point cloud data corresponding to the current boom region from the point cloud data (step S46).

[0096] The controller 11 calculates the proportion of the boom point cloud data in the current boom area, i.e., the occupancy rate of the boom point cloud data (step S47). The controller 11 determines whether the occupancy rate is above a pre-set threshold (step S48). If the occupancy rate is above the threshold (step S48 is "Yes"), the controller 11 repeats the processing after step S44. On the other hand, if the occupancy rate is below the threshold (step S48 is "No"), the controller 11 sends an error message to the portable terminal 7 (step S49).

[0097] Next, refer to Figure 10 and Figure 11 The operating system 301 according to the fourth embodiment will be described. The detection object according to this fourth embodiment is the upper rotating body 22 of the construction machinery 20, and the reliability of the detection value of the rotation angle sensor 54 that detects the rotation angle of the upper rotating body 22 is evaluated.

[0098] The rotation angle sensor 54 detects the rotation angle of the upper rotating body 22 of the construction machinery 20 relative to the lower traveling body 21. That is, the rotation angle sensor 54 detects the rotational posture of the upper rotating body 22 relative to the lower traveling body 21. See the top view of the construction machinery 20. Figure 10As shown by the double-dotted line and solid line respectively, the rotation angles of the upper rotating body 22, corresponding to the different first and second rotation postures, are detected by the rotation angle sensor 54. The first rotation posture is a reference posture, and the center line of the auxiliary device 30, i.e., the auxiliary device center line, in this reference posture is set as the reference line 77. The angle between the auxiliary device center line 78 and the reference line 77 in the second rotation posture corresponds to the difference between the detection value (first detection value) in the first rotation posture and the detection value (second detection value) in the second rotation posture.

[0099] On the other hand, the operating system 301 has the same as Figure 1 The controller 11 shown is the same controller 11, which includes a differential calculation unit, an attitude calculation unit, and a deviation calculation unit.

[0100] The difference calculation unit calculates the difference between the first detection value and the second detection value as the rotation angle of the upper rotating body 22 measured from the reference line 77. That is, in this fifth embodiment, the rotation angle sensor 54 and the difference calculation unit constitute a posture detector, which detects the difference, i.e., the rotation angle of the upper rotating body 22 measured from the reference line 77, as a value representing the posture of the upper rotating body 22.

[0101] The posture calculation unit of the controller 11 commands Figure 1 The camera device 5 shown, or other information acquisition device, acquires three-dimensional information of the construction machinery 20. Based on this three-dimensional information, the rotation angle of the upper rotating body 22, measured from the reference line 77, is calculated as a value representing the posture of the upper rotating body 22. That is, the controller 11 calculates the angle between the reference line 77 and the center line 78 of the auxiliary device in the second rotation posture. The deviation calculation unit of the controller 11 calculates the difference between the rotation angle detected by the rotation angle sensor 54 and the difference calculation unit, and the deviation between the rotation angle calculated by the posture calculation unit. Thus, the reliability of the detection value of the rotation angle sensor 54 can be evaluated.

[0102] Next, refer to Figure 11 The flowchart is used to illustrate the processing performed in this fifth embodiment.

[0103] First, the controller 11 sets the upper rotating body 22 as... Figure 10After the first turning posture, indicated by the double-dotted line, which is the reference posture, is obtained, the camera device 5 acquires reference point cloud data (step S61). The reference point cloud data is the point cloud data of the construction machinery 20 at the reference position. Next, the controller 11 starts the operation of the construction machinery 20 (step S62). During the operation, the orientation of the lower walking body 21 remains unchanged.

[0104] The controller 11 obtains a detection value from the sensor 50, specifically from the rotation angle sensor 54 (step S63). The detection value is the rotation angle of the upper rotating body 22 relative to the lower traveling body 21.

[0105] The controller 11 instructs the camera device 5 to acquire current point cloud data (step S64). The current point cloud data is the point cloud data of the engineering machinery 20 at the current location. The controller 11 calculates the difference between the reference point cloud data and the current point cloud data, thereby extracting the portion of the current point cloud data corresponding to the auxiliary device 30 (step S65). This extraction may also be accompanied by clustering or denoising.

[0106] The controller 11 projects the point cloud data extracted as described above onto a pre-set projection surface (step S66). The projection surface is, for example, a horizontal plane. For the projection surface, the controller 11 calculates the center line of the auxiliary device 30 in the first rotation posture, i.e., the auxiliary device center line (step S67), and sets the auxiliary device center line as the reference line 77 (step S68).

[0107] Next, the controller 11 sets the upper rotating body 22 to rotate by a predetermined angle. Figure 10 After the second rotation posture shown, a detection value is obtained again from the sensor 50, namely the rotation angle sensor 54 (step S69). The detection value is the rotation angle of the upper rotating body 22 relative to the lower walking body 21.

[0108] At this point in time, the controller 11 instructs the camera device 5 to acquire the current point cloud data again (step S70). The controller 11 calculates the difference between the reference point cloud data and the newly acquired current point cloud data, thereby extracting the portion of the current point cloud data corresponding to the auxiliary device 30 (step S71).

[0109] The controller 11 projects the point cloud data extracted as described above onto the projection surface (step S72). For the projection surface, the controller 11 calculates the center line of the auxiliary device 30, i.e., the auxiliary device center line 78 (step S73), and calculates the angle between the auxiliary device center line 78 and the reference line 77 (step S74).

[0110] The controller 11 determines whether the calculated rotation angle, calculated based on point cloud data as described above, i.e., the angle between the center line 78 of the auxiliary device and the reference line 77, is consistent with the detected rotation angle. This detected rotation angle is the difference between the rotation angle of the upper rotating body 22 detected by the rotation angle sensor 54 when the reference line 77 is set in the first rotation posture, and the rotation angle of the upper rotating body 22 detected by the rotation angle sensor 54 when the center line 78 of the auxiliary device is calculated in the second rotation posture (step S75). If it is determined that the calculated rotation angle is consistent with the detected rotation angle, i.e., the difference between the detected values ​​of the rotation angle sensor 54 in the first and second rotation postures (step S75 is "yes"), the controller 11 repeatedly performs the processing after step S69. On the other hand, if it is determined that the calculated rotation angle is not consistent with the detected rotation angle (step S75 is "No"), the controller 11 sends an error message to the portable terminal 7 (step S76).

[0111] According to the operating system 301 described above, the rotation angle sensor 54 and the differential calculation unit of the controller 11 detect the rotation angle of the upper rotating body 22, which is the object of detection. On the other hand, the posture calculation unit of the controller 11 calculates the rotation angle of the upper rotating body 22 from the reference line 77 based on the three-dimensional information. Therefore, the deviation calculation unit of the controller 11 calculates the deviation between the former rotation angle (detected rotation angle) and the latter rotation angle (calculated rotation angle). Thus, the reliability of the detection value of the rotation angle sensor 54 can be evaluated based on the magnitude of this deviation.

[0112] Next, refer to Figure 12 and Figure 13 The fifth embodiment describes the operating system 401. The object of detection in this fifth embodiment is the lower traveling body 21 of the construction machinery 20, and the reliability of the detection value of the tilt angle sensor 55, which detects the tilt angle of the lower traveling body 21 relative to the horizontal plane, is evaluated.

[0113] The operating system 401 has the capability of Figure 1The controller 11 shown is the same controller 11, which includes a posture calculation unit and a deviation calculation unit. The posture calculation unit calculates the tilt angle of the lower walking body 21 relative to the horizontal plane based on the acquired three-dimensional information, as a value representing the posture of the lower walking body 21. The deviation calculation unit calculates the deviation between the tilt angle of the lower walking body 21 detected by the tilt angle sensor 55 (the detected value) and the tilt angle of the lower walking body 21 calculated by the posture calculation unit (the calculated value).

[0114] The lower traveling body 21 is marked with multiple symbols. (See the perspective view of the engineering machinery 20.) Figure 12 In the example shown, the pair of left and right tracks 60 of the lower walking body 21 are respectively marked with a front side mark 88F and a rear side mark 88R. The front side mark 88F is located on the central axis of the idler wheel, which serves as the front wheel of the track 60, on the outer side surface of the idler wheel. The rear side mark 88R is located on the central axis of the drive wheel, which serves as the rear wheel of the track 60, on the outer side surface of the drive wheel.

[0115] The controller 11 extracts the portion corresponding to the lower walking body 21 from the point cloud data obtained by the camera device 5 and projects it onto a pre-set projection surface. The projection surface is, for example, a vertical plane. The controller 11 extracts data related to the front side mark 88F and the rear side mark 88R set on the same track 60, and based on this data, calculates the straight line connecting the front side mark 88F and the rear side mark 88R to each other, which is the track centerline 91.

[0116] The posture calculation unit calculates the angle between the track centerline 91 and the horizontal plane, i.e., the tilt angle. The deviation calculation unit calculates the deviation between the tilt angle calculated by the posture calculation unit (i.e., the calculated tilt angle) and the tilt angle detected by the tilt angle sensor 55 (i.e., the detected tilt angle), thereby enabling the evaluation of the reliability of the detection value of the tilt angle sensor 55.

[0117] Alternatively, the shape of the lower walking body 21 can be determined based on the point cloud data without using the front side mark 88F and the rear side mark 88R, and the track centerline 91 can be calculated. The shape information stored in the storage device 13 is used to determine the shape.

[0118] Next, refer to Figure 13 The flowchart is used to illustrate the processing performed in this fifth embodiment.

[0119] The controller 11 instructs the camera device 5 to acquire reference point cloud data (step S81). The reference point cloud data is the point cloud data of the engineering machinery 20 at the reference position. Then, the controller 11 does not cause the lower walking body 21 to perform a walking action, but causes the upper rotating body 22 to start moving (step S82).

[0120] The controller 11 obtains the detection value from the tilt angle sensor 55 (step S83). The detection value is the tilt angle of the lower walking body 21 relative to the horizontal plane in the current posture of the engineering machinery 20.

[0121] The controller 11 instructs the camera device 5 to acquire the current point cloud data (step S84). The current point cloud data is the point cloud data of the engineering machinery 20 at the current location. The controller 11 calculates the difference between the reference point cloud data and the current point cloud data, thereby extracting the portion of the current point cloud data corresponding to the lower walking body 21 (step S85). This extraction may also be accompanied by clustering or noise reduction.

[0122] The controller 11 projects the point cloud data extracted as described above onto a pre-set projection surface (step S86). The projection surface is, for example, a vertical plane. The controller 11 determines the positions of the front mark 88F and the rear mark 88R (step S87). The controller 11 repeats the processing after step S83 until all positions of the front mark 88F and the rear mark 88R are determined (step S88 is "No"). On the other hand, after determining all positions of the marks 88F and 88R (step S88 is "Yes"), the controller 11 calculates the track centerline 91 that connects the front mark 88F and the rear mark 88R to each other (step S89), and calculates the angle between the track centerline 91 and the horizontal plane (step S90).

[0123] The controller 11 determines whether the tilt angle calculated as described above, i.e., the calculated tilt angle, is consistent with the tilt angle detected by the tilt angle sensor 55, i.e., the detected tilt angle (step S91). If it is determined that the calculated tilt angle is consistent with the detected tilt angle (step S91 is "yes"), the controller 11 repeats the processing after step S83. On the other hand, if it is determined that the calculated tilt angle is not consistent with the detected tilt angle (step S91 is "no"), the controller 11 sends an error message to the portable terminal 7 (step S92).

[0124] When using the front side marker 88F and the rear side marker 88R, the action of extracting the part corresponding to the lower walking body 21 from the point cloud data can also be omitted, that is, in Figure 13In this process, steps S81 and S84 to S86 can also be omitted.

[0125] According to the operating system 401 described above, the tilt angle sensor 55, which constitutes the posture detector, detects the tilt angle of the lower walking body 21 relative to the horizontal plane as the detection object, i.e., the posture of the lower walking body 21. The posture calculation unit calculates the tilt angle of the lower walking body 21 relative to the horizontal plane based on the acquired three-dimensional information. The deviation calculation unit calculates the deviation between the tilt angle detected by the tilt angle sensor 55 and the tilt angle calculated based on the three-dimensional information. Therefore, the reliability of the detection value of the tilt angle sensor 55 can be evaluated based on the magnitude of the deviation.

[0126] The embodiments described above are specific examples and do not imply any limitation on the scope of the present invention. The specific structures disclosed may be appropriately modified. Furthermore, the effects and benefits of the present invention are not limited to those described in the embodiments.

[0127] As described above, a work system is provided capable of monitoring detection values ​​related to the posture of a target object. The work system includes a lower traveling body, an upper slewing body, a boom, a stick, a remote attachment, a posture detector, an information acquisition device, a posture calculation unit, and a deviation calculation unit. The upper slewing body is rotatably mounted on the lower traveling body. The boom is rotatably connected to the upper slewing body. The stick is rotatably connected to the boom. The remote attachment is rotatably connected to the stick. The posture detector detects the posture of at least one target object, selected from the lower traveling body, the upper slewing body, the boom, the stick, and the remote attachment. The information acquisition device acquires three-dimensional information of the target object. The posture calculation unit calculates the posture of the target object based on the three-dimensional information acquired by the information acquisition device. The deviation calculation unit calculates the deviation between the posture detected by the posture detector and the posture calculated by the posture calculation unit.

[0128] The operating system calculates the deviation between the posture detected by the posture detector and the posture calculated based on the three-dimensional information. Based on the magnitude of the deviation, it can evaluate the reliability of the detection value detected by the posture detector. Thus, for example, it can monitor whether the detection value detected by the posture detector has deviated from the allowable range set for the detection value.

[0129] When the at least one detection object includes the boom, the stick, and the distal attachment, preferably: the posture detector detects the tilt angle of the detection object relative to the horizontal plane based on the posture of the detection object, and the posture calculation unit calculates the tilt angle of the detection object relative to the horizontal plane or the angle corresponding to the tilt angle based on the posture of the detection object.

[0130] For example, when the at least one detection object includes the upper rotating body, it is preferable that: the posture detector detects the rotation angle of the upper rotating body from the reference line based on the posture of the upper rotating body, and the posture calculation unit calculates the rotation angle of the upper rotating body from the reference line based on the posture of the upper rotating body.

[0131] For example, when the at least one detection object includes the lower walking body, it is preferable that: the posture detector detects the tilt angle of the lower walking body relative to the horizontal plane based on the posture of the lower walking body, and the posture calculation unit calculates the tilt angle of the detection object relative to the horizontal plane based on the posture of the lower walking body.

[0132] The operating system preferably also includes a mark attached to the object being detected, and the information acquisition device acquires the three-dimensional information of the mark.

[0133] The operating system preferably further includes a storage device for storing shape information about the shape of the object being detected. The posture calculation unit uses the three-dimensional information and the shape information stored in the storage device to calculate the posture of the object being detected.

Claims

1. An operating system, characterized in that... include: Lower walking body; The upper rotating body is rotatably mounted on the lower traveling body; The boom is rotatably connected to the upper rotating body; The stick is rotatably connected to the boom; A remote auxiliary device is rotatably connected to the boom; A posture detector detects the posture of at least one object selected from the lower walking body, the upper rotating body, the boom, the stick, and the distal attachment. The information acquisition device acquires the three-dimensional information of the object being detected independently of the posture detector; The posture calculation unit calculates the posture of the detected object based on the three-dimensional information acquired by the information acquisition device. as well as The deviation calculation unit calculates the deviation between the posture detected by the posture detector and the posture calculated by the posture calculation unit.

2. The operating system according to claim 1, characterized in that, The at least one detection object includes the boom, the stick, and the distal attachment. The posture detector detects the tilt angle of the object relative to the horizontal plane based on the object's posture. The posture calculation unit calculates the tilt angle of the detected object relative to the horizontal plane or the angle corresponding to the tilt angle, based on the posture of the detected object.

3. The operating system according to claim 1, characterized in that, The at least one detection object includes the upper rotating body. The posture detector detects the rotation angle of the upper rotating body, calculated from the baseline, based on the posture of the upper rotating body. The posture calculation unit calculates the rotation angle of the upper rotating body from the baseline based on the posture of the upper rotating body.

4. The operating system according to claim 1, characterized in that, The at least one detection object includes the lower walking body. The posture detector detects the tilt angle of the lower walking body relative to the horizontal plane based on the posture of the lower walking body. The posture calculation unit calculates the tilt angle of the detected object relative to the horizontal plane based on the posture of the lower walking body.

5. The operating system according to any one of claims 1 to 4, characterized in that... Also includes: A mark is attached to the object being detected. The information acquisition device acquires the three-dimensional information of the mark.

6. The operating system according to any one of claims 1 to 4, characterized in that... Also includes: The storage device stores shape information about the shape of the detected object. The posture calculation unit uses the three-dimensional information and the shape information stored in the storage device to calculate the posture of the detected object.

Images