Real machine status monitoring system and real machine status monitoring method
Through the real machine status monitoring system, it is solved to identify the unstability of the excavator and output the continuous change information, which is difficult to judge the floating body of the excavator with high accuracy in the prior art, and improves the operator's identification ability and Work efficiency.
Patent Information
- Application Number
- CN202180040200.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-06-04
- Filing Date
- 2021-05-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-05-20
AI Technical Summary
The prior art is difficult to judge the floating condition of the lower driving body of the excavator with high accuracy, which makes it difficult for the operator to accurately grasp the degree of movement of the boom and bucket, which in turn affects the operation efficiency.
A real-machine state monitoring system is designed to identify the base posture and external forces acting on the work part through the real-machine state recognition element. The unstability evaluation element is used to evaluate the instability as a continuous variable, and the output control element is continuously The variable outputs instability information, including images, sounds and vibrations, helping the operator identify instability with high accuracy.
It improves the operator's accuracy of the excavator instability, avoids the instability of the substrate, ensures that the operation mechanism is within the allowable range, and improves the working efficiency.
Smart Images

Figure CN115698438B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a system for monitoring the state of a working machine (actual machine). Background Art
[0002] There has been proposed a shovel technique in which the instability of the shovel is presented to the operator, thereby enabling accurate determination of an operation not intended by the operator (for example, see Patent Document 1).
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-112783 Summary of the invention
[0006] Problems to be solved by the invention
[0007] However, since the instability is presented as a discrete variable represented by three regions, for example, even if the operator refers to the instability, it is difficult to accurately grasp the extent to which the lower traveling body of the excavator will float when the boom, arm, and bucket are moved. Therefore, even if the possibility of the lower traveling body floating is low, that is, the possibility of the excavator becoming unstable is low, the operator will stop further operation of the boom, etc., and there is a possibility that the working efficiency will be reduced.
[0008] Therefore, an object of the present invention is to provide a system or the like that can improve the accuracy of information on the instability of a working machine such as a shovel provided to an operator of the working machine.
[0009] Means for solving problems
[0010] The real machine status monitoring system of the present invention is a real machine status monitoring system for transmitting the status of the working machine to the operator of the working machine and outputting the status of the working machine to an information output device, wherein the working machine has a base, a working mechanism extending from the base, and a working part installed at the top end of the working mechanism.
[0011] The real machine status monitoring system comprises:
[0012] A real machine state recognition element for recognizing the posture of the base and the external force acting on the working part;
[0013] an instability evaluation element that evaluates the instability of the base as a continuous variable based on the posture of the base recognized by the actual machine state recognition element and the external force acting on the working unit; and
[0014] An output control element causes instability information to be output to the information output device in a manner in which the output mode of the instability information changes continuously in response to the continuous change of the instability, wherein the instability information is information indicating the instability of the substrate evaluated by the instability evaluation element.
[0015] According to the real machine state monitoring system having such a configuration, the instability information indicating the instability of the matrix whose numerical value is evaluated as a continuous variable is output to the information output device in such a manner that its output form changes continuously in accordance with the continuous change of the instability.
[0016] Therefore, the operator of the working machine can accurately recognize how close the current instability of the base is to the threshold indicating that the base will become unstable, thereby preventing the base from becoming unstable and allowing the operator to accurately recognize the allowable range of operation of the working mechanism, etc.
[0017] In order to allow the operator to recognize the instability through his vision, a diagram may be outputted in the following manner: the output control element causes the diagram representing the instability of the substrate to be outputted to the image output device constituting the information output device in a manner that the diagram continuously changes in form based on the threshold value of the instability. In order to allow the operator to recognize the instability through his hearing, a sound may be outputted in the following manner: the output control element causes the sound representing the instability of the substrate to be outputted to the sound output device constituting the information output device in a manner that the volume, frequency, or combination of volume and frequency of the sound continuously changes. In order to allow the operator to recognize the instability through his tactile sense, a vibration may be outputted in the following manner: the output control element causes the vibration representing the instability of the substrate to be outputted to the vibration output device constituting the information output device in a manner that the amplitude, vibration frequency, or combination of amplitude and vibration frequency of the vibration continuously changes.
[0018] The real machine status monitoring system of the present invention may also be composed of the following remote operation assistance server: the remote operation assistance server assists the remote operation device in remotely operating the working machine based on the communication with the working machine and the remote operation device for remotely operating the working machine. The information output device may also be composed of the remote operation device for remotely operating the working machine.
[0019] Brief description of the attached figure
[0020] Figure 1 This is an explanatory diagram regarding the configuration of a real machine state monitoring system as one embodiment of the present invention.
[0021] Figure 2This is an explanatory diagram regarding the structure of the remote control device.
[0022] Figure 3 This is an explanatory diagram relating to the structure of a working machine.
[0023] Figure 4 This is an explanatory diagram of functions related to the remote operation system.
[0024] Figure 5 This is an explanatory diagram related to the functions of the real machine status monitoring system.
[0025] Figure 6 This is an explanatory diagram related to the work environment image.
[0026] Figure 7 It is an explanatory diagram concerning the first instability evaluation method when the ground surface is flat.
[0027] Figure 8 It is an explanatory diagram concerning a first instability evaluation method when the ground is inclined.
[0028] Fig. 9 It is an explanatory diagram concerning the second instability evaluation method when the ground surface is flat.
[0029] Fig.10 It is an explanatory diagram of the second instability evaluation method when the ground is inclined.
[0030] Fig.11 It is an explanatory diagram concerning the third instability evaluation method when the ground surface is flat.
[0031] Fig.12 It is an explanatory diagram concerning the third instability evaluation method when the ground is inclined.
[0032] Fig.13 This is an explanatory diagram regarding the output method of the instability information. DETAILED DESCRIPTION
[0033] (Composition of remote operating system)
[0034] Figure 1 The real machine state monitoring system 110 shown as one embodiment of the present invention is composed of a remote operation support server 10 for supporting the remote operation of the working machine 40 by the remote operation device 20. The remote operation support server 10 and the remote operation device 20 are configured to communicate with each other via a first network. The remote operation support server 10 and the working machine 40 are configured to communicate with each other via a second network. The first network and the second network may be networks with a common communication standard or the like, or networks with different communication standards or the like.
[0035] (Configuration of remote operation support server)
[0036] The remote operation support server 10 includes a database 102, a real machine state monitoring system 110, a first auxiliary processing element 121, and a second auxiliary processing element 122. The database 102 stores and saves captured image data, etc. The database 102 may also be constituted by a database server that is independent of the remote operation support server 10. Each auxiliary processing element is constituted by an operation processing device (a single-core processor or a multi-core processor, or a processor core constituting the processor), and reads necessary data and software from a storage device such as a memory, and performs the operation processing described later according to the software on the data.
[0037] (Structure of the real machine status monitoring system)
[0038] The real machine state monitoring system 110 includes a real machine state recognition element 111, an instability evaluation element 112, and an output control element 114. Each element is composed of an operation processing device (a single-core processor or a multi-core processor, or a processor core constituting the processor), and reads necessary data and software from a storage device such as a memory, and performs operation processing described later in accordance with the software based on the data.
[0039] (Structure of remote control device)
[0040] The remote operation device 20 includes a remote control device 200, a remote input interface 210, and a remote output interface 220. The remote control device 200 is composed of a processing device (a single-core processor or a multi-core processor, or a processor core constituting the processor), and reads necessary data and software from a storage device such as a memory, and performs processing in accordance with the software on the data.
[0041] The remote input interface 210 includes a remote operating mechanism 211. The remote output interface 220 includes a remote image output device 221, a sound output device 222, a vibration output device 223, and a remote wireless communication device 224. The remote image output device 221, the sound output device 222, and the vibration output device 223 each constitute an "information output device". Some of the remote image output device 221, the sound output device 222, and the vibration output device 223 may be omitted.
[0042] The remote operating mechanism 211 includes an operating device for traveling, an operating device for rotating, an operating device for an arm, an operating device for an arm, and an operating device for a bucket. Each operating device has an operating lever that receives a rotation operation. The operating lever (travel lever) of the operating device for traveling is operated to move the lower traveling body 410 of the working machine 40. The traveling lever may also serve as a traveling pedal. For example, a traveling pedal fixed to the base or the lower end of the traveling lever may also be provided. The operating lever (rotation lever) of the operating device for rotating is operated to actuate the hydraulic rotating motor of the rotating mechanism 430 constituting the working machine 40. The operating lever (boom lever) of the operating device for the arm is operated to actuate the boom cylinder 442 of the working machine 40. The operating lever (arm lever) of the operating device for the arm is operated to actuate the arm cylinder 444 of the working machine 40. The operating lever (bucket lever) of the operating device for the bucket is operated to actuate the bucket cylinder 446 of the working machine 40.
[0043] like Figure 2 As shown, the operating levers constituting the remote operating mechanism 211 are arranged around a seat St for the operator to sit on, for example. The seat St may be a high-back chair with armrests, a low-back chair without a headrest, or a chair without a backrest, etc., and may be any seat portion in which the operator can sit.
[0044] A pair of left and right travel levers 2110 corresponding to the left and right crawlers are arranged side by side in front of the seat St. One operating lever can also serve as multiple operating levers. Figure 2 The left operating lever 2111 provided in front of the left frame of the seat St shown in the figure functions as a boom lever when operated in the front-rear direction, and functions as a swing lever when operated in the left-right direction. Figure 2 The right operating lever 2112 provided in front of the right frame of the seat St shown functions as a boom lever when operated in the front-rear direction, and functions as a bucket lever when operated in the left-right direction. The lever mode can be arbitrarily changed according to the operator's operation instruction.
[0045] For example, Figure 2 As shown, the remote image output device 221 is composed of a central remote image output device 2210, a left remote image output device 2211, and a right remote image output device 2212, each of which has a substantially rectangular screen and is disposed in front of, obliquely in front of, and obliquely in front of the seat St. The shapes and sizes of the screens (image display areas) of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be the same or different.
[0046] like Figure 2 As shown, the right edge of the left remote image output device 2211 is adjacent to the left edge of the central remote image output device 2210 in such a manner that the inclination angle between the screen of the central remote image output device 2210 and the screen of the left remote image output device 2211 is θ1 (for example, 120°≤θ1≤150°). Figure 2 As shown, the left edge of the right remote image output device 2212 is adjacent to the right edge of the central remote image output device 2210 in such a manner that the inclination angle between the screen of the central remote image output device 2210 and the screen of the right remote image output device 2212 is θ2 (for example, 120°≤θ2≤150°). The inclination angles θ1 and θ2 may be the same angle or different angles.
[0047] The screens of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be parallel to the vertical direction or may be inclined to the vertical direction. At least one of the central remote image output device 2210, the left remote image output device 2211, and the right remote image output device 2212 may be composed of a plurality of image output devices divided into a plurality of parts. For example, the central remote image output device 2210 may be composed of a pair of image output devices adjacent to each other in the upper and lower directions and having a substantially rectangular screen.
[0048] The sound output device 222 is composed of one or more speakers, for example, Figure 2 As shown, the sound output device 222 is composed of a central sound output device 2220 disposed behind the seat St, a left sound output device 2221 disposed behind the left armrest, and a right sound output device 2222 disposed behind the right armrest. The specifications of the central sound output device 2220, the left sound output device 2221, and the right sound output device 2222 may be the same or different.
[0049] The vibration output device 223 is composed of a piezoelectric element and is disposed or embedded in one or more parts of the seat St. The vibration output device 223 vibrates, and the operator sitting on the seat St can recognize the vibration form through the sense of touch. The vibration output device 223 can also be provided on all parts that the operator can recognize by touch, such as the remote operating lever constituting the remote operating mechanism 211.
[0050] (Structure of operating machinery)
[0051] The working machine 40 includes a real machine control device 400, a real machine input interface 41, a real machine output interface 42, and a working mechanism 440. The real machine control device 400 is composed of a computing device (a single-core processor or a multi-core processor, or a processor core constituting the processor), reads necessary data and software from a storage device such as a memory, and performs computing processing in accordance with the software on the data.
[0052] The working machine 40 is, for example, a hydraulic, electric, or hybrid-driven crawler excavator (construction machine), such as Figure 3 As shown, it has a crawler-type lower traveling body 410 and an upper revolving body 420 rotatably mounted on the lower traveling body 410 via a revolving mechanism 430. A cab 424 (driver's cab) is provided on the front left side of the upper revolving body 420. A working mechanism 440 is provided in the front center of the upper revolving body 420.
[0053] The real machine input interface 41 includes a real machine operating mechanism 411, a real machine photographing device 412, and a real machine state sensor group 414. The real machine operating mechanism 411 includes a plurality of operating levers arranged around a seat arranged inside the cab 424 in the same manner as the remote operating mechanism 211. A driving mechanism or a robot is provided in the cab 424 to receive a signal corresponding to the operating form of the remote operating lever and to operate the real machine operating lever based on the received signal. The real machine photographing device 412 is, for example, provided inside the cab 424 to photograph an environment including at least a portion of the operating mechanism 440 through the front window and a pair of left and right side windows. The front window (or window frame) and part or all of the side windows may also be omitted. The real machine state sensor group 414 is composed of an angle sensor, a rotation angle sensor, an external force sensor, a three-axis acceleration sensor, etc., wherein,
[0054] The angle sensor is used to measure the rotation angle (lifting and lowering angle) of the boom 441 relative to the upper rotating body 420, the rotation angle of the arm 443 relative to the boom 441, and the rotation angle of the bucket 445 relative to the arm 443; the rotation angle sensor is used to measure the rotation angle of the upper rotating body 420 relative to the lower traveling body 410; the external force sensor is used to measure the external force acting on the bucket 445; the 3-axis acceleration sensor is used to measure the 3-axis acceleration acting on the upper rotating body 420.
[0055] The real machine output interface 42 includes a real machine image output device 421 and a real machine wireless communication device 422. The real machine image output device 421 is disposed, for example, inside the cab 424 and near the front window (see Figure 6 and Fig. 9 ). The actual machine image output device 421 may also be omitted.
[0056] The working mechanism 440 as a working mechanism includes a boom 441 which is raisably mounted on the upper rotating body 420, an arm 443 which is rotatably connected to the top of the boom 441, and a bucket 445 which is rotatably connected to the top of the arm 443. The working mechanism 440 is equipped with a boom cylinder 442, an arm cylinder 444, and a bucket cylinder 446 which are composed of telescopic hydraulic cylinders. As the working part, various attachments such as a hydraulic shear (nibbler), a cutter, and a magnet can be used in addition to the bucket 445.
[0057] The boom cylinder 442 is interposed between the boom 441 and the upper slewing body 420, so that it can be extended and retracted by receiving the supply of hydraulic oil, thereby rotating the boom 441 in the lifting direction. The arm cylinder 444 is interposed between the arm 443 and the boom 441, so that it can be extended and retracted by receiving the supply of hydraulic oil, thereby rotating the arm 443 around the horizontal axis relative to the boom 441. The bucket cylinder 446 is interposed between the bucket 445 and the arm 443, so that it can be extended and retracted by receiving the supply of hydraulic oil, thereby rotating the bucket 445 around the horizontal axis relative to the arm 443.
[0058] (First function)
[0059] use Figure 4 The flowchart shown illustrates the first function of the remote operation support system composed of the remote operation support server 10, the remote operation device 20 and the working machine 40 having the above-mentioned structure. In the flowchart, the block "C●" is used to simplify the description, and refers to the conditional branch of sending and / or receiving data, and the processing in the branch direction is executed based on the sending and / or receiving of the data.
[0060] In the remote operation device 20, it is determined whether there is a designated operation ( Figure 4 / Step (STEP) 210). "Specified operation" refers to, for example, an operation such as clicking performed by the operator on the remote input interface 210 in order to specify the working machine 40 to be remotely operated. Figure 4 / Step (STEP) 210 ... No (NO)), a series of processing ends. On the other hand, if the determination result is positive ( Figure 4 / Step (STEP) 210 ... Yes (YES)), send an environment confirmation request to the remote operation assistance server 10 through the remote wireless communication device 224 ( Figure 4 / Step (STEP) 212).
[0061] In the remote operation support server 10, when receiving the environment confirmation request, the first auxiliary processing component 121 sends the environment confirmation request to the corresponding working machine 40 ( Figure 4 / C10).
[0062] In the working machine 40, when an environment confirmation request is received via the real machine wireless communication device 422 ( Figure 4 / C40), the real machine control device 400 obtains the captured image through the real machine capturing device 412 ( Figure 4 / Step (STEP) 410). The real machine control device 400 transmits the captured image data ( Figure 4 / Step (STEP) 412).
[0063] In the remote operation assistance server 10, when the first auxiliary processing component 121 receives the camera image data ( Figure 4 / C11), the second auxiliary processing component 122 sends the environment image data corresponding to the camera image to the remote control device 20 ( Figure 4 / Step (STEP) 110). The environment image data includes, in addition to the captured image data itself, image data representing a simulated environment image generated based on the captured image.
[0064] In the remote operation device 20, when the environment image data is received through the remote wireless communication device 224 ( Figure 4 / C21), outputs the environment image corresponding to the environment image data to the remote image output device 221 ( Figure 4 / Step (STEP) 214).
[0065] Thus, for example, Figure 6 As shown, an environment image in which a boom 441 , an arm 443 , and a bucket 445 , which are part of the working mechanism 440 , are reflected is output to the remote image output device 221 .
[0066] In the remote operation device 20, the operation mode of the remote operation mechanism 211 is recognized by the remote control device 200 ( Figure 4 / Step (STEP) 216), and a remote operation instruction corresponding to the operation mode is sent to the remote operation assistance server 10 through the remote wireless communication device 224 ( Figure 4 / Step (STEP) 218).
[0067] In the remote operation support server 10, when the second auxiliary processing component 122 receives the remote operation command, the first auxiliary processing component 121 sends the remote operation command to the working machine 40 ( Figure 4 / C12).
[0068] In the working machine 40, when the real machine control device 400 receives an operation instruction via the real machine wireless communication device 422 ( Figure 4 / C41), controlling the action of the operating mechanism 440, etc. ( Figure 4 / Step (STEP) 414). For example, an operation is performed in which the soil in front of the working machine 40 is scooped up by the bucket 445 and the soil is dumped from the bucket 445 after the upper rotating body 420 is rotated.
[0069] use Figure 5 The flowchart shown is used to explain the second function of the remote operation support system having the above configuration (mainly the function of the real machine status monitoring system 110 constituted by the remote operation support server 10). In the flowchart, the "C●" block is used to simplify the description and refers to the conditional branch for executing the processing in the branch direction based on the sending and / or receiving of data.
[0070] In the working machine 40, the real machine control device 400 obtains the real machine state data ( Figure 5 / Step (STEP) 420). The operating state of the working machine 40 includes: the rotation angle (lifting and lowering angle) of the boom 441 relative to the upper rotating body 420; the rotation angle of the bucket 443 relative to the boom 441; the rotation angle of the bucket 445 relative to the bucket 443; the rotation angle of the upper rotating body 420 relative to the lower traveling body 410; and the external force F acting on the bucket 445, etc.
[0071] The real machine control device 400 sends the real machine status data ( Figure 5 / Step (STEP) 422).
[0072] In the remote operation assistance server 10, when receiving the real machine status data ( Figure 5 / C14), the actual machine state recognition element 111 recognizes the state of the working machine 40 based on the actual machine state data ( Figure 5 / Step (STEP) 120).
[0073] Specifically, the time series of the external force F acting on the bucket 445 is identified. The external force F may be identified based on the hydraulic pressure of at least one of the boom cylinder 442 , the arm cylinder 444 , and the bucket cylinder 446 .
[0074] Furthermore, the coordinate values of the center of gravity P0, the floating fulcrum P1, and the external force application point P2 (the top point of the bucket 445) of the base body composed of the lower traveling body 410 and the upper rotating body 420 in the real machine coordinate system are identified, wherein the position and posture of the real machine coordinate system are fixed relative to the working machine 40. The coordinate value of the center of gravity P0 of the base body in the real machine coordinate system is pre-registered in the database 102 in accordance with the type and / or specification of the working machine 40. The coordinate value of the floating fulcrum P1 in the real machine coordinate system is identified based on the rotation angle of the upper rotating body 420 relative to the lower traveling body 410 (refer to the floating fulcrum T1 f in Patent Document 1). The external force acting point P2 in the real machine coordinate system is geometrically identified based on the rotation angle (lifting and lowering angle) of the boom 441 relative to the upper revolving body 420, the rotation angle of the arm 443 relative to the boom 441, and the rotation angle of the bucket 445 relative to the arm 443, and the link lengths of the boom 441, the arm 443, and the bucket 445. The link length of the boom 441 (the distance from the joint mechanism on the upper revolving body 420 side to the joint mechanism on the arm 443 side), the link length of the arm 443 (the distance from the joint mechanism on the boom 441 side to the joint mechanism on the bucket 445 side), and the link length of the bucket 445 (the distance from the joint mechanism on the arm 443 side to the top end of the bucket 445) are pre-registered in the database 102 according to the type and / or specification of the working machine 40.
[0075] The actual machine state recognition element 111 determines whether the working machine 40 is using the bucket 445 (working unit) to perform a designated work ( Figure 5 / Step (STEP) 121). For example, when the designated work is an excavation work, whether the working machine 40 is performing the designated work is identified based on whether the external force F acting on the bucket 445 repeatedly increases and decreases.
[0076] If the determination result is negative ( Figure 5 / Step (STEP) 121 ... No (NO)), the series of processing in this control cycle ends. On the other hand, if the judgment result is positive ( Figure 5 / Step (STEP) 121 ... Yes (YES)), based on the actual machine state recognized by the actual machine state recognition element 111, the first instability I s1, the second instability I s2 and the third instability I s3 of the upper rotating body 420 (base) of the working machine 40 are evaluated by the instability evaluation element 112 ( Figure 5 / Step (STEP) 122).
[0077] The first instability Is1 is defined from the viewpoint that the lower traveling structure 410 (base) of the working machine 40 becomes unstable due to the base being lifted off the ground. Figure 7 The external force F and the angle θ between the external force vector and the horizontal plane are shown f , the distance l between the center of gravity P0 of the base and the floating support point P1 located behind the center of gravity P0 g , the distance l between the floating fulcrum P1 and the external force application point P2 t , the angle θ between the line segment P0-P1 (or the plane containing the line segment) and the horizontal plane g , the angle θ between the line segment P1-P2 (or the plane containing the line segment) and the horizontal plane t , the weight m of the substrate and the gravitational acceleration g, and the first instability I s1 is obtained according to the relation (11). That is, the continuous variable l t , F, θ f ,θ t , l g and θ g As the main variable, a first degree of instability Is1 is defined as a continuous function or a continuous subordinate variable.
[0078] I s1=l t ·Fs in(θ t +θ f ) / l g mgcosθ g … (11).
[0079] like Figure 8 As shown, the inclination angle at the ground is θ m In the case of , the first instability I s1 is defined by the relation (21). The inclination angle θ of the ground m The measurement can be performed based on the output signal of the three-axis acceleration sensor for measuring the three-axis acceleration acting on the upper rotating body 420 constituting the actual machine state sensor group 414 .
[0080] I s1=l t ·Fs in(θ t +θ f ) / l g mgcos(θ g +θ m ) … (twenty one).
[0081] The second instability Is2 is an instability defined from the viewpoint that the lower traveling structure 410 (base) of the working machine 40 becomes unstable due to the base being lifted off the ground. Fig. 9 The external force F and the angle θ between the external force vector and the horizontal plane are shown f , the distance l between the center of gravity P0 of the base and the floating fulcrum P1 located in front of the center of gravity P0 fg , the distance l between the floating fulcrum P1 and the external force application point P2 ft , the angle θ between the line segment P0-P1 (or the plane containing the line segment) and the horizontal plane fg , the angle θ between the line segment P1-P2 (or the plane containing the line segment) and the horizontal plane ft , the weight m of the substrate and the gravitational acceleration g, and the second instability I s2 is obtained according to the relation (12). That is, the continuous variable l is defined as ft , F, θ f ,θ ft , l fg and θ fg The second instability I s2 of the primary variable as a continuous function or a continuous secondary variable.
[0082] I s2=l ft ·Fs in(θ f -θ ft ) / l fg mgcosθ fg … (12).
[0083] like Fig.10 As shown, the inclination angle on the ground is θ m In the case of , the second instability I s2 is defined by equation (22).
[0084] I s2=l ft ·Fs in(θ f -θ ft ) / l fg mgcos(θ fg +θ m ) … (twenty two).
[0085] The third instability Is3 is defined from the viewpoint that the lower traveling structure 410 (base) of the working machine 40 becomes unstable due to sliding relative to the ground. Fig.11 The external force F, the angle θf formed by the external force vector and the horizontal plane, the weight m of the substrate, the gravitational acceleration g, and the static friction coefficient μ (or dynamic friction coefficient) between the substrate and the ground are used to calculate the third instability Is3 according to the relationship (13). That is, the continuous variables F and θ are defined as fThe third instability Is3 as a continuous function of the main variable or a continuous dependent variable. It should be noted that the friction coefficient μ uses a standard value of the work site, but a value different from the above standard value can also be used according to different meteorological conditions (precipitation, temperature, humidity, etc.) and / or soil conditions and foundation conditions (sand, clay, gravel, sand, rubble, etc.).
[0086] I s3=Fcosθ f / μmg… (13).
[0087] like Fig.12 As shown, the inclination angle on the ground is θ m In the case of , the third instability Is3 is defined by equation (23).
[0088] I s3=Fcosθ f / (μmgcosθ m -mgs i nθ m ) … (twenty three).
[0089] The output control element 114 determines which instability value is the largest among the first instability Is1, the second instability Is2 and the third instability Is3. Figure 5 / Step (STEP) 124).
[0090] In the case where the first instability Is1 is determined to be the maximum instability Ismax ( Figure 5 / Step (STEP) 124 ... 1), the output control element 114 generates first instability information ( Figure 5 / Step (STEP) 125). When it is determined that the second instability Is2 is the maximum instability Ismax ( Figure 5 / Step (STEP) 124 ... 2), the output control element 114 generates the second instability information ( Figure 5 / Step (STEP) 126). When it is determined that the third instability Is3 is the maximum instability Ismax ( Figure 5 / Step (STEP) 124 ... 3), the output control element 114 generates the third instability information ( Figure 5 / Step (STEP) 127). Then, the output control element 114 sends the first instability information, the second instability information or the third instability information to the remote control device 20 ( Figure 5 / Step (STEP) 128).
[0091] In the remote operation device 20, when the first instability information, the second instability information or the third instability information is received through the remote wireless communication device 224 ( Figure 5 / C22), the instability information is output to the remote image output device 221 through the remote control device 200 ( Figure 5 / Step (STEP) 224).
[0092] Thus, for example, Fig.13 As shown, in the remote image output device 221, a diagram f(x) or a bar graph showing the length change from the lower edge of the window f according to the degree of instability in the window f is outputted in an overlapping manner with the environment image. The size of the diagram f(x) is defined by an increasing function such as a linear function, an exponential function, a logarithmic function, etc., with the degree of instability as a variable. The scale at the upper edge of the window f or a position lower than the upper edge is a threshold value f indicating the predicted occurrence of the following situations: th : When the first instability I s1, the second instability I s2 or the third instability I s3 reaches the threshold value f th In the case of a substrate, the substrate floats from the ground or the substrate slides relative to the ground.
[0093] The shape of the diagram f(x) may be a circle, a sector, a rhombus, or any other shape other than a rectangle. The diagram f(x) may also be output in the following manner: the size, shape, color (brightness, saturation, and hue), or pattern of the diagram f(x), or any combination thereof, may be changed continuously in accordance with the continuous change of the instability I s1, I s2, and I s3.
[0094] (Effect)
[0095] According to the real machine state monitoring system 110 constituting the remote operation support system having the above-mentioned configuration, the output mode of the instability information is output to the remote image output device 221 (information output device) in a manner that continuously changes in accordance with the continuous changes of the instabilities Is1, Is2, and Is3 (see Figure 5 / Step (STEP) 122 →…→STEP 224, Fig. 9 ). The instability information indicates the instabilities I s1 , I s2 , and I s3 of the base body (the lower traveling body 410 and the upper rotating body 420 ) whose numerical values are evaluated as continuous variables.
[0096] Therefore, the operator of the working machine 40 can accurately recognize how close the current instability of the base is to the threshold indicating that the base is becoming unstable, and can further recognize the permissible range for operating the working mechanism, etc. while avoiding instability of the base.
[0097] By confirming the instability information (first instability information) indicating the first instability output by the information output device, the operator of the working machine can recognize with high accuracy how close the first instability of the base is to the threshold value (first threshold value), thereby preventing the base from becoming unstable due to floating from the ground with the floating fulcrum P1 behind the center of gravity P0 as the origin, and at the same time, the operator of the working machine can recognize with high accuracy the allowable range of the action of the working mechanism, etc. (refer to Figure 7 , Figure 8 and Fig.13 ). Similarly, by confirming the instability information (second instability information) indicating the second instability output by the information output device, the operator of the working machine can accurately recognize the degree of proximity of the second instability of the base to the threshold value (second threshold value), thereby preventing the base from becoming unstable due to floating from the ground with the floating fulcrum P1 in front of the center of gravity P0 as the origin, and at the same time, the operator of the working machine can accurately recognize the allowable range of the action of the working mechanism, etc. (refer to Fig. 9 , Fig.10 and Fig.13 By confirming the instability information (third instability information) indicating the third instability output by the information output device, the operator of the working machine can accurately recognize the degree of proximity of the third instability of the base to the threshold value (third threshold value), thereby preventing the base from becoming unstable due to sliding relative to the ground, and can accurately recognize the allowable range of the action of the working mechanism, etc. (refer to Fig.11 , Fig.12 and Fig.13 ).
[0098] In addition, only when the working machine 40 is performing excavation work as a designated work while applying force to the working object (sand, rubble, etc.) using the bucket 445 (working unit), that is, when there is a possibility that the base may become unstable, the instability information is transmitted to the operator through the information output device (see Figure 5 / STEP121…YES→…→STEP224). Thus, the usefulness of the instability information is improved.
[0099] (Other embodiments of the present invention)
[0100] In the above embodiment, the real machine state monitoring system 110 is constituted by the remote operation support server 10, but as another embodiment, the real machine state monitoring system 110 may be constituted by the remote operation device 20 and / or the working machine 40. That is, the remote operation device 20 and / or the working machine 40 may also have the functions of the real machine state recognition element 111, the instability evaluation element 112, and the output control element 114.
[0101] In the above embodiment, the instability information is outputted through the remote image output device 221, but the instability information may be outputted through the sound output device 222 and / or the vibration output device 223 in addition to or instead of the remote image output device 221. The sound indicating the instability of the substrate may be outputted through the sound output device 222 in such a manner that the volume, frequency, or a combination of the volume and frequency of the sound continuously changes. The vibration indicating the instability of the substrate may be outputted through the vibration output device 223 in such a manner that the amplitude, frequency, or a combination of the amplitude and frequency of the vibration continuously changes.
[0102] In the above embodiment, the first instability Is1, the second instability Is2, and the third instability Is3 are evaluated (refer to Figure 5 / STEP 122, Figure 7 to Figure 12 ), but as another embodiment, only one of the first instability I s1, the second instability I s2 and the third instability I s3 may be evaluated, and the instability information indicating the one instability may be output to the information output device. Alternatively, the average value or weighted sum of at least two of the first instability I s1, the second instability I s2 and the third instability I s3 may be evaluated as a single instability.
[0103] In the above embodiment, only the instability information indicating one of the first instability I s1, the second instability I s2 and the third instability I s3 is output to the information output device (refer to Figure 5 / Step 124…1→Step 125→Step 128→…→Step 224, Figure 5 / Step 124…2→Step 126→Step 128→…→Step 224, Figure 5 / Step 124…3→Step 126→Step 127→…→Step 224), but three instability information or two instabilities representing all instabilities or two instabilities among the first instability I s1, the second instability I s2 and the third instability I s3 may be output to the information output device. In this case, two diagrams f(x) representing the first instability I s1, the second instability I s2 and the third instability I s3 may be output. The determination process of the maximum instability I smax is omitted (refer to Figure 5 / Step (STEP) 124).
[0104] In the above embodiment, only when the working machine 40 uses the bucket 445 (working unit) to perform a designated work (for example, excavation work), the instability information is transmitted to the operator through the information output device (see Figure 5 / Step (STEP) 121…Yes→…→STEP 224), but as another embodiment, the instability information may be transmitted to the operator through the information output device regardless of whether the working machine 40 performs the designated work.
[0105] In the actual machine state monitoring system of the present invention, it is preferred that the instability evaluation element evaluates at least one of a first instability based on the base not floating from the ground and a second instability based on the base not sliding relative to the ground as the instability.
[0106] According to the real machine state monitoring system of this structure, by confirming the instability information (first instability information) indicating the first instability outputted by the information output device, the operator of the working machine can recognize with high accuracy the degree of proximity of the first instability of the base to the threshold value (first threshold value), thereby preventing the base from floating from the ground and becoming unstable, and allowing the operator of the working machine to recognize with high accuracy the permissible range for the operation of the working mechanism, etc. Similarly, by confirming the instability information (second instability information) indicating the second instability outputted by the information output device, the operator of the working machine can recognize with high accuracy the degree of proximity of the instability of the base to the threshold value (second threshold value), thereby preventing the base from becoming unstable due to sliding relative to the ground, and allowing the operator of the working machine to recognize with high accuracy the permissible range for the operation of the working mechanism, etc.
[0107] In the real machine state monitoring system of the present invention, it is preferred that the real machine state identification element identifies whether the working machine is performing a designated work while causing the working part to apply force to the working object, and the output control element outputs the instability information to the information output device on the necessary condition that the real machine state identification element identifies that the working machine is performing the designated work.
[0108] According to the real machine state monitoring system having such a configuration, only when the working machine is performing a designated work while causing the working part to apply force to the work object, that is, when the base body may become unstable, the instability information is transmitted to the operator through the information output device, thereby achieving the improvement of the usefulness of the instability information.
[0109] Explanation of symbols
[0110] 10…remote operation assisting server, 20…remote operation device, 200…remote control device, 40…operating machine, 210…remote input interface, 211…remote operation mechanism, 220…remote output interface, 221…remote image output device (information output device), 222…sound output device (information output device), 223…vibration output device (information output device), 224…remote wireless communication device, 41…real machine input interface, 412…real machine imaging device, 414…real machine state sensor group, 42…real machine output interface, 421…real machine image output device (information output device), 422…real machine wireless communication device, 440…operating mechanism (operating attachment), 445…bucket (operating part), 110…real machine state monitoring system, 111…real machine state identification element, 112…instability evaluation element, 114…output control element, 410…lower traveling body (base), I s1…first instability, I s2…the second degree of instability, I s3…the third degree of instability.
Claims
1. A real machine status monitoring system for transmitting the status of a working machine to an operator of the working machine and outputting the status of the working machine to an information output device, wherein the working machine comprises a base, a working mechanism extending from the base, a real machine photographing device, and a working unit installed at the top end of the working mechanism, The real machine status monitoring system is characterized by comprising: A real machine state recognition element for recognizing the posture of the base and the external force acting on the working part; an instability evaluation element that evaluates the instability of the base as a continuous variable of a first instability based on the base not floating from the ground and a second instability based on the base not sliding relative to the ground, based on the posture of the base recognized by the actual machine state recognition element and the external force acting on the working unit; and An output control element selectively causes instability information to be output to the information output device in an overlapping manner with an environmental image in a manner in which the output mode of the instability information continuously changes corresponding to the continuous change of the maximum instability, wherein the instability information is information representing the maximum instability of the first instability and the second instability of the substrate evaluated by the instability evaluation element, and the environmental image is an image based on a captured image of at least a portion of the operating mechanism captured by the actual machine shooting device.
2. The real machine status monitoring system according to claim 1, characterized in that: The first instability includes a first instability based on the front side of the base not floating from the ground, and another first instability based on the rear side of the base not floating from the ground. The output control element selectively causes the instability information to be output to the information output device in an overlapping manner with the environmental image in a manner in which the output mode of the instability information continuously changes corresponding to the continuous change of the maximum instability, the instability information being information representing the maximum instability among a first instability, other first instabilities and a second instability of the substrate evaluated by the instability evaluation element, and the environmental image being an image based on a captured image of at least a portion of the operating mechanism captured by the actual machine shooting device.
3. The real machine status monitoring system according to claim 1, characterized in that: The output control element outputs the graph indicating the instability of the substrate to the image output device constituting the information output device in such a manner that the graph continuously changes based on the threshold value of the instability.
4. The real machine status monitoring system according to claim 3, characterized in that: The output control element causes the diagram to be output to an image output device constituting the information output device in a manner that indicates the first degree of instability and the second degree of instability, respectively.
5. The real machine status monitoring system according to any one of claims 1 to 4, characterized in that: The output control element outputs a sound indicating the instability of the substrate to the sound output device constituting the information output device in a manner such that the volume, frequency, or a combination of the volume and frequency of the sound continuously changes.
6. The real machine status monitoring system according to any one of claims 1 to 4, characterized in that: The output control element outputs the vibration representing the instability of the substrate to the vibration output device constituting the information output device in a manner in which the amplitude, frequency, or a combination of the amplitude and frequency of the vibration continuously changes.
7. The real machine status monitoring system according to any one of claims 1 to 4, characterized in that: The actual machine state recognition element recognizes whether the working machine is performing a designated work while causing the working unit to apply force to a work object. The output control element outputs the instability information to the information output device on the condition that the actual machine state recognition element recognizes that the working machine is performing the designated work.
8. The real machine status monitoring system according to any one of claims 1 to 4, characterized in that: The remote operation assisting server assists the remote operation device in remotely operating the working machine based on communication with the working machine and the remote operation device for remotely operating the working machine.
9. The real machine status monitoring system according to any one of claims 1 to 4, characterized in that: The information output device is constituted by a remote operation device for remotely operating the working machine.
10. A real machine status monitoring method for transmitting the status of a working machine to an operator of the working machine and outputting the status of the working machine to an information output device, wherein the working machine comprises a base, a working mechanism extending from the base, a real machine photographing device, and a working unit installed at the top end of the working mechanism, The real machine status monitoring method is characterized by comprising the following steps: a real machine state recognition step of recognizing the posture of the base body and the external force acting on the working part; an instability evaluation step of evaluating the instability of the base as a continuous variable of a first instability based on the base not floating from the ground and a second instability based on the base not sliding relative to the ground, based on the posture of the base recognized in the actual machine state recognition step and the external force acting on the working unit; and An output control process selectively causes the instability information to be output to the information output device in an overlapping manner with an environmental image in a manner that the output mode of the instability information continuously changes corresponding to the continuous change of the maximum instability, wherein the instability information is information representing the maximum instability of the first instability and the second instability of the substrate evaluated in the instability evaluation process, and the environmental image is an image based on a captured image of at least a portion of the operating mechanism captured by the actual machine shooting device.
