Remote operation system having swing assist function for construction machine

Abstract

An aspect of the present disclosure provides a remote operation system having a swing assist function for a construction machine, the system including: a construction machine including a lower traveling body, an upper rotating body rotatably coupled to the lower traveling body, and a working device mounted on the upper rotating device and including a boom, an arm, and a bucket; and a remote operation device for remotely operating the construction machine, wherein the remote operation system having a swing assist function for a construction machine includes: a photographing unit provided in the construction machine to photograph a working area of the construction machine; a sensor unit provided in the construction machine to scan the working area of the construction machine and obtain necessary information; a remote display unit provided in the remote operation device to output a photographed image of a peripheral area photographed by the photographing unit; a remote operation unit provided in the remote operation device and having a swing lever for remotely operating a swing of the construction machine; and an image control unit provided in the remote operation device, generating a first projection image obtained by projecting a swing path of the bucket onto the ground on the basis of information obtained by the sensor unit and a swing operation command of the swing lever, and superimposing and displaying the first projection image on the photographed image of the remote display unit.

Description

Remote control system for construction machinery with swing assist function

[0001] The present disclosure relates to a remote control system for construction machinery having a swing assist function. In a specific aspect, the present disclosure relates to a remote control system for construction machinery that displays the swing path of the construction machinery on a display when an operator operates the swing lever of a remote control device at a remote control site, thereby allowing the operator to intuitively recognize information about the swing. The present disclosure can be applied to large vehicles, such as trucks, buses, and construction equipment, among other vehicle types. While the present disclosure may be described with respect to a specific vehicle, the present disclosure is not limited to any particular vehicle.

[0002] An excavator is a type of construction machine that performs various tasks, such as digging the ground at construction sites, loading work to transport soil, excavating work to create foundations, crushing work to dismantle buildings, leveling work to prepare the ground, and leveling work to level the ground.

[0003] In the case of work using excavators, work is often carried out in hazardous areas or under harsh environments depending on the work location, environment, and conditions, and in such cases, it is often difficult for the operator to directly board and control the excavator.

[0004] Recently, a tele-operation system has emerged that allows the operator to control the excavator without having to board the excavator directly.

[0005] This remote control system outputs images captured by multiple cameras installed on the excavator to a display at the remote operation site, and the operator remotely controls the excavator while viewing the images displayed on the display.

[0006] Meanwhile, one of the biggest challenges with remote control systems is the 2D display. This is because it's difficult for operators to intuitively perceive the distance between the work device and the workpiece on a 2D display.

[0007] For example, when an operator swings an excavator using a remote control device, if separate image information indicating the swing path or distance is not displayed on the remote display, a dangerous situation may occur in which the work device collides with a dumping bed or wall when the excavator is swinging.

[0008] A first aspect of the present disclosure is a remote control system for a construction machine having a swing assist function, which includes a construction machine including a lower body, an upper swing body rotatably coupled to the lower body, a work device including a boom, an arm, and a bucket mounted on the upper swing body, and a remote control device for remotely controlling the construction machine, the remote control system including a swing assist function, the remote control system including a shooting unit provided in the construction machine for shooting a work area of ​​the construction machine, a sensor unit provided in the construction machine for scanning the work area of ​​the construction machine and acquiring necessary information, a remote display unit provided in the remote control device for outputting a shooting image of a surrounding area shot by the shooting unit, a remote control unit provided in the remote control device for remotely operating a swing of the construction machine, and an image control unit provided in the remote control device for generating a first projection image projecting a swing path of the bucket onto the ground based on information acquired from the sensor unit and a swing operation command of the swing lever, and superimposing the first projection image on a shooting image of the remote display unit. Provides a remote control system for construction machinery.

[0009] In some examples, optionally including at least one preferred example, the image control unit may be a remote control system for a construction machine having a swing assist function, characterized in that it maps visual information according to the distance between the ground and the bucket calculated based on information obtained from the sensor unit onto the first projection image.

[0010] In some examples, optionally including at least one preferred example, the image control unit may be a remote control system for a construction machine having a swing assist function, characterized in that, when an object exists on the swing path of the bucket and the lowest position of the bucket is lower than the highest position of the object, the image control unit generates a second projection image projecting the bucket onto a side surface of the object, and superimposes the second projection image on the captured image of the remote display unit.

[0011] In some examples, optionally including at least one preferred example, the image control unit may be a remote control system for a construction machine having a swing assist function, characterized in that it maps visual information according to the distance between the side of the object and the bucket calculated based on information obtained from the sensor unit onto the second projection image.

[0012] In some examples, optionally including at least one preferred example, the remote control system for a construction machine having a swing assist function may be characterized in that the visual information is color information.

[0013] In some examples, optionally including at least one preferred example, a remote control system for a construction machine having a swing assist function may further include a setting unit capable of setting one or more display conditions for the first projection image.

[0014] In some examples, optionally including at least one preferred example, the setting unit may be a remote control system for a construction machine having a swing assist function, characterized in that it includes a first display condition that can be set not to display the first projection image for an operation amount below a specific ratio compared to the maximum operation amount of the swing lever.

[0015] In some examples, optionally including at least one preferred example, the setting unit may be a remote control system for a construction machine having a swing assist function, characterized in that it includes a second display condition that can set a swing projection angle limit of the first projection image displayed on the remote display.

[0016] In some examples, optionally including at least one preferred example, the image control unit may be a remote control system for a construction machine having a swing assist function, characterized in that it calculates a swing projection angle of the first projection image based on an operation amount of the swing lever, and displays the first projection image in which the swing projection angle is reflected on the remote display unit.

[0017] In some examples, optionally including at least one preferred example, the image control unit may be a remote control system for a construction machine having a swing assist function, characterized in that the image control unit calculates a swing projection angle of the first projection image based on a swing speed, an inertia of the upper swing body, and a swing brake torque, and displays the first projection image in which the swing projection angle is reflected on the remote display unit.

[0018] In some examples including at least one preferred example, the information obtained from the sensor unit may be a remote control system for a construction machine having a swing assist function, characterized in that it includes at least one of position, attitude and orientation information of the work device, topographic information of the work area, object information of the work area, distance information between the work device and the work area, and distance information between the work device and the object.

[0019] A remote control system for a construction machine having a swing assist function according to the present disclosure generates a first projection image projecting the swing path of the bucket onto the ground when an operator operates a swing lever at a remote control site, and displays the first projection image on a captured image of a remote display unit, thereby enabling the operator to intuitively recognize the swing path of the bucket and the distance between the bucket and the ground even at a remote control site.

[0020] In addition, a remote control system of a construction machine having a swing assist function can generate a second projection image projected on the side of an object when an object exists on the swing path of the bucket, and display the second projection image on a captured image of a remote display unit, thereby allowing an operator to intuitively recognize the distance between the bucket and the object.

[0021] The effects of the present disclosure are not limited to the effects described above, but should be understood to include all effects that can be inferred from the detailed description of the present disclosure or the composition of the disclosure described in the claims.

[0022] The disclosed aspects, embodiments (including any preferred embodiments), and / or accompanying claims may be suitably combined with one another as would be apparent to one skilled in the art.

[0023] Additional features and advantages are set forth in the following description, claims and drawings, and in part will be readily apparent to those skilled in the art or may be recognized by practicing the teachings herein.

[0024] The example is described in more detail below with reference to the attached drawings.

[0025] FIG. 1 is a drawing illustrating a construction machine according to one aspect of the present disclosure.

[0026] FIG. 2 is a block diagram schematically illustrating the configuration of a remote control system for a construction machine having a swing assist function according to one aspect of the present disclosure.

[0027] FIG. 3 illustrates an example of a remote display unit and a remote operation unit of a remote control device according to one aspect of the present disclosure.

[0028] FIG. 4 is a drawing exemplarily showing a remote display unit on which a first projection image is displayed according to one aspect of the present disclosure.

[0029] Figures 5 (a) and (b) are drawings for explaining a first projection image according to one aspect of the present disclosure.

[0030] Figures 6 (a) and (b) are drawings for explaining the swing projection angle of the first projection image according to one aspect of the present disclosure.

[0031] FIG. 7 is a drawing for explaining a first projection image reflecting visual information according to one aspect of the present disclosure.

[0032] FIG. 8 is a drawing for explaining a second projection image according to one aspect of the present disclosure.

[0033] FIG. 9 is a diagram showing the use of a swing assist function in which a first projection image and a second projection image are displayed according to one aspect of the present disclosure.

[0034] The detailed description set forth below provides information and examples of the disclosed technology with sufficient detail to enable those skilled in the art to practice the present disclosure. However, the present disclosure may be implemented in many different forms and is therefore not limited to the aspects described herein. In addition, for the purpose of clearly illustrating the present disclosure, parts irrelevant to the description are omitted in the drawings, and like reference numerals are used throughout the specification to designate like parts.

[0035] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the present disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The terms "comprises," "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0036] While terms such as "first" and "second" may be used herein to describe various elements, it is to be understood that these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the present disclosure, a first element could be referred to as a "second element," and similarly, a second element could also be referred to as a "first element."

[0037] Relative terms such as "below" or "above" or "upper" or "lower" or "horizontal" or "vertical" may be used herein to describe one element's relationship to another, as illustrated in the drawings. It will be understood that these terms and the terms discussed above are intended to encompass different orientations of the device in addition to the orientations depicted in the drawings. When a component is referred to as being "connected" or "coupled" to another component, it will be understood that it may be directly connected or coupled to the other component, or that intervening components may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, no intervening elements are present.

[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should be further understood that terms used herein should be interpreted to have a meaning consistent with their meaning within the context of this specification and related technologies, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0039] Hereinafter, one aspect of the present disclosure will be described in detail with reference to the attached drawings.

[0040] FIG. 1 is a drawing illustrating a construction machine according to one aspect of the present disclosure, and FIG. 2 is a block diagram schematically illustrating a configuration of a remote control system of a construction machine having a swing assist function according to one aspect of the present disclosure.

[0041] Referring to FIGS. 1 and 2, a remote control system (1000) for a construction machine having a swing assist function according to one aspect of the present disclosure may include a construction machine (100) and a remote control device (200) for remotely controlling the construction machine (100).

[0042] The construction machine (100) can be operated at a work site. The remote control device (200) can be installed at a location far from the work site. That is, the operator can remotely control the construction machine (100) from a distance where the operator cannot see the construction machine (100).

[0043] The construction machine (100) and the remote control device (200) may be configured to communicate with each other through a common or separate network. The construction machine (100) may be remotely controlled based on an operation signal transmitted from the remote control device (200).

[0044] A construction machine (100) according to one aspect of the present disclosure may be, for example, an excavator. While the present disclosure may be described with respect to an excavator, it is not limited thereto. An excavator is a device capable of excavating an excavation target, and may include various types of excavators capable of performing excavation work in various ways, such as soil transport work, building demolition work, and ground clearing work.

[0045] A construction machine (100) may include an upper swing body (120) that is supported so as to be able to swing on a lower travel body (110), and a work device connected to the upper swing body (120). The work device may include, for example, a boom (130), an arm (140), and a bucket (150).

[0046] The lower drive unit (110) supports the loads of the upper slewing unit (120), boom (130), arm (140), and bucket (150), and is configured to move the excavator for work purposes. The lower drive unit includes a pair of left and right drive sections, and can drive in a straight line in the forward and backward direction, turn left and right by steering, or turn in the opposite direction of travel.

[0047] The upper swing body (120) is a structure supported on the lower driving body (110) and is designed to swing on the lower driving body (110) by a swing device including a swing motor, a swing reduction gear, etc.

[0048] The arm (140) is connected to the bucket (150) and the boom (130), respectively, and in one embodiment, the upper swivel body (120) is connected in the order of the boom (130), the arm (140), and the bucket (150) through joints, and each joint can be moved by a hydraulic cylinder.

[0049] For example, the arm (140) is connected to a boom (130) which is connected to the upper slewing body (120) of the excavator at one end, and is connected to a bucket (150) at the other end, and each of the boom (130), the arm (140) and the bucket (150) can rotate about one or more axes by the boom cylinder, the arm cylinder and the bucket cylinder, and the bucket (150) can hold an excavation target (e.g., soil) on the ground inside as it rotates.

[0050] Since the boom (130), arm (140) and bucket (150) are installed while being supported toward the front on the upper swivel body (120), when the upper swivel body (120) swings, the boom (130), arm (140) and bucket (150) swing together with the upper swivel body (120).

[0051] The construction machine (100) may be configured with, for example, an electro-hydraulic system, and its drive may be electronically controlled by a controller.

[0052] Meanwhile, the construction machine (100) may further include a photographing unit (160), a sensor unit (170), a communication unit (180), and a machine control unit (190).

[0053] The photographing unit (160) may include a plurality of cameras mounted on the construction machine (100) to photograph the surrounding area of ​​the construction machine (100). The photographing unit (160) may acquire photographing data of the front, rear, left, right, and upper sides of the construction machine (100) during driving and operation using the plurality of cameras. The photographing data acquired by the photographing unit (160) may be provided to the machine control unit (190) or may be provided to the remote control device (200) via the communication unit (190). The photographing data may be output to the remote display unit (220) of the remote control device (200).

[0054] The sensor unit (170) is mounted on a construction machine (100) to scan the work area of ​​the construction machine (100) and obtain information necessary to perform the swing assist function.

[0055] The information acquired from the sensor unit (170) may include one or more of topographic information of the work area, information on an object of the work area, distance information between the work device and the work area, distance information between the work device and the object, and position and posture information of the work device.

[0056] More specifically, the sensor unit (170) may include a terrain detection sensor, a location information acquisition sensor, and a posture measurement sensor.

[0057] For example, the terrain detection sensor may include a lidar. The lidar can scan the surrounding environment and work area within a certain radius centered on the construction machine. In addition, the lidar may be installed to measure the displacement between the construction machine (100) and an object. The object may include a pedestrian, a vehicle, another construction machine, a wall, an obstacle, etc. Measurement data (e.g., point cloud data) generated by the lidar may be provided to the machine control unit (190) or transmitted to the remote control device (200) via the communication unit (180). The measurement data may be output to the remote display unit (220) of the remote control device (200).

[0058] The image control unit (240) of the remote control device (200) can calibrate and align the coordinates of the photographing data received from the camera and the measurement data received from the lidar. The remote display (220) of the remote control device (200) can receive the alignment data analyzed by the image control unit (240) and display it to the operator.

[0059] For example, the location information acquisition sensor may include a GNSS (Global Navigation Satellite System) sensor. The GNSS sensor may include a GNSS antenna. The GNSS sensor can calculate the location of the construction machine (100) in a global coordinate system in real time. The global coordinate system is a coordinate system defined by GNSS. GNSS refers to a global navigation satellite system. As a global navigation satellite system, the Global Positioning System (GPS) can be an example. The GNSS can detect a location defined by coordinate data of latitude, longitude, and altitude. The GNSS sensor can receive radio waves from the GNSS and calculate the current location of the construction machine (100) in the global coordinate system based on the received radio waves.

[0060] In one example, the attitude measurement sensor may include a plurality of inertial measurement units (IMUs). The inertial measurement units may be mounted on each of the upper swivel body (120), the boom (130), the arm (140), and the bucket (150). The inertial measurement units may measure displacement and / or position and / or attitude of one or more of the upper swivel body (120), the boom (130), the arm (140), and the bucket (150).

[0061] Information measured by the sensor unit (170) may be provided to the machine control unit (190) or may be provided to the remote control device (200) via the communication unit (180).

[0062] The communication unit (180) may be configured to communicate with a remote control device (200) located outside the construction machine (100) and may be connected to a commercial communication network. Examples of commercial communication networks include a 3G mobile communication network, a 4G LTE mobile communication network, and a 5G mobile communication network. The communication unit (180) may be connected to a commercial communication network, i.e., a mobile communication network, to provide an Internet connection service to the construction machine (100).

[0063] In addition, the communication unit (180) can be connected to the internal network of the construction machine (100) and communicate with the machine control unit (190). The internal network of the construction machine (100) can be any one of CAN (Controller Area Network), Ethernet, FlexRay, LIN (Local Interconnect Network), etc., but it is preferable to use Ethernet.

[0064] The machine control unit (190) is configured by an operation processing unit (a single-core processor or a multi-core processor or a processor core constituting the same), and can read necessary data and software from a storage device such as a memory, and execute operation processing according to the software on the data.

[0065] The machine control unit (190) can generate an electrical signal corresponding to the operation of the remote control unit (210) of the remote control device (200) to control the operation of the lower driving body (110), the upper swing body (120), and the work device.

[0066] In addition, the machine control unit (190) can calculate the distance between the bucket (150) and the ground or the distance between the bucket (150) and the object based on the information received from the sensor unit (170). Information regarding the distance between the bucket (150) and the ground or the distance between the bucket (150) and the object can be provided to the remote control device (200) via the communication unit (180).

[0067] FIG. 3 illustrates an example of a remote display unit and a remote operation unit of a remote control device according to one aspect of the present disclosure.

[0068] Referring to FIGS. 2 and 3, the remote control device (200) is for remotely controlling the construction machine (100) from a location away from the construction machine (100).

[0069] The remote control device (200) may include a remote control unit (210), a remote display unit (220), a communication unit (230), and an image control unit (240).

[0070] The remote control unit (210) may be an electric joystick, and preferably, the remote control unit (210) may generate an electrical signal in proportion to the amount of manipulation by the operator.

[0071] The remote control unit (210) can control the swing of the upper swivel body (120), movement of the boom (130), arm (140), and bucket (150), etc. The remote control unit (210) can include a first remote control lever (211) and a second remote control lever (212).

[0072] The first remote control lever (211) can be used to operate the arm (140) and the swing operation of the upper swivel body (120). When the first remote control lever (211) is operated in the forward direction of the operator, the arm actuator is operated to perform an arm-out operation, that is, to open the arm. When the first remote control lever (211) is operated in the rearward direction of the operator, the arm actuator is operated to perform an arm-in operation, that is, to close the arm. In addition, when the first remote control lever (211) is operated in the left direction of the operator, the swing hydraulic motor is operated to perform a left swing operation, that is, to swing the upper swivel body to the left. When the first remote control lever (211) is operated in the right direction of the operator, the swing hydraulic motor is operated to perform a right swing operation, that is, to swing the upper swivel body to the right. That is, the first remote control lever (211) can be operated as a swing lever.

[0073] The second remote control lever (212) can be used to operate the boom (130) and the bucket (150). When the second remote control lever (212) is operated in the forward direction of the operator, the boom actuator is operated to perform a boom-down operation, i.e., to lower the boom. When the second remote control lever (212) is operated in the rearward direction of the operator, the boom actuator is operated to perform a boom-up operation, i.e., to raise the boom. In addition, when the second remote control lever (212) is operated in the left direction of the operator, the bucket actuator is operated to perform a bucket-in operation, i.e., to close the bucket. When the second remote control lever (212) is operated in the right direction of the operator, the bucket actuator is operated to perform a bucket-out operation, i.e., to open the bucket.

[0074] However, this is not limited thereto, and the operation methods of the first remote control lever (211) and the second remote control lever (212) can of course be changed by the operator. For example, the first remote control lever (211) can be used to operate the boom and bucket, and the second remote control lever (212) can be used to operate the arm and the upper swivel body.

[0075] The remote display unit (220) outputs a captured image of a surrounding area captured by the shooting unit (160) of the construction machine (100) and a projected image generated by the image control unit (240), and may include one or more displays.

[0076] For example, the remote display unit (220) may include a first display (221) arranged in front of the seat, a second display (222) arranged above the first display (221), a third display (223) arranged on the left side of the first display (221), a fourth display (224) arranged on the right side of the first display (221), and a fifth display (225) arranged below the first display (221).

[0077] Here, the first display (221) can output a captured image for the front view of the construction machine (100), the second display (222) can output captured images for the rear view and upper view of the construction machine (100), the third display (223) can output a captured image for the left mirror view of the construction machine (100), the fourth display (224) can output a captured image for the right mirror view of the construction machine (100), and the fifth display (225) can output a captured image for the lower view of the construction machine (100).

[0078] The communication unit (230) may be configured to communicate with the construction machine (100) and may be connected to a commercial communication network or a dedicated communication network. Examples of commercial communication networks include a 3G mobile communication network, a 4G LTE mobile communication network, and a 5G mobile communication network. Examples of dedicated communication networks include a private 5G network, a WiFi network, and the like. The communication unit (230) may be connected to a commercial communication network, i.e., a mobile communication network, to provide Internet connection services to the construction machine (100).

[0079] Additionally, the communication unit (230) can be connected to the internal network of the remote control device (200) and communicate with the image control unit (2400). The internal network of the remote control device (200) can be any one of CAN (Controller Area Network), Ethernet, FlexRay, LIN (Local Interconnect Network), etc., but it is preferable to use Ethernet.

[0080] The image control unit (240) is configured by an operation processing unit (a single-core processor or a multi-core processor or a processor core constituting the same), and can read necessary data and software from a storage device such as a memory, and execute operation processing according to the software on the data.

[0081] The image control unit (240) can generate a first projection image that projects the swing path of the work device onto the ground and display the first projection image on the remote display unit (220).

[0082] FIG. 4 is a drawing exemplarily showing a remote display unit on which a first projection image is displayed according to one aspect of the present disclosure, and FIG. 5 (a) and (b) are drawings for explaining the first projection image according to one aspect of the present disclosure.

[0083] Referring to FIG. 4, the image control unit (240) can generate a first projection image (BSI-1) projecting the swing path of the bucket (150) onto the ground based on the information acquired from the sensor unit (170) and the swing operation command of the swing lever. More specifically, the image control unit (240) can determine the current position coordinates and the swing path of the bucket (150) based on the information acquired from the sensor unit (170) and the swing operation command of the swing lever, obtain coordinates at which the current position coordinates and the swing path of the bucket (150) are projected onto the ground, and convert them into 2D coordinates to generate the first projection image (BSI-1).

[0084] The image control unit (240) can display the first projection image (BSI-1) generated by superimposing it on the remote display unit (220) that outputs the image captured by the shooting unit (160).

[0085] Referring to FIG. 5, the width (Pw) of the first projection image (BSI-1) may be set to have a size (Pw-1) corresponding to the vertical width of the bucket (150). Alternatively, the width (Pw) of the first projection image (BSI-1) may be set to have a size (Pw-2) that adds a portion of the margin to the vertical width of the bucket (150). Alternatively, the width of the first projection image (BSI-1) may be set to have a size (Pw-3) that takes into account the arm-in or arm-out motion in the vertical width of the bucket (150). In this way, the width of the first projection image (BSI-1) may be adjusted in various ways according to the operator's intention. Here, the vertical width of the bucket (150) may be the distance between the first side on which the bucket teeth are formed and the second side facing the first side among the sides defining the open surface of the bucket (150).

[0086] Figures 6 (a) and (b) are drawings for explaining the swing projection angle of the first projection image according to one aspect of the present disclosure.

[0087] Referring to FIG. 6, the image control unit (240) calculates the swing projection angle (θ) of the first projection image (BSI-1) based on the amount of operation of the swing lever, and can display the first projection image (BSI-1) in which the swing projection angle (θ) is reflected on the remote display unit (220). The swing projection angle (θ) means the swing angle of the first projection image (BSI-1) projected onto the ground based on the swing center (CS) of the upper swing body.

[0088] Here, swing projection angle (θ) = lever demand X maximum swing projection angle (Max Projection Angle, θ) max ) can be calculated. The range of lever operation can be from 0 to 100%. The maximum swing projection angle (θ max ) can be set by the operator.

[0089] For example, the maximum swing projection angle (θ) when the lever operation amount is 100% max ) is set to 90° and the lever operation amount is 100%, the swing projection angle (θ) of the first projection image (BSI-1) max ) can be 90°. And, the maximum swing projection angle (θ) when the lever operation amount is 100% max ) is set to 90° and the lever operation amount is 50%, the swing projection angle (θ) of the first projection image (BSI-1) can be 45°.

[0090] For example, the image control unit (240) may calculate the swing projection angle (θ) of the first projection image (BSI-1) based on the swing speed, the upper frame inertia, and the swing braking torque, and display the first projection image (BSI-1) in which the swing projection angle (θ) is reflected on the remote display unit (220).

[0091] To calculate the swing projection angle (θ), the angular acceleration (α) can first be calculated as follows.

[0092] [Formula 1]

[0093] Here, τ is the swing brake torque and Ι is the inertia of the upper swing body.

[0094] The change in angular velocity (Δω) can be expressed as the product of angular acceleration (α) and time (t).

[0095] [Formula 2]

[0096] The braking time (t) can be calculated using the initial swing speed (ω0) and angular acceleration (α). The swing speed at the end of braking is 0, (Assuming that the swing speed decreases due to braking), the braking time can be calculated as follows.

[0097] [Formula 3]

[0098] The swing angle of the upper body during braking, i.e., the swing projection angle (θ), can be expressed as the integral of the angular velocity over time, taking into account the initial swing speed (ω0), as follows:

[0099] [Formula 4]

[0100] In this way, the accurate swing projection angle (θ) calculated using [Equation 1] to [Equation 4] can be reflected in the first projection image (BSI-1) so that the operator can predict the stopping point during the swing motion.

[0101] FIG. 7 is a drawing for explaining a first projection image reflecting visual information according to one aspect of the present disclosure.

[0102] Referring to FIG. 7, the image control unit (240) can map visual information according to the distance between the ground (G) and the bucket (150), calculated based on information obtained from the sensor unit (170), to the first projection image (BSI-1). Here, the visual information may be color information.

[0103] That is, as illustrated in FIG. 7, the image control unit (240) can generate different colors of the first projection image (BSI-1) depending on the distance between the ground (G) and the bucket (150). For example, various methods, such as a gradient method in which colors gradually change depending on the distance, can be applied. In addition, the image control unit (240) can also generate a distance information image that represents the correlation between distance and color.

[0104] FIG. 8 is a drawing for explaining a second projection image according to one aspect of the present disclosure, and FIG. 9 is a diagram showing a state of use of a swing assist function in which a first projection image and the second projection image according to one aspect of the present disclosure are displayed.

[0105] Referring to FIGS. 8 and 9, when an object (O) exists on the swing path of the bucket (150) and the lowest position of the bucket (150) is lower than the highest position of the object (O), the image control unit (240) can generate a second projection image (BSI-2) that projects the bucket (150) onto the side of the object (O), and superimpose the second projection image (BSI-2) on the captured image of the remote display unit (220).

[0106] For example, during a digging and dumping operation, after digging, the operator inputs commands to the remote control unit for swing and boom-up operations. During this time, the swing is directed toward a truck positioned to the left or right. Since a collision with the truck may occur during the swing, the distance between the truck and the bucket is a critical factor. Therefore, if there is an object (e.g., a truck, a wall, etc.) to the left or right of the swing, a second projection image (BSI-2) needs to be displayed on the object to provide the operator with distance information between the bucket and the object.

[0107] That is, the image control unit (240) can display a second projection image (BSI-2) on the object (O) on the remote display (220) if a swing command is input and an object (O) exists in the swing direction of the bucket (150) (which can be measured by a sensor such as a lidar).

[0108] Referring to FIG. 8, the image control unit (240) can map visual information according to the distance between the object (O) and the bucket (150), calculated based on information obtained from the sensor unit (170), to the second projection image (BSI-2). Here, the visual information may be color information.

[0109] That is, as illustrated in FIG. 8, the image control unit (240) can generate different colors of the second projection image (BSI-2) depending on the distance between the object (O) and the bucket (150). For example, various methods can be applied, such as a gradient method in which the color gradually changes depending on the distance.

[0110] Meanwhile, the remote control device (200) may further include a setting unit (250) capable of setting one or more display conditions for the first projection image (BSI-1).

[0111] For example, the setting unit (250) may include a first display condition that can be set not to display the first projection image (BSI-1) for an operation amount below a certain ratio compared to the maximum operation amount of the swing lever.

[0112] The swing lever can be manipulated minutely, even when the operator does not intend to perform a swing motion. If these minute manipulations frequently change the first projection image (BSI-1), this can be a distraction for the operator. Therefore, if the operator sets the first display condition, the first projection image (BSI-1) can be suppressed for manipulations below a certain percentage of the maximum manipulation amount, thereby improving operator convenience.

[0113] As an example, the setting unit (250) may include a second display condition that can set a swing projection angle limit of the first projection image (BSI-1) displayed on the remote display (220).

[0114] When remotely controlling, the operator basically drives while looking at the remote display unit where the camera image is output, and the screen display area may vary depending on the FOV (Field of View) angle of the camera. In other words, the first projection image (BSI-1) area cannot but be limited to the camera's display area. Therefore, when the first display condition is set by the operator, the first projection image (BSI-1) area displayed on the remote display unit can be limited, thereby preventing unnecessary swing path calculation or display.

[0115] In this way, the remote control system of a construction machine having a swing assist function according to the present disclosure generates a first projection image projecting the swing path of the bucket onto the ground when an operator operates the swing lever at a remote control site, and displays the first projection image on a captured image of a remote display unit, thereby enabling the operator to intuitively recognize the swing path of the bucket and the distance between the bucket and the ground even at a remote control site.

[0116] In addition, a remote control system of a construction machine having a swing assist function can generate a second projection image projected on the side of an object when an object exists on the swing path of the bucket, and display the second projection image on a captured image of a remote display unit, thereby allowing an operator to intuitively recognize the distance between the bucket and the object.

[0117] It should be understood that the present disclosure is not limited to the embodiments described above and illustrated in the drawings. Rather, those skilled in the art will recognize that numerous modifications and variations can be made within the scope of the present disclosure and the appended claims. In the drawings and specification, aspects are disclosed for illustrative purposes only, not for purposes of limitation, and the scope of the present disclosure is set forth in the claims below.

[0118]

[0119] Description of the symbol

[0120] Remote control system for construction machinery with 1000 swing assist function

[0121] 100 construction equipment

[0122] 110 lower drivetrain

[0123] 120 upper swivel

[0124] 130 Boom

[0125] 140 Aam

[0126] 150 buckets

[0127] 160 Filming Department

[0128] 170 sensor section

[0129] 180 Communications Department

[0130] 190 Machine Control Unit

[0131] 200 remote control devices

[0132] 210 Remote Control Unit

[0133] 211 1st remote control lever

[0134] 212 Second remote control lever

[0135] 220 remote display unit

[0136] 221 First display

[0137] 222 Second display

[0138] 223 Third display

[0139] 224 4th Display

[0140] 225 5th Display

[0141] 230 Communications Department

[0142] 240 Image Control Unit

[0143] 250 Settings

[0144] BSI-1 1st Projection Film Award

[0145] BSI-2 2nd Projection Film Award

Claims

1. A remote control system for a construction machine having a swing assist function, including a lower drive body, an upper swing body rotatably connected to the lower drive body, a work device including a boom, an arm, and a bucket mounted on the upper swing body, and a remote control device for remotely controlling the construction machine, A photographing unit equipped in the above construction machine to photograph the work area of ​​the above construction machine; A sensor unit equipped in the above construction machine to scan the work area of ​​the above construction machine and obtain necessary information; A remote display unit provided in the above remote control device for outputting a captured image of a surrounding area captured by the above shooting unit; A remote control unit having a swing lever provided in the above remote control device for remotely controlling the swing of the construction machine; and A remote control system for a construction machine having a swing assist function, comprising an image control unit provided in the remote control device, which generates a first projection image projecting the swing path of the bucket onto the ground based on information obtained from the sensor unit and a swing operation command of the swing lever, and superimposes the first projection image on a captured image of the remote display unit.

2. In paragraph 1, The above image control unit, A remote control system for a construction machine having a swing assist function, characterized in that visual information according to the distance between the ground and the bucket calculated based on information obtained from the sensor unit is mapped onto the first projection image.

3. In paragraph 1, The above image control unit, A remote control system for a construction machine having a swing assist function, characterized in that when an object exists on the swing path of the bucket and the lowest position of the bucket is lower than the highest position of the object, a second projection image is generated by projecting the bucket onto the side of the object, and the second projection image is superimposed on the captured image of the remote display unit.

4. In paragraph 3, The above image control unit, A remote control system for a construction machine having a swing assist function, characterized in that visual information according to the distance between the side of the object and the bucket calculated based on information obtained from the sensor unit is mapped onto the second projection image.

5. In paragraph 2 or paragraph 4, A remote control system for a construction machine having a swing assist function, characterized in that the above visual information is color information.

6. In paragraph 1, A remote control system for a construction machine having a swing assist function, characterized in that it further includes a setting unit capable of setting one or more display conditions for the first projection image.

7. In paragraph 6, The above setting section, A remote control system for a construction machine having a swing assist function, characterized in that it includes a first display condition that can be set not to display the first projection image for an operation amount below a specific ratio compared to the maximum operation amount of the swing lever.

8. In paragraph 6, The above setting section, A remote control system for a construction machine having a swing assist function, characterized in that it includes a second display condition that can set a swing projection angle limit of the first projection image displayed on the remote display.

9. In paragraph 1, The above image control unit, A remote control system for a construction machine having a swing assist function, characterized in that the swing projection angle of the first projection image is calculated based on the amount of operation of the swing lever, and the first projection image reflecting the swing projection angle is displayed on the remote display unit.

10. In paragraph 1, The above image control unit, A remote control system for a construction machine having a swing assist function, characterized in that the swing projection angle of the first projection image is calculated based on the swing speed, the inertia of the upper swing body, and the swing brake torque, and the first projection image reflecting the swing projection angle is displayed on the remote display unit.

11. In paragraph 1, A remote control system for a construction machine having a swing assist function, characterized in that the information acquired from the sensor unit includes at least one of position, posture and direction information of the work device, topographic information of the work area, object information of the work area, distance information between the work device and the work area, and distance information between the work device and the object.

Images