Control systems and methods for construction machinery

Abstract

The control system of the engineering machinery of the present invention includes: an upper camera mounted on the cab of the rear vehicle body to capture images of the front of the cab; a lower camera mounted on the front body rotatably connected to the rear vehicle body to capture images of the front of the front vehicle body; an angle information detection unit for detecting bending angle information of the front vehicle body relative to the rear vehicle body; an image processing device for combining a first image and a second image captured by the upper camera and the lower camera into a single image, and setting the position of a transparent processing area of ​​at least one of the first image and the second image in the composite image based on the bending angle information obtained from the angle information detection unit; and a display device for displaying the composite image synthesized by the image processing device.

Description

Technical Field

[0001] This invention relates to a control system and method for construction machinery. More specifically, it relates to a control system for recognizing obstacles ahead during the operation or travel of construction machinery such as wheel loaders and excavators, and a control method for construction machinery utilizing the same. Background Technology

[0002] Construction machinery such as excavators and wheel loaders are widely used to perform tasks such as digging and transporting materials like soil and sand at construction sites and loading them onto cargo vehicles such as dump trucks. These tasks are performed by working devices, such as buckets and booms, mounted on the construction machinery. However, these working devices may obstruct or limit the operator's forward view during driving or travel, and such interference with the operator's forward view, as described above, can be a cause of safety accidents. Summary of the Invention

[0003] Technical issues

[0004] One objective of this invention is to provide a control system for construction machinery that can improve forward visibility limited by the working device.

[0005] Another objective of this invention is to provide a control method for engineering machinery utilizing the above-described control system.

[0006] Technical solution

[0007] An exemplary embodiment of a control system for an engineering machine used to achieve one of the objectives of the present invention includes: an upper camera mounted on the cab of a rear-mounted vehicle to capture images of the front of the cab; a lower camera mounted on a front body rotatably connected to the rear-mounted vehicle to capture images of the front of the front-mounted vehicle; an angle information detection unit for detecting bending angle information of the front-mounted vehicle relative to the rear-mounted vehicle; an image processing device for combining a first image and a second image captured by the upper and lower cameras into a single image, and setting the position of a transparent processing area in the synthesized image where at least one of the first and second images is transparent, based on the bending angle information obtained from the angle information detection unit; and a display device for displaying the synthesized image synthesized by the image processing device.

[0008] In some exemplary embodiments, the image processing apparatus may include: a steering angle calculation unit that determines the steering hinge angle of the front vehicle body based on the bending angle information obtained from the angle information detection unit; and a transparency processing unit that determines the position of the transparency processing area in the synthesized image based on the determined steering hinge angle.

[0009] In some exemplary embodiments, the steering angle calculation unit can convert the detection value obtained from the angle information detection unit into the steering hinge angle value of the vehicle body in front.

[0010] In some exemplary embodiments, the angle information detection unit may include a center pin angle sensor, a steering cylinder displacement sensor, or a gyroscope sensor.

[0011] In some exemplary embodiments, the control system of the construction machinery may further include a working device posture detection unit for detecting the posture of the forward working device, and the image processing device performs transparency processing on at least one of the first image and the second image in the transparency processing area based on the posture of the forward working device detected by the working device posture detection unit.

[0012] In some exemplary embodiments, the image processing device may make the first image transparent in the transparency processing area when at least a portion of the front working device infringes upon the predetermined position, and may make the second image transparent in the transparency processing area when the front working device does not infringe upon the predetermined position.

[0013] In some exemplary embodiments, the control system of the construction machinery may further include an input unit for setting image processing conditions in the image processing device.

[0014] In some exemplary embodiments, the image processing conditions may include the transparency transition time points of the first image and the second image, or the area occupied by the transparency processing area in the entire display area of ​​the display device.

[0015] In some exemplary embodiments, the image processing apparatus may process the image in a manner that displays the outline of the boom or bucket appearance captured and made transparent in the first or second image using lines or dashed lines in the transparent processing area.

[0016] In some exemplary embodiments, the outline of the boom or bucket can be displayed by making the boom or bucket image captured in the actual first or second image transparent.

[0017] In some exemplary embodiments, the image processing device may selectively perform transparency processing on the boom or bucket image of the first image and the second image combined with the front vehicle body in the transparency processing area.

[0018] In some exemplary embodiments, the composite image may include objects identified by the image processing device within the first image and the second image.

[0019] In some exemplary embodiments, the image processing apparatus can identify people, animals, buildings, or equipment as objects using a pre-determined algorithm.

[0020] In some exemplary embodiments, the image processing device may extract a cropped image from one of the first image and the second image, make a portion of the other image transparent, and composite the extracted cropped image into the transparent region to generate the transparent image.

[0021] In some exemplary embodiments, the transparent portion may be the transparent processing area.

[0022] In some exemplary embodiments, the image processing apparatus may set a portion of the first image as a first composite region and a portion of the second image as a second composite region, and perform semi-transparent processing on the second composite region of the second image, and composite the semi-transparent second composite region into the first composite region of the first image to create the composite image.

[0023] An exemplary embodiment of a control method for engineering machinery used to achieve another objective of the present invention includes: acquiring a first image of the front of the cab from an upper camera mounted on the cab of a rear-mounted vehicle; acquiring a second image of the front of the front vehicle from a lower camera mounted on a front vehicle rotatably connected to the rear-mounted vehicle; acquiring bending angle information of the front vehicle relative to the rear-mounted vehicle; combining the first image and the second image into a single image; setting the position of a transparency processing area in the combined image based on the acquired bending angle information of the front vehicle; performing transparency processing on at least one of the first image and the second image within the transparency processing area; and displaying the transparency-processed image via a display device.

[0024] In some exemplary embodiments, the step of setting the position of the transparency processing area based on the obtained bending angle information of the front vehicle body may include: determining the steering hinge angle of the front vehicle body from the bending angle information; and determining the position of the transparency processing area in the synthesized image based on the steering hinge angle.

[0025] In some exemplary embodiments, in the step of determining the steering hinge angle of the front vehicle body from the bending angle information, the bending angle detection value can be converted into the steering hinge angle value of the front vehicle body.

[0026] In some exemplary embodiments, the step of obtaining bending angle information of the front vehicle body relative to the rear vehicle body may include: using a bending angle detection sensor, which includes a center pin angle sensor, a steering cylinder displacement sensor, or a gyroscope sensor.

[0027] In some exemplary embodiments, the control method for the construction machinery may further include: detecting the posture of the front working device, and the step of making at least one of the first image and the second image transparent within the transparency processing area may include: making at least one of the first image and the second image transparent according to the detected posture of the front working device.

[0028] In some exemplary embodiments, the control method for the engineering machinery may further include: setting image processing conditions that make the first image and the second image transparent.

[0029] In some exemplary embodiments, the image processing conditions may include the transparency transition time points of the first image and the second image, or the area occupied by the transparency processing area in the entire display area of ​​the display device.

[0030] An exemplary embodiment of a control method for engineering machinery used to achieve another objective of the present invention includes: acquiring a first image of the front of the cab from an upper camera mounted on the cab of a rear-mounted vehicle; acquiring a second image of the front of the front vehicle from a lower camera mounted on a front vehicle rotatably connected to the rear-mounted vehicle; acquiring bending angle information of the front vehicle relative to the rear-mounted vehicle; setting the position of a transparent processing area in the first image based on the acquired bending angle information of the front vehicle; setting a portion of the second image corresponding to the transparent processing area as a cropping area to extract a cropped image; adjusting the transparency of the transparent processing area in the first image; compositing the cropped image extracted from the second image into the transparent processing area of ​​the first image; and displaying the composite image via a display device.

[0031] In some exemplary embodiments, the step of compositing the extracted cropped image into the transparent processing area of ​​the first image may include: enlarging the cropped image extracted from the second image; and compositing the enlarged cropped image into the transparent processing area of ​​the first image.

[0032] In some exemplary embodiments, the step of adjusting the transparency of the transparent area of ​​the first image may include: displaying the outline of the boom or bucket appearance in the transparent area of ​​the first image using lines or dashed lines.

[0033] In some exemplary embodiments, the control method for the engineering machinery may further include: identifying people, animals, buildings, or equipment as objects in the synthesized image using a pre-determined algorithm.

[0034] The effects of the invention

[0035] According to some exemplary embodiments, the control system of construction machinery can combine a first image and a second image captured by an upper camera mounted on the cab and a lower camera mounted on the front of the vehicle body into a single image. The system determines the position of a transparency processing area in the synthesized image based on the steering hinge angle of the front of the vehicle body. Based on the position of the bucket or boom connected to the front of the vehicle body, the system performs transparency processing on at least one of the first image and the second image within the transparency processing area. The system then displays the transparent image through a display device.

[0036] That is, the position of the transparent processing area in the image synthesized in a manner matching the steering hinge angle of the vehicle body in front is determined, and at least one of the first and second images is made transparent in the transparent processing area according to the posture of the working device, such as the position of the bucket. This eliminates blind spots where the working device in front obstructs the forward view, even when the construction machinery is turning. Therefore, it increases the operator's cognitive ability to ensure stability, thereby preventing accidents before they occur.

[0037] Furthermore, the transparency processing area can be set according to the operator's choice, thus increasing the flexibility of using the transparent image and forming an effective system.

[0038] However, the effects of the present invention are not limited to those mentioned above, and can be extended in various ways without departing from the spirit and scope of the present invention. Attached Figure Description

[0039] Figure 1 This is a side view of an engineering machine illustrating an exemplary embodiment.

[0040] Figure 2 It shows the corresponding Figure 1 A side view of the bucket rising position at the boom rotation angle.

[0041] Figure 3 It is shown Figure 1 A plan view showing the horizontal field of view angles of the upper and lower cameras when the construction machinery is traveling straight and turning to the left.

[0042] Figure 4 It is shown Figure 1 A block diagram of the control system of the engineering machinery.

[0043] Figure 5 It is shown Figure 4 A block diagram of an image processing device.

[0044] Figure 6 This is a diagram showing the first image captured by the upper camera.

[0045] Figure 7 This is a diagram showing a second image captured by the lower camera.

[0046] Figure 8 It shows through Figure 5 Image processing device synthesis Figure 6 First image and Figure 7 The second image of the image.

[0047] Figure 9 This is a sequence diagram illustrating a control method for a wheel loader according to an exemplary embodiment.

[0048] Figure 10 It is shown Figure 3 The image captured by the upper camera when the construction machinery is traveling straight (state A) is displayed on the screen of the display device in the cab.

[0049] Figure 11 It is shown Figure 3 The image captured by the upper camera when the construction machinery turns to the left (state B) is displayed on the screen of the display device in the cab.

[0050] Figure Labels

[0051] 10: Wheel loader; 12: Front body; 14: Rear body; 20: Boom; 22: Boom cylinder; 24: Boom angle sensor; 30: Bucket; 32: Bucket cylinder; 34: Tilt stick; 40: Cab; 70: Front wheel; 100: Camera unit; 110: First camera; 120: Second camera; 150: Angle information detection unit; 200: Image processing device; 210: Steering angle calculation unit; 220: Image compositing unit; 230: Transparency processing unit; 240: Image rendering unit; 250: Storage unit; 300: Display device; 400: Input unit. Detailed Implementation

[0052] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0053] In the figures of this invention, the dimensions of the structures are shown enlarged from their actual dimensions for clarity.

[0054] In this invention, terms such as "first" and "second" can be used to describe various constituent elements, but these constituent elements should not be limited by these terms. These terms are used only for the purpose of distinguishing one constituent element from another.

[0055] The terminology used in this invention is for illustrative purposes only and is not intended to limit the invention. Unless explicitly defined differently in the context, singular expressions include plural expressions. In this application, terms such as "comprising" or "having" should be understood as specifying the presence of features, numbers, steps, actions, constituent elements, components, or combinations thereof as described in the specification, and not as precluding the presence or addability of one or more other features or numbers, steps, actions, constituent elements, components, or combinations thereof.

[0056] The specific structural and even functional descriptions of the embodiments of the present invention disclosed herein are merely illustrative for the purpose of explaining the embodiments of the present invention. The embodiments of the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein.

[0057] That is, the present invention can be modified in various ways and can have multiple forms. Some specific embodiments will be illustrated in the accompanying drawings and described in detail herein. However, this is not intended to limit the present invention to a specific disclosed form, but should be understood to include all modifications, equivalents, and even substitutions that fall within the spirit and technical scope of the present invention.

[0058] Figure 1 This is a side view of an engineering machine illustrating an exemplary embodiment. Although Figure 1 A wheel loader 10 is shown, but this does not mean that the control device of the exemplary embodiment of the construction machinery is limited to use in wheel loaders, but can be applied to a variety of industrial vehicles, etc. Hereinafter, only the wheel loader 10 will be described for ease of explanation.

[0059] Reference Figure 1 The construction machinery 10 may include a body 12, 14, a cab 40, and a front-mounted work device. Figure 1 Taking a wheel loader 10 as an example, the body 12 and 14 may include a front body 12 and a rear body 14 that are rotatably connected to each other. The front body 12 may include a front working device and front wheels 70. The rear body 14 may include a cab 40, an engine compartment 50, and rear wheels 72.

[0060] The front working device may include a boom 20 and a bucket 30. The boom 20 may be rotatably connected to the front vehicle body 12, and the bucket 30 may be rotatably connected to one end of the boom 20. The boom 20 is connected to the front vehicle body 12 via a pair of boom cylinders 22, and the boom 20 may be rotated vertically by the drive of the boom cylinders 22. A tilting stick 34 is rotatably connected to the boom 20 at approximately the center, and one end of the tilting stick 34 is connected to the front vehicle body 12 via a pair of bucket cylinders 32. The bucket 30, connected to the other end of the tilting stick 34 via a tilting rod, may be rotated vertically (unloading or digging) by the drive of the bucket cylinders 32.

[0061] Furthermore, the front body 12 and the rear body 14 are rotatably connected to each other by a center pin 16, and the front body 12 can bend left and right relative to the rear body 14 by the extension and retraction of a steering cylinder (not shown).

[0062] A travel device for driving the wheel loader 10 can be mounted on the rear of the vehicle body 14. An engine (not shown) can be located in the engine compartment and supply power output to the travel device. The travel device may include a torque converter, a transmission, a auger shaft, and an axle. The power output of the engine is transmitted to the front wheels 70 and the rear wheels 72 via the torque converter, the transmission, the auger shaft, and the axle, enabling the wheel loader 10 to travel.

[0063] A hydraulic pump (not shown) for supplying pressurized oil to the boom cylinder 22 and bucket cylinder 32 of the working device can be mounted on the rear of the vehicle body 14. The hydraulic pump can be driven using a portion of the power output from the engine. For example, with respect to the engine output, the hydraulic pumps for the working device and the steering pump can be driven by a power transmission device such as a gear system connected to the engine.

[0064] The hydraulic pump supplies working oil to drive the working device and can be classified as either a variable-capacity type or a constant-capacity type. A pump control device is connected to the variable-capacity type hydraulic pump and can control the discharge flow rate of the variable-capacity type hydraulic pump. The hydraulic pump can be connected to a main control valve (MCV) including a boom control valve and a bucket control valve via a predetermined hydraulic circuit. The discharge oil from the hydraulic pump can be supplied to the boom cylinder 22 and the bucket cylinder 32 through the boom control valve and the bucket control valve of the main control valve (MCV). The main control valve (MCV) can supply working oil discharged from the hydraulic pump to the boom cylinder 22 and the bucket cylinder 32 based on a pilot pressure signal based on the operation input of the operating lever. Thus, the boom 20 and the bucket 30 can be driven by the hydraulic pressure of the working oil discharged from the hydraulic pump.

[0065] The cab 40 is mounted on the body of the construction machinery, and in the case of a wheel loader, it may be mounted on the upper part of the rear body 14. A driving control device may be provided inside the cab 40. The driving control device may include a travel pedal, a brake pedal, an FNR travel lever, a control lever for operating work cylinders such as the boom cylinder 22 and the bucket cylinder 32, and a steering device such as a steering wheel for operating the steering cylinders.

[0066] As described above, the wheel loader 10 may include a driving system that uses the output of the engine to drive the driving device via a power transmission device, and a hydraulic system for driving front working devices such as the boom 20 and the bucket 30.

[0067] The control system of the construction machinery will be described below using a wheel loader as an example.

[0068] Figure 2 It shows the corresponding Figure 1 A side view of the bucket rising position at the boom rotation angle. Figure 3 It is shown Figure 1 A plan view showing the horizontal field of view angles of the upper and lower cameras when the construction machinery is traveling straight and turning to the left. Figure 4 It is shown Figure 1 A block diagram of the control system of the engineering machinery. Figure 5 It is shown Figure 4 A block diagram of an image processing device. Figure 6 This is a diagram showing the first image captured by the upper camera. Figure 7 This is a diagram showing a second image captured by the lower camera. Figure 8 It shows through Figure 5 Image processing device synthesis Figure 6 First image and Figure 7 The second image is a diagram of the image. (Refer to...) Figures 1 to 8 The control system of the wheel loader may include: a camera unit 100, mounted on the wheel loader 10, for capturing images of the front of the wheel loader 10; an angle information detection unit 150, for acquiring bending angle information of the front vehicle body 12 relative to the rear vehicle body 14; an image processing device 200, for processing images from the camera unit 100 in real time; and a display device 300, for displaying the images processed by the image processing device 200. Furthermore, the control system of the wheel loader may also include: a working device posture detection unit, for detecting the posture of the front working device connected to the front vehicle body 12; and an input unit 400, for setting image processing conditions in the image processing device 200.

[0069] The image processing unit 200 of the wheel loader 10 may be mounted on the rear body 14 as part of an engine control unit (ECU) or vehicle control unit (VCU) or as a separate controller. The image processing unit 200 may include specified hardware, software, and circuitry for performing the functions described herein. These components may be physically executed by electrical circuits such as logic circuits, microprocessors, and storage devices.

[0070] In some exemplary embodiments, the camera unit 100 is used to monitor the front of the wheel loader 10 when it is traveling or operating, and may include multiple cameras. Specifically, the camera unit 100 may include: an upper camera 110, which is disposed above the cab 40, for capturing images of the front of the cab 40 to obtain a first image IM1; and a lower camera 120, which is disposed on the front of the vehicle body 12, for capturing images of the front of the vehicle body 12 to obtain a second image IM2. For example, the upper camera and the lower camera may be fisheye cameras equipped with fisheye lenses. Although Figure 1 and Figure 2 The image shows one upper camera and one lower camera, but it is not limited to this; multiple upper cameras and multiple lower cameras may be provided.

[0071] The upper camera 110 may have a first vertical field of view (FoV) and a first horizontal field of view (θh1) with reference to the forward direction of the wheel loader. For example, the first vertical and first horizontal field of view may have an angle range of 60 degrees to 120 degrees. The lower camera 120 may have a second vertical field of view (θv2) and a second horizontal field of view (θh2). For example, the second vertical and second horizontal field of view may have an angle range of 60 degrees to 120 degrees.

[0072] The first image may be an image captured by the upper camera 110 with the upper part in front as the focus, and the second image may be an image captured by the lower camera 120 with the lower part in front as the focus.

[0073] It can be set to partially overlap the first vertical field of view θv1 of the upper camera 110 and the second vertical field of view θv2 of the lower camera 120, and to partially overlap the first horizontal field of view θh1 of the upper camera 110 and the second horizontal field of view θh2 of the lower camera 120, so that the first image and the second image partially overlap each other.

[0074] In some exemplary embodiments, the upper camera 110 may be aligned with or positioned behind the central axis (steering center axis) of the center pin 16, and the lower camera 120 may be positioned in front of the center axis relative to the center pin 16.

[0075] Since the upper camera 110 and the lower camera 120 are positioned at different locations relative to the steering center axis, such as Figure 3 As shown, when the vehicle body 12 bends to the left (or right), the direction observed by the upper camera 110 and the direction observed by the lower camera 120 will be different from each other. As described later, the image processing device 200 can combine the first image IM1 and the second image IM2 into a single image, and process the image so that the position of the transparent area of ​​at least one of the first image IM1 and the second image IM2 in the synthesized image is matched with the steering hinge angle θs of the wheel loader 10.

[0076] In some exemplary embodiments, the working device posture detection unit can detect whether the forward working device encroaches on the transparent processing area within the display area of ​​the display device 300. As described later, the transparency processing of the captured image is performed when the forward working device encroaches on a predetermined position (a position matching the steering hinge angle), that is, the actual position of the entire display area of ​​the display device 300 corresponding to the predetermined transparent processing area, thereby ensuring the operator's field of vision. The posture of the forward working device can include the position of the bucket 30 (distance from the bucket height) or the posture of the boom 20 (boom rotation angle). For this purpose, the working device posture detection unit can include a boom angle sensor 24 for detecting the position of the bucket 30 or the posture of the boom 20. At the same time, a bucket angle sensor (not shown) can be included for detecting the relative rotation angle of the boom 20 and the bucket 30. Instead of the boom angle sensor 24, a displacement sensor for detecting the stroke of the cylinder driving the boom 20 can be included.

[0077] In addition, the posture detection unit of the working device may include an image analysis device (e.g., a shape recognition unit), which determines the posture of the working device by analyzing the image of the working device captured by the camera.

[0078] The boom angle sensor 24 can detect the rotation angle of the boom 20 and provide information on the position of the bucket 30 based on this. For example... Figure 2As shown, the rotation angle of boom 20 can be the angle θ between the extension line L of boom 20 (bucket 30) at its lowest position (0%) and the extension line R of boom 20. The rotation angle of boom 20 at its highest position (max boom height) is θmax.height, at which point the position of boom (bucket) can be at its maximum height (100%).

[0079] In some exemplary embodiments, the angle information detection unit 150 may include a sensor for detecting the bending angle information of the rear vehicle body 14 relative to the front vehicle body 12 caused by the steering device. The detection values ​​from the sensor can be transmitted to the image processing device 200 via a CAN network.

[0080] For example, the angle information detection unit 150 may include a center pin angle sensor, a steering cylinder displacement sensor, or a gyroscope sensor. The center pin angle sensor may be disposed on the center pin 16 connecting the front vehicle body 12 and the rear vehicle body 14 to detect the steering hinge angle of the front vehicle body 12. The steering cylinder displacement sensor may include a stroke sensor for detecting the stroke of the cylinder inside the steering cylinder. The gyroscope sensor may be disposed on both the front vehicle body 12 and the rear vehicle body 14, and includes a first sensor and a second sensor that identify absolute angles of six axes or more to detect the relative angle between the first and second sensors, thereby identifying the steering hinge angle.

[0081] In some exemplary embodiments, the image processing apparatus 200 can combine a first image IM1 and a second image IM2 captured by the upper camera 110 and the lower camera 120 into a single image, determine the steering hinge angle θs of the front vehicle body 12 obtained from the angle information detection unit 150, and set the position of the transparent processing area of ​​at least one of the first and second images in the synthesized image based on the steering hinge angle. The image processing apparatus 200 may include a steering angle calculation unit 210, an image synthesis unit 220, a transparency processing unit 230, an image rendering unit 240, and a storage unit 250. The image processing apparatus 200 may be provided as a control device built into a control device or display device of engineering machinery.

[0082] Specifically, the steering angle calculation unit 210 can determine the steering hinge angle θs of the front vehicle body 12 from the bending angle information detected by the angle information detection unit 150. The steering angle calculation unit 210 can convert the detected value from the angle information detection unit 150 into the steering hinge angle using a conversion table stored in the storage unit 250.

[0083] The image compositing unit 220 can combine the first image IM1 and the second image IM2 into a single image. The image compositing unit 220 matches and combines repeated images from the first image and the second image captured by the upper camera 110 and the lower camera 120 into a single image. The transparency processing unit 230 can perform transparency processing on at least one of the first and second images in the transparency processing area. The image rendering unit 240 can render the image-processed composite image into a three-dimensional image. The image rendering unit 240 can process the composite image in a way that displays it as a realistic image and output it to the display device 300. The functions of the image compositing unit 220, the transparency processing unit 230, and the image rendering unit 240 can be implemented through computer processing using a single processor such as a GP or CPU for image processing, or through separate processors.

[0084] In some exemplary embodiments, the transparency processing unit 230 may perform transparency processing on one of the first image and the second image in the synthesized image based on the detected posture of the working device. The transparency processing unit 230 may perform image processing on at least one of the first image and the second image in such a way that the transparency processing area, which is a portion of the entire display area of ​​the display device 300, is made transparent. The transparency processing area may be defined as the area where the forward view is obstructed by the forward working device, including the lifting boom 20 and the bucket 30.

[0085] In the transparency processing, the first image or the second image can be removed or made semi-transparent within the transparency processing area of ​​the synthesized image to overlap with the background image, or the outline of the appearance can be displayed in two dimensions using lines or dashed lines so that only the shape can be identified. For example, the first image or the second image within the transparency processing area can be removed from the synthesized image using techniques such as AlphaBlending.

[0086] In some exemplary embodiments, the transparency processing unit 220 can perform transparency processing corresponding to the location where at least a portion of the forward working device infringes upon the transparency processing area. For example, when it can be determined that the position of the bucket or boom, where at least a portion of the forward working device does not infringe upon the transparency processing area, is lower than a predetermined position (transparency conversion position), the second image in the synthesized image can be made transparent. Simultaneously, when the position of the bucket or boom, where it can be determined that it infringes upon the transparency processing area, is higher than a predetermined position (transparency conversion position), the first image in the synthesized image can be made transparent. For example, the predetermined boom position can be set such that the rotation angle θ of the boom 20 is within the range of 15 to 20 degrees.

[0087] When the bucket 30 is at its lowest position (0%) and between the transparent transition position, which is the boundary of the transparent processing area (as defined by the pre-set bucket position), the subject captured by the upper camera 110 can be highlighted by making the second image captured by the lower camera 120 transparent. When the bucket 30 is in a relatively low position, the forward view of the front vehicle body 12 may be obstructed by the forward working device, including the boom 20 and the bucket 30, in the second image captured by the lower camera 120. The transparent processing unit 230 can prevent the view from being obstructed by the forward working device by making the second image transparent and highlighting the first image.

[0088] When the bucket 30 is located between the pre-set bucket position and the highest position (100%) of the transparency processing area, by making the first image captured by the upper camera 110 located within the transparency processing area transparent, the subject captured by the lower camera 120 can be displayed as the focus. When the bucket 30 is in a relatively high position, the forward view of the cab 40 may be obstructed by the forward working device including the boom 20 and the bucket 30 in the first image captured by the upper camera 110. The transparency processing unit 230 can prevent the view from being obstructed by the forward working device by making the first image transparent and displaying it with the second image as the focus.

[0089] When the bucket 30 rises or falls and passes through the pre-set bucket position (transparency conversion position), the image located in the transparency processing area where the transparency processing unit 230 performs transparency processing can be converted from the second image to the first image, or from the first image to the second image.

[0090] Unlike other methods, when the rotation angle θ of the boom is within a first angle range, the transparency processing unit 220 can make the second image in the synthesized image transparent; when the rotation angle θ of the boom is within a second angle range, the transparency processing unit 230 can make both the first image and the second image transparent in the transparency processing area of ​​the synthesized image; and when the rotation angle θ of the boom is within a third angle range, the transparency processing unit 230 can make the first image in the synthesized image transparent. For example, the first angle range can be set to be within 0 to 15 degrees, the second angle range to be within 15 to 25 degrees, and the third angle range to be within 25 to 45 degrees. In some exemplary embodiments, the image processing conditions in the image processing apparatus 200 can be set through the input unit 400. For example, the image processing conditions can be for the area occupied by the transparency processing area in the entire display area. By determining the transparency processing area, the transparency transition positions of the first image and the second image, as well as the transparency processing area within the entire display area of ​​the display device 300, can be set. For example, the transparency transition position can refer to the boundary position of the transparency processing area. When the bucket 30 moves and is located at the boundary of the transparency processing area, it can be considered that the bucket 30 is located at the pre-set position for transparency transition. The area and position of the transparency processing area, the transparency transition time point, etc., can be fixedly set by the manufacturer according to the equipment model, and can be freely changed and set by the operator or maintenance personnel.

[0091] For example, the input unit 400 can be implemented as a dashboard option, through which the operator can change the transparency transition time point, the transparency processing area, etc.

[0092] As described above, when the transparency processing area and the transparency transition time point are set, the display device 300 can distinguish between the transparency processing area R, where the image captured by the camera unit is transparently processed, and the outer area of ​​the transparency processing area R, and display the image accordingly. The display device 300 can either additionally display the outline of the transparency processing area R in a manner that allows for the distinction of the transparency processing area R, or it can connect the image corresponding to the transparency processing with the image of the outer area of ​​the transparency processing area R without the outline and display them together.

[0093] Furthermore, the display device 300 can display a first image in the outer region of the transparency processing area R, and display a transparent image of the first image, the second image, or at least one of the first image and the second image in the inner region of the transparency processing area R as the transparency processing proceeds.

[0094] For example, when the bucket 30 is located outside the transparent processing area R, the display device 300 may only display the first image that connects the transparent processing area R and the outer area of ​​the transparent processing area R. Conversely, for the interior of the transparent processing area R, a transparent image highlighting the first image may also be displayed. In this case, the operator may perceive that the display device 300 is displaying the first image as a whole because of the highlighted transparent image. Furthermore, when at least a portion of the bucket 30 is located inside the transparent processing area R, for the interior of the transparent processing area R, the display device 300 may display a transparent processing image highlighting a second image or a second image; for the outer area of ​​the transparent processing area R, the display device 300 may display the first image excluding the image within the transparent processing area R.

[0095] In some exemplary embodiments, the transparency processing unit 230 can set the position of the transparent processing area in the synthesized image based on the bending angle information of the front vehicle body obtained from the angle information detection unit 150. The transparency processing unit 230 can adjust the position of the transparent processing area based on the steering hinge angle θs of the front vehicle body calculated by the steering angle calculation unit 210.

[0096] The position of the transparent processing area can be adjusted to follow the calculated steering hinge angle. As the steering mechanism causes the stroke of the steering cylinder to change, the front vehicle body 12 bends to the left (or right). At this time, the front working device, including the boom 20 and bucket 30, moves out of the initially set transparent processing area, potentially obstructing the forward view. In the case of a steering cylinder displacement sensor, the steering angle calculation unit 210 can convert the stroke value of the steering cylinder into a steering hinge angle. The transparent processing unit 230 can automatically change the position of the transparent processing area in a manner that follows the converted steering hinge angle. Figures 6 to 8 As shown, when a part of the boom or bucket is within the transparent processing area R of the first image M1, a part of the second image can be cropped based on the first image and then composited into the first image.

[0097] First, the position of the transparent processing area R can be set based on the detected posture of the working device, the position of the boom or bucket, and the bending angle information of the front vehicle body. Figures 6 to 8 For ease of explanation, the following is shown: Figure 3The image includes a first image, a second image, and a composite image captured by an upper camera and a lower camera when the construction machinery is traveling in a straight line (state A). In this case, the transparent processing area R can be located in its initial position, for example, in the center of the screen 300 of the display device.

[0098] After that, as Figure 7 As shown, a portion of the second image corresponding to the transparent processing area can be determined as the cropping region CR, and a cropped image can be extracted based on the determined cropping region CR. Then, the area corresponding to... Figure 6 The transparency of the transparent processing area R of the first image can be adjusted. For example, the transparency of the transparent processing area R of the first image can be adjusted to a level of 180 / 255. In this case, the transparency of the cropped silhouette image extracted from the second image does not need to be adjusted.

[0099] Next, as Figure 8 As shown, the cropped image extracted from the second image can be composited into the transparent processing area R of the first image. In this case, when the area of ​​the cropped area CR is smaller than the area of ​​the transparent processing area R in the first image, the cropped image can be enlarged and composited into the first image.

[0100] When there is a person O in front of the wheel loader, Figure 6 In the first image, person O cannot be accurately identified due to obstruction by the boom or bucket, but... Figure 7 Person O can be identified in the second image. Figure 7 The cropped silhouette image extracted from the second image may include an image of person O. Because the region is synthesized into the transparent area of ​​the first image corresponding to the cropped silhouette image, therefore... Figure 8 The transparency processing area of ​​the synthesized image can be confirmed from Figure 7 The extracted image of person O can thus improve the forward field of vision limited by the working device.

[0101] when Figure 3 When the construction machinery is turning left or right, the transparent processing area R can move to the left or right side of the display device screen 300 according to the bending angle information of the vehicle body in front. In this case, the position of the transparent processing area R can move to the left or right side according to the bending angle information, and at least one of the first image and the second image can be transparentized in the moving transparent processing area R.

[0102] In some exemplary embodiments, the image processing apparatus 200 can identify people or other objects (equipment, vehicles, etc.) in the first image, the second image, and the composite image, and use methods such as bounding boxes ( Figure 8 The object identified is marked with a D) mark, or the outline of the identified object is displayed with a dashed line, etc.

[0103] The image processing apparatus 200 can compare the actual image in the image with the learned image of the object stored in the storage unit 250. When the actual image and the image of the object are the same, the object can be identified. Here, the learned image of the object may include images of various shapes captured by a camera that have undergone machine learning and been stored. Machine learning is a field of artificial intelligence and can refer to algorithms that enable processing devices such as computers to learn.

[0104] The following describes the use of Figure 4 The control system of construction machinery and the methods for controlling construction machinery will be described below. The following description will also use a wheeled loader as a reference, similar to the methods described above.

[0105] Figure 9 This is a sequence diagram illustrating a control method for a wheel loader according to an exemplary embodiment. Figure 10 It is shown Figure 3 The image captured by the upper camera when the construction machinery is traveling straight (state A) is displayed on the screen of the display device in the cab. Figure 11 It is shown Figure 3 The image captured by the upper camera when the construction machinery turns to the left (state B) is displayed on the screen of the display device in the cab.

[0106] Reference Figures 1 to 11 First, the upper camera 110 and the lower camera 120 mounted on the wheel loader 10 can acquire a first image IM1 and a second image IM2 (S100), and obtain the bending angle information of the front vehicle body 12 relative to the rear vehicle body 14 (S110). The first image IM1 and the second image IM2 can be combined into a single image (S120), and the position of the transparent processing area can be set according to the bending angle information (S130).

[0107] In some exemplary embodiments, a first image IM1 of the front of the cab 40 can be acquired using an upper camera 110 mounted on the cab 40. A second image IM2 of the front of the front vehicle 12 can be acquired using a lower camera 120 mounted on the front vehicle body 12.

[0108] The first image can be an image captured with the upper front part as the focus point via the upper camera 110, and the second image can be an image captured with the lower front part as the focus point via the lower camera 120. It can be configured such that the first vertical field of view θv1 of the upper camera 110 and the second vertical field of view θv2 of the lower camera 120 partially overlap, and that the first horizontal field of view θh1 of the upper camera 110 and the second horizontal field of view θh2 of the lower camera 120 partially overlap, so that the first image and the second image partially repeat each other.

[0109] For example, the upper camera 110 may be aligned with or positioned behind the central axis (rotational central axis) of the central pin 16, while the lower camera 120 may be positioned in front of the central axis relative to the central pin 16. Because the upper camera 110 and the lower camera 120 are positioned differently relative to the rotational central axis, as... Figure 3 As shown, when the front vehicle body 12 connected to the front working device rotates to the left (or right), the direction observed by the upper camera 110 and the direction observed by the lower camera 120 will be different from each other.

[0110] In some exemplary embodiments, bending angle information of the front vehicle body 12 relative to the rear vehicle body 14 can be obtained from the angle information detection unit 150. The angle information detection unit 150 may include a sensor for detecting the bending angle information of the front vehicle body 12 relative to the rear vehicle body 14 caused by the steering device. The detection value from the sensor can be transmitted to the image processing device 200 via a CAN network. For example, the angle information detection unit 150 may include a center pin angle sensor, a steering cylinder displacement sensor, or a gyroscope sensor.

[0111] In some exemplary embodiments, the image processing apparatus 200 can match and combine the first image IM1 and the second image IM2 into a single image. Furthermore, the image processing apparatus 200 can determine the steering hinge angle θs of the front vehicle body 12 obtained from the angle information detection unit 150.

[0112] The steering angle calculation unit 210 of the image processing device 200 can determine the steering hinge angle θs of the front vehicle body 12 from the bending angle information detected by the angle information detection unit 150. The steering angle calculation unit 210 can convert the detected value from the angle information detection unit 150 into the steering hinge angle using a conversion table stored in the storage unit 250.

[0113] The transparency processing unit 230 of the image processing apparatus 200 can set the position of the transparency processing area in the synthesized image based on the bending angle information of the front vehicle body obtained from the angle information detection unit 150. The transparency processing unit 230 can adjust the position of the transparency processing area based on the steering hinge angle θs of the front vehicle body calculated by the steering angle calculation unit 210.

[0114] The image processing device 200 can set a portion of the first image IM1 from the upper camera 110 as a first composite region, set a portion of the second image IM2 from the lower camera 120 as a second composite region, and after performing semi-transparent processing on the second composite region of the second image IM2, composite the semi-transparent second composite region into the first composite region of the first image IM1.

[0115] like Figure 10 As shown, when the wheel loader 10 is traveling straight, the transparency processing unit 230 of the image processing device 200 can maintain the position of the transparency processing area R at its initial position by following the calculated steering hinge angle.

[0116] Furthermore, regarding the transparent processing area R, one of the first image and the second image captured by the upper camera 110 and the lower camera 120 can be selected, or a portion of the first image and the second image can be selectively made transparent. For example, a portion of the image including the boom or bucket in the front working device can be selectively made transparent.

[0117] like Figure 11 As shown, when the vehicle body 12 is turned to the left by the steering mechanism, the front working device, including the boom 20 and bucket 30, will move out of the initially set transparency processing area, potentially obstructing the forward view. At this time, the steering angle calculation unit 210 of the image processing device 200 can calculate the steering hinge angle θs from the detection value from the angle information detection unit 150, and the transparency processing unit 230 of the image processing device 200 can automatically change the position of the transparency processing area R in a manner that follows the calculated steering hinge angle.

[0118] Next, at least one of the first image and the second image can be made transparent within the transparency processing area R (S140), and the composite image after transparency processing can be displayed on the display device 300 (S150).

[0119] In some exemplary embodiments, the posture of the forward working device can be detected. The rotation angle of the boom 20 connected to the forward body 12 can be detected. The position of the bucket 30 relative to the boom angle sensor 24, i.e., the height of the bucket 30 above the ground, can be detected. The lifting height of the bucket can be determined from the rotation angle of the boom 20 measured by the boom angle sensor 24.

[0120] In some exemplary embodiments, the image portion of the boom or bucket that is part of the forward working device in the first image and the second image may be selectively transparent in the transparency processing area R (S140), and the composite image after transparency processing may be displayed by the display device 300 (S150).

[0121] In the transparent processing area R, the outline of the boom or bucket that was photographed and made transparent in the first or second image can be displayed using lines, dashed lines, or shadows, so that only the shape can be identified.

[0122] The outline of the boom or bucket displayed in the transparent processing area R is derived from an image actually taken in the first or second image, so that it can be consistent with the movement of the boom or bucket corresponding to the operation of the actual operator.

[0123] In some exemplary embodiments, the image processing device 200 can identify objects within a transparent first or second forward image and display the outline of the identified objects using dashed lines or similar methods, or mark the identified objects with markers such as bounding boxes. For example, a computing device can generate a composite image by identifying objects within an image based on a pre-determined algorithm and displaying the outline of the corresponding object. In this case, the driver can easily confirm the shape of parts of the transparent wheel loader 100 using outline dashed lines or similar methods.

[0124] like Figure 2 As shown, the rotation angle of boom 20 can be the angle θ between the extension line L of boom 20 at its lowest position (0%) and the extension line R of boom 20. The rotation angle of boom 20 at its highest position (max boom height) is θmax.height, at which point the bucket position can be at its maximum height (100%).

[0125] Next, it can be confirmed whether the bucket position is higher or lower than a preset position (transparency conversion position). The preset position can be the transparency conversion position, that is, the boundary of the transparency processing area R. That is, comparing the bucket position with the preset position can include confirming whether a part of the bucket 30 or boom 20 is located inside the transparency processing area R. When the position of the bucket or boom is lower than the preset position, the second image in the synthesized image can be made transparent; and when the position of the bucket or boom is higher than the preset position, the first image in the synthesized image can be made transparent. Here, based on the image displayed by the display device 300, the preset position can be the lower boundary of the preset transparency processing area R. Afterwards, the synthesized image after transparency processing can be displayed by the display device 300. At this time, the display device 300 can display the first image in the outer area of ​​the transparency processing area A.

[0126] In some exemplary embodiments, the image processing apparatus 200 may perform transparency processing on at least one of the first image and the second image in the synthesized image based on the detected boom position.

[0127] The transparency processing unit 230 can perform image processing on the first image and the second image in such a way that only a portion of the transparency processing area R, which is part of the entire display area of ​​the display device 300, is made transparent. The transparency processing area R can be defined as the area where the forward view is obstructed by the forward working device, including the lifting boom 20 and the bucket 30.

[0128] In the transparency processing, the first image or the second image can be removed or made semi-transparent within the transparency processing area R of the synthesized image so as to overlap with the background image, or the outline of the appearance can be displayed in two dimensions with lines or dashed lines so that only the shape can be identified. For example, the first image or the second image within the transparency processing area can be removed from the synthesized image by using alpha blending technology or the like.

[0129] When the bucket 30 or boom 20 is located between its lowest position (0%) and the pre-set bucket or boom position, by making the second image captured by the lower camera 120 transparent, a transparent image focusing on the subject captured by the upper camera 110 can be displayed inside the transparent processing area R of the display device 300. When the bucket 30 or boom 20 is located at a relatively low position, by making the portion of the forward working device in the second image that obstructs the forward view transparent, the forward object can be identified in the synthesized image.

[0130] When the bucket 30 or boom 20 is at its highest position (100%) from the predetermined position, by making the first image captured by the upper camera 110 transparent, a transparent image focusing on the subject captured by the lower camera 120 can be displayed inside the transparent processing area R of the display device 300. When the bucket 30 or boom 20 is at a relatively high position, by making the portion of the forward working device in the first image that obstructs the forward view transparent, the forward object can be identified in the synthesized image.

[0131] For example, the position of the boom can be set such that the rotation angle θ of the boom 20 is within the range of 15 degrees to 20 degrees.

[0132] Unlike the previous method, when the boom's rotation angle θ is within a first angle range, the second image in the synthesized image can be made transparent; when the boom's rotation angle θ is within a second angle range, both the first and second images can be made transparent in the transparency processing area of ​​the synthesized image; and when the boom's rotation angle θ is within a third angle range, the first image in the synthesized image can be made transparent. For example, the first angle range can be set to be within 0 to 15 degrees, the second angle range to be within 15 to 25 degrees, and the third angle range to be within 25 to 45 degrees.

[0133] In some exemplary embodiments, image processing conditions for making the first image and the second image transparent can be set. The image processing conditions in the image processing apparatus 200 can be set via the input unit 400. For example, the image processing conditions may include the transparency transition time point of the first image and the second image, the area occupied by the transparency processing area in the entire display area of ​​the display device 300, etc. The transparency transition time point of the first image and the second image can be determined by the position of the bucket 30 or the boom 20 and the pre-set position of the bucket or boom. The transparency processing area can be selected according to the model of the equipment.

[0134] For example, the input unit 400 can be implemented as a dashboard option, allowing the operator to change the transparency transition time point, the area of ​​the transparency processing region, etc. The input unit 400 can be provided as a separate operating device located inside the driver's cab, an operating device integrated with the display device, or a touchscreen constituting the display screen of the display device. Therefore, the operator can set various image processing conditions, such as setting the area around objects requiring attention during operation as a transparency processing area.

[0135] As described above, the first image and the second image captured by the upper camera 110 mounted on the cab 40 of the wheel loader 10 and the lower camera 120 mounted on the front body 12 can be combined into a single image. The position of the transparency processing area R in the combined image is determined according to the steering hinge angle θs of the front body 12. The first image and the second image are made transparent within the transparency processing area R according to the position of the bucket 30 or the boom 20 connected to the front body 12. The transparent image is then displayed through the display device 300.

[0136] When the bucket 30 or boom 20 is in a relatively low position between its lowest position (0%) and the previously set bucket or boom position, the forward view may be obstructed by the forward working device, including the boom 20 and bucket 30, in the second image captured by the lower camera 120. When the bucket 30 or boom 20 is in a relatively high position between the previously set bucket or boom position and the highest position (100%) of the transparent display area, the forward view may be obstructed by the forward working device, including the boom 20 and bucket 30, in the first image captured by the upper camera 110.

[0137] By making the first or second image transparent in the synthesized image according to the position of the bucket 30 or the boom 20, blind spots where the forward view is obstructed by the forward working device can be eliminated.

[0138] Furthermore, since the upper camera 110 and the lower camera 120 are installed at different positions, the bucket 30 may detach from the transparent processing area R when turning to the left or right, resulting in inconsistency. When the wheel loader 10 turns, the steering hinge angle θs of the front vehicle body 12 can be determined based on the bending angle information of the front vehicle body 12 relative to the rear vehicle body 14, and the images are processed in a way that matches the position of the transparent processing area R with the steering hinge angle θs in the composite images of the first image IM1 and the second image IM2.

[0139] Therefore, even when the wheel loader 10 is turning, it can prevent the forward view from being obstructed by the forward working device, including the boom 20 and bucket 30. This increases the operator's awareness, ensuring stability and preventing accidents.

[0140] Although the invention has been described above with reference to embodiments thereof, those skilled in the art will understand that various modifications and alterations can be made to the invention without departing from the spirit and scope of the invention as set forth in the following claims.

Claims

1. A control system for engineering machinery, characterized in that, include: An upper camera is mounted on the driver's cab at the rear of the vehicle to capture images of the area in front of the driver's cab; A lower camera is mounted on the front body of the rear vehicle that is rotatably connected to the front of the front vehicle to capture images of the front of the front vehicle. An angle information detection unit is used to detect the bending angle information of the front vehicle body relative to the rear vehicle body; An image processing apparatus is used to combine a first image and a second image captured by the upper camera and the lower camera into a single image, and to set the position of a transparent processing area in the synthesized image by at least one of the first image and the second image being transparent, based on the bending angle information obtained from the angle information detection unit. as well as A display device for displaying a composite image synthesized by the image processing device. The image processing device includes: The steering angle calculation unit determines the steering hinge angle of the front vehicle body based on the bending angle information obtained from the angle information detection unit. as well as The transparency processing unit determines the position of the transparency processing area in the synthesized image based on the determined steering hinge angle. The transparency processing unit performs image processing on at least one of the first image and the second image in such a way that the transparency processing area, which is a part of the entire display area of ​​the display device, is made transparent.

2. The control system for engineering machinery according to claim 1, characterized in that, The steering angle calculation unit converts the detection value obtained from the angle information detection unit into the steering hinge angle value of the front vehicle body.

3. The control system for engineering machinery according to claim 1, characterized in that, The angle information detection unit includes a center pin angle sensor, a steering cylinder displacement sensor, or a gyroscope sensor.

4. The control system for engineering machinery according to claim 1, characterized in that, It also includes a work device posture detection unit, which is used to detect the posture of the front work device. The image processing device performs transparency processing on at least one of the first image and the second image in the transparency processing area based on the posture of the front working device detected by the working device posture detection unit.

5. The control system for engineering machinery according to claim 4, characterized in that, When at least a portion of the forward working device infringes upon the predetermined position, the image processing device makes the first image transparent in the transparency processing area, and when the forward working device does not infringe upon the predetermined position, it makes the second image transparent in the transparency processing area.

6. The control system for engineering machinery according to claim 1, characterized in that, It also includes an input unit for setting image processing conditions in the image processing apparatus.

7. The control system for engineering machinery according to claim 6, characterized in that, The image processing conditions include the time points of the transparency transition of the first image and the second image, or the area occupied by the transparency processing area in the entire display area of ​​the display device.

8. The control system for engineering machinery according to claim 1, characterized in that, The image processing device processes the image in a manner that displays the outline of the boom or bucket that was captured and made transparent in the first or second image using lines or dashed lines in the transparent processing area.

9. The control system for engineering machinery according to claim 8, characterized in that, The outline of the boom or bucket is displayed by making the actual first or second image of the boom or bucket transparent.

10. The control system for engineering machinery according to claim 1, characterized in that, The image processing device selectively performs transparency processing on the boom or bucket images combined with the front vehicle body in the first image and the second image in the transparency processing area.

11. The control system for engineering machinery according to claim 1, characterized in that, The composite image includes objects identified by the image processing device within the first image and the second image.

12. The control system for engineering machinery according to claim 1, characterized in that, The image processing device identifies people, animals, buildings, or equipment as objects using a pre-determined algorithm.

13. The control system for engineering machinery according to claim 1, characterized in that, The image processing device extracts a cropped image from one of the first image and the second image, makes a portion of the other image transparent, and then composites the extracted cropped image into the transparent portion to generate the transparent image.

14. The control system for engineering machinery according to claim 13, characterized in that, The transparent portion is the transparent processing area.

15. The control system for engineering machinery according to claim 1, characterized in that, The image processing device sets a portion of the first image as a first composite region and a portion of the second image as a second composite region, performs semi-transparent processing on the second composite region of the second image, and composites the semi-transparent second composite region onto the first composite region of the first image to create the composite image.

16. A control method for engineering machinery, characterized in that, include: A first image of the front of the driver's cab is obtained from an upper camera mounted on the driver's cab at the rear of the vehicle. A second image of the front of the front vehicle is obtained from a lower camera mounted on the front vehicle body that is rotatably connected to the rear vehicle body; Obtain the bending angle information of the front vehicle body relative to the rear vehicle body; Combine the first image and the second image into a single image; Based on the obtained bending angle information of the front vehicle body, the position of the transparent processing area is set in the synthesized image; Transparency processing is performed on at least one of the first image and the second image within the transparency processing area; as well as The image after transparency processing is displayed on a display device. The step of setting the position of the transparency processing area based on the obtained bending angle information of the front vehicle body includes: The steering hinge angle of the vehicle body at the front is determined from the bending angle information; as well as Based on the steering hinge angle, the position of the transparent processing area is determined in the synthesized image. When setting the position of the transparency processing area, image processing is performed on at least one of the first image and the second image in such a way that the transparency processing area, which is a part of the entire display area of ​​the display device, is transparent.

17. The control method for engineering machinery according to claim 16, characterized in that, In the step of determining the steering hinge angle of the front vehicle body from the bending angle information, the bending angle detection value is converted into the steering hinge angle value of the front vehicle body.

18. The control method for engineering machinery according to claim 16, characterized in that, The step of obtaining bending angle information of the front vehicle body relative to the rear vehicle body includes: using a bending angle detection sensor, the bending angle detection sensor including a center pin angle sensor, a steering cylinder displacement sensor or a gyroscope sensor.

19. The control method for engineering machinery according to claim 16, characterized in that, Also includes: Detect the posture of the front working device. The step of making at least one of the first image and the second image transparent within the transparency processing area includes: making at least one of the first image and the second image transparent according to the detected posture of the front working device.

20. The control method for engineering machinery according to claim 16, characterized in that, Also includes: Set image processing conditions that make the first image and the second image transparent.

21. The control method for engineering machinery according to claim 20, characterized in that, The image processing conditions include the time points of the transparency transition of the first image and the second image, or the area occupied by the transparency processing area in the entire display area of ​​the display device.

22. A control method for engineering machinery, characterized in that, include: A first image of the front of the driver's cab is obtained from an upper camera mounted on the driver's cab at the rear of the vehicle. A second image of the front of the front vehicle is obtained from a lower camera mounted on the front vehicle body that is rotatably connected to the rear vehicle body; Obtain the bending angle information of the front vehicle body relative to the rear vehicle body; Based on the obtained bending angle information of the front vehicle body, the position of the transparent processing area is set in the first image. In the second image, a portion of the area corresponding to the transparent processing area is set as the cropping area to extract the cropped image; Adjust the transparency of the transparent area of ​​the first image; The cropped image extracted from the second image is composited into the transparent area of ​​the first image; and The synthesized image is displayed on a display device.

23. The control method for engineering machinery according to claim 22, characterized in that, The step of compositing the extracted silhouette image into the transparent area of ​​the first image includes: Adjust the size of the cropped silhouette image extracted from the second image; and The adjusted silhouette image is composited into the transparent area of ​​the first image.

24. The control method for engineering machinery according to claim 22, characterized in that, The step of adjusting the transparency of the transparent area of ​​the first image includes: The outline of the boom or bucket is displayed in the transparent area of ​​the first image using lines or dashed lines.

25. The control method for engineering machinery according to claim 22, characterized in that, Also includes: In the synthesized image, people, animals, buildings, or equipment are identified as objects using a pre-determined algorithm.

Images