Around view image display device for articulated dump truck and around view image display method

Abstract

According to one embodiment of the present invention, an around view image display device comprises a processing device, the processing device comprising: a plurality of cameras installed at locations that enable capturing of a blind area between a front frame and a rear frame of an articulated dump truck and surroundings of the articulated dump truck; and a processing device communicatively connected with the plurality of cameras, wherein the processing device can acquire a plurality of images from the plurality of cameras, synthesize the plurality of images by using a previously implemented 3D model to generate a synthesized image, and output the generated synthesized image by using a preset device.

Description

Around-view image display device and around-view image display method for articulated dump truck

[0001] The following embodiments relate to an around-view image display device for an articulated dump truck and an around-view image display method for an articulated dump truck.

[0002]

[0003] In general, construction machinery such as dozers, loaders, excavators, graders, dump trucks, and articulated dump trucks are primarily utilized in earthmoving environments. In particular, dump trucks with hinged front and rear frames and steering cylinders that allow left-right directional control are called articulated dump trucks.

[0004] While a typical dump truck turns by turning its front wheels, an articulated dump truck turns by turning its front end, reducing the turning radius and enabling it to move easily in narrow work sites and secure an efficient moving route.

[0005] Because existing articulated dump trucks do not have a camera installed between the cabin included in the front frame and the dump included in the rear frame, a blind area that cannot be seen by the articulated dump truck driver is created between the cabin included in the front frame and the dump included in the rear frame.

[0006]

[0007] The present invention was invented to solve the above-mentioned problem, and its purpose is to provide an around-view image display device and an around-view image display method that can display an image of a blind area between a cabin included in a front frame of an articulated dump truck and a dump included in a rear frame using a composite image that synthesizes a plurality of images.

[0008] In addition, another object of the present invention is to provide an around-view image display device and an around-view image display method capable of displaying a forward composite image generated by synthesizing images generated by at least two cameras positioned on the same line based on information on the moving speed of an articulated dump truck.

[0009] In addition, another object of the present invention is to provide an around-view image display device and an around-view image display method that can display a partially synthesized image generated by synthesizing images generated by four cameras located in preset areas based on transmission information and an image generated by a camera located at the rear of a construction machine.

[0010]

[0011] According to one embodiment of the present invention, an around-view image display device for an articulated dump truck includes a plurality of cameras installed at positions capable of photographing a blind area between a front frame and a rear frame of the articulated dump truck and the surroundings of the articulated dump truck, and a processing device connected to the plurality of cameras so as to be communicable, wherein the processing device can obtain a plurality of images from the plurality of cameras, synthesize the plurality of images using a pre-implemented 3D model to generate a synthesized image, and output the generated synthesized image using a pre-set device.

[0012] In addition, the processing device can obtain a first image and a second image generated by two cameras positioned on the same line in the longitudinal direction of the articulated dump truck and photographing the front of the articulated dump truck, one of which photographs the front blind area where the driver's view is obstructed by the cabin, and synthesize the first image and the second image using the pre-implemented 3D model to generate a front composite image without a blind area.

[0013] In addition, the processing device can obtain the moving speed of the articulated dump truck, compare the obtained moving speed of the articulated dump truck with preset speed information, and output the forward composite image using the preset device according to the comparison result.

[0014] In addition, the processing device can obtain images generated by at least four cameras positioned in a preset area among the plurality of cameras, and generate a partial composite image by synthesizing the four images determined using the pre-implemented 3D model.

[0015] Additionally, the processing device may include a blind area between the front frame and the rear frame of the articulated dump truck in the partial composite image.

[0016] In addition, the processing device can obtain transmission information of the articulated dump truck, compare the obtained transmission information with preset transmission information, and simultaneously output the generated partial composite image and a preset image among the plurality of images based on the comparison result.

[0017] In addition, the processing device can simultaneously output the partial composite image and an image capturing the rear of the articulated dump truck among the plurality of images when the acquired transmission information and the preset transmission information are the same.

[0018] According to another embodiment of the present invention, a method for displaying an around-view image of an articulated dump truck may include a step of acquiring a plurality of images corresponding to each of a plurality of cameras installed at positions capable of photographing a blind area between a front frame and a rear frame of the articulated dump truck and the surroundings of the articulated dump truck, a step of generating a composite image by synthesizing the plurality of images using a pre-implemented 3D model, and a step of outputting the generated composite image using a pre-set device.

[0019] In addition, the step of generating the composite image may include a step of acquiring a first image and a second image generated by two cameras, one of which is positioned on the same line in the longitudinal direction of the articulated dump truck among the plurality of cameras and photographs the front of the articulated dump truck, one of which photographs so as to be identical to the driver's field of view and the other of which photographs a front blind area where the driver's field of view is obstructed by the cabin, and a step of generating a front composite image without a blind area by synthesizing the first image and the second image using the pre-implemented 3D model.

[0020] In addition, the step of outputting the generated synthetic image using a preset device may include the step of obtaining a moving speed of the articulated dump truck, and if the obtained moving speed of the articulated dump truck is faster than the preset speed, the step of outputting a forward synthetic image without the blind area using the preset device.

[0021] In addition, the step of generating the synthetic image may include the step of acquiring images generated by at least four cameras positioned in a preset area among the plurality of cameras, and the step of generating a partial synthetic image by synthesizing the four acquired images using the pre-implemented 3D model.

[0022] Additionally, the partial composite image may include a blind area between the front frame and the rear frame of the articulated dump truck.

[0023] In addition, the step of outputting the generated partial synthetic image using a preset device may include the step of obtaining transmission information of the articulated dump truck, comparing the obtained transmission information with preset transmission information, and simultaneously outputting the generated partial synthetic image and a preset image among the plurality of images based on the comparison result.

[0024] In addition, the preset transmission information indicates reverse, and the step of outputting the generated partial synthetic image using a preset device may include a step of simultaneously outputting the partial synthetic image and an image capturing the rear of the articulated dump truck among the plurality of images when the acquired transmission information and the preset transmission information are the same.

[0025]

[0026] According to one embodiment of the present invention, there is an effect of being able to display an image of a blind area between a cabin included in a front frame of an articulated dump truck and a dump included in a rear frame using a composite image obtained by synthesizing a plurality of images.

[0027] In addition, there is an effect that can display a forward composite image created by synthesizing images created by at least two cameras positioned on the same line based on information on the moving speed of the construction equipment.

[0028] In addition, there is an effect that can display a partially synthesized image created by synthesizing images generated by four cameras located in preset areas based on transmission information and an image generated by a camera located at the rear of the construction machine.

[0029]

[0030] FIG. 1A and FIG. 1B are drawings showing how an around view image display device operates according to one embodiment.

[0031] FIG. 2 is a drawing showing the configuration of an around view image display device according to one embodiment.

[0032] FIGS. 3A and 3B are drawings showing a camera included in an around view image display device mounted on a construction machine according to one embodiment.

[0033] FIG. 4 is a drawing showing an around view image display device simultaneously outputting a partial composite image and an image generated from the rear of a construction machine according to one embodiment.

[0034] Figure 5 is a flow chart showing an around view image display method according to one embodiment.

[0035] Fig. 6 is a flow chart illustrating an around view image display method according to another embodiment.

[0036] Fig. 7 is a flow chart illustrating an around view image display method according to another embodiment.

[0037]

[0038] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed in this specification are merely illustrative for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described in this specification.

[0039] Embodiments according to the concept of the present invention may have various modifications and take various forms, and thus, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit embodiments according to the concept of the present invention to specific disclosed forms, but rather includes all modifications, equivalents, or alternatives falling within the spirit and technical scope of the present invention.

[0040] While terms such as "first" or "second" may be used to describe various components, these components should not be limited by these terms. These terms are intended solely to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component, without departing from the scope of the invention.

[0041] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0042] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0043] In this specification, it should be understood that terms such as “comprise” or “have” are intended to specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0044] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0045] Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0046] In the following description, identical identifiers indicate identical configurations, and unnecessary redundant descriptions and explanations of known technologies are omitted.

[0047] In embodiments of the present invention, the terms "communication," "communication network," and "network" may be used interchangeably. These three terms refer to wired and wireless local and wide-area data transmission and reception networks that enable the transmission and reception of files between user terminals, other users' terminals, and download servers.

[0048] Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the attached drawings.

[0049] FIG. 1A and FIG. 1B are drawings showing how an around view image display device operates according to one embodiment.

[0050] Referring to FIGS. 1A and 1B, the around view image display device can output different images using preset devices according to the moving speed of the construction machine.

[0051] Referring to FIG. 1a, the around view image display device can output an around view image when the moving speed of the construction machine is low (e.g., less than 10 km).

[0052] The around view video display device can be installed at a location (e.g., outside a cabin included in a front frame of an articulated dump truck, etc.) where the camera included in the around view video display device can capture the front, left side, right side, and blind area of ​​the construction machine (100) of the construction machine.

[0053] The around view image display device can acquire four images corresponding to each of the four cameras installed on the construction machine (100), and can generate a composite image by synthesizing the four acquired images using a pre-implemented 3D model. In addition, the around view image display device can output the generated composite image using a pre-configured device.

[0054] The image output by the around view image display device may include a blind area image of the construction machine (100). According to one embodiment, when the construction machine is an articulated dump truck in the image output by the around view image display device, the blind area image may be an image (130) of the area between the cabin (110) included in the front frame and the dump (120) included in the rear frame.

[0055] Referring to FIG. 1b, the around view image display device can output an image of the front blind area of ​​the construction machine when the moving speed of the construction machine is high (e.g., 10 km or more).

[0056] Since the driver's seat of a construction machine (e.g., an articulated dump truck, etc.) (140) is located very high above the ground, the driver's field of vision (150) for a long distance from the construction machine (e.g., an articulated dump truck, etc.) (100) is very good.

[0057] However, if a protruding part such as a bonnet (170) exists on a construction machine (e.g., an articulated dump truck, etc.) (140), a front blind area that cannot be seen by the construction machine operator sitting in the driver's seat may be formed (160) right in front of the construction machine (e.g., an articulated dump truck, etc.) (140).

[0058] Referring to FIG. 1b, the around view image display device can generate a composite image by synthesizing respective images acquired by cameras installed on the upper part of the driver's seat of a construction machine (e.g., an articulated dump truck, etc.) (140) and the front bumper of the construction machine (e.g., an articulated dump truck, etc.) (140), and can display (180) the generated composite image on the bonnet (170) of the construction machine (e.g., an articulated dump truck, etc.) (140). In this way, when a composite image is generated by synthesizing the respective images acquired by cameras installed on the upper part of the driver's seat of a construction machine (e.g., an articulated dump truck, etc.) (140) and the front bumper of the construction machine (e.g., an articulated dump truck, etc.) (140), and displaying (180) the composite image on the bonnet (170) of the construction machine (e.g., an articulated dump truck, etc.) (140), when the construction machine is moving at a high speed (e.g., 10 km or more), it is easy to identify surrounding objects existing in the front blind area of ​​the construction machine, thereby enabling safe operation of the construction machine.

[0059] FIG. 2 is a drawing showing the configuration of an around view image display device according to one embodiment, and FIGS. 3a and 3b are drawings showing a camera included in an around view image display device mounted on a construction machine according to one embodiment.

[0060] According to one embodiment, the around view image display device (200) may be coupled to the construction machine as a separate, independent device or may be communicatively connected to the construction machine.

[0061] According to another embodiment, the around view image display device (200) may be included in a construction machine.

[0062] Referring to FIG. 2, an around view image display device (200) according to various embodiments may include a first camera (210), a second camera (220), a third camera (230), a fourth camera (240), a fifth camera (250), a sixth camera (260), a processing device (270), and a display device (280).

[0063] The first camera (210), the second camera (220), the third camera (230), the fourth camera (240), the fifth camera (250), the sixth camera (260), the processing device (270), and the display device (280) included in the around view image display device (200) are interconnected and may be capable of transmitting data to each other.

[0064] According to various embodiments, a plurality of cameras included in the around view video display device can be installed at preset locations of the construction machine.

[0065] Referring to FIGS. 3A and 3B , the first camera (210) may be installed above the driver's seat of the construction machine (e.g., an articulated dump truck, etc.). According to one embodiment, the first camera (210) may be installed above the driver's seat vertically above the head of the driver riding in the driver's seat of the construction machine (e.g., an articulated dump truck, etc.). According to another embodiment, the first camera (210) may be installed above the driver's seat of the construction machine (e.g., an articulated dump truck, etc.) so as to be in the same direction as the line of sight of the driver riding in the driver's seat of the construction machine (e.g., an articulated dump truck, etc.). According to the position of the first camera (210) described above, the first camera (210) may generate a first image corresponding to the field of view of the driver riding in the driver's seat of the construction machine (e.g., an articulated dump truck, etc.).

[0066] Referring to FIGS. 3A and 3B , the second camera (220) may be installed on the front bumper of a construction machine (e.g., an articulated dump truck, etc.). The second camera (220) may be installed on the front bumper of the construction machine (e.g., an articulated dump truck, etc.) so as to have a preset angle (e.g., 45 degrees, etc.). Here, the preset angle may be changed in consideration of the shape, size, etc. of the construction machine.

[0067] The second camera (220) may be installed on the front bumper of the construction machine (e.g., an articulated dump truck, etc.) along the same line as the first camera (210) in the longitudinal direction of the construction machine. In this way, if the first camera (210) and the second camera (220) are positioned along the same line, camera calibration is easy, and since the cameras are positioned based on the same axis, there is a physical reference coordinate system for camera parameter modeling, which can be facilitated.

[0068] The second camera (220) can generate a second image corresponding to the front image of the construction machine (e.g., an articulated dump truck, etc.). In one embodiment, the second camera (220) can generate a second image corresponding to the front portion of the construction machine that is not visible to the driver due to the hood (170).

[0069] According to various embodiments, the third camera (230) may be installed on the left outer side of the driver's seat of the construction machine (e.g., an articulated dump truck, etc.), and the fourth camera (240) may be installed on the right outer side of the driver's seat of the construction machine (e.g., an articulated dump truck, etc.). According to one embodiment, the third camera (230) and the fourth camera (240) may be installed on the left and right outer sides of the driver's seat of the construction machine (e.g., an articulated dump truck, etc.) so as to be on the same line as the first camera (210) in the width direction of the construction machine. In this way, when the first camera (210), the third camera (230), and the fourth camera (240) are positioned on the same line, camera calibration is easy, and since the cameras are arranged based on the same axis, there is a physical reference coordinate, which can be easy when modeling camera parameters.

[0070] According to the positions of the cameras described above, the third camera (230) can generate a third image corresponding to the left external image of the driver's seat of the construction machine (e.g., an articulated dump truck, etc.), and the fourth camera (240) can generate a fourth image corresponding to the right external image of the driver's seat of the construction machine (e.g., an articulated dump truck, etc.).

[0071] According to various embodiments, the fifth camera (250) may be installed at a location where a blind area of ​​the construction machine that cannot be seen in a typical around-view image display device (e.g., when the construction machine is an articulated dump truck, the area between the cabin included in the front frame and the dump included in the rear frame, etc.) can be captured, for example, when the construction machine is an articulated dump truck, the outside of the cabin included in the front frame, etc.) may be captured. According to one embodiment, the fifth camera (250) may be installed at the top of the outside of the cabin so as to be able to look down from above. The fifth camera (250) may generate a fifth image corresponding to an image of a blind area of ​​the construction machine (e.g., the area between the cabin included in the front frame of the articulated dump truck and the dump included in the rear frame, etc.).

[0072] According to various embodiments, the sixth camera (260) may be installed at the rear of the construction machine (e.g., when the construction machine is an articulated dump truck, a dump included in the rear frame, etc.). According to one embodiment, the sixth camera (260) may be installed at a location where the rear bottom of the construction machine can be photographed (e.g., when the construction machine is an articulated dump truck, under the dump included in the rear frame, etc.). Accordingly, the sixth camera (260) may generate a sixth image corresponding to an image of the rear bottom of the construction machine.

[0073] According to various embodiments, the processing device (270) can acquire at least two of a first image generated by a first camera (210), a second image generated by a second camera (210), a third image generated by a third camera (210), a fourth image generated by a fourth camera (210), a fifth image generated by a fifth camera (210), and a sixth image generated by a sixth camera (210).

[0074] The processing device (270) may generate a composite image by synthesizing at least two images among the first to sixth images using a pre-implemented 3D model. According to another embodiment, the processing device (270) may generate a composite image by synthesizing at least two images among the first to sixth images using a pre-implemented 3D model. The processing device (270) may generate a composite image by curvedly projecting at least two acquired images onto a pre-implemented 3D model.

[0075] According to various embodiments, the processing device (270) can match each of the first to sixth images with each of the preset portions of the pre-implemented 3D model. More specifically, the processing device (270) can generate a composite image by projecting each of the first to sixth images, each of which is matched to each of the preset portions of the pre-implemented 3D model, onto a curved surface.

[0076] According to various embodiments, the processing device (270) may acquire an image of a calibration panel installed at the front of the construction machine to calibrate a pre-implemented 3D model. The processing device (270) may calculate parameters of the 3D model based on the image of the calibration panel installed at the front of the construction machine. The processing device (270) may compare the parameters of the calculated 3D model with the parameters of the pre-implemented 3D model, and update the parameters of the pre-implemented 3D model based on the comparison result.

[0077] According to various embodiments, the processing device (270) can output the generated synthetic image using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a synthetic image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0078] According to various embodiments, the processing device (270) may be communicatively connected to the construction machine and may obtain information on the movement speed of the construction machine from the construction machine. If the obtained movement speed of the construction machine is slower than a preset speed (e.g., 10 km, etc.), the processing device (270) may output the generated synthetic image using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a synthetic image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0079] According to various embodiments, the processing device (270) may determine images generated by at least two cameras positioned on the same line among a plurality of cameras. In one embodiment, the processing device (270) may determine the first image and the second image as images positioned on the same line. The processing device (270) may generate a forward composite image by combining the first image and the second image using a pre-implemented 3D model.

[0080] The processing device (270) can compare the acquired moving speed of the construction machine with the preset speed, and output a forward composite image using the preset device based on the comparison result. If the acquired moving speed of the construction machine is faster than the preset speed (e.g., 10 km), the processing device (270) can output the generated forward composite image using the preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0081] According to various embodiments, the processing device (270) may determine images generated by at least four cameras positioned in preset areas among a plurality of cameras. The processing device (270) may determine the second to fifth images as images generated by at least four cameras positioned in preset areas. The processing device (270) may generate a partial composite image by combining the second to fifth images using a pre-implemented 3D model. The processing device (270) may generate a partial composite image by curvedly projecting the second to fifth images onto the pre-implemented 3D model. According to one embodiment, the processing device (270) may match each of the second to fifth images with each of the pre-implemented portions of the pre-implemented 3D model. More specifically, the processing device (270) may generate a partial composite image by curvedly projecting each of the second to fifth images matching each of the pre-implemented portions of the pre-implemented 3D model.

[0082] According to various embodiments, the processing device (270) can obtain transmission information of the construction machine from the construction machine. The processing device (270) can compare the obtained transmission information of the construction machine with preset transmission information, and can simultaneously output a generated partial synthetic image and a preset image based on the comparison result using a preset device. Here, the preset image may be the sixth image, which is an image taken of the rear of the construction machine, but the preset image is not limited thereto. When the obtained transmission information of the construction machine matches the preset transmission information (e.g., reverse), the processing device (270) can simultaneously output the generated partial synthetic image and the preset image using a preset device. Here, the preset image may be the sixth image, but the preset image is not limited thereto. In addition, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the bonnet of the construction machine, etc., but the preset device is not limited thereto.

[0083] According to various embodiments, the display device (280) can display an image generated by the processing device (270). The display device (280) may be a display device equipped in a construction machine driver's seat, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the bonnet of the construction machine, etc., but the display device (280) is not limited thereto.

[0084] According to various embodiments, the display device (280) may include a monitor, a touchscreen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, earphones, headphones, or a touchpad, and may obtain user input.

[0085] The display device (280) may be provided as an integral part of the around-view image display device or may be provided separately from the around-view image display device. According to one embodiment, the display device (280) may be provided in a construction machine.

[0086] It is apparent to those skilled in the art that the term 'device or module' used herein refers to a logical configuration unit and not necessarily a physically distinct component.

[0087] FIG. 4 is a drawing showing an around view image display device simultaneously outputting a partial composite image and an image generated from the rear of a construction machine according to one embodiment.

[0088] Referring to FIG. 4, the around-view image display device (200) can obtain a first image and / or a second image capturing the front of the construction machine from the first camera (210) and / or the second camera (220) of the construction machine (e.g., an articulated dump truck, etc.). In addition, the around-view image display device (200) can obtain a third image and a fourth image from the third camera (230) and the fourth camera (240) installed on the left and right exterior sides of the driver's seat of the construction machine. The around-view image display device (200) can obtain a fifth image from the fifth camera (250) installed at a position capable of capturing a blind area of ​​the construction machine (e.g., an area (411) between a cabin (110) included in a front frame and a dump (120) included in a rear frame).

[0089] The around view image display device (200) can create a partial composite image by combining the first to fifth images using a pre-implemented 3D model.

[0090] According to one embodiment, the around view image display device (200) can generate a partial composite image by projecting the first to fifth images onto a pre-implemented 3D model on a curved surface. The around view image display device (200) can match each of the first to fifth images with each of the preset portions of the pre-implemented 3D model. In more detail, the around view image display device (200) can generate a partial composite image (410) by projecting each of the first to fifth images matched to each of the preset portions of the pre-implemented 3D model on a curved surface.

[0091] Referring to FIG. 4, when the construction machine is an articulated dump truck in the partial composite image (410) output by the around view image display device (200), the partial composite image (410) generated by the around view image display device (200) may include an image for the area (411) between the cabin included in the front frame and the dump included in the rear frame.

[0092] Referring to FIG. 4, the around view image display device (200) can simultaneously output a generated partial composite image (410) and a sixth image (420) acquired from a sixth camera installed at the rear of the construction machine (e.g., a dump included in a rear frame when the construction machine is an articulated dump truck) using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0093] Figure 5 is a flow chart showing an around view image display method according to one embodiment.

[0094] Referring to FIG. 5, an around view image display method according to an embodiment may include a step (S500) in which an around view image display device (200) acquires a plurality of images corresponding to each of a plurality of cameras installed at preset locations of a construction machine, a step (S510) in which the around view image display device (200) synthesizes the plurality of images using a pre-implemented 3D model to generate a composite image, and a step (S520) in which the around view image display device (200) outputs the composite image generated using a preset device.

[0095] In step S500, the around view image display device (200) can obtain the first to sixth images from six cameras installed at preset locations of the construction machine.

[0096] In step S510, the around view image display device (200) can generate a composite image by synthesizing multiple images acquired using a pre-implemented 3D model.

[0097] Additionally, the around view image display device (200) can create a composite image by projecting the second to sixth images onto a pre-implemented 3D model.

[0098] Additionally, the around view image display device (200) can match each of the first to sixth images with each of the preset portions of the pre-implemented 3D model.

[0099] In addition, the around view image display device (200) can generate a composite image by projecting each of the second to sixth images matching each preset part of a pre-implemented 3D model onto a curved surface.

[0100] In step S520, the around view image display device (200) can output the generated composite image using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0101] Fig. 6 is a flow chart illustrating an around view image display method according to another embodiment.

[0102] Referring to FIG. 6, an around view image display method according to an embodiment may include a step (S600) in which an around view image display device (200) acquires a plurality of images corresponding to each of a plurality of cameras installed at preset locations of a construction machine, a step (S610) in which an around view image display device (200) determines images generated by at least two cameras positioned on the same line among the plurality of cameras, a step (S620) in which an around view image display device (200) synthesizes a plurality of images using a pre-implemented 3D model to generate a forward composite image, a step (S630) in which an around view image display device (200) acquires moving speed information of a construction machine, and a step (S640) in which an around view image display device (200) compares the moving speed information of the construction machine acquired by the around view image display device (200) with preset speed information and outputs a forward composite image using a preset device according to a result of the comparison.

[0103] In step S600, the around view image display device (200) can obtain a first image from a first camera installed above the driver's seat of the construction machine.

[0104] Additionally, the around view image display device (200) can obtain a second image from a second camera installed on the front bumper of the construction machine.

[0105] Additionally, the around view image display device (200) can determine the first image and the second image as images generated by two cameras positioned on the same line.

[0106] In step S620, the around view image display device (200) can generate a front composite image by combining the first image and the second image using a pre-implemented 3D model.

[0107] In step S630, the around view image display device (200) is communicatively connected to the construction machine and can obtain the moving speed of the construction machine from the construction machine.

[0108] Although steps S600 to S620 are illustrated as being performed before step S630 in FIG. 6, steps S600 to S620 and S630 may be performed simultaneously, in parallel, or overlapping, and steps S600 to S620 may be performed after step S630.

[0109] In step S640, if the acquired moving speed of the construction machine is faster than a preset speed (e.g., 10 km), the around view image display device (200) can output the generated forward composite image using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0110] Fig. 7 is a flow chart illustrating an around view image display method according to another embodiment.

[0111] Referring to FIG. 7, an around view image display method according to an embodiment includes a step (S700) in which an around view image display device (200) acquires a plurality of images corresponding to each of a plurality of cameras installed at preset locations of a construction machine, a step (S710) in which an around view image display device (200) determines images generated by at least four cameras located at preset areas, a step (S720) in which an around view image display device (200) synthesizes a plurality of images using a pre-implemented 3D model to generate a partial synthesized image, a step (S730) in which an around view image display device (200) acquires transmission information of a construction machine, and a step (S740) in which an around view image display device (200) compares the acquired transmission information with preset transmission information and simultaneously outputs a generated partial synthesized image and a preset image among the plurality of images based on the comparison result.

[0112] In step S700, the around view image display device (200) can obtain a second image from a second camera installed on the front bumper of a construction machine (e.g., an articulated dump truck, etc.).

[0113] In addition, the around view image display device (200) can obtain third and fourth images from the third camera (230) and the fourth camera (240) installed on the left and right exterior sides of the construction equipment driver's seat, respectively.

[0114] In addition, the around view image display device (200) can obtain a fifth image from a fifth camera (250) installed in a position capable of capturing a blind area of ​​the construction machine (e.g., an area between a cabin included in the front frame and a dump included in the rear frame when the construction machine is an articulated dump truck).

[0115] In step S710, the around view image display device (200) can determine the second to fifth images as images generated by at least four cameras positioned in a preset area.

[0116] In step S720, the around view image display device (200) can generate a partial composite image by combining the second to fifth images using a pre-implemented 3D model.

[0117] Additionally, the around view image display device (200) can create a partial synthetic image by projecting the second to fifth images onto a pre-implemented 3D model.

[0118] Additionally, the around view image display device can match each of the preset portions of the pre-implemented 3D model with each of the second to fifth images.

[0119] Additionally, the around view image display device can generate a partial composite image by projecting each of the second to fifth images matching each preset part of a pre-implemented 3D model onto a curved surface.

[0120] In step S730, the around view image display device (200) is communicatively connected to the construction machine to obtain transmission information of the construction machine.

[0121] Although steps S700 to S720 are illustrated as being performed before step S730 in FIG. 7, steps S700 to S720 and S730 may be performed simultaneously, in parallel, or overlapping, and steps S700 to S720 may be performed after step S730.

[0122] In step S740, the around view image display device (200) compares the acquired transmission information of the construction machine with preset transmission information, and can simultaneously output a partial synthetic image generated based on the comparison result and a sixth image acquired from a sixth camera installed at the rear of the construction machine (e.g., a dump truck included in the rear frame when the construction machine is an articulated dump truck) using a preset device. Here, the preset device may be a display device equipped on the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a synthetic image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0123] Although all components constituting the embodiments of the present invention have been described as being combined or operated in combination as one, the present invention is not necessarily limited to such embodiments. That is, within the scope of the present invention, all components may be selectively combined and operated at least as one.

[0124] Additionally, while each of these components may be implemented as a single, independent piece of hardware, some or all of these components may be selectively combined and implemented as a computer program having program modules that perform some or all of the functions of one or more hardware components. The codes and code segments that constitute the computer program will be readily inferred by those skilled in the art.

[0125] These computer programs can be stored on computer-readable media and read and executed by a computer, thereby implementing embodiments of the present invention. Storage media for the computer program may include magnetic recording media, optical recording media, etc.

[0126] In addition, terms such as “include,” “comprise,” or “have” described above, unless specifically stated to the contrary, should be interpreted to mean that the corresponding component may be included, and thus should not be interpreted to exclude other components, but rather to include other components.

[0127] All terms, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains, unless otherwise defined. Commonly used terms, such as terms defined in dictionaries, should be interpreted in a way consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0128] The above description is merely an example of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present invention.

[0129] Accordingly, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be construed as being included within the scope of the present invention.

Claims

1. In an around-view image display device for an articulated dump truck, A plurality of cameras installed in positions capable of photographing the blind area between the front frame and the rear frame of the articulated dump truck and the surroundings of the articulated dump truck; and A processing device connected to communicate with the plurality of cameras, The above processing device, Acquire multiple images from the multiple cameras, Generate a composite image by synthesizing the above multiple images using a pre-implemented 3D model, Outputting the generated synthetic image using a preset device, Around view video display device.

2. In paragraph 1 The above processing device, Obtaining first and second images generated by two cameras, one of which is positioned along the same line in the longitudinal direction of the articulated dump truck among the plurality of cameras and photographs the front of the articulated dump truck, one of which photographs the front blind area where the driver's view is obstructed by the cabin, By synthesizing the first image and the second image using the above-described 3D model, a forward composite image without a blind area is generated. Around view video display device.

3. In paragraph 2 The above processing device, Obtain the moving speed of the above articulated dump truck, Comparing the movement speed of the acquired articulated dump truck with the preset speed information, and outputting the forward composite image using the preset device according to the comparison result. Around view video display device.

4. In paragraph 1 The above processing device, Acquire images generated by at least four cameras located in a preset area among the above multiple cameras, Generating a partial synthetic image by synthesizing at least four images obtained using the above pre-implemented 3D model, Around view video display device.

5. In paragraph 4 The above processing device, In order to include a blind area between the front frame and the rear frame of the articulated dump truck in the above partial composite image, Around view video display device.

6. In paragraph 5 The above processing device, Obtain transmission information of the above articulated dump truck, Comparing the acquired transmission information with the preset transmission information, and simultaneously outputting the generated partial synthetic image and the preset image among the plurality of images based on the comparison result. Around view video display device.

7. In paragraph 6 The above preset transmission information indicates reverse, The above processing device, If the acquired transmission information and the preset transmission information are the same, the partial composite image and the image capturing the rear of the articulated dump truck among the plurality of images are simultaneously output. Around view video display device.

8. A step of acquiring a plurality of images corresponding to each of the plurality of cameras installed at a position capable of photographing a blind area between the front frame and the rear frame of the articulated dump truck and the surroundings of the articulated dump truck; A step of generating a composite image by synthesizing the plurality of images using a pre-implemented 3D model; and A step of outputting the generated synthetic image using a preset device, Method for displaying around view video.

9. In paragraph 8, The step of generating the above synthetic image is: A step of acquiring first and second images generated by two cameras, one of which is positioned along the same line in the longitudinal direction of the articulated dump truck among the plurality of cameras and photographs the front of the articulated dump truck, one of which photographs the front blind area where the driver's view is obstructed by the cabin; and A step of generating a forward composite image without a blind area by synthesizing the first image and the second image using the pre-implemented 3D model, Method for displaying around view video.

10. In paragraph 9, The step of outputting the above-generated synthetic image using a preset device is as follows: A step of obtaining the moving speed of the above articulated dump truck; If the movement speed of the acquired articulated dump truck is faster than a preset speed, a step of outputting a forward synthetic image without the blind area using the preset device is included. Method for displaying around view video.

11. In paragraph 8, The step of generating the above synthetic image is: A step of acquiring images generated by at least four cameras located in a preset area among the plurality of cameras; and A step of generating a partial synthetic image by synthesizing at least four images obtained using the above pre-implemented 3D model, Method for displaying around view video.

12. In paragraph 11, The above partial composite image includes a blind area between the front frame and the rear frame of the articulated dump truck. Method for displaying around view video.

13. In paragraph 12, The step of outputting the above-generated partial synthetic image using a preset device is as follows: A step of obtaining transmission information of the above articulated dump truck; and Comprising a step of comparing the acquired transmission information with preset transmission information, and simultaneously outputting the generated partial synthetic image and the preset image among the plurality of images based on the comparison result. Method for displaying around view video.

14. In paragraph 13, The above preset transmission information indicates reverse, The step of outputting the above-generated partial synthetic image using a preset device is as follows: In a case where the acquired transmission information and the preset transmission information are the same, a step of simultaneously outputting the partial synthetic image and the image capturing the rear of the articulated dump truck among the plurality of images, Method for displaying around view video.

Images