Blind area display device for construction machinery, and blind area display method for construction machinery

Abstract

According to one embodiment of the present invention, a blind area display device for construction machinery comprises at least one processor, and the at least one processor acquires a first image from a first camera provided at a first position of the construction machinery, acquires a second image from a second camera provided at a second position of the construction machinery, combines the first image and the second image and generate a composite image by using a previously implemented 3D model, and outputs the generated composite image by using a preset device.

Description

Construction machinery blind area display device and construction machinery blind area display method

[0001] The following embodiments relate to a construction machinery blind area display device and a construction machinery blind area display method.

[0002]

[0003] In general, construction machinery such as dozers, loaders, excavators, graders, dump trucks, and articulated dump trucks are primarily utilized in earthmoving environments. In particular, dump trucks with hinged front and rear frames and steering cylinders that allow left-right directional control are called articulated dump trucks.

[0004] While a typical dump truck turns by turning its front wheels, an articulated dump truck turns by turning its front end, reducing the turning radius and enabling it to move easily in narrow work sites and secure an efficient moving route.

[0005] Construction equipment, such as articulated dump trucks, are increasingly being manufactured larger for reasons such as increased efficiency. These larger machines create a blind spot directly in front of the machine, obscuring the view from the driver's seat.

[0006]

[0007] The present invention was invented to solve the above-mentioned problem, and its purpose is to provide a construction machinery blind area display device and a construction machinery blind area display method that can display an image of a construction machinery blind area using a composite image obtained by synthesizing a first image and a second image.

[0008] In addition, another object of the present invention is to provide a construction machinery blind area display device and a construction machinery blind area display method capable of recognizing an object included in an image and outputting a synthetic image that implements an object warning for a stingy object.

[0009]

[0010] According to one embodiment of the present invention, a construction machine blind area display device includes a first camera installed at a first location of the construction machine so as to be able to photograph an area that can be seen by a construction machine operator, a second camera installed at a second location of the construction machine so as to be able to photograph a blind area that cannot be seen by the construction machine operator, and at least one processor, wherein the at least one processor can obtain a first image from the first camera, obtain a second image from the second camera, generate a composite image by synthesizing the first image and the second image using a pre-implemented 3D model, and output the generated composite image using a pre-set device.

[0011] Additionally, the at least one processor may detect an object included in an image from at least one of the first image and the second image, implement an object warning for the detected object, and combine the implemented object warning with the composite image.

[0012] Additionally, the at least one processor may determine the first image as a background image and project the second image onto the background image to generate the composite image.

[0013] In addition, the at least one processor may obtain an image of two panels positioned at predefined positions for each of the first camera and the second camera, and when the two panels are recognized at the predefined positions in the image for each camera, calculate a distance to the two panels using triangulation, and update parameters of the pre-implemented 3D model using the calculated distance.

[0014] Additionally, the first position and the second position may be located on the same line in the front-rear direction of the construction machine.

[0015] Additionally, the first position may be the upper part of the driver's seat, and the second position may be the front bumper of the construction machine.

[0016] According to another embodiment of the present invention, a method for displaying a blind area of ​​a construction machine may include the steps of: obtaining a first image from a first camera installed at a first location of the construction machine so as to be able to photograph an area that a construction machine operator can see; obtaining a second image from a second camera installed at a second location of the construction machine so as to be able to photograph a blind area that the construction machine operator cannot see; generating a composite image by synthesizing the first image and the second image using a pre-implemented 3D model; and outputting the generated composite image using a pre-set device.

[0017] In addition, the construction equipment blind area display method may further include a step of detecting an object included in an image from at least one of the first image and the second image, a step of implementing an object warning for the detected object, and a step of combining the implemented object warning with the composite image.

[0018] Additionally, the step of generating the synthetic image may include a step of determining the first image as a background image and a step of projecting the second image onto the background image.

[0019] In addition, the construction equipment blind area display method may further include an operation of updating parameters of the pre-implemented 3D model, and the operation of updating parameters of the pre-implemented 3D model may include an operation of acquiring images of two panels positioned at pre-defined positions for each of the first camera and the second camera, an operation of calculating a distance to the two panels using triangulation when the two panels are recognized at the pre-defined positions in the images for each camera, and an operation of updating parameters of the pre-implemented 3D model using the calculated distance.

[0020]

[0021] According to one embodiment of the present invention, there is an effect of being able to display an image of a construction machinery blind area using a composite image obtained by synthesizing a first image and a second image.

[0022] Additionally, it has the effect of recognizing objects contained in the image and outputting a synthetic image that implements object warnings for stingy objects.

[0023]

[0024] FIG. 1 is a drawing showing how a construction machinery blind area display device operates according to one embodiment.

[0025] FIG. 2 is a drawing showing the configuration of a construction machinery blind area display device according to one embodiment.

[0026] FIGS. 3A and 3B are drawings showing a position where a camera of a construction machinery blind area display device is mounted on a construction machinery according to one embodiment.

[0027] FIG. 4 is a flowchart illustrating a method for updating parameters of a 3D model for image synthesis according to one embodiment.

[0028] Fig. 5 is a flow chart showing a construction machinery blind area display method according to one embodiment.

[0029] Fig. 6 is a flow chart showing a construction machinery blind area display method according to another embodiment.

[0030]

[0031] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed in this specification are merely illustrative for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described in this specification.

[0032] Embodiments according to the concept of the present invention may have various modifications and take various forms, and thus, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit embodiments according to the concept of the present invention to specific disclosed forms, but rather includes all modifications, equivalents, or alternatives falling within the spirit and technical scope of the present invention.

[0033] While terms such as "first" or "second" may be used to describe various components, these components should not be limited by these terms. These terms are intended solely to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component, without departing from the scope of the invention.

[0034] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0035] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0036] In this specification, it should be understood that terms such as “comprise” or “have” are intended to specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0038] Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0039] In the following description, identical identifiers indicate identical configurations, and unnecessary redundant descriptions and explanations of known technologies are omitted.

[0040] In embodiments of the present invention, the terms "communication," "communication network," and "network" may be used interchangeably. These three terms refer to wired and wireless local and wide-area data transmission and reception networks that enable the transmission and reception of files between user terminals, other users' terminals, and download servers.

[0041] Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the attached drawings.

[0042] FIG. 1 is a drawing showing how a construction machinery blind area display device operates according to one embodiment.

[0043] Referring to FIG. 1, according to various embodiments, since the driver's seat of a construction machine (e.g., an articulated dump truck, etc.) (100) is located very high from the ground, the driver's field of vision (110) for a long distance from the construction machine (100) is very good.

[0044] On the other hand, if there is a protruding part such as a bonnet (130) on the construction machine (100), a blind area that cannot be seen by the construction machine operator sitting in the driver's seat is formed (120) right in front of the construction machine (100).

[0045] According to various embodiments, the cameras included in the construction machinery blind area display device may be installed on the upper part of the driver's seat of the construction machinery (100) and the front bumper of the construction machinery (100), respectively.

[0046] The construction equipment blind area display device can generate a composite image by synthesizing each image acquired by a camera installed on the upper part of the driver's seat of the construction equipment (100) and the front bumper of the construction equipment (100).

[0047] A construction machinery blind area display device can display a generated composite image on a display (140) of a construction machinery (100). In this way, by synthesizing the respective images acquired by cameras installed on the upper part of the driver's seat of the construction machinery (100) and the front bumper of the construction machinery (100), and displaying the generated composite image on the display (140) of the construction machinery (e.g., an articulated dump truck, etc.) (100), when a pedestrian enters the construction machinery blind area, even in a situation where it is difficult to identify the presence of the pedestrian because the pedestrian is obscured by the bonnet (130) of the construction machinery (e.g., an articulated dump truck, etc.) (100), it can help the construction machinery driver identify the pedestrian, and it can also help to safely operate the equipment by making it easy to identify objects around the construction machinery.

[0048] FIG. 2 is a drawing showing the configuration of a construction machinery blind area display device according to one embodiment.

[0049] According to various embodiments, the construction machinery blind area display device (200) may be coupled to the construction machinery as a separate, independent device. Alternatively, the construction machinery blind area display device (200) may be communicatively connected to the construction machinery as a separate, independent device. Alternatively, the construction machinery blind area display device (200) may be included in the construction machinery.

[0050] Referring to FIG. 2, a construction machinery blind area display device (200) according to various embodiments includes a first camera (210), a second camera (220), a processor (230), an input / output interface module (240), and a memory (250).

[0051] The first camera (210), the second camera (220), the processor (230), the input / output interface module (240), and the memory (250) included in the construction machinery blind area display device (200) are interconnected and can transmit data to each other.

[0052] According to various embodiments, the processor (230) may execute programs or commands stored in the memory (250). Here, the memory (250) may store an operation program (e.g., OS) for operating the construction equipment blind area display device (200).

[0053] The processor (230) can execute a program for managing information about the construction equipment blind area display device (200).

[0054] The processor (230) can execute a program for managing the operation of the construction equipment blind area display device (200).

[0055] The processor (230) can execute a program to manage the operation of the input / output interface module (240).

[0056] According to various embodiments, the first camera (210) may be installed above the driver's seat of the construction machine, which is the first location. According to one embodiment, the first camera (210) may be installed above the driver's seat, vertically above the head of the driver sitting in the driver's seat of the construction machine. According to another embodiment, the first camera (210) may be installed above the driver's seat of the construction machine so that the direction of the driver's gaze is the same as the direction of the driver's gaze.

[0057] The first camera (210) can generate a first image corresponding to the field of view of a driver sitting in the driver's seat of the construction machine.

[0058] According to various embodiments, the second camera (220) may be installed on the front bumper of the construction machine, which is the second location. The second camera (220) may be installed on the front bumper of the construction machine, which is the second location, at a preset angle (e.g., 45 degrees). Here, the preset angle may be changed depending on the shape, size, etc. of the construction machine.

[0059] According to various embodiments, the second camera (220) may be installed on the front bumper of the construction machine, which is located on the same line as the first camera (210). In this way, if the first camera (210) and the second camera (220) are located on the same line, camera calibration is easy, and since the cameras are arranged based on the same axis, there is a physical reference coordinate system, which can be used for camera parameter modeling.

[0060] According to various embodiments, the second camera (220) can generate a second image corresponding to a front image of the construction machine.

[0061] According to various embodiments, the processor (230) can obtain a first image generated by a first camera (210) and a second image generated by a second camera (220).

[0062] The processor (230) may determine either the first image or the second image as a background image. According to one embodiment, the processor (230) may determine the first image generated by the first camera (210) installed above the driver's seat of the construction machine, which is the first location, as the background image, and the processor (230) may determine the second image generated by the second camera (220) installed on the front bumper of the construction machine, which is the second location, as the image to be projected onto the background image.

[0063] The processor (230) can generate a composite image by synthesizing the first image and the second image using a pre-implemented 3D model. The processor (230) can update the parameters of the pre-implemented 3D model. The processor (230) can acquire an image of a calibration panel installed at the front of the construction machine to calibrate and update the parameters of the pre-implemented 3D model.

[0064] According to one embodiment, the processor (230) may calculate parameters of a 3D model based on an image captured of a calibration panel installed in front of the acquired construction machine. The processor (230) may compare the parameters of the calculated 3D model with the parameters of a pre-implemented 3D model, and update the parameters of the pre-implemented 3D model based on the comparison result.

[0065] According to various embodiments, the processor (230) can project a first image generated by a first camera (210) determined as a background image onto a pre-implemented 3D model.

[0066] The processor (230) can generate a composite image by projecting the second image onto the first image so that the second image generated by the second camera (220) is overlaid on the first image on a pre-implemented 3D model that is a curved projection of the first image.

[0067] The processor (230) can synchronize the first image and the second image using a preset method. Here, the preset method may be frame sync, but the preset method is not limited thereto.

[0068] According to various embodiments, the processor (230) may output the generated synthetic image using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a synthetic image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0069] According to various embodiments, the processor (230) may output the generated synthetic image to an external device. Here, the external device may be a control device capable of remotely monitoring and controlling construction machinery, but the external device is not limited thereto.

[0070] According to various embodiments, the processor (230) may recognize an object included in an image from at least one of the first image and the second image. According to one embodiment, the processor (230) may detect an object in a second image generated by a second camera (220) installed on the front bumper of the construction machine (which is a second location). When an object is detected, the processor (230) may display an object warning so that the construction machine driver can recognize the detected object. The processor (230) may display an object warning with an image indicating the detected object (e.g., a red square image surrounding the object, etc.) so that the construction machine driver can recognize the detected object, but the warning for object recognition that the processor (230) may implement is not limited thereto.

[0071] According to various embodiments, the processor (230) may combine the displayed object alert with the synthetic image.

[0072] According to various embodiments, the processor (230) may output a composite image combined with an object warning using a preset device. Here, the preset device may be a display device equipped in the driver's seat of the construction machine, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the hood of the construction machine, etc., but the preset device is not limited thereto.

[0073] The processor (230) can output a composite image combined with an object warning to an external device. Here, the external device may be a control device capable of remotely monitoring and controlling construction equipment, but the external device is not limited thereto.

[0074] According to various embodiments, the input / output interface module (240) may be connected to an external device (e.g., a control device that can remotely monitor and control construction equipment, a work information transmission server, etc.) via a network. The input / output interface module (240) may obtain data (e.g., work information, etc.) from the external device. The input / output interface module (240) may be communicatively connected to the first camera and the second camera. The input / output interface module (240) may obtain data (e.g., a first image, a second image, etc.) from the first camera and the second camera. The input / output interface module (240) may obtain a control signal transmitted by the external device. The input / output interface module (240) may transmit data to the external device.

[0075] According to various embodiments, the input / output interface module (240) can obtain user input. The input / output interface module (240) can be provided as an integral part of the construction machinery blind area display device. The input / output interface module (240) can be provided separately from the construction machinery blind area display device. The input / output interface module (240) can be a separate device that is communicatively connected to the construction machinery blind area display device.

[0076] According to various embodiments, the input / output interface module (240) may include a port (e.g., a USB port) for connecting to an external device. The input / output interface module (240) may include a monitor, a touchscreen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, earphones, headphones, or a touchpad.

[0077] According to various embodiments, the memory (250) can store data acquired through the input / output interface module (240). The memory (250) can store data transmitted through the input / output interface module (240). The memory (250) can store data output through the input / output interface module (240). The memory (250) can store a background image determined by the processor (230). The memory (250) can store a projection image determined by the processor (230). The memory (250) can store a user's input. The memory (250) can store parameters of a pre-implemented 3D model updated by the processor (230). The memory (250) can store a composite image synthesized by the processor (230). The memory (250) can store an object warning implemented by the processor (230). The memory (250) can store a composite image combined with an object warning. Memory (250) can store applications.

[0078] It is apparent to those skilled in the art that the term 'device or module' used herein refers to a logical configuration unit and not necessarily a physically distinct component.

[0079] FIGS. 3A and 3B are drawings showing a position where a camera of a construction machinery blind area display device is mounted on a construction machinery according to one embodiment.

[0080] Figure 3a is a drawing showing the position where the camera is mounted as seen from the side of the construction machine, and Figure 3b is a drawing showing the position where the camera is mounted from the top of the construction machine.

[0081] Referring to FIGS. 3A and 3B, the construction machinery blind area display device may include a first camera (310) and a second camera (320).

[0082] The first camera (310) included in the construction machinery blind area display device may be installed at the center (301) of the upper part of the driver's seat of the construction machinery (300), which is the first location. According to one embodiment, the first camera (310) included in the construction machinery blind area display device may be installed at the upper part of the driver's seat (301) vertically above the head of the driver who sits in the driver's seat of the construction machinery (300). In addition, the first camera (310) included in the construction machinery blind area display device may be installed at the upper part of the driver's seat (301) of the construction machinery (300) so as to be in the same direction as the line of sight of the driver who sits in the driver's seat of the construction machinery (300).

[0083] The second camera (320) included in the construction machinery blind area display device may be installed at the center (302) of the front bumper of the construction machinery (300), which is the second location. According to one embodiment, the second camera (320) included in the construction machinery blind area display device may be installed at the second location so as to have a 45 degree angle in the downward direction.

[0084] The first camera (340) and the second camera (350) included in the construction machinery blind area display device can be installed on the same straight line (330).

[0085] FIG. 4 is a flowchart illustrating a method for updating parameters of a 3D model for image synthesis according to one embodiment.

[0086] The method of updating the parameters of the 3D model of Fig. 4 can be executed at the user's request, periodically, or when a large number of errors occur.

[0087] Referring to FIG. 4, in operation S400, an inspector may position calibration panels around the construction machine to calibrate a pre-implemented 3D model. In one embodiment, two calibration panels may be positioned at predefined locations for each camera, relative to the first camera (340) and the second camera (350).

[0088] In operation S410, the construction machinery blind area display device can capture two calibration panels corresponding to each of the first camera (340) and the second camera (350), find the two calibration panels corresponding to each camera in the captured images, and perform automatic correction. If the construction machinery blind area display device normally finds the panels at predefined locations in the images of each camera, it can calculate the actual distance using triangulation with respect to each panel based on the installation location of each camera, and display the result on the display.

[0089] In operation S420, the administrator or user can review the displayed results and, based on the reviewed results, select to re-execute, terminate the task without updating the parameters, or update the parameters and terminate the task. In one embodiment, if the results displayed on the display are not valid, the administrator or user can instruct to re-execute the task for updating the parameters. Alternatively, if the results displayed on the display are not significantly different from the current parameters, the task can be terminated without updating. Alternatively, if the results displayed on the display are valid and there is a difference when compared to the current parameters, the administrator or user can instruct to update the model parameters.

[0090] According to another embodiment, operation S420 may be executed by a construction machinery blind area display device.

[0091] In operation S430, the construction machinery blind area display device can update the parameters of the 3D model by storing the calculated distance results if the result of operation S420 instructs to update the model parameters.

[0092] Fig. 5 is a flow chart showing a construction machinery blind area display method according to one embodiment.

[0093] Referring to FIG. 5, a construction machinery blind area display method according to an embodiment includes a step (S500) in which a construction machinery blind area display device acquires a first image from a first camera installed at a first location of the construction machinery and acquires a second image from a second camera installed at a second location of the construction machinery, a step (S510) in which the construction machinery blind area display device generates a composite image by synthesizing the first image and the second image using a pre-implemented 3D model, and a step (S520) in which the construction machinery blind area display device outputs the composite image generated using a pre-set device.

[0094] In step S500, the construction machinery blind area display device can acquire a first image generated by a first camera and a second image generated by a second camera. The second camera can be installed to capture the blind area.

[0095] In step S510, the construction machinery blind area display device may determine one of the first image and the second image as the background image. According to one embodiment, the construction machinery blind area display device may determine the first image generated by the first camera installed above the driver's seat of the construction machinery, which is the first location, as the background image. In addition, the construction machinery blind area display device may determine the second image generated by the second camera installed on the front bumper of the construction machinery, which is the second location, to capture the blind area as the image to be projected on the background image.

[0096] In step S510, the construction machinery blind area display device can generate a composite image by synthesizing the first image and the second image using a pre-implemented 3D model.

[0097] According to one embodiment, a construction machinery blind area display device may curve-project a first image generated by a first camera determined as a background image onto a pre-implemented 3D model. In addition, the construction machinery blind area display device may curve-project a second image onto the first image so that the second image generated by the second camera is overlaid on the first image on the pre-implemented 3D model that is the curve-projected first image, thereby generating a composite image.

[0098] In step S520, the construction machinery blind area display device can output the generated composite image using a preset device. Here, the preset device may be a display device equipped in the construction machinery driver's seat, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the bonnet of the construction machinery, etc., but the preset device is not limited thereto.

[0099] Fig. 6 is a flow chart showing a construction machinery blind area display method according to another embodiment.

[0100] Referring to FIG. 6, a construction machinery blind area display method according to an embodiment includes a step (S600) in which a construction machinery blind area display device acquires a first image from a first camera installed at a first location of the construction machinery and acquires a second image from a second camera installed at a second location of the construction machinery, a step (S610) in which the construction machinery blind area display device generates a composite image by synthesizing the first image and the second image using a pre-implemented 3D model, a step (S620) in which the construction machinery blind area display device detects an object included in an image from at least one of the first image and the second image, a step (S630) in which the construction machinery blind area display device implements an object warning for the object detected, a step (S640) in which the construction machinery blind area display device combines the object warning with the composite image, and a step (S650) in which the construction machinery blind area display device outputs the composite image generated using a pre-set device.

[0101] In step S600, the construction machinery blind area display device can obtain a first image generated by a first camera and a second image generated by a second camera.

[0102] In step S610, the construction machinery blind area display device may determine one of the first and second images as the background image. According to one embodiment, the construction machinery blind area display device may determine the first image generated by the first camera installed above the driver's seat of the construction machinery, which is the first location, as the background image.

[0103] The construction machinery blind area display device can determine a second image generated by a second camera installed on the front bumper of the construction machinery, which is a second location, as an image to be projected on the background image.

[0104] In step S610, the construction machinery blind area display device can generate a composite image by synthesizing the first image and the second image using a pre-implemented 3D model.

[0105] According to one embodiment, the construction machinery blind area display device may curve-project a first image generated by a first camera determined as a background image onto a pre-implemented 3D model. Furthermore, the construction machinery blind area display device may curve-project a second image onto the first image so that the second image generated by the second camera is overlaid on the first image on the pre-implemented 3D model that is the curve-projected first image, thereby generating a composite image.

[0106] In step S620, the construction machinery blind area display device can recognize an object included in an image from at least one of the first image and the second image.

[0107] According to one embodiment, a construction machinery blind area display device is installed on the front bumper of the construction machinery at a second location and can detect objects only in a second image generated by a second camera that photographs the blind area.

[0108] In step S630, the construction machinery blind area display device can implement an object warning for a detected object so that a construction machinery operator can recognize the detected object.

[0109] According to one embodiment, the construction machinery blind area display device may implement an object warning as an image (e.g., a red square image surrounding the object) for displaying a detected object so that a construction machinery operator can recognize the detected object.

[0110] In step S640, the construction machinery blind area display device can combine the implemented object warning with a synthetic image.

[0111] In step S650, the construction machinery blind area display device can output a composite image combined with an object warning using a preset device. Here, the preset device may be a display device installed in the construction machinery driver's seat, a head-up display (HUD) device, a head-mounted display (HMD) device, an image output device capable of outputting a composite image to the bonnet of the construction machinery, etc., but the preset device is not limited thereto.

[0112] Although all components constituting the embodiments of the present invention have been described as being combined or operated in combination as one, the present invention is not necessarily limited to such embodiments. That is, within the scope of the present invention, all components may be selectively combined and operated at least as one.

[0113] Additionally, while each of these components may be implemented as a single, independent piece of hardware, some or all of these components may be selectively combined and implemented as a computer program having program modules that perform some or all of the functions of one or more hardware components. The codes and code segments that constitute the computer program will be readily inferred by those skilled in the art.

[0114] These computer programs can be stored on computer-readable media and read and executed by a computer, thereby implementing embodiments of the present invention. Storage media for the computer program may include magnetic recording media, optical recording media, etc.

[0115] In addition, terms such as “include,” “comprise,” or “have” described above, unless specifically stated to the contrary, should be interpreted to mean that the corresponding component may be included, and thus should not be interpreted to exclude other components, but rather to include other components.

[0116] All terms, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains, unless otherwise defined. Commonly used terms, such as terms defined in dictionaries, should be interpreted in a way consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0117] The above description is merely an example of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present invention.

[0118] Accordingly, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be construed as being included within the scope of the present invention.

Claims

1. In the construction machinery blind area display device, A first camera installed at a first position of the construction machine to capture an area visible to the construction machine operator; A second camera installed at a second location of the construction machine to capture a blind area that cannot be seen by the construction machine operator; and Contains at least one processor, At least one processor, Acquire a first image from the first camera, Acquire a second image from the second camera, Generating a composite image by synthesizing the first image and the second image using a pre-implemented 3D model, A construction machinery blind area display device that outputs the above-mentioned generated synthetic image using a preset device.

2. In paragraph 1, At least one processor, Detecting an object included in an image from at least one of the first image and the second image, Implement object warnings for the objects detected above, A construction machinery blind area display device that combines the above-described object warning with the above-described synthetic image.

3. In paragraph 1, At least one processor, A construction machinery blind area display device that determines the first image as a background image and projects the second image onto the background image to generate the composite image.

4. In paragraph 1, At least one processor, Obtaining images of two panels placed at predefined locations by the first camera and the second camera, When the two panels are recognized at the predefined locations in the images for each camera, the distance to the two panels is calculated using triangulation, A construction machinery blind area display device that updates the parameters of the pre-implemented 3D model using the calculated distance.

5. In paragraph 1, A construction machine blind area display device in which the first position and the second position are located on the same line in the front-rear direction of the construction machine.

6. In paragraph 5, A construction equipment blind area display device, wherein the first position is the upper part of the driver's seat and the second position is the front bumper of the construction equipment.

7. A step of acquiring a first image from a first camera installed at a first location of the construction machine so as to be able to capture an area visible to the construction machine operator; A step of acquiring a second image from a second camera installed at a second location of the construction machine so as to be able to capture a blind area that cannot be seen by the construction machine operator; A step of generating a composite image by synthesizing the first image and the second image using a pre-implemented 3D model; and A construction machinery blind area display method, comprising a step of outputting the above-mentioned generated synthetic image using a preset device.

8. In paragraph 7, A step of detecting an object included in an image from at least one of the first image and the second image; A step of implementing an object warning for the above-detected object; and A construction machinery blind area display method further comprising a step of combining the above-described object warning with the above-described synthetic image.

9. In paragraph 7, The step of generating the above synthetic image is: A step of determining the first image as a background image; and A construction machinery blind area display method, comprising a step of projecting the second image onto the background image.

10. In paragraph 7, A construction machinery blind area display method further comprising an operation of updating parameters of the above-described pre-implemented 3D model.

11. In paragraph 7, The operation of updating the parameters of the above pre-implemented 3D model is An operation of acquiring an image by photographing two panels placed at predefined positions by the first camera and the second camera; When the two panels are recognized at the predefined locations in the images for each camera, an operation of calculating the distance to the two panels using triangulation; A construction machinery blind area display method, comprising an operation of updating parameters of the pre-implemented 3D model using the calculated distance.

12. In paragraph 7, A construction machine blind area display method, wherein the first position and the second position are located on the same line in the front and rear directions of the construction machine.

13. In paragraph 12, A construction machine blind area display method, wherein the first position is the upper part of the driver's seat and the second position is the front bumper of the construction machine.

Images