Avm calibration system and avm calibration method using the same

Abstract

An AVM tolerance correction system according to various embodiments of the present invention, in an AVM (Around View Monitoring) tolerance correction system applied to construction machinery, comprises: a plurality of cameras that generate image information; an AVM controller that stores at least one parameter data for each of the plurality of cameras and generates AVM image information based on the image information; and a terminal device that receives at least one previously stored initial parameter data for the plurality of cameras from an external device, wherein the AVM controller can correct the tolerance of the plurality of cameras based on the received initial parameter data when distortion occurs in the AVM image information.

Description

Technology Field

[0001] The present invention relates to a method for correcting AVM tolerances, and more specifically, to a method for efficiently correcting the AVM tolerances of construction machinery by linking parameters for AVM tolerance correction with a TMS. Background Technology

[0002] Generally, since safety accidents can occur when operators of construction equipment, such as excavators, fail to detect obstacles located in the direction of movement, a display system capable of assessing the situation not only in front but also behind and to the sides of the equipment is a critical factor. To this end, Around View Monitor (AVM) systems are being researched, which involve installing multiple cameras in various directions of the construction equipment and displaying the images captured by each camera individually or combining them into a single image.

[0003] Four or more cameras are mounted along the perimeter of construction equipment, and images collected from each camera can be combined and synthesized to produce highly intuitive real-time video. However, because multiple cameras are used, errors in the mounted AVM images compared to the design reference images may occur due to mounting tolerances during the inline process or the replacement of individual cameras. Therefore, when applying an AVM system to construction equipment, it is necessary to correct for errors that occur during the process of installing the cameras on the construction machinery.

[0004] Meanwhile, generally, when correcting tolerances in an AVM system, an inline tolerance correction method is used, which involves correcting using a tolerance correction pattern placed at a designated location indoors. In actual construction sites, cases where cameras of AVM systems mounted on construction equipment are damaged are very frequent. When only some of the multiple cameras are damaged, or when a controller that generates AVM images by combining multiple camera images with some of the cameras is damaged, there was a problem in that even if only the damaged cameras or controller were replaced on-site, the system had to move away from the construction site to an indoor tolerance correction location to correct the tolerances. Prior art literature

[0005] Registered Patent No. 10-1515482 (April 29, 2015) The problem to be solved

[0006] The present invention was devised to solve the above problem, and aims to provide an AVM tolerance correction system for correcting the tolerance of an AVM system at a construction site using previously stored parameter values ​​in the event of damage to an AVM system mounted on construction machinery. means of solving the problem

[0007] An AVM tolerance correction system according to various embodiments of the present invention, in an AVM (Around View Monitoring) tolerance correction system applied to construction machinery, comprises: a plurality of cameras that generate image information; an AVM controller that stores at least one parameter data for each of the plurality of cameras and generates AVM image information based on the image information; and a terminal device that receives at least one previously stored initial parameter data for the plurality of cameras from an external device, wherein the AVM controller can correct the tolerance of the plurality of cameras based on the received initial parameter data when distortion occurs in the AVM image information.

[0008] Preferably, the parameter data may include parameters for the position and rotation of each of the plurality of cameras.

[0009] More preferably, the parameter data may include parameters representing displacement along the X-axis, Y-axis, and Z-axis, as well as yawing, pitching, and rolling for each of the plurality of cameras.

[0010] More preferably, the initial parameter may include parameter data for the plurality of cameras in a state where the tolerance is corrected.

[0011] Preferably, the external device includes a TMS server, and the terminal device can communicate with the TMS server via an OTA (Over-The-Air) method.

[0012] Preferably, the AVM controller can store parameter data of the current state of the plurality of cameras.

[0013] AVM tolerance correction method according to various embodiments of the present invention, in an Around View Monitoring (AVM) tolerance correction method applied to a construction machine, comprises: an operation of transmitting initial parameter data for each of a plurality of cameras to a server when the tolerance correction of the AVM of the construction machine is completed; an operation of generating AVM image information based on image information generated through the plurality of cameras; an operation of receiving the initial parameter data for each of the plurality of cameras from the server when distortion occurs in the generated AVM image information; and correcting the tolerance of the plurality of cameras based on the received initial parameter data. Effects of the invention

[0014] The present invention has the effect of enabling a rapid on-site response to damage to construction machinery by using previously stored parameter values ​​to correct the tolerance of the AVM system at the construction site in the event of damage to the AVM system mounted on the construction machinery.

[0015] In addition, since tolerance correction of the AVM system is possible at the construction site, it has the effect of shortening construction time and reducing costs. Brief explanation of the drawing

[0016] FIG. 1 is a drawing illustrating an AVM tolerance compensation system according to various embodiments of the present invention. FIG. 2 is a drawing illustrating an AVM controller according to various embodiments of the present invention. FIG. 3 is a diagram illustrating camera parameters according to various embodiments of the present invention. FIG. 4 is a drawing illustrating an AVM tolerance correction method according to various embodiments of the present invention. Specific details for implementing the invention

[0017] For convenience of explanation, some embodiments of the present invention will be described below with reference to exemplary drawings. In labeling the components of each drawing, the same components are indicated by the same reference numeral whenever possible, even if they are shown in different drawings.

[0018] Terms and words used in this specification and claims shall not be limited to their ordinary or dictionary meanings, and shall be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor may appropriately define the concept of the terms to best describe his invention. Furthermore, terms such as first, second, A, B, (a), (b), etc., may be used when describing the components of the embodiments of the invention. These terms are intended merely to distinguish the component from other components, and the essence, order, or sequence of the component is not limited by the term. Where it is stated that a component is 'connected' or 'combined' with another component, it should be understood that the component may be directly connected or combined with the other component, but that another component may also be 'connected' or 'combined' between the component and the other component.

[0019] Accordingly, the embodiments described in this specification and the configurations illustrated in the drawings are merely the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention; therefore, it should be understood that various equivalents and modifications capable of replacing them may exist at the time of filing this application. Furthermore, detailed descriptions of known functions and configurations that may unnecessarily obscure the essence of the present invention are omitted.

[0020] Hereinafter, an AVM tolerance compensation system according to various embodiments of the present invention will be described in detail with reference to the attached drawings.

[0021] FIG. 1 is a drawing illustrating an AVM tolerance correction system according to various embodiments of the present invention, FIG. 2 is a drawing illustrating an AVM controller (150) according to various embodiments of the present invention, and FIG. 3 is a drawing illustrating camera parameters according to various embodiments of the present invention.

[0022] Referring to FIGS. 1 to 3, an AVM tolerance correction system according to various embodiments of the present invention may include an AVM system (100) composed of a plurality of cameras (120) and an AVM controller (150), a terminal device (180), and a server (200).

[0023] An AVM tolerance correction system according to various embodiments of the present invention may include a plurality of cameras (120). The plurality of cameras (120) may be installed on the outside or on the outside of the construction machine (101) so as to be able to photograph the outer perimeter of the construction machine (101). In one embodiment, the plurality of cameras (120) may include a first camera installed in front of the construction machine (101) (e.g., the first camera (121) of FIG. 2), a second camera installed on the left and right sides of the construction machine (101) (e.g., the second camera (122) of FIG. 2), a third camera (e.g., the third camera (123) of FIG. 2), and a fourth camera installed at the rear of the construction machine (101) (e.g., the fourth camera (124) of FIG. 2). The first to fourth cameras may each photograph the situation in front and behind and the situation on the left and right sides of the construction machine (101) and generate captured image information. The image information generated from each camera can be transmitted to the AVM controller (150) via wired or wireless communication, and the image information generated from each camera can be combined through the controller to create a single image information of a top-view perspective looking down from the top of the construction machine (101).

[0024] An AVM tolerance correction system according to various embodiments of the present invention may include an AVM controller (150). The AVM controller (150) may include a storage unit (152), an AVM image generation unit (154), a correction unit (155), and a communication unit (158).

[0025] The storage unit (152) can store image information regarding the external conditions of the construction machine (101) generated by a plurality of cameras (120). In one embodiment, the storage unit (152) can temporarily store image information of each camera (e.g., the first camera (121), the second camera (122), the third camera (123), and the fourth camera (124) of FIG. 2). Additionally, the storage unit (152) can store camera parameter data for each of the plurality of cameras (120). The camera parameter data may be related to focus information of each camera (121, 122, 123, 124). That is, it may be focus information of the plurality of cameras (120) for distortion-free alignment of image information captured by the plurality of cameras (120). Specifically, referring to FIGS. 3(a), FIGS. 3(b), and FIGS. 3(c), camera parameter data may include information representing the displacement and rotational momentum of each camera in the X-axis, Y-axis, and Z-axis directions relative to a reference point in the world coordinate system, as well as yawing, pitching, and rolling.

[0026] In one embodiment, the storage unit (152) can store camera parameter data in the current state. The storage unit (152) can store initial parameter data for each camera (121, 122, 123, 124) when the construction machine (101) is shipped out. The storage unit (152) can store parameter data for each camera (121, 122, 123, 124) in the current state according to a preset cycle after the construction machine (101) is shipped out. In one embodiment, parameter data for each camera (121, 122, 123, 124) stored at the time of shipment of the construction machine (101) can be stored in the server (200) via the terminal device (180).

[0027] The AVM image generation unit (154) can generate a single AVM image information by combining image information generated from each camera (121, 122, 123, 124). The AVM image information may include image information converted into a highly intuitive real-time image by combining and synthesizing images collected through each camera (121, 122, 123, 124). In this case, the AVM image information may include a single image information from a top-view perspective looking down from above the construction machine (101). The single image information generated by the AVM image generation unit (154) can be displayed to a user through a display unit (300) installed in the cabin of the construction machine (101). Therefore, if the generated AVM image information contains a distorted part, the user can check it through the display unit (300).

[0028] The correction unit (155) can perform correction of the tolerance of each camera (121, 122, 123, 124) when the AVM image information displayed on the display unit (300) is displayed in a distorted manner. According to one embodiment, the correction unit (155) can correct the tolerance of each camera (121, 122, 123, 124) based on the initial parameter data of each camera (121, 122, 123, 124) received from the server (200). In one embodiment, the correction unit (155) may include a tolerance correction program (not shown), and the tolerance of each camera (121, 122, 123, 124) can be corrected by inputting the initial parameter data received from the terminal device (180) into the tolerance correction program. Accordingly, in the present invention, when a tolerance correction is required because a camera or AVM controller (150) is damaged and newly installed during the operation of a construction machine (101), the correction procedure can be performed at the site using the initial parameter data of the camera received from the server (200) without moving to a separate facility equipped with a correction facility or installing a correction facility at the construction site.

[0029] The communication unit (158) can communicate with the terminal device (180) via wired or wireless communication. An AVM controller (150) according to various embodiments of the present invention can transmit initial parameter data for each camera (121, 122, 123, 124) at the time of shipment of the construction machine (101) to the terminal device (180) via the communication unit (158). The initial parameter data can be stored in an external server (200) via the terminal device (180). Additionally, the communication unit (158) can receive the initial parameter data for each camera (120) from the terminal device (180) that the terminal device (180) received from the server (200) when a request for tolerance correction occurs.

[0030] An AVM tolerance correction system according to various embodiments of the present invention may include a terminal device (180). The terminal device (180) may be installed on the construction machine (101) for remote management of the construction machine (101). The terminal device (180) may transmit and receive data via wireless communication with a server (200) located outside the construction machine (101). In this case, the terminal device (180) may serve to transmit the data received from the server (200) to the AVM control via wired or wireless communication. In one embodiment, parameter data of each AVM camera (121, 122, 123, 124) may be transmitted to the server (200) when the construction machine (101) is shipped. In this case, the parameter data transmitted to the server (200) may include parameter data of each AVM camera (121, 122, 123, 124) in a state where tolerance correction has been completed in the inline process. In one embodiment, when a request for tolerance correction for a plurality of cameras (120) occurs, the terminal device (180) can receive parameter data of the AVM camera (120) at the time of shipment for each construction machine (101) from the server (200). In one embodiment, when a user confirms through the display unit (300) that distortion is included in the AVM image information, the user can send an initial parameter data request signal to an external server (200) by operation. In one embodiment, the terminal device (180) can send an initial parameter data request signal to the server (200) according to a preset period. In this case, the terminal device (180) can receive the initial parameter data from the server (200) and transmit it to the AVM controller (150).

[0031] An AVM tolerance correction system according to various embodiments of the present invention may include a server (200). The server (200) may wirelessly transmit and receive data with a terminal device (180) installed on a construction machine (101). In one embodiment, the server (200) may include a server (200) of a tele-monitoring-system (TMS) for remote management of the construction machine (101). In one embodiment, the server (200) may receive and store initial parameter data for an AVM camera (120) through a terminal device installed on the construction machine (101) at the time of shipment of each construction machine (101). In one embodiment, when there is a request from the terminal device, the server (200) may transmit the initial parameter data for the AVM camera (120) of the construction machine (101) that was previously stored to the terminal device (180) of the construction machine (101). In one embodiment, the server (200) is a terminal device installed on each construction machine (101) via an over-the-air (OTA) method, and can transmit initial parameter data of cameras (120) installed on the construction machine (101) according to a preset period.

[0032] FIG. 4 is a diagram illustrating an AVM tolerance correction method according to various embodiments of the present invention.

[0033] Referring to FIG. 4, an AVM tolerance correction method according to various embodiments of the present invention may include: an operation (510) of transmitting initial parameter data for a plurality of cameras (120) installed on a construction machine (101) to a server (200); an operation (520) of generating AVM image data based on image data generated through the plurality of cameras (120); an operation (530) of receiving the initial parameter data for each of the plurality of cameras (120) from the server (200) when distortion occurs in the generated AVM image data; and an operation (540) of correcting the tolerance of the plurality of cameras (120) based on the received initial parameter data.

[0034] (510) Initial parameter data for each of the multiple cameras (120) can be transmitted to an external server (200). The cameras (120) installed on each construction machine (101) are shipped out with the tolerance correction completed in the inline process, and once the tolerance correction is completed, the parameter data of each camera (121, 122, 123, 124) can be determined. This can be described as initial parameter data. In the AVM tolerance correction method of the present invention according to various embodiments, the initial parameter data with the tolerance correction completed at the time of shipment is transmitted to the server (200). The initial parameter data can be stored in the storage unit (152) of the AVM controller (150) and can be transmitted to a terminal device (180) through the communication unit (158). The initial parameter data of each camera (121, 122, 123, 124) transmitted to the terminal device (180) in this way can be transmitted to and stored in the server (200) according to the user's operation or a preset period.

[0035] (520) In operation, the AVM image generation unit (154) can generate a single AVM image information by combining image data generated by each camera (121, 122, 123, 124) for which tolerance correction has been completed. The AVM image information generated in this way can be displayed to a user through a display unit (300) placed inside the cabin of the construction machine (101). In one embodiment, since the AVM image information is generated by combining image information generated by four different cameras (121, 122, 123, 124), if the focus of each camera (121, 122, 123, 124) is misaligned, distortion may occur in the AVM image information. If the AVM controller (150) is damaged along with some of the multiple cameras due to an impact that occurs during the operation of the construction machine (101), the damaged camera and the AVM controller (150) can be replaced and installed on-site, and in this case, the generated AVM image information may include a distorted image. In this case, the user can check whether distortion of the image information has occurred through the display unit (300).

[0036] (530) When partially distorted AVM image information is displayed through the display unit (300) during operation, the user can generate a signal requesting initial parameter data for each camera (121, 122, 123, 124) stored in the server (200) by operating the terminal device (180). By sending the request signal to the server (200) through the terminal device (180), the previously stored initial parameter data can be received.

[0037] (540) In this operation, the initial parameter data for the cameras (120) installed on the construction machine (101) received in this way can be transmitted to the AVM controller (150). By executing the correction program stored in the correction unit (155), a tolerance correction procedure for each camera (121, 122, 123, 124) can be performed based on the received initial parameter data.

[0038] As described above, the AVM tolerance correction method according to various embodiments of the present invention has the effect of enabling rapid on-site response to damage to the AVM system (100) by correcting the tolerance of the AVM system (100) at the construction site using the initial parameter value that was previously stored when the AVM system (100) mounted on the construction machine (101) is damaged.

[0039] In addition, since tolerance correction of the AVM system is possible at the construction site, it has the effect of shortening construction time and reducing costs.

[0040] In the foregoing, although all components constituting an embodiment of the present invention have been described as being combined or operating in combination, the present invention is not necessarily limited to such embodiments. That is, within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate. Furthermore, terms such as "include," "constitute," or "have" described above, unless specifically stated otherwise, mean that the relevant component may be inherent; thus, they should be interpreted as allowing for the inclusion of additional components rather than excluding other components. All terms, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains, unless otherwise defined. Terms commonly used, such as those defined in advance, should be interpreted in accordance with their meaning in the context of the relevant technology and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the present invention.

[0041] The foregoing description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0042] 100: AVM System 101: Construction Machinery 120: Camera 150: AVM controller 152: Storage section 1521: Video storage unit 1522: Parameter storage section 154: AVM Image Generation Unit 155: Correction section 158: Communications Department 180: Terminal device 200: Server 300: Display section

Claims

Claim 1 An Around View Monitoring (AVM) tolerance correction system applied to construction machinery comprises: a plurality of cameras that generate image information; an AVM controller that stores at least one parameter data for each of the plurality of cameras and generates AVM image information based on the image information; and a terminal device that receives at least one previously stored initial parameter data for the plurality of cameras from an external device, wherein the AVM controller corrects the tolerance of the plurality of cameras based on the received initial parameter data when distortion occurs in the AVM image information, and the parameter data includes parameters for the position and rotation of each of the plurality of cameras, and the initial parameter is parameter data for the plurality of cameras in a state where the tolerance is corrected. Claim 2 In claim 1, the parameter data comprises an AVM tolerance correction system including parameters for the position and rotation of each of the plurality of cameras. Claim 3 In paragraph 2, the parameter data comprises an AVM tolerance correction system including parameters representing displacement of the X-axis, Y-axis, and Z-axis and yawing, pitching, and rolling for each of the plurality of cameras. Claim 4 In paragraph 2, the initial parameter is an AVM tolerance correction system, which is parameter data for the plurality of cameras in a state where the tolerance is corrected. Claim 5 In claim 1, the external device includes a TMS server, and the terminal device is an AVM tolerance correction system that communicates with the TMS server via an OTA (Over-The-Air) method. Claim 6 In claim 1, the AVM controller is an AVM tolerance correction system that stores parameter data of the current state of the plurality of cameras. Claim 7 A method for correcting tolerances for Around View Monitoring (AVM) applied to construction machinery, comprising: transmitting initial parameter data for each of a plurality of cameras to a server when the tolerance correction of the AVM of the construction machinery is completed; generating AVM image information based on image information generated through the plurality of cameras; receiving the initial parameter data for each of the plurality of cameras from the server when distortion occurs in the generated AVM image information; and correcting the tolerances of the plurality of cameras based on the received initial parameter data, wherein the parameter data includes parameters for the position and rotation of each of the plurality of cameras, and the initial parameter is parameter data for the plurality of cameras in a state where the tolerances are corrected.

Images