Camera calibration system for correcting camera image

Abstract

The present invention relates to a camera calibration system for correcting a camera image. The camera calibration system comprises: a reference target unit including a laser module which acquires a plurality of reference light patterns or a test chart on which a checkerboard pattern is printed; a camera configured to acquire the original data or correction data by photographing the reference target unit at one or more time points; and a correction result processing and display device configured to acquire result data on the basis of the acquired original data or correction data, wherein the correction result processing and display device comprises: a communication unit for receiving the correction data from the camera; and a processor unit for processing the received correction data.

Description

Camera calibration system for performing correction of camera images

[0001] The present invention relates to a camera calibration system for performing correction of a camera image.

[0002] Cameras are essential components in diverse industries, and applications such as autonomous driving, augmented reality (AR), 3D scanning, and factory automation (FA) require high-precision image calibration. However, camera lenses introduce optical distortion, requiring a process of correcting this distortion for accurate image processing. Typically, a calibration technique utilizing a checkerboard pattern is used, calculating correction parameters such as focal length, distortion center, and radial and tangential distortion coefficients. However, existing camera calibration systems have several limitations.

[0003] Conventional camera calibration methods often rely on manual methods for collecting and analyzing calibration data. Because users must manually measure data and apply it to calibration algorithms, the calibration process is cumbersome and time-consuming. Furthermore, results can vary depending on the operator's experience, and the need for repeated adjustments reduces efficiency and increases the likelihood of measurement and calibration errors.

[0004] Furthermore, existing calibration systems lack the ability to evaluate calibration quality and detect errors. In typical calibration processes, users must manually review the calibration results to determine whether the correction was performed properly, and even after calibration is complete, lens distortion may not be completely eliminated. Furthermore, they lack the ability to immediately detect and address abnormal calculations of values ​​such as Focal Length and Distortion Center, making it difficult to quickly resolve calibration errors.

[0005] Therefore, a technology is required to solve the problem of such camera correction systems.

[0006] Existing correction methods are inefficient because they rely on manual work to collect and analyze correction data. The present invention aims to simplify the correction process by collecting and processing correction data and to minimize repetitive manual work.

[0007] Existing correction methods lack the ability to evaluate whether correction has been performed normally, making error detection difficult. Therefore, the present invention aims to evaluate correction quality by analyzing the corrected Focal Length, Distortion Center, and distortion coefficient.

[0008] In the existing correction method, the correction result is output as a simple number, making it difficult for the user to intuitively check the correction quality. Therefore, the present invention provides a visual output function such as a graph, chart, or color display through a display unit, so as to easily check the correction status and intuitively provide information on whether additional correction is necessary.

[0009] The present invention aims to solve the problems of inefficiency and low accuracy of existing calibration methods that require manually placing reference targets for camera calibration, repeatedly acquiring images at multiple viewpoints, and then manually processing calibration data.

[0010] The present invention relates to a camera calibration system for performing correction of a camera image, comprising: a reference target section including a laser module for obtaining a plurality of reference light patterns or a test chart having a checkerboard pattern printed thereon; a camera configured to photograph the reference target section at one or more viewpoints to obtain original data or correction data; a correction result processing and display device configured to obtain result data based on the obtained original data or correction data; wherein the correction result processing and display device includes a communication section for receiving the correction data from the camera; and a processor section for processing the received correction data.

[0011] In addition, the camera correction device is further included, wherein the camera correction device is configured to receive the original data from the camera and obtain correction data, and the communication unit is configured to receive the correction data from the camera correction device.

[0012] In addition, the processor unit is configured to analyze at least one parameter among the Focal Length (fx, fy) parameter, the Distortion Center (cx, cy) parameter, the Radial Distortion Coefficients parameter, and the Tangential Distortion Coefficients (P1, P2) parameter with respect to the received correction data.

[0013] In addition, the correction result processing and display device further includes a display unit, and is configured to output the acquired result data in the form of at least one of numerical data, graph data, chart data, and image comparison data.

[0014] Additionally, the display unit is configured to output a warning message when an error is detected based on the acquired result data.

[0015] In addition, the correction result processing and display device further includes a memory unit, and the memory unit is configured to store result data including at least one of corrected Focal Length numerical data, Distortion Center numerical data, Radial numerical data, and Tangential distortion coefficient numerical data.

[0016] Additionally, the laser module is configured to generate a plurality of reference light patterns arranged in a grid shape and is configured to project 70 or more dots at regular intervals.

[0017] Additionally, the laser module includes a VCSEL or DOE-based light source element and is configured to generate dots in a fixed pattern.

[0018] Additionally, the test chart is positioned at a fixed position so that it can be photographed at different points in time, and the camera is configured to acquire multiple images while performing a tilt operation.

[0019] Additionally, the test chart includes reference points for calculating reprojection errors based on three-dimensional reference coordinates corresponding to two-dimensional coordinates of grid corners.

[0020] In addition, the correction result processing and display device is configured to transmit the acquired result data to an external server through the communication unit.

[0021] The present invention can provide an effect of quantitatively evaluating the quality of correction by collecting and analyzing correction data from a camera correction device or a camera.

[0022] The present invention can provide an effect of improving correction accuracy through analysis of Focal Length, Distortion Center, and distortion coefficient.

[0023] The present invention provides the effect of enabling a user to immediately confirm whether correction has been performed correctly through automated quality analysis.

[0024] The present invention provides an effect of visually outputting correction data in the form of a graph, chart, color code, image comparison, etc. through a display unit.

[0025] The present invention provides the effect of evaluating whether a camera recognizes a reference pattern with a constant accuracy under various angles, distances, and field of view conditions, compared to a conventional calibration method that relies on shooting at a single point in time.

[0026] The present invention can provide the effect of automating the existing manual data acquisition process by changing the camera's posture and collecting image data captured at multiple points in time.

[0027] The present invention provides the effect of quantitatively securing the reliability of calibration results by calculating reprojection errors through data analysis at multiple points in time.

[0028] The present invention provides an effect of performing the entire process from creation, processing, evaluation, and visualization of correction data within a single system by integrating a reference target, a correction device, a camera, and a result processing and display device.

[0029] FIG. 1 is a drawing for explaining the configuration of a camera correction data collection and correction result display system according to the present invention.

[0030] FIG. 2 is a flowchart illustrating a correction data processing method of a camera correction data collection and correction result display system according to one embodiment of the present invention.

[0031] FIG. 3 is a drawing for explaining an embodiment of performing calibration using a plurality of laser dots in a grid shape radiating from a laser module of the present invention.

[0032] FIG. 4 is a drawing for explaining an example of utilizing a test chart for performing calibration of a camera according to the present invention.

[0033] FIG. 5 is a diagram showing a test environment of a camera calibration system according to one embodiment of the present invention.

[0034] Specific details of the embodiments are included in the detailed description and drawings.

[0035] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0036] FIG. 1 is a drawing for explaining the configuration of a camera correction data collection and correction result display system according to the present invention.

[0037] Camera calibration result processing and display device (100)

[0038] Memory unit (110)

[0039] As illustrated, the camera calibration system (1) of the present invention includes a camera calibration result processing and display device (100). Specifically, the camera calibration result processing and display device (100) includes a memory unit (110). The memory unit (110) serves to store and manage correction data (D2) received from the camera calibration device (10) or the camera (11).

[0040] According to one embodiment of the present invention, the memory unit (110) can manage not only the temporary storage of correction data (D2), but also various information for analysis and comparison of correction results. The memory unit (110) can store the correction data (D2) for a long period of time. Accordingly, a user can evaluate whether the correction quality is continuously improving through comparison with previous correction results. The memory unit (110) can provide a cumulative storage function for the correction data (D2).

[0041] According to one embodiment of the present invention, the memory unit (110) can store result data (D3) including the correction quality evaluation result. The memory unit (110) can store the quality analysis results such as the correction reliability score, error rate, and quality index performed by the processor unit (130). Visual correction data output from the display unit (140) can also be stored in the memory unit (110). The memory unit (110) can store UI display information such as graphs, comparison images, and numerical data obtained based on the post-correction data (D2).

[0042] According to one embodiment of the present invention, the memory unit (110) can store user settings and system configuration information. For example, the memory unit (110) can store information such as display formats selected by the user, such as graph mode and number mode, data update cycle, and network settings. In addition, the memory unit (110) can perform a function of temporarily storing result data (D3) to be transmitted to the server (20) via the communication unit (120). The memory unit (110) can store the correction data (D2) for a certain period of time to prevent it from being lost when the network connection is not smooth. Accordingly, the communication unit (120) can transmit the result data (D3) from the memory unit (110) to the server (20) when the connection is restored.

[0043] Communications Department (120)

[0044] As illustrated, the camera calibration system (1) of the present invention includes a camera calibration result processing and display device (100). Specifically, the camera calibration result processing and display device (100) includes a communication unit (120).

[0045] According to one embodiment of the present invention, the communication unit (120) receives correction data (D2) from the camera correction device (10) or the camera (11), and transmits the analyzed correction result to the server (20). The communication unit (120) can transmit and receive data through a wired or wireless network, and can utilize various communication protocols for reliable data transmission. Specifically, the communication unit (120) can be configured to transmit and receive correction data through not only wired communication methods such as Ethernet, USB, serial communication (RS-232, UART), but also wireless communication methods such as Wi-Fi, Bluetooth, LTE, and 5G. In addition, the communication unit (120) can selectively apply protocols such as TCP / IP, MQTT, HTTP(S), and FTP, and when real-time communication is required, a QoS (Quality of Service)-based protocol or a lightweight protocol supporting real-time data transmission can be applied.

[0046] According to one embodiment of the present invention, when receiving correction data (D2), the communication unit (120) may receive data directly from the camera correction device (10), or in some cases, may receive correction data (D2) acquired by the camera (11) itself. In addition, the communication unit (120) performs a function of transmitting result data (D3) to the server (20). After correction is completed, the result data (D3) analyzed by the processor unit (130) is transmitted to the server (20), thereby enabling long-term data storage and remote monitoring.

[0047] Processor section (130)

[0048] As illustrated, the camera calibration system (1) of the present invention includes a camera calibration result processing and display device (100). Specifically, the camera calibration result processing and display device (100) includes a processor unit (130).

[0049] Example of Correction Data Analysis and Geometric Correction Status Evaluation

[0050] According to one embodiment of the present invention, the processor unit (130) analyzes correction data (D2) received from the camera correction device (10) or the camera (11), evaluates the quality of the correction result, and performs a function of visually converting the same. To this end, the processor unit (130) processes the correction data to calculate major correction indices such as Focal Length (fx, fy), Distortion Center (cx, cy), Radial Distortion Coefficients (K1 to K6), and Tangential Distortion Coefficients (P1, P2), and based on this, the geometric correction state of the camera can be quantitatively evaluated.

[0051] Internal parameter analysis and focal length evaluation example

[0052] According to one embodiment of the present invention, the processor unit (130) accurately estimates the internal parameters of the camera (11) based on the received correction data (D2) and performs an operation to confirm whether the correction process has been performed properly. In particular, the Focal Length (fx, fy) is an element indicating the change in the focal length and optical center of the camera, and the processor unit (130) can analyze this to evaluate whether the correction has been performed normally.

[0053] Example of Distortion Center Analysis and Precision Improvement

[0054] According to one embodiment of the present invention, the processor unit (130) can analyze the Distortion Center (cx, cy) to evaluate how closely the lens distortion center aligns with the center of the image sensor. If the Distortion Center value shows a significant difference from the reference value even after correction, the processor unit (130) can warn the user or apply an automatic correction algorithm to perform additional adjustments.

[0055] Specifically, the processor unit (130) compares the Distortion Center (cx, cy) value extracted from the received correction data (D2) with the theoretical center coordinates of the image sensor to calculate the error range, and if this error exceeds a preset threshold, determines that the correction quality is insufficient. At this time, the processor unit (130) may output a visual warning message to the user through the display unit (140) or repeatedly perform the process of reapplying the automatic correction algorithm to correct the position of the Distortion Center closer to the center of the sensor. In addition, if the Distortion Center tends to continuously deviate from the reference value, the processor unit (130) may diagnose the possibility of a hardware alignment problem or an offset between the lens and the sensor and output a message to the user indicating that a technical inspection is necessary.

[0056] Nonlinear Distortion (Radial and Tangential Distortion) Correction and Analysis Example

[0057] According to one embodiment of the present invention, the processor unit (130) precisely analyzes the nonlinear distortion of the camera lens by utilizing the Radial Distortion Coefficients (K1 to K6) and the Tangential Distortion Coefficients (P1, P2). The Radial Distortion Coefficients indicate the degree to which image distortion occurs along a circular pattern, and the Tangential Distortion Coefficients indicate the asymmetric distortion that occurs due to the alignment error between the lens and the image sensor. The processor unit (130) compares and analyzes these distortion coefficients to evaluate whether the distortion has been reduced after correction, and to optimize the corrected data.

[0058] Specifically, the processor unit (130) can quantitatively compare the radial and tangential distortion coefficient values ​​before and after correction to evaluate the actual improvement effect of the distortion correction using a numerical index. For example, if the absolute values ​​of the K1 to K6 values ​​are significantly smaller after correction, it means that the radial distortion has been effectively reduced, and if the changes in the P1 and P2 values ​​are minimal or adjusted to within the reference value, it means that the tangential distortion has also been stably corrected. Based on such analysis results, the processor unit (130) can calculate the reliability of the distortion correction result, re-execute the correction if the correction quality falls below a specific threshold, or use it as a judgment criterion for guiding additional user operations.

[0059] Real-time correction possibility assessment and application of automatic correction algorithm

[0060] According to one embodiment of the present invention, the processor unit (130) can analyze the correction result to determine whether the internal parameters of the camera (11) have been accurately corrected, and maintain a consistent correction process even in various environments. This simplifies the correction process and reduces the time for repetitive correction work. In addition, the processor unit (130) can calculate the distortion coefficient on a pixel-by-pixel basis, which is advantageous for implementing a system capable of real-time correction. Specifically, the processor unit (130) can analyze the image stream input from the camera (11) in real time and dynamically calculate the distortion correction parameter applied to each pixel in the image. Through this, the processor unit (130) can readjust the parameters of the correction algorithm to adapt to various external environmental changes such as lighting conditions, shooting distance, and lens replacement.

[0061] Additionally, the processor unit (130) continuously monitors the correction quality of each image frame received in real time, and if it falls below a certain quality threshold, the correction algorithm can be immediately updated or previously stored optimal correction parameters can be loaded and applied. This method provides the advantage of maintaining consistency in image quality even when photographing objects moving at high speed or in environments where the position of the camera (11) frequently changes.

[0062] Display section (140)

[0063] As illustrated, the camera calibration system (1) of the present invention includes a camera calibration result processing and display device (100). Specifically, the camera calibration result processing and display device (100) includes a display unit (140).

[0064] According to one embodiment of the present invention, the display unit (140) visually outputs the result data (D3) analyzed by the processor unit (130), thereby allowing the user to intuitively check the correction status. The result data (D3) may be provided to the user in various ways, such as numerical information, graphics, color display, comparison images, etc.

[0065] According to one embodiment of the present invention, the display unit (140) performs a function of visually representing the results of the correction quality assessment. For example, the display unit (140) can organize data such as the corrected Focal Length, Distortion Center, Radial and Tangential distortion coefficients, and output them in the form of a chart, graph, color code, or numerical data. This allows the user to easily check whether the correction has been performed properly.

[0066] Additionally, the display unit (140) can visually provide result data (D3) including comparison results before and after correction. The display unit (140) includes a function for comparing data before and after correction is applied, allowing the user to intuitively check how much image distortion has been improved. For example, the display unit (140) can compare images before and after correction by placing them side by side, allowing the user to immediately evaluate the effect of the correction.

[0067] According to one embodiment of the present invention, the display unit (140) may also perform a function of providing a warning when a calibration error occurs or additional calibration is required. If, as a result of analysis by the processor unit (130), a specific calibration coefficient value exceeds a standard or the calibration quality is low, the display unit (140) may output a warning message or a visual signal to inform the user that calibration adjustment is required. Specifically, the display unit (140) may provide a visual notification in the form of a warning icon, color change, pop-up message, etc., based on the result data (D3) received from the processor unit (130), if the calibration quality is not within a normal range.

[0068] For example, if a specific Radial Distortion Coefficient (K1 to K6) exceeds a reference value, or if the Distortion Center deviates from the center of the image sensor by a specified threshold distance, a red warning sign or a phrase such as “Additional correction required” may be displayed on the screen. In addition, the display unit (140) supports the user to intuitively understand the current correction status and take appropriate action by displaying the status of the correction quality in real time in the form of a graph or gauge. For example, if the correction quality score is low, the gauge color may change from green to yellow or red, or a warning light may light up as the quality score decreases. In addition, the display unit (140) may not simply warn the user of an error, but may also provide the user with specific guidance information such as how to perform additional correction, items for which re-calibration is recommended, and causes of expected problems. Through this, the user can quickly and accurately perform the correction task based on a clear understanding of the correction status.

[0069] Real-time correction data monitoring function example

[0070] According to one embodiment of the present invention, the display unit (140) can provide a real-time correction data monitoring function. In certain environments, the correction data is continuously updated, and the correction process can be performed in real time. In this case, the display unit (140) can dynamically output the correction status by reflecting the data updated in real time.

[0071] Specifically, the display unit (140) can output the change trend of each parameter in the form of a time series graph, real-time numerical change display, warning signal, etc. based on the real-time correction result data (D3) transmitted from the processor unit (130) at regular intervals or when an event occurs. For example, if the value of Focal Length or Distortion Coefficients changes rapidly during correction, the corresponding value is reflected in real time and displayed on the screen, and if it exceeds the threshold, a visual warning is immediately output so that the user can quickly recognize the abnormality.

[0072] Additionally, the display unit (140) can express a correction quality index or correction accuracy graph that changes in real time through animation effects or color changes. For example, the display unit (140) can change the color to green when the quality score is high, yellow when it is close to the threshold, and red when a possibility of an error is detected.

[0073] Additionally, the display unit (140) can output a status message such as “Calibration complete” or “Calibration status good” when the calibration has reached a certain level or when the calibration has stabilized. Conversely, the display unit (140) can provide immediate feedback by displaying a warning message in real time such as “Recalibration recommended” or “Quality degradation detected” when the calibration status is unstable or quality deteriorates.

[0074] FIG. 2 is a flowchart illustrating a correction data processing method of a camera correction data collection and correction result display system according to one embodiment of the present invention.

[0075] Correction data collection and storage step (S100)

[0076] The correction data collection and storage step (S100) is a step of collecting and storing correction data (D2) from the camera correction device (10) or the camera (11). First, the communication unit (120) serves to receive the correction data (D2) from the camera correction device (10) or the camera (11). The correction data (D2) is composed of numerical values ​​such as Focal Length (fx, fy), Distortion Center (cx, cy), Radial Distortion Coefficients (K1 to K6), and Tangential Distortion Coefficients (P1, P2), and can be collected via a wired (USB, Ethernet, etc.) or wireless (Wi-Fi, Bluetooth, etc.) network. Since there is a possibility that an error may occur during the process of transmitting the correction data (D3), the communication unit (120) performs a function of verifying the integrity of the received data. For example, the reliability of data can be verified using methods such as checksum or CRC (Cyclic Redundancy Check), and if an error is detected, a resend request can be performed or erroneous data can be filtered.

[0077] In the correction data collection and storage step (S100), the memory unit (110) plays a role in storing and managing the collected correction data. Since the correction data (D2) is utilized in the subsequent correction data processing and analysis step (S200), it is important to store it stably. The memory unit (110) stores data using volatile memory (RAM) or non-volatile memory (Flash, SSD, etc.), and can store log information such as the correction execution time, data collection time, and data source (which camera it was collected from).

[0078] In the correction data collection and storage step (S100), the communication unit (120) can receive the correction data (D2) in one of two ways as follows.

[0079] Embodiment of receiving correction data (D2) from a camera correction device (10)

[0080] According to one embodiment of the present invention, when receiving correction data (D2) through the camera correction device (10), since the camera (11) itself does not have a correction function built in, the camera correction device (10) receives the original data (D1), corrects it, and then acquires the correction data (D2) and transmits it to the communication unit (120). In this process, the camera (11) acquires the original data (D1), such as a photographed image or a pattern image for calibration, and this data is the original data to which correction has not been applied, with lens distortion included. Thereafter, the camera correction device (10) receives the original data (D1) from the camera (11), performs a correction process, such as lens distortion correction, optical center correction, and focal length adjustment, and then acquires the correction data (D2). The data to which correction has been completed includes values ​​to which correction has been applied, such as the corrected Focal Length and the corrected Distortion Coefficients, and is converted to an optimized state. The acquired correction data (D2) is transmitted from the camera correction device (10) to the communication unit (120), and after receiving the data, the communication unit (120) verifies the data integrity and stores only normal data in the memory unit (110).

[0081] Embodiment of receiving correction data (D2) directly from the camera (11)

[0082] According to one embodiment of the present invention, in the case of receiving correction data directly from the camera (11), the camera (11) itself has a built-in correction function, so that correction data (D2) is acquired directly within the camera (11) without going through a separate camera correction device (10) and transmitted to the communication unit (120). In this method, the camera (11) performs processes such as lens distortion correction, Distortion Center adjustment, and Focal Length correction through its own correction algorithm, and immediately acquires data after correction is completed. Thereafter, the correction data (D2) acquired by the camera (11) itself is transmitted to the communication unit (120) through a network (wired / wireless) or serial communication, and the communication unit (120) verifies the data integrity after receiving it and stores normal data in the memory unit (110).

[0083] The method of receiving correction data through a camera correction device (10) has the advantage of allowing more precise adjustment of the correction algorithm, as the camera (11) only acquires the original data (D1), and the correction is entirely performed by the camera correction device (10). On the other hand, the method of acquiring correction data directly from the camera (11) can simplify the system configuration because correction is performed only by the camera (11) without a separate correction device, but there is a difference in that the correction quality may be limited due to the limitations of the internal correction function of the camera (11).

[0084] Accordingly, the communication unit (120) receives correction data (D2) in a different manner depending on whether the correction was performed in the camera correction device (10) or the camera (11), and stores the received data in the memory unit (110) after integrity verification so that it can be utilized in the subsequent correction data processing and analysis step (S200).

[0085] Correction data processing and analysis step (S200)

[0086] Step for performing filtering on correction data (S210)

[0087] The filtering performance step (S210) for the correction data includes a process of evaluating the reliability of the correction data (D2) received through the communication unit (120), detecting error data, and excluding it from a subsequent analysis process. According to the present invention, the processor unit (130) analyzes the correction data (D2) to determine whether it is a value within a normal range. That is, the processor unit (130) can individually examine correction parameters such as Focal Length (fx, fy), Distortion Center (cx, cy), Radial Distortion Coefficients (K1 to K6), and Tangential Distortion Coefficients (P1, P2) included in the correction data (D2), detect values ​​that are outside the set allowable range, and filter the corresponding data.

[0088] When correction data (D2) is received through the communication unit (120), the processor unit (130) can analyze the correction data (D2) to perform initial verification to determine whether to filter. At this time, the processor unit (130) can check whether the format of the data is correct and check whether all expected parameters are included. For example, the processor unit (130) can determine that there is a high possibility that the data is erroneous if the Focal Length value is a physically impossible value or is abnormally large or small. Thereafter, the processor unit (130) can analyze whether there is an error by comparing individual parameters of the correction data (D2) with an allowable range. At this time, if the Focal Length (fx, fy) value is too large or small, if the Distortion Center (cx, cy) is at an extreme position off the center of the image sensor, or if the Radial and Tangential Distortion Coefficients show unexpectedly rapid changes, the data can be determined to be erroneous. The processor unit (130) detects such outliers and selects data suspected of being erroneous so that it can be excluded from the subsequent analysis process.

[0089] When error data is detected, the processor unit (130) can process the error data by applying filtering rules. There are various filtering methods, and the processor unit (130) can discard the data if the correction data (D2) contains a value exceeding the allowable range by using a simple removal method. In addition, when the processor unit (130) applies a partial correction method, if only a specific parameter (such as Focal Length) exceeds the allowable range, the processor unit (130) can use the corrected previous data for correction. If data errors occur frequently, the processor unit (130) can request new correction data from the camera correction device (10) or the camera (11) through the communication unit (120) and re-collect it.

[0090] Distortion correction analysis execution step (S220)

[0091] The distortion correction analysis performance step (S220) is a step in which the processor unit (130) precisely analyzes the quality of the camera correction based on the received correction data (D2). Specifically, the processor unit (130) can first analyze the Radial Distortion Coefficients (K1 to K6) and Tangential Distortion Coefficients (P1, P2) included in the correction data (D2). The radial distortion coefficients indicate the degree to which the image is distorted radially from the center, and are generally related to distortion in the form of expansion or contraction caused by the lens curvature. The tangential distortion coefficients indicate asymmetrical distortion that occurs when the lens and the image sensor are not aligned. The processor unit (130) compares the distortion coefficients with a reference value or a previous correction result to determine whether the correction has been performed appropriately. If the distortion coefficients are excessively large or change rapidly, there is a possibility that the correction quality has deteriorated or there is a problem with the lens condition, and thus it can be determined that additional inspection of the correction result is necessary.

[0092] In addition, the processor unit (130) analyzes the Distortion Center (cx, cy) value to evaluate how much the center of the lens distortion matches the center of the image sensor of the camera (11). Theoretically, the Distortion Center must match the center of the image sensor to be considered accurate in correction, and if the center is significantly off, there is a possibility that the correction was not sufficiently performed or distortion remains due to a structural problem in the camera. Therefore, the processor unit (130) can quantitatively measure the deviation between the coordinates of the Distortion Center and the coordinates of the center of the image sensor, and if it exceeds the allowable standard, it can be determined as a correction error.

[0093] In addition, the processor unit (130) compares the corrected Focal Length (fx, fy) values ​​with the reference values ​​to check whether the correction has been performed stably. Focal Length is the distance between the optical center of the lens and the image sensor, and a change in the focal length directly affects the correction quality. The processor unit (130) evaluates whether the Focal Length obtained as a result of the correction conforms to the general camera characteristics, or whether it has changed excessively or represents a physically impossible value, and detects it as an error if it exceeds the reference value or goes out of the expected range.

[0094] Step for detecting correction errors and determining the need for additional correction (S230)

[0095] The step of detecting a correction error and determining the need for additional correction (S230) is a step in which the processor unit (130) finally determines whether the correction quality is appropriate based on the distortion correction analysis results, and if the reliability of the correction results is determined to be insufficient, provides the user with information that additional correction is necessary.

[0096] According to the present invention, the processor unit (130) filters various correction parameters included in the correction data (D2) and performs a precise analysis through a filtering performance step (S210) and a distortion correction analysis performance step (S220) for the correction data, and then can comprehensively determine whether the data satisfies the correction criteria. At this time, if all major correction indices such as Focal Length (fx, fy), Distortion Coefficients (K1 to K6, P1, P2), and Distortion Center (cx, cy) exist within the allowable range and the correction quality is evaluated to be above a certain standard, the processor unit (130) can determine that the correction has been successfully performed and obtain result data (D3).

[0097] The result data (D3) includes the corrected parameters, and the processor unit (130) can output the result data (D3), which is information on the completed correction result, to the display unit (140) or transmit it to the server (20) via the communication unit (120). The result data (D3) includes the finally confirmed corrected Focal Length, Radial and Tangential distortion coefficients, Distortion Center coordinates, etc., and can be used as important reference information in the subsequent camera performance diagnosis, correction quality monitoring, or system maintenance process.

[0098] On the other hand, if the quality of the analysis result correction data (D2) is judged to be below the standard, or if a specific correction value exceeds the allowable range or deviates sharply from the expected range, the processor unit (130) determines that the correction was not sufficiently performed or that an error exists. For example, this may be the case when the Distortion Center is excessively displaced from the center of the image sensor, the distortion coefficient jumps sharply, or the Focal Length is analyzed to be a physically impossible value.

[0099] In such cases, instead of obtaining result data (D3), the processor unit (130) obtains an error message or warning message for conveying the error status to the user or recording it in the system. The message may include information regarding the cause of the calibration failure or the direction of action, such as "additional calibration required," "calibration failed," or "distortion center mismatch detected," and may be conveyed to the user via the display unit (140) or stored in the server (20).

[0100] Step of performing result data acquisition and storage (S240)

[0101] The result data acquisition and storage execution step (S240) is a step of storing the result data (D3) acquired by the processor unit (130) in the memory unit (110). Specifically, if the processor unit (130) determines that the correction quality is normal through analysis of the correction data (D2), it acquires the result data (D3) including the main indicators of the completed correction state. The result data (D3) includes information such as the corrected Focal Length (fx, fy), Radial and Tangential distortion coefficients (K1 to K6, P1, P2), and Distortion Center (cx, cy) coordinates, for example. In addition, analysis indicators such as the quality score, reliability index, and error rate from the reference value derived from the correction quality evaluation results may also be included.

[0102] The result data (D3) thus obtained is stored in the memory unit (110). The memory unit (110) is designed not only to simply store this result data, but also to manage it together with previous correction history for the same camera (11). For example, if correction is performed multiple times for the same camera (11), each correction result is individually stored, thereby allowing analysis of changes in correction quality over time. Such correction history data can be utilized to compare past correction results with current results to determine whether the correction quality has improved or deteriorated.

[0103] Additionally, the memory unit (110) can also store a history of calibration failures or errors. For example, if an error is detected in a specific calibration and result data (D3) is not obtained and an error message is obtained, the memory unit (110) can also store cause information related to this, such as excessive distortion center deviation, sudden change in distortion coefficient, etc.

[0104] Step for displaying the results of correction data analysis through the display unit (S300)

[0105] The step (S300) of displaying the results of the correction data analysis through the display unit is a step of visually providing the user with the result data (D3) obtained through the correction data processing and analysis step (S200). The display unit (140) can receive the result data (D3) obtained through analysis and calculation in the processor unit (130), and perform output by visualizing it in a form that the user can easily understand. The display unit (140) can perform output including not only simple numbers or text information, but also various visual expression means such as graphs, charts, color indicators, and image comparisons. This visualization allows the correction results to be understood at a glance, and is effective in intuitively conveying whether the correction status is good, the degree of correction quality, and whether there is a warning for a specific item.

[0106] Specifically, the display unit (140) can organize the Focal Length (fx, fy), Distortion Center (cx, cy), Radial and Tangential distortion coefficients (K1 to K6, P1, P2), etc. included in the correction result and output them as a numerical table or graphic elements. For example, if the Focal Length is within a reference value, it can be displayed in green, and if it exceeds the reference value, it can be displayed in a visual warning color such as yellow or red. In addition, it can output how much the Distortion Center deviates from the center of the image sensor in the form of a coordinate comparison graph or image overlay, and it can also provide an image comparison viewer function that compares the distortion difference between images before and after correction.

[0107] According to one embodiment of the present invention, the display unit (140) can also perform a notification function regarding the occurrence of a calibration error and the need for additional calibration. If the processor unit (130) detects an error or quality degradation, the display unit (140) provides clear feedback to the user by displaying messages such as "Calibration failed," "Additional calibration required," or "Excessive distortion center deviation" on the screen. Furthermore, the display unit (140) can not only simply output results, but also dynamically update the UI by reflecting a data stream updated in real time.

[0108] Correction result saving step (S400)

[0109] The correction result storage step (S400) is a step for continuously storing and managing the correction result output to the user through the display unit (140). The memory unit (110) stores the result data (D3) acquired by the processor unit (130) in the correction result storage step for a long period of time, thereby accumulating the correction history for the same camera (11) and providing a basis for comparison and analysis with subsequent correction results. Specifically, the memory unit (110) can store key correction parameters such as the corrected Focal Length (fx, fy), Distortion Coefficients (K1~K6, P1, P2), and Distortion Center (cx, cy), as well as the quality evaluation results, whether an error was detected, the error rate from the reference value, etc.

[0110] In addition, the memory unit (110) can also perform a history management function of the correction results. When the same camera (11) is corrected multiple times, the result data (D3) at each correction point is independently stored and organized into time series data, enabling analysis of correction quality trends, correction algorithm effects, hardware status changes, etc. This function can serve as important reference material for long-term quality management, system inspection, and correction algorithm improvement. In addition, the memory unit (110) can also serve as an intermediate storage for transmitting the result data (D3) to the server (20) as needed. In case of unstable network connection, the data can be stored locally until communication is restored.

[0111] FIG. 3 is a drawing for explaining an embodiment of performing calibration using a plurality of laser dots in a grid shape radiating from a laser module of the present invention.

[0112] The present invention relates to a method and device for setting lighting conditions for performing calibration of a camera (11), and is configured to perform high-precision calibration using a plurality of laser dots in a grid shape radiating from a laser module (12), instead of a checkerboard or flat marker.

[0113] As illustrated in FIG. 3, the laser module (12) is a light source device configured to project a plurality of laser dots in a grid shape forward, thereby forming a plurality of dot patterns with a predetermined interval to provide reference position information within the imaging area of ​​the camera. The test chart (13) illustrated in FIG. 4 is configured as a flat substrate on which a checkerboard or standardized grid pattern is printed, and enables the camera to obtain correction reference coordinates through the vertices, intersections, or reference points of the pattern when shooting. The camera calibration system (1) of the present invention includes a reference target portion (not illustrated), and the reference target portion is configured to include at least one of the laser module (12) and the test chart (13). The reference target portion is arranged within the field of view of the camera, and functions as a reference reference element that provides position information of a reference point when acquiring original image data, and enables precise calibration to be performed from a plurality of image data acquired according to changes in the attitude of the camera (11).

[0114] As illustrated in FIG. 3, the laser module (12) includes a VCSEL (Vertical Cavity Surface Emitting Laser) array or a single laser light source and a diffractive optical element (DOE) therein, and can uniformly spread multiple laser beams forward and form multiple dots in a grid pattern.

[0115] At this time, the laser module (12) acquires, for example, 70 or more laser dots, and these dots are projected forward while maintaining a fixed pattern such as a regular interval and a rectangular grid arrangement, an equilateral triangular grid, etc. The acquired laser dots are designed to be evenly distributed throughout the field of view (FOV) of the camera (11), and the positions and number of dots detected on the screen at the time the camera takes a picture can be compared with a fixed reference position.

[0116] In particular, the present invention is characterized by performing calibration based on the number and arrangement of dots included within the camera's field of view. For example, even if a laser module emits 70 or more dots, the actual number of dots detected on the image sensor may be only a portion, depending on the camera's focal length, sensor size, field of view, and other field of view characteristics. In this case, it is possible to calculate and correct internal / external parameters based on the coordinates of dots observable within the camera image.

[0117] According to one embodiment of the present invention, the reference position of the dot (the coordinates of the fixed pattern from the laser module) is pre-stored in the camera calibration system (1), and compared with the coordinates of the dot actually detected in the image captured by the image sensor of the camera (11), thereby calculating the pixel error. The calculated error is used to estimate or correct internal parameters such as the focal length, principal point coordinates, and lens distortion coefficient of the camera (11), and external parameters such as the posture and position of the camera (11). In addition, when the dot is not limited to a two-dimensional plane, and various depth directions or parallax information between multiple cameras are utilized together, it is also possible to increase the accuracy of the three-dimensional pose alignment and spatial position estimation of the camera system.

[0118] Conventional flat marker-based methods are easily affected by lighting reflection, marker printing conditions, and shooting distance, but the grid-type laser dot-based method of the present invention provides a constant light output and a clear dot shape, thereby providing a more stable and precise calibration environment.

[0119] FIG. 4 is a drawing for explaining an example of utilizing a test chart for performing calibration of a camera according to the present invention.

[0120] The present invention can precisely correct internal and external parameters of the camera based on image data acquired at multiple points in time by arranging a test chart (13) and a camera (11) in a specific spatial relationship.

[0121] As illustrated in Fig. 4, the test chart (13) is formed in a checkerboard pattern or a similar planar reference pattern including a grid pattern repeated at regular intervals, and the test chart (13) is positioned within a predetermined distance from the camera (11). At this time, the camera (11) is installed in a structure that is supported so as to be able to perform tilt rotation around the X-axis and the Y-axis. In addition, the camera (11) is configured so as to be able to photograph the test chart (13) at different observation points while gradually changing the posture of the camera (11) through tilt rotation.

[0122] According to one embodiment of the present invention, a user can sequentially change the viewpoint of a camera (11) to capture a test chart (13) at different viewpoints corresponding to, for example, a total of nine locations, thereby acquiring each image. The images acquired at each viewpoint are used to analyze and extract two-dimensional coordinates (pixel coordinates) of vertices, intersections, or reference points of a grid pattern included in the test chart (13).

[0123] At this time, the actual physical location of the test chart (13) is stored as a location on a predefined three-dimensional reference coordinate system, and based on the three-dimensional coordinate information of the reference point corresponding to the 2D coordinate detected in each image, the projection matrix or calibration matrix of the camera can be mathematically interpreted. Through this, internal parameters such as the focal length, principal point coordinates, and lens distortion coefficient of the camera (11) and external parameters such as the position and attitude of the camera can be precisely estimated and corrected.

[0124] This calibration process may include a step of sequentially changing the posture of the camera (11) and capturing a total of about 9 images by photographing the test chart (13) at various viewpoints, a step of automatically or manually extracting 2D coordinates of reference points or vertices on the pattern of the test chart (13) from each image, a step of calculating theoretical projection coordinates by referring to the reference 3D coordinates (position on the model coordinate system) of the test chart (13) corresponding to the extracted 2D coordinates, and a step of evaluating the accuracy of the corresponding calibration state by calculating the error (reprojection error) between the coordinates on the actual image and the theoretical projection coordinates. In addition, the camera (11) may calculate an average reprojection error for all images acquired at multiple viewpoints.

[0125] In addition, in addition to the above configuration, the present invention may additionally include a configuration in which the pattern of the test chart (13) is replaced with a grid-type dot pattern obtained by a laser module (12) instead of a physically printed correction pattern. In this case, the laser module (12) is configured to project 70 or more dots at regular intervals in front using an optical element based on a VCSEL, a DOE (Diffractive Optical Element), or a MEMS.

[0126] This laser-based calibration method enables the extraction of identical reference points without the need for repeated physical pattern placement, simultaneously improving both the efficiency and accuracy of the calibration process. In particular, it is less affected by external lighting conditions and even enables depth estimation in three-dimensional space, enabling faster and more precise calibration.

[0127] As illustrated in FIG. 1, the camera calibration system (1) is configured to include a camera (11), a camera calibration device (10), and a camera calibration result processing and display device (100), and is configured to acquire original data (D1) required for calibration of the camera (11) by utilizing a laser module (12) or a test chart (13).

[0128] As illustrated, the camera (11) acquires a reference pattern image to be corrected by photographing a laser module (12) or a test chart (13) as a reference light source placed externally. The laser module (12) is a light source capable of projecting a plurality of dots forward in a grid shape, and acquires a dot pattern having a certain interval, and the test chart (13) is configured as a flat target in which a checkerboard or grid pattern is formed.

[0129] Such a reference light source or target is acquired as an image by the imaging image sensor of the camera (11), and the image signal acquired through this is defined as original data (D1). Specifically, the original data (D1) indicates how the reference dot or reference pattern is shaped on the image based on the current calibration status of the camera (11), and is used as basic information required for subsequent calibration processing. In addition, the original data (D1) is transmitted to the camera calibration device (10), and the camera calibration device (10) extracts the positions of the laser dot or reference points on the test chart from the image, and calculates the internal parameters and external parameters of the camera based on this to acquire the calibration data (D2). In this way, the present invention uses the laser module (12) or the test chart (13) as a reference light source or target, acquires the original data (D1) from the image captured by the camera (11), and performs the camera calibration process based on this.

[0130] FIG. 5 is a diagram showing a test environment of a camera calibration system according to one embodiment of the present invention.

[0131] As illustrated, a plurality of test charts (checkerboard pattern) are fixedly installed on a wall, and a movable device equipped with a camera, which is a calibration target, is placed on the opposite side. The camera (11) of the present invention is configured to capture test charts from different viewpoints through various posture changes, such as tilting and moving, and acquire a plurality of image data therefrom.

[0132] The experimental environment was designed as a darkroom to minimize the influence of external lighting and enable repeatable calibration experiments.

[0133] The scope of the present invention is not limited to the embodiments described above, but can be implemented in various forms within the scope of the appended claims. It is contemplated that the scope of the claims encompasses various modifications that can be made by anyone skilled in the art without departing from the spirit of the invention as claimed.

[0134] [Explanation of symbols]

[0135] 1: Camera calibration data collection and calibration result display system

[0136] 10: Camera Calibration Device

[0137] 11: Camera

[0138] 12: Laser module

[0139] 13: Test Chart

[0140] 100: Camera calibration result processing and display device

[0141] 110: Memory section

[0142] 120: Communications Department

[0143] 130: Processor section

[0144] 140: Display section

[0145] D1: Original data

[0146] D2: Correction data

[0147] D3: Result data

[0148] 200: Server

Claims

1. In a camera calibration system for performing correction of a camera image, A reference target portion including a laser module for acquiring multiple reference light patterns or a test chart having a checkerboard pattern printed thereon; A camera configured to photograph the reference target portion at one or more viewpoints to obtain original data or correction data; A correction result processing and display device configured to obtain result data based on the above-mentioned acquired original data or correction data; The above correction result processing and display device a communication unit for receiving the correction data from the camera; and A processor unit for processing the received correction data; A camera calibration system for performing correction of camera images.

2. In paragraph 1, further comprising a camera correction device; The above camera correction device, configured to receive the original data from the camera and obtain correction data, The above communication department, configured to receive the correction data from the camera correction device, A camera calibration system for performing correction of camera images.

3. In paragraph 1, The above processor unit, Configured to analyze at least one parameter among the Focal Length (fx, fy) parameter, the Distortion Center (cx, cy) parameter, the Radial Distortion Coefficients parameter, and the Tangential Distortion Coefficients (P1, P2) parameter for the above received correction data. A camera calibration system for performing correction of camera images.

4. In paragraph 1, The above correction result processing and display device is, further including a display section; Configured to output the above-obtained result data in at least one format among numerical data, graph data, chart data, and image comparison data. A camera calibration system for performing correction of camera images.

5. In paragraph 4, The above display unit, Configured to output a warning message when an error is detected based on the above-obtained result data, A camera calibration system for performing correction of camera images.

6. In paragraph 1, The above correction result processing and display device is, Including further memory section; The above memory section, Configured to store result data including at least one of corrected Focal Length numerical data, Distortion Center numerical data, Radial numerical data, and Tangential distortion coefficient numerical data, A camera calibration system for performing correction of camera images.

7. In paragraph 1, The above laser module, configured to generate a plurality of reference light patterns arranged in a grid shape, Configured to project more than 70 dots at regular intervals, A camera calibration system for performing correction of camera images.

8. In paragraph 7, The above laser module, Contains a VCSEL or DOE-based light source device, Configured to generate dots in a fixed pattern, A camera calibration system for performing correction of camera images.

9. In paragraph 1, The above test chart is placed in a fixed position so that it can be photographed at different points in time. wherein the camera is configured to acquire multiple images while performing a tilt operation; A camera calibration system for performing correction of camera images.

10. In paragraph 9, The above test chart is Including a reference point for calculating the reprojection error based on the 3D reference coordinate corresponding to the 2D coordinate of the grid corner, A camera calibration system for performing correction of camera images.

11. In paragraph 1, The above correction result processing and display device is, Configured to transmit the obtained result data to an external server through the above communication unit, A camera calibration system for performing correction of camera images.

Images