Method for adjusting the output video of a rear view camera for vehicles

Abstract

Video adjustment procedure of a rear view camera for a vehicle, comprising: a pattern detecting step (S10) for detecting a predetermined pattern by an intrinsic parameter of a rear view camera and by geometry information; a camera posture estimation step (S20) for estimating a posture of the rear view camera currently provided in the vehicle (hereinafter simply referred to as “an installed camera”) from a reference pattern coordinate system by the predetermined pattern detected by the pattern detecting step (S10); a homography obtaining step (S30) for obtaining a homography corresponding to an angular error of the installed camera, which is checked by the camera pose estimation step (S20); and a video calibration step (S40) for calibrating the video photographed by the installed camera based on a pose of a design-mounted camera (hereinafter simply referred to as “design-mounted camera”) using the homography obtained by the homography obtaining step (S30).

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Korean Patent Application No. 10-2014-0148539 filed October 29, 2014. TECHNICAL FIELD

[0002] The present invention relates to a video adjustment method of a rear view camera for a vehicle, and more particularly to a video adjustment method of a rear view camera for a vehicle that adjusts a video photographed by an angle error of a currently installed rear view camera based on the video information photographed by a rear view camera and outputs the video to prevent a negligent accident. BACKGROUND

[0003] A rear view camera installed in a vehicle ensures that the driver can see the rear of the vehicle via a monitor when the driver is reversing or reversing the vehicle, thereby enabling the driver to reverse or park the vehicle safely.

[0004] Generally, the rear view camera is hidden in or mounted on a bracket, license plate, trunk, emblem, spoiler or bumper, and the rear view camera is usually permanently mounted to a rear of the vehicle in such a way that the lens is exposed.

[0005] A video shot by the rear view camera is output to a display unit of an audio and video (AV) system of a driver's seat, so that the driver can check a situation at a rear of the vehicle displayed on the display unit while reversing or reversing the vehicle.

[0006] A rearview camera disclosed in Korean Unexamined Patent Application KR 10 2003 0 017 772 A (March 4, 2003) is designed to provide video information from the rear of the vehicle at various vehicle positions. However, an optical axis is off-center due to the change in the position of the rearview camera, so video information cannot be provided at a precise point.

[0007] In a rear view camera disclosed in an unexamined Korean patent application KR 10 2013 0 005 159 A (January 15, 2013), when the position of the rear view camera is changed, the recording position is changed by an actuator so that an optical axis of the rear view camera is adjusted while making maximum use of the video information recorded by the rear view camera, thereby solving the problem of the above-described prior art.

[0008] However, according to the state of the art, another problem arises that an additional actuator must be provided to change the recording position of the rear view camera, so that the number of components as well as the man-hours required for assembly increase, which may lead to an increase in product costs.

[0009] The Fig. 1A and Fig. 1B are views schematically showing an installed state of a rear view camera for a vehicle according to the prior art, an image taken thereby, and a rearward direction of the vehicle.

[0010] As in the Fig. As shown in Figure 1A, when the rear view camera is mounted on a rear side of a vehicle body in accordance with design criteria, an actual reversing direction of the vehicle and a traveling direction displayed in the reversing image are the same. In contrast, as shown in Fig. 1 B, if the rear view camera is mounted off-center by a predetermined angle deviating from the design criteria, the actual reversing direction of the vehicle and the direction of travel shown in the reversing image are different, so that a negligent accident may occur during parking of the vehicle.

[0011] US 2008 / 0 186 384 A1 describes a camera calibration device. The camera calibration device has a parameter deriver adapted to find parameters for projecting an image captured by a camera onto a predetermined surface. The parameter deriver finds the parameters based on a calibration image from the camera, and the calibration image contains a plurality of calibration patterns with previously known shapes arranged at various positions within the camera's recording range.

[0012] US 2008 / 0205706 A1 describes a device for monitoring the surroundings of a vehicle. The device has: a point-of-view converter that converts an image captured with a camera installed on a vehicle into a bird's-eye view image through point-of-view conversion; a position adjuster that performs position adjustment between two bird's-eye view images generated from two images captured at different times, the position adjuster performing the position adjustment based on the two bird's-eye view images; and a three-dimensional object detector that detects a three-dimensional object around the vehicle based on differences between the two bird's-eye view images that have undergone the position adjustment.

[0013] US 2012 / 0002057 A1 describes a method for obtaining a captured image having a preferred calibration index located in an exclusive region on a first plane and a non-preferential calibration index located in a common region. First, the coordinate position of the preferred calibration index is acquired, and then the coordinate position of the non-preferential calibration index is calculated based on the positional relationship with the acquired preferred calibration index. Based on the actual coordinate positions and the calculated coordinate positions representing the coordinates of at least four points, a homography between a captured image area of ​​the image area and the first plane is calculated, and camera calibration is performed using the homography. SUMMARY

[0014] The present invention was made in an effort to provide a video adjustment method of a rear view camera for a vehicle, which adjusts a video photographed by an angle error of a currently installed rear view camera based on the video information of a rear view camera without a separate actuator and outputs the video to prevent a negligent accident.

[0015] The above technical problem is solved by a method according to claim 1. Claims 2 to 7 relate to particularly advantageous implementations of the method according to claim 1.

[0016] An exemplary embodiment of the present invention provides a method for video adjustment of a rear view camera for a vehicle, comprising: a pattern detection step of detecting a predetermined pattern by an intrinsic parameter of a rear view camera and geometric information; a step of estimating the orientation of the rear view camera currently installed in the vehicle (hereinafter referred to simply as "installed camera") from a reference pattern coordinate system by the patterns predetermined in a pattern detection step; a homography obtaining step of obtaining a homography corresponding to an angular error of the installed camera, which is verified by the camera posture estimation step;and a video calibration step for calibrating the video recorded by the installed camera based on a posture of a camera in accordance with the design criteria using the homography obtained by the homography obtaining step (hereinafter simply referred to as “design criteria mounted camera”).

[0017] The camera pose estimation step may be a step that calculates a pose of the installed camera from the reference pattern coordinate system based on the pose of the camera mounted according to the design criteria (hereinafter referred to as "first calculation") and the pose of the mounted camera using a pattern detected by a camera mounted in the process line (hereinafter referred to as "a process line-mounted camera") (hereinafter referred to as "second calculation").

[0018] The homography may be video information obtained by rotating the image by the position of the camera (hereinafter referred to as a "second position") calculated by the first calculation by the position of the camera calculated by the second calculation (hereinafter referred to as a "first position").

[0019] The first position and the second position may include a camera position obtained by the reference pattern coordinate system and an angle (roll, pitch and yaw angles) (X, Y and Z coordinates).

[0020] If the error of the camera mounted in the process line is equal to or greater than the set angle compared to the camera mounted according to design criteria without performing the camera pose estimation step, the method may further comprise a warning step for issuing a warning to a user.

[0021] The setting angle can be adjusted so that it does not exceed 5 degrees in any direction with respect to the reference pattern coordinate system.

[0022] The homography can be obtained by an operation that calculates a matrix R that rotates the camera to the angular direction of a normally installed camera using a difference between the angular information of the installed camera and the angular information of the camera installed in the process line.

[0023] An embodiment of a video adjustment method of a rear view camera for a vehicle according to the present invention does not need to include a separate actuator that adjusts a recording angle of the rear view camera, so that the number of components is reduced, assembly man-hours are reduced, and overall increased costs of a product can be prevented. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1A and Fig. 1B are views schematically showing an installed state of a rear view camera for a vehicle according to the prior art and a photographed image thereof, and illustrating a reversing direction of the vehicle. Fig. 2 is a control and flow chart showing an embodiment of a video adjustment method of a rear view camera for a vehicle according to the present invention. DETAILED DESCRIPTION

[0024] Hereinafter, an embodiment of a video adjustment method of a rear view camera for a vehicle according to the present invention will be described in detail with reference to the accompanying drawings.

[0025] Fig. 2 is a control and flow chart showing an embodiment of a video adjustment method of a rear view camera for a vehicle according to the present invention.

[0026] The embodiment of the video adjustment method of a rear view camera for a vehicle according to the present invention comprises, as shown in Fig.2, the following: a pattern detection step S10 for detecting a predetermined pattern by an intrinsic parameter of a rear-view camera and by geometric information, a camera pose estimation step S20 for estimating a pose of the rear-view camera currently installed in the vehicle (hereinafter simply referred to as "installed camera") from a reference pattern coordinate system by the predetermined pattern detected by the reference pattern detection step S10, a homography obtaining step S30 for obtaining homography corresponding to an angular error of the installed camera, which is checked by the camera pose estimation step S20, and a video calibration step S40 for calibrating the video recorded by the installed camera based on a pose of a camera (hereinafter simply referred to as "design-installed camera").which is installed in accordance with the design criteria using the homography obtained by the homography obtaining step S30.,

[0027] In general, to calibrate an image of the camera, one needs to obtain an intrinsic parameter and an extrinsic parameter, and the pattern detecting step S10, which detects the predetermined pattern, is a step for obtaining the extrinsic parameter.

[0028] The extrinsic parameter refers to the information that indicates how much the camera is spatially separated from an original point of a current world and how much the camera rotates, but the extrinsic factor is not determined at the time of camera manufacture.

[0029] In contrast, the intrinsic factor refers to a structural error of the camera caused by lens distortion or internal information of the camera caused by a focus distance caused by zooming in or out.

[0030] Here, the pattern detecting step S10 may be a step of detecting the predetermined pattern based on an intrinsic parameter item and geometry information of the camera having the same data without distinguishing the type of the currently installed camera and the camera installed according to design criteria.

[0031] More specifically, the camera can display the image using a predetermined pattern shape in accordance with the data and establish a reference pattern coordinate system through the pattern. The reference pattern coordinate system established as described above becomes a reference for estimating the positions of the installed camera and the camera installed according to the design criteria.

[0032] The camera pose estimation step may be a step that calculates a pose of the installed camera from a reference pattern coordinate system based on the pose of the camera mounted according to the design criteria (hereinafter referred to as "first calculation") and calculates the pose of the mounted camera using a pattern detected by a camera (hereinafter referred to as "camera mounted in the process line") (hereinafter referred to as "second calculation").

[0033] Meanwhile, the homography may be image information obtained by rotating the image by the posture (hereinafter referred to as "second posture") of the camera calculated by the first calculation from the posture of the camera (hereinafter referred to as "first posture") calculated by the second calculation.

[0034] Homography becomes a subject for calibrating the rear video that is mistakenly photographed due to a physically off-center angle of the installed camera. Specifically, if the Z-axis value of a real-world three-dimensional coordinate is assumed to be zero, a two-dimensional plane with an XY coordinate is provided. The two-dimensional plane can be a two-dimensional video recorded by the camera. The feature points of the two-dimensional video can be changed as if the feature points were viewed from a different angle, which is called homography.

[0035] Meanwhile, the first position and the second position may include a position (X, Y, and Z coordinates) of the camera obtained from the reference pattern coordinate system and an angle (roll, pitch, and yaw). That is, the first position and the second position refer to positions including an omnidirectional angle for photographing the rear video by the installed camera.

[0036] If the camera mounted in the process line has an error equal to or greater than the setting angle compared to the camera mounted according to design criteria, as a result of calculating the first pose and the second pose, the camera pose estimation step S20 cannot be performed. If the error of the installed camera is equal to or greater than the setting angle compared to the camera installed according to design criteria, the image quality of the rear video calibrated using homography will be reduced, or the video may be processed as a distorted video, which could give the driver a sense of deviation. The reduced image quality or the sense of deviation could cause an accident caused by the driver's negligence.In this case, instead of performing the video calibration step S40, a predetermined warning unit is used to issue a warning to the user (a warning step S50), the user then observes the surroundings with the naked eyes to try to reverse park the vehicle without depending on the video displayed from the installed camera.

[0037] When the driver receives a predetermined warning through warning step S50, the driver immediately attempts to reattach the camera and ignores the displayed video to attempt to reverse park the vehicle safely.

[0038] The setting angle can be adjusted so that it does not exceed 5 degrees in any direction with respect to the reference pattern coordinate system.

[0039] The homography can be obtained by a process that calculates a matrix R that rotates the camera to the normally installed angular direction using a difference between angular information of the installed camera and angular information of the camera installed in the process line.

[0040] More specifically, the following equation is an equation that calibrates a video using homography. P'=KRK−1P

[0041] Here, P denotes a video coordinate of the camera mounted in the process line, P' denotes a video coordinate in one direction of the camera mounted according to design criteria, and K denotes the intrinsic parameter of the camera. KRK - refers to the homography and is a matrix R as described above and a 3 x 3 matrix.

[0042] In particular, according to the present invention, when there is only a stationary rear camera, the coordinate on the third dimension cannot be calculated, but as described above, if the intrinsic parameter is obtained exactly in accordance with the data of the camera, the homography can be obtained by the equation.

[0043] More specifically, although the equation does not represent exact three-dimensional information, there may be a method that assumes the field of view seen by the rear camera is a plane (called the ground plane) and estimates the ground plane coordinates of a target detected from the image. This can be realized if there is a projective transformation between the image plane and the ground plane. Such a homography is called a plane-to-plane homography.

[0044] A commonly used method for obtaining homography is a direct linear transformation (DLT) algorithm. When the DLT method is used, given four or more corresponding pairs of points between the two planes, the homography is obtained.

[0045] According to one embodiment of the video adjustment method of a rearview camera for a vehicle according to the present invention, it is possible to precisely obtain the intrinsic parameter according to the camera data. Furthermore, since it is possible to accurately know the reference pattern coordinate system, the DLT method is very efficient, and the video can be easily calibrated using homography through the simple equation mentioned above.

[0046] According to the video adjustment method of a rearview camera for a vehicle according to the present invention configured as described above, when the installed camera is mounted off-center so that the adjustment angle is not exceeded to photograph an incorrect reversing video, the incorrect reversing video is calibrated and displayed to the driver. Therefore, the driver can reverse park the vehicle while ensuring reliable reversing video. Furthermore, there is no need to provide a separate actuator that physically rotates the installed camera to calibrate the video, which can lead to further convenience for the user.

[0047] Hereinafter, the embodiment of the video adjustment method of a rear view camera for a vehicle according to the present invention has been described in detail with reference to the accompanying drawings. However, it should be understood that the embodiment of the present invention is not limited to the above-described embodiment, but various modifications and implementations are possible within the scope of the invention. Therefore, a true scope of the invention can be determined by the following claims.

Claims

[1] Video adjustment method of a rear view camera for a vehicle, comprising: a pattern detecting step (S10) for detecting a predetermined pattern by an intrinsic parameter of a rear view camera and by geometry information; a camera posture estimation step (S20) for estimating a posture of the rear view camera currently provided in the vehicle (hereinafter simply referred to as “an installed camera”) from a reference pattern coordinate system by the predetermined pattern detected by the pattern detecting step (S10); a homography obtaining step (S30) for obtaining a homography corresponding to an angular error of the installed camera, which is checked by the camera pose estimation step (S20); and a video calibration step (S40) for calibrating the video photographed by the installed camera based on a pose of a design-mounted camera (hereinafter simply referred to as “design-mounted camera”) using the homography obtained by the homography obtaining step (S30). [2] The method according to claim 1, wherein in the camera pose estimation step (S20), a pose of the installed camera is calculated from the reference pattern coordinate system based on a pose of the camera mounted according to design criteria (hereinafter referred to as a “first calculation”), and the posture of the installed camera is calculated using the pattern (hereinafter referred to as a “second calculation”) detected from the camera mounted in the process line (hereinafter simply referred to as a “process line-mounted camera”). [3] A method according to any one of the preceding claims, wherein the homography is image information obtained by rotating the image by the posture of the camera (hereinafter referred to as a "second posture") calculated by the first calculation from the posture (hereinafter referred to as a "first posture") of the camera calculated by the second calculation. [4] A method according to any one of the preceding claims, wherein the first position and the second position comprise a position (X, Y and Z coordinates) of the camera obtained from the reference pattern coordinate system and an angle (roll, pitch and yaw angles). [5] A method according to any one of the preceding claims, further comprising: if the error of the camera mounted in the process line is equal to or greater than the setting angle compared to the camera mounted according to design criteria, without performing the camera pose estimation step, a warning step of issuing a warning to a user. [6] A method according to any one of the preceding claims, wherein the setting angle is adjusted so that it does not exceed 5 degrees in any direction with respect to the reference pattern coordinate system. [7] A method according to any one of the preceding claims, wherein the homography is obtained by a process that calculates a matrix R that rotates the camera to the normally installed camera angular direction using a difference between the angular information of the installed camera and the angular information of the camera mounted in the process line.

Images