Parking space-based automobile panoramic camera external parameter optimization method

Abstract

The invention provides an automobile panoramic camera external parameter optimization method based on a parking space, and relates to the technical field of automobile electronics and intelligent driving. The method comprises the steps that a vehicle is parked in the middle of a standard parking space, a vehicle-mounted display screen enters a lane line correction interface for correction, image data are collected through cameras around the vehicle, an OpenCV visual operator is used for processing the image data, and interference lines are recognized and eliminated. And two longest line segments of the parking space calibration line are calculated and prolonged, if the extension lines are not parallel, a rotation matrix is determined by calculating the vanishing point angle, and the external parameters of the camera are iteratively adjusted. The method improves camera correction precision and panoramic view angle accuracy, adapts to various environments, optimizes vehicle parking position detection, adjusts camera parameters in real time, remarkably improves panoramic image quality and driving safety, and has wide application prospects.

Description

Technical Field

[0001] The present invention relates to the field of automotive electronics and intelligent driving technology, and in particular to a method for optimizing and correcting external parameters of a panoramic camera of an automobile. Background Art

[0002] With the continuous development of automotive electronics and intelligent driving technology, panoramic camera systems are increasingly used in modern cars. The panoramic camera system can collect images around the vehicle through four cameras and stitch these images into a 360-degree panoramic view, providing the driver with a "God's perspective" and significantly improving driving safety and convenience.

[0003] The image stitching accuracy of a 360-degree panoramic imaging system mainly depends on the external parameters (extrinsic parameters) of the camera, which determine the relative position and angle between the images of each camera. During the automobile manufacturing process, each car will undergo an external parameter calibration of the camera before leaving the factory to ensure the accuracy of the stitching image. However, with the long-term use of the car, the installation position of the camera may change slightly due to factors such as road bumps and vehicle vibrations. These position changes will cause image stitching errors, resulting in inaccurate stitching of the panoramic image, as shown in Figure (1), which will affect the driver's visual experience and driving safety.

[0004] Based on the above problems, it is urgent to design a highly applicable panoramic camera extrinsic parameter correction method so that when the camera extrinsic parameters are offset, correction can be performed in time to ensure the stitching accuracy and image quality of the panoramic image, and ensure the safety and convenience of driving. This method should not only have efficient correction capabilities, but also be able to adapt to a variety of environmental conditions to ensure that high-quality panoramic images can be provided in various complex usage scenarios. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in order to solve the problem that the camera position may be offset, the present invention provides an external parameter optimization method of a car panoramic camera based on a parking space to solve the above problem.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a method for optimizing external parameters of a panoramic camera of an automobile based on a parking space, comprising the following steps:

[0007] S1. Park the vehicle in the center of a standard parking space;

[0008] S2. After receiving the control command, the on-board display screen enters the lane line correction interface and performs external parameter correction;

[0009] S3, collecting four-channel image data through cameras around the vehicle;

[0010] S4, judging whether the wheels and the vehicle body are both located in the standard parking space and the camera is not blocked according to the four-channel image data, and if so, using OpenCV visual operators to process the four-channel image data, identifying the outlines of the parking space demarcation lines in the four-channel image data, and excluding the interference lines of the non-parking space demarcation lines;

[0011] S5. Calculate the two longest line segments of the outline of the parking space demarcation line, record the two longest line segments as the rising edge and the falling edge respectively, extend the two ends of the rising edge and the falling edge, and determine whether the extended lines are parallel; if the extended lines are not parallel, determine the rotation matrix by calculating the angle of the vanishing point, and adjust the external parameters of the camera through an iterative method until the extended lines are parallel.

[0012] Preferably, the parking space marking line is white in color and has a line width greater than 10 cm.

[0013] Preferably, in step S4, the contours of the parking space marking lines in the four-channel image data are calculated using a Sobel operator, and interference lines of non-parking space marking lines are excluded.

[0014] Preferably, in step S5, the rising edge and the falling edge are respectively fitted as straight lines, and the intersection of the two straight lines, i.e., the vanishing point, is calculated; if the vanishing point is at infinity, it means that the rising edge and the falling edge are parallel, and no external parameters need to be corrected; if the vanishing point is not at infinity, it means that the parking space calibration line deviates at this time, and the current camera external parameters need to be corrected.

[0015] Preferably, in step S5, v p The position of the camera is used to calculate the camera's movement angle As shown in formula (1) and formula (2):

[0016]

[0017]

[0018] in, v ∞ =[0 1 0] T ;

[0019] v is the vanishing point, ‖v‖ is the modulus of v;

[0020] According to v p and Calculate the rotation matrix R, as shown in formula (3):

[0021]

[0022] The change of the rotation matrix R will cause the position of the points on the parking space calibration line to change. The value of is offset with a variable step length. When the condition of formula (4) is satisfied, it can be considered that the parking space marking line is parallel. The external parameter corresponding to the current rotation matrix is ​​the corrected external parameter; where I is the unit matrix;

[0023] argmin(|D G ―D R |+|D G ―D L |+|p 4 ―p 1 |+|p 3 ―p 2 |) Formula (4)

[0024] Among them, D G Represents the distance between the rising edge and the falling edge when they are parallel, D R Represents the distance between the shorter sides when the rising and falling edges are not parallel, D L Represents the distance between the longer sides when the rising and falling edges are not parallel, p 1 Corresponding to the left end point of the rising edge, p 4 Corresponding to the right end point of the rising edge, p 2 Corresponding to the left end point of the falling edge, p 3 The right endpoint corresponding to the falling edge;

[0025] The relationship between the image coordinate system and the vehicle coordinate system is shown in formula (5):

[0026] x i =K[R|T]x v =Hx v Formula (5)

[0027] Among them, x v Represents a point in the world coordinate system, x i Represents a point in the image coordinate system; K represents the intrinsic parameter of the camera, R represents the rotation matrix, T represents the translation matrix, [R|T] represents the external parameter of the camera, and H is the homography matrix, which is equal to the intrinsic parameter of the camera multiplied by the external parameter.

[0028] The beneficial effect of the present invention is that the present invention uses the external parameter optimization method of the automobile panoramic camera based on the parking space demarcation line:

[0029] Improve camera calibration accuracy: By accurately identifying and correcting the parking space marking lines, it is possible to ensure that the position offset of the panoramic camera system caused by vibration or other reasons during long-term use can be corrected in a timely manner, ensuring the stitching accuracy and image quality of the panoramic image.

[0030] Enhance the accuracy of the panoramic view: By iteratively adjusting the camera's external parameters, the extension lines of the parking space marking lines are made parallel, ensuring accurate camera view correction, thereby improving the accuracy of the 360-degree view provided by the panoramic camera system and enhancing driving safety and convenience.

[0031] Adapt to various environmental conditions: Using OpenCV visual operators to process image data, it can accurately identify parking space marking lines under complex environmental conditions and eliminate interference lines of non-parking space marking lines, thereby improving the adaptability and stability of the system.

[0032] Optimize the detection of vehicle parking positions: By calculating the two longest line segments of the contour of the parking space demarcation line and using their rising and falling edges for extension and judgment, it is possible to accurately determine whether the vehicle is parked in a standard parking space, thereby providing reliable basic data for external parameter optimization.

[0033] Real-time adjustment of camera parameters: By calculating the angle and rotation matrix of the vanishing point and adjusting the camera's external parameters using an iterative method, real-time adjustment can be achieved to ensure that the panoramic camera system can provide high-quality panoramic images in various usage scenarios.

[0034] In summary, the present invention can significantly improve the correction accuracy and image quality of the automobile panoramic camera system through the external parameter optimization method based on the parking space calibration line, enhance driving safety and convenience, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0036] Figure 1 It is a schematic diagram of surround view stitching of the optimal embodiment of the external parameter optimization method of the automobile panoramic camera based on the parking space of the present invention.

[0037] Figure 2 It is a schematic diagram of external parameter correction of the optimal embodiment of the external parameter optimization method of the automobile panoramic camera based on parking space of the present invention. DETAILED DESCRIPTION

[0038] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0039] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0040] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0041] like Figure 1 and 2 As shown, the present invention provides an embodiment of a method for optimizing external parameters of a panoramic camera of an automobile based on a parking space, comprising the following steps:

[0042] S1. Park the vehicle in the center of a standard parking space. The parking space marking line is white and the line width is greater than 10 cm to ensure visibility and recognition accuracy of the marking line.

[0043] S2. After receiving the control command, the on-board display screen enters the lane line correction interface and performs external parameter correction.

[0044] S3. Collect four-channel image data through cameras around the vehicle.

[0045] S4. Determine whether the wheels and the vehicle body are both located in the standard parking space and the camera is not blocked based on the four-channel image data. If yes, use OpenCV operators to process the four-channel image data, identify the contours of the parking space demarcation lines in the four-channel image data, and exclude the interference lines of the non-parking space demarcation lines. The processing methods of OpenCV operators include linear filtering, morphological operations, contour search and other methods. The following are the specific steps and implementation process:

[0046] Image preprocessing

[0047] Grayscale conversion: Convert the collected color image into a grayscale image to reduce computational complexity and improve processing efficiency.

[0048] Linear filtering

[0049] Gaussian Blur: Use a Gaussian blur filter to smooth the grayscale image and reduce the interference of noise and details on edge detection. Gaussian blur is achieved through convolution operation to smooth the image.

[0050] Edge Detection

[0051] Sobel operator: Use the Sobel operator to calculate the gradient of the image to identify the edges in the image. The Sobel operator calculates the gradient of the image in the horizontal and vertical directions through convolution operations, and then combines the gradients in these two directions to obtain the edge of the image.

[0052] Morphological operations

[0053] Binarization: Binarize the gradient image and convert the grayscale value in the image into black and white to make the edge more obvious. By setting a threshold, pixels greater than the threshold are set to white, and pixels below the threshold are set to black.

[0054] Dilation and erosion: Use morphological operations (dilation and erosion) to remove small noise and connect broken lines. Dilation expands the white area to fill small black holes; erosion shrinks the white area to remove small white noise. Usually, the combination of dilation and erosion is called closing operation, which is used to eliminate small black noise and connect broken white lines.

[0055] Contour Finder

[0056] Find contours: Use OpenCV's contour finding algorithm to identify all contours in the image. The algorithm traverses the binary image, finds all continuous white pixels, and connects them into contours.

[0057] Contour Filtering

[0058] Filter contours: Filter out contours that are not parking lines based on their shape and size. For example, you can determine whether a contour is a parking line based on its area, length, aspect ratio, and other features. Usually, parking lines are long and thin, while other interfering lines may be short or wide.

[0059] S5. Calculate the two longest line segments of the contour of the parking space calibration line, record the two longest line segments as the rising edge and the falling edge respectively, fit the rising edge and the falling edge into straight lines respectively, and calculate the intersection of the two straight lines, that is, the vanishing point; if the vanishing point is at infinity, it means that the rising edge and the falling edge are parallel, and no external parameters need to be corrected; if the vanishing point is not at infinity, it means that the parking space calibration line deviates at this time, and the current camera external parameters need to be corrected.

[0060] If the extension lines are not parallel, use v p The position of the camera is used to calculate the camera's movement angle As shown in formula (1) and formula (2):

[0061]

[0062]

[0063] in, v ∞ =[0 1 0] T ;

[0064] v is the vanishing point, ‖v‖ is the modulus of v;

[0065] According to v p and Calculate the rotation matrix R, as shown in formula (3):

[0066]

[0067] The change of the rotation matrix R will cause the position of the points on the parking space demarcation line in Figure (2) to change. The value of is offset with a variable step length. When the condition of formula (4) is satisfied, it can be considered that the parking space marking line is parallel. The external parameter corresponding to the current rotation matrix is ​​the corrected external parameter; where I is the unit matrix;

[0068] argmin(|D G ―D R |+|D G ―D L |+|p 4 ―p 1 |+|p 3 ―p 2 |) Formula (4)

[0069] Among them, D G Represents the distance between the rising edge and the falling edge when they are parallel, D R Represents the distance between the shorter sides when the rising and falling edges are not parallel, D L Represents the distance between the longer sides when the rising and falling edges are not parallel, p 1Corresponding to the left end point of the rising edge, p 4 Corresponding to the right end point of the rising edge, p 2 Corresponding to the left end point of the falling edge, p 3 The right endpoint corresponding to the falling edge;

[0070] The relationship between the image coordinate system and the vehicle coordinate system is shown in formula (5):

[0071] x i =K[R|T]x v =Hx v Formula (5)

[0072] Among them, x v Represents a point in the world coordinate system, x i Represents a point in the image coordinate system; K represents the intrinsic parameters of the camera, R represents the rotation matrix, T represents the translation matrix, [R|T] represents the camera extrinsic parameters, and H is the homography matrix, which is equal to the camera intrinsic parameters multiplied by the extrinsic parameters. In the present invention, K and T are both unchanged, so updating R is equivalent to updating the camera extrinsic parameters.

[0073] The present invention uses an external parameter optimization method for a car panoramic camera based on parking space calibration lines, and adopts the steps of vehicle parking, image acquisition and preprocessing, edge detection and morphological operation, contour search and filtering, and external parameter calculation and correction to accurately identify and correct parking space calibration lines, ensure that the position offset of the panoramic camera system caused by vibration or other reasons during long-term use is corrected in time, and improve the stitching accuracy and image quality of the panoramic image. Specifically, by parking the vehicle in the center of the standard parking space, using OpenCV visual operators for image processing, identifying the contour of the parking space calibration line, and iteratively adjusting the camera external parameters by calculating the vanishing point and rotation matrix, ensuring that the extension line of the parking space calibration line is parallel, thereby achieving accurate correction of the camera external parameters. This method can improve the camera correction accuracy and the accuracy of the panoramic viewing angle, adapt to a variety of environmental conditions, optimize the vehicle parking position detection, and realize real-time adjustment of camera parameters, thereby significantly enhancing driving safety and convenience, and has broad application prospects.

[0074] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0075] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for optimizing external parameters of a panoramic camera of a car based on parking spaces, characterized in that: The following steps are involved: S1. Park the vehicle in the center of a standard parking space; S2. After receiving the control command, the on-board display screen enters the lane line correction interface and performs external parameter correction; S3, collecting four-channel image data through cameras around the vehicle; S4, judging whether the wheels and the vehicle body are both located in the standard parking space and the camera is not blocked according to the four-channel image data, and if so, using OpenCV visual operators to process the four-channel image data, identifying the outlines of the parking space demarcation lines in the four-channel image data, and excluding the interference lines of the non-parking space demarcation lines; S5. Calculate the two longest line segments of the outline of the parking space demarcation line, record the two longest line segments as the rising edge and the falling edge respectively, extend the two ends of the rising edge and the falling edge, and determine whether the extended lines are parallel; if the extended lines are not parallel, determine the rotation matrix by calculating the angle of the vanishing point, and adjust the external parameters of the camera through an iterative method until the extended lines are parallel.

2. The method according to claim 1, characterized in that The parking space marking line is white in color and has a line width greater than 10 cm.

3. The method according to claim 1, characterized in that In step S4, the Sobel operator is used to calculate the contours of the parking space marking lines in the four-channel image data, and the interfering lines of the non-parking space marking lines are excluded.

4. The method according to claim 3, characterized in that In step S5, the rising edge and the falling edge are fitted as straight lines respectively, and the intersection of the two straight lines, i.e., the vanishing point, is calculated; if the vanishing point is at infinity, it means that the rising edge and the falling edge are parallel, and no external parameters need to be corrected; if the vanishing point is not at infinity, it means that the parking space calibration line deviates at this time, and the current camera external parameters need to be corrected.

5. The method according to claim 4, characterized in that In step S5, using v p The position of the camera is used to calculate the camera's movement angle As shown in formula (1) and formula (2): in, v ∞ =[0 1 0] T ; v is the vanishing point, ‖v‖ is the modulus of v; According to v p and Calculate the rotation matrix R, as shown in formula (3): The change of the rotation matrix R will cause the position of the points on the parking space calibration line to change. The value of is offset with a variable step length. When the condition of formula (4) is satisfied, it can be considered that the parking space marking line is parallel. The external parameter corresponding to the current rotation matrix is ​​the corrected external parameter; where I is the unit matrix; argmin(|D G ―D R |+|D G ―D L |+|p4 - p1| + |p3 - p2|) Equation (4) Among them, D G Represents the distance between the rising edge and the falling edge when they are parallel, D R Represents the distance between the shorter sides when the rising and falling edges are not parallel, D L Represents the distance between the longer sides when the rising edge and the falling edge are not parallel, p1 corresponds to the left end point of the rising edge, p4 corresponds to the right end point of the rising edge, p2 corresponds to the left end point of the falling edge, and p3 corresponds to the right end point of the falling edge; The relationship between the image coordinate system and the vehicle coordinate system is shown in formula (5): x i =K[R|T]x v =Hx v shì(5) Among them, x v Represents a point in the world coordinate system, x i Represents a point in the image coordinate system; K represents the intrinsic parameter of the camera, R represents the rotation matrix, T represents the translation matrix, [R|T] represents the external parameter of the camera, and H is the homography matrix, which is equal to the intrinsic parameter of the camera multiplied by the external parameter.

Images