Intelligent identification system using a wide-angle camera

The intelligent identification system corrects distorted wide-angle images using a correction formula to enhance the on-board computer's ability to accurately calculate distances, thereby improving driving safety.

DE202024103391U1Active Publication Date: 2024-11-14CHIMEI MOTOR ELECTRONICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE202024103391
Authority / Receiving Office
DE · DE
Patent Type
Utility models
Current Assignee / Owner
Filing Date
2024-06-21
Publication Date
2024-11-14
Estimated Expiration
2034-06-30

AI Technical Summary

Technical Problem

Wide-angle cameras capture distorted images, making it difficult for on-board computers to accurately judge road conditions and calculate distances between vehicles, which affects driving safety.

Method used

An intelligent identification system and method using a wide-angle camera that corrects distorted images through image rotation, lens tilt angle, inverse perspective projection, and lens yaw angle corrections, employing a correction formula to convert wide-angle image coordinates to world coordinates.

Benefits of technology

The system accurately calculates distances between a vehicle and surrounding objects, enhancing driving safety by improving the on-board computer's judgment of road conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Intelligent identification system (900) using a wide-angle camera (410), comprising: a memory (910) configured to store a large number of instructions; the wide-angle camera (410) mounted on one side of a vehicle (420), wherein the wide-angle camera (410) is configured to capture an image of an image correction pattern (300) in order to obtain a correction pattern image; and a processor (920) which is electrically connected to the memory (910) and configured to load the instructions and is in signal communication with the wide-angle camera (410) and is configured to receive the correction pattern image, wherein the processor (920) is further configured to perform a correction step on the wide-angle camera (410).
Need to check novelty before this filing date? Find Prior Art

Description

STATE OF THE ARTField of the utility model

[0001] The present utility model relates to an intelligent identification system and an intelligent identification method using a wide-angle camera. Description of the state of the art

[0002] Driving safety is a very important issue for drivers and passengers. Currently, the industry provides many technologies to improve driving safety. For example, multiple wide-angle cameras are installed on a vehicle to capture images from different directions. As another example, an onboard computer is provided to receive the images captured by the wide-angle cameras and determine whether the vehicle's current driving situation is safe or unsafe.

[0003] However, the images captured by the wide-angle camera are distorted images, meaning that the images captured by the wide-angle camera are distorted, making it difficult for the onboard computer to accurately assess the current road conditions. For example, it is difficult for the onboard computer to accurately calculate the distance between the driver's vehicle and surrounding objects (vehicles), which impairs the onboard computer's assessment of the current road conditions.

[0004] Therefore, an intelligent identification system and an intelligent identification method using a wide-angle camera are needed to correct the distorted images captured by the wide-angle camera to facilitate image identification. SUMMARY

[0005] Embodiments of the present utility model propose an intelligent identification system and an intelligent identification method using a wide-angle camera that can correct distorted images captured by the wide-angle camera.

[0006] According to some examples of the present utility model, an intelligent identification method using a wide-angle camera includes: providing an image correction pattern; capturing an image of the image correction pattern using a wide-angle camera to obtain a correction pattern image; and performing a correction step of performing image rotation correction, lens tilt angle correction, inverse perspective projection, and yaw angle correction on the wide-angle camera using a correction formula to straighten grid lines in the correction pattern image and convert wide-angle image coordinates in the correction pattern image into world coordinates. The correction formula used in the correction step is: [XwYwZw1]=ABCD[XiYi11] where X w , Y w , Z w represent three-dimensional coordinates of a world coordinate axis; X i , Y irepresent two-dimensional coordinates of a wide-angle image coordinate axis, and a matrix A is a lens rotation correction matrix, a matrix B is a lens tilt correction and inverse perspective projection matrix, a matrix C is a lens yaw angle correction, and a matrix D is a coordinate transformation matrix, which are expressed as follows: the matrix A=[cos(θroll)−sin(θroll)00sin(θroll)cos(θroll)0000100001] the matrix B=[10000cos(θpitch)sin(θpitch)−H⋅cos(θpitch)0−sin(θpitch)cos(θpitch)H⋅sin(θpitch)0001]−1 the matrix C=[cos(θyaw)sin(θyaw)00−sin(θyaw)cos(θyaw)0000100001] the matrix D=[1ZcKx01Zcw200−1ZcKy1Zch2000100001]−1 where θ yaw is a lens yaw angle; θ Pitch is a lens tilt angle; θ Roll is a lens roll angle; H is a height of the wide-angle camera; Z c is a camera depth; K xis a proportionality constant between an image pixel coordinate in an X-axis direction and the world coordinates; K y is a proportionality constant between an image pixel coordinate in a Y-axis direction and the world coordinates; w is an image width (e.g., in pixels), and h is an image height (e.g., in pixels).

[0007] According to one embodiment, the wide-angle camera is a fisheye lens camera.

[0008] According to one embodiment, when the wide-angle camera is used to capture the image correction pattern, the wide-angle camera is mounted on a vehicle, and the image correction pattern is placed on a floor adjacent to the vehicle.

[0009] According to one embodiment, the image correction pattern extends along the floor.

[0010] According to some embodiments of the present utility model, an intelligent identification system using a wide-angle camera includes a memory and a processor. The memory is configured to store a plurality of instructions.The processor is configured to load the instructions to perform the steps of: providing an image correction pattern; capturing an image of the image correction pattern using a wide-angle camera to obtain a correction pattern image; and performing a correction step to perform image rotation correction, lens tilt angle correction, inverse perspective projection, and lens yaw angle correction on the wide-angle camera using a correction formula to straighten a plurality of grid lines in the correction pattern image and to convert a plurality of wide-angle image coordinates in the correction pattern image to a plurality of world coordinates, wherein the correction formula used in the correction step is:. [XwYwZw1]=ABCD[XiYi11] where X w , Y w , Z w represent three-dimensional coordinates of a world coordinate axis; X i , Y irepresent two-dimensional coordinates of a wide-angle image coordinate axis, and a matrix A is a lens rotation correction matrix, a matrix B is a lens tilt correction and inverse perspective projection matrix, a matrix C is a lens yaw angle correction, and a matrix D is a coordinate transformation matrix, which are expressed as follows: the matrix A=[cos(θroll)−sin(θroll)00sin(θroll)cos(θroll)0000100001] the matrix B=[10000cos(θpitch)sin(θpitch)−H⋅cos(θpitch)0−sin(θpitch)cos(θpitch)H⋅sin(θpitch)0001]−1 the matrix C=[cos(θyaw)sin(θyaw)00−sin(θyaw)cos(θyaw)0000100001] the matrix D=[1ZcKx01Zcw200−1ZcKy1Zch2000100001]−1 where θ yaw is a lens yaw angle; θ Pitch is a lens tilt angle; θ Roll is a lens roll angle; H is a height of the wide-angle camera; Z c is a camera depth; K xis a proportionality constant between an image pixel coordinate in an X-axis direction and the world coordinates; K y is a proportionality constant between an image pixel coordinate in a Y-axis direction and the world coordinates; w is an image width (e.g., in pixels), and h is an image height (e.g., in pixels).

[0011] In some embodiments, the wide-angle camera is a fisheye lens camera.

[0012] In some embodiments, the image correction pattern comprises a black and white checkerboard pattern.

[0013] In some embodiments, when the wide-angle camera is used to capture the image correction pattern, the wide-angle camera is mounted on a vehicle, and the image correction pattern is placed on the ground adjacent to the vehicle.

[0014] In some embodiments, the image correction pattern extends along the floor. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a schematic diagram showing an image taken with a wide-angle camera. Fig. 2 is a schematic flowchart illustrating an intelligent identification method using a wide-angle camera according to an embodiment of the present utility model. Fig. 3 illustrates an image correction pattern according to an embodiment of the present invention. Fig. 4 illustrates a method of capturing an image correction pattern according to an embodiment of the present invention. Fig. 5 illustrates the straightening of a wide-angle image according to an embodiment of the present utility model. Fig. 6 illustrates the grid width restoration of a wide-angle image according to an embodiment of the present invention. Fig. 7 illustrates the grid length restoration of a wide-angle image according to an embodiment of the present invention. Fig. 8 illustrates the roll angle rotation of a wide-angle image according to an embodiment of the present utility model. Fig. 9 illustrates an intelligent identification system using a wide-angle camera according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] With reference to Fig. 1 is Fig. 1 is a schematic diagram showing an image 100 captured by a wide-angle camera. The wide-angle camera is mounted on a driver's vehicle to capture the road conditions around the driver's vehicle and transmits the captured image 100 to an onboard computer so that the onboard computer can determine whether the driver's vehicle is too close to other vehicles / objects. Specifically, the onboard computer detects objects 110 in the image 100 and determines whether the objects 110 are too close to the driver's vehicle to further decide whether to issue a warning. However, the image 100 captured by the wide-angle camera is distorted, making it difficult for the onboard computer to accurately calculate the distances between the driver's vehicle and the surrounding objects 110 and accurately issue a warning.

[0016] With reference to Fig. 2 is Fig. 2 is a schematic flowchart illustrating an intelligent identification method 200 using a wide-angle camera according to an embodiment of the present invention. The intelligent identification method 200 using a wide-angle camera is used to correct the image data of the image 100 captured by the wide-angle camera. For example, the coordinate data in the image 100 is corrected so that it can exactly correspond to the camera coordinates, and the on-board computer can thus accurately calculate the distances between the driver's vehicle and surrounding objects. In the intelligent identification method 200 using a wide-angle camera, a step 210 is first performed to provide an image correction pattern 300, as shown in Fig. 3. In the present embodiment, the image correction pattern 300 includes a black and white checkerboard pattern. Specifically, black grids 312 and white grids 314 are alternately arranged in a matrix to form the image correction pattern 300, but embodiments of the present invention are not limited thereto. In some embodiments, the image correction pattern 300 is formed on a correction plate, but embodiments of the present invention are not limited thereto.

[0017] Returning to Fig. 2, after step 210, a step 220 is performed to capture an image of the image correction pattern 300 on the correction plate using a wide-angle camera 410, as shown in Fig. 4. In the present embodiment, the wide-angle camera 410 is mounted on a side of a vehicle 420, such as a side mirror. The wide-angle camera 410 may be, for example, a fisheye lens, but the embodiments of the present utility model are not limited thereto. The correction plate is placed on the ground at a side of the vehicle 420 so that the image correction pattern 300 is aligned with the vehicle 420 and extends along the ground. In the present embodiment, a center line 310C of the image correction pattern 300 and a center line 420C of the vehicle 420 are parallel to each other, so that the wide-angle camera 410 can accurately capture the image correction pattern 300.

[0018] In some embodiments, the wide-angle camera 410 is arranged behind the vehicle 420, and the correction plate is placed on the ground behind the vehicle 420 such that the image correction pattern 300 is aligned with the vehicle 420 and extends along the ground to enable the wide-angle camera 410 to accurately capture the image correction pattern 300.

[0019] Returning to Fig. 2, after step 220, a step 230 is performed to correct the correction pattern image acquired in step 220. In the correction step, intrinsic parameters and extrinsic parameters of the wide-angle camera 410 are calculated using a correction formula to obtain a corresponding coordinate transformation matrix. Specifically, the correction step of step 230 uses the correction formula to perform image rotation correction, lens tilt angle correction, inverse perspective projection, and lens yaw angle correction on the wide-angle camera 410 to straighten several grid lines in the correction pattern image and convert wide-angle image coordinates in the correction pattern image into world coordinates. The world coordinates are based on the camera as the origin, so the distances between the driver's vehicle and surrounding objects can be calculated.The correction formula used in the present embodiment is as follows:. [XwYwZw1]=ABCD[XiYi11] where X w , Y w , Z w represent three-dimensional coordinates of a world coordinate axis; X i , Y i represent two-dimensional coordinates of a wide-angle image coordinate axis, and a matrix A is a lens rotation correction matrix, a matrix B is a lens tilt correction and inverse perspective projection matrix, a matrix C is a lens yaw angle correction, and a matrix D is a coordinate transformation matrix, which are expressed as follows: the matrix A=[cos(θroll)−sin(θroll)00sin(θroll)cos(θroll)0000100001] the matrix B=[10000cos(θpitch)sin(θpitch)−H⋅cos(θpitch)0−sin(θpitch)cos(θpitch)H⋅sin(θpitch)0001]−1 the matrix C=[cos(θyaw)sin(θyaw)00−sin(θyaw)cos(θyaw)0000100001] the matrix D=[1ZcKx01Zcw200−1ZcKy1Zch2000100001]−1

[0020] In matrices A to D, θ yaw a yaw angle of the lens; is θ Pitch a lens tilt angle; is θ Roll is a lens roll angle; H is a height of the wide-angle camera; Z is c a camera depth; is K x a proportionality constant between an image pixel coordinate in an X-axis direction and the world coordinates; K y a proportionality constant between an image pixel coordinate in a y-axis direction and the world coordinates; w is an image width (e.g., in pixels) and h is an image height (e.g., in pixels).

[0021] The design of Matrix A, Matrix B, Matrix C, and Matrix D (hereinafter collectively referred to as the conversion matrix) considers the intrinsic parameters (correction of camera distortion) and the extrinsic parameters (calculation of pitch / yaw / roll angle) to correct the distorted wide-angle image. For example, as shown in Fig. 5, the distorted wide-angle image can be corrected and straightened after the processing in step 230 of the present embodiment. In particular, the wide-angle image in Fig. 5 grid lines, and the grid lines represent the distortion of the wide-angle image. After the processing in step 230 of the present embodiment, the grid lines are straightened. As another example, as in Fig. 6, the optically distorted grid line width can be restored by the yaw angle correction in step 230 of the present embodiment. As yet another example, as shown in Fig. 7, the optically distorted grid line length can be restored by the tilt angle correction in step 230 of the present embodiment. As yet another example, as shown in Fig. As shown in Figure 8, step 230 of the present embodiment performs a roll angle rotation on the entire image.

[0022] By substituting the coordinate values ​​(world coordinates / wide-angle image coordinates) of the known positions on the correction plate into the above correction formula, the transformation matrix can be obtained, and then the relationship between X w , Y w , Z w (the three-dimensional coordinates of the world coordinate axis) and X i , Y i (the two-dimensional coordinates of the wide-angle image coordinate axis). When the user uses the wide-angle camera 410 to capture images, the on-board computer can use the above transformation matrix to correct the wide-angle image captured by the wide-angle camera 410 and calculate the distances between the driver's vehicle and surrounding objects.

[0023] For example, in some embodiments, the transformation matrix of the wide-angle camera 410 may be obtained after the wide-angle camera 410 has been corrected by the intelligent identification method 200 using a wide-angle camera. After the transformation matrix has been written to the on-board computer, the on-board computer may use the stored transformation matrix during the subsequent driving phase of the vehicle to correct the wide-angle image captured by the wide-angle camera 410 and calculate the distances between the driver's vehicle and surrounding objects.

[0024] With reference to Fig. 9 illustrates Fig.9 shows an intelligent identification system 900 using a wide-angle camera according to an embodiment of the present invention. The intelligent identification system 900 using a wide-angle camera includes a memory 910 and a processor 920. The memory 910 stores a plurality of instructions, and the processor 920 is electrically connected to the memory 910 to load the instructions for performing the above intelligent identification method 200 using a wide-angle camera. The processor 920 is in signal communication with the wide-angle camera 410 and can receive the image correction pattern 300 and the correction pattern image captured by the wide-angle camera 410.In some embodiments, the intelligent identification system 900 using a wide-angle camera may be implemented as a computing device that can receive the wide-angle image captured by the wide-angle camera 410 and apply the intelligent identification method 200 using a wide-angle camera to correct the wide-angle image captured by the wide-angle camera 410 and obtain the above transformation matrix.

[0025] According to the above description, the intelligent identification method 200 using a wide-angle camera and the intelligent identification system 900 using a wide-angle camera of the embodiment of the present utility model use a lens rotation correction matrix (matrix A), a lens tilt correction and inverse perspective projection matrix (matrix B), a lens yaw angle correction matrix (matrix C), and a coordinate transformation matrix (matrix D) to correct the image of the wide-angle camera to eliminate image distortion and calculate the distance between each object in the image and the coordinate origin (camera).The application of such a method can improve the accuracy of the on-board computer in determining the distances between the driver's vehicle and surrounding objects and enable the on-board computer to identify each object in the image, thereby greatly improving the driver's driving safety.

Claims

[1] An intelligent identification system (900) using a wide-angle camera (410), comprising: a memory (910) configured to store a plurality of instructions; the wide-angle camera (410) mounted on a side of a vehicle (420), wherein the wide-angle camera (410) is configured to capture an image of an image correction pattern (300) to obtain a correction pattern image; and a processor (920) electrically connected to the memory (910) and configured to load the instructions and in signal communication with the wide-angle camera (410) and configured to receive the correction pattern image, the processor (920) further configured to perform a correction step on the wide-angle camera (410). [2] An intelligent identification system using the wide-angle camera (410) according to claim 1, wherein the wide-angle camera (410) is a fisheye lens camera. [3] An intelligent identification system using the wide-angle camera (410) according to claim 1, wherein the image correction pattern (300) comprises a black and white checkerboard pattern. [4] An intelligent identification system using the wide-angle camera (410) according to claim 1, wherein, when the wide-angle camera (410) is used to capture the image correction pattern (300), the image correction pattern (300) is placed on a ground adjacent to the vehicle (420). [5] An intelligent identification system using the wide-angle camera (410) according to claim 4, wherein the image correction pattern (300) extends along the ground.