Intelligent recognition system and intelligent recognition method using wide-angle camera

Abstract

To provide an intelligent identification system and an intelligent identification method to which a wide-angle camera is applied.SOLUTION: A system, comprising: a memory; and a processor, configured to load instructions stored in the memory to perform the following intelligent recognition method. Providing an image correction pattern, photographing the image correction pattern by a wide-angle camera to obtain a correction pattern image, and performing screen rotation correction, lens pitch angle correction, inverse perspective projection and lens yaw angle correction on the wide-angle camera according to a lens rotation correction matrix, a lens pitch correction matrix, a lens yaw correction and inverse perspective projection matrix and a coordinate transformation matrix to straighten a plurality of grids in the correction pattern image and transform a plurality of wide-angle image coordinates in the correction pattern image into a plurality of world coordinates.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an intelligent identification system and an intelligent identification method using a wide-angle camera. [Background technology]

[0002] Safe driving is a very important issue for vehicle drivers and passengers. Currently, the industry provides many technologies to assist safe driving. For example, a vehicle is equipped with multiple wide-angle cameras to capture images of the vehicle in various directions. In addition, an on-board computer receives the images captured by the wide-angle cameras and uses them to determine whether the current vehicle is proceeding safely.

[0003] However, the image captured by the wide-angle camera is subject to image distortion, that is, the distorted image makes it difficult for the onboard computer to accurately judge the current road conditions. For example, the onboard computer has difficulty in accurately calculating the distance between the driver's vehicle and surrounding objects (vehicles), which affects the onboard computer's judgment of the current road conditions.

[0004] Therefore, in order to perform image recognition appropriately, an intelligent recognition system and an intelligent recognition method that employ a wide-angle camera that corrects image distortions captured by the wide-angle camera are desired. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present invention provide an intelligent identification system and method using a wide-angle camera that can correct image distortions captured by the wide-angle camera. [Means for solving the problem]

[0006] According to some embodiments of the present invention, an intelligent identification method using a wide-angle camera of the present invention includes: a step of providing an image correction pattern; a step of photographing the image correction pattern by a wide-angle camera to obtain a correction pattern image; and a step of performing screen rotation correction, lens pitch angle correction, inverse perspective projection, and lens yaw angle correction on the wide-angle camera based on a correction formula to straighten a plurality of grids in the correction pattern image and convert a plurality of wide-angle image coordinates in the correction pattern image into a plurality of world coordinates, wherein the correction formula used in the correction step is as follows:

number

number

[0007] In some embodiments, the wide-angle camera is a fisheye lens camera.

[0008] In some embodiments, when the image correction pattern is photographed by a wide-angle camera, the wide-angle camera is mounted on a vehicle and the image correction pattern is mounted on the ground beside the vehicle.

[0009] In some embodiments, the image correction pattern extends along the ground.

[0010] According to some embodiments of the present invention, an intelligent identification system using a wide-angle camera includes a memory for storing a plurality of commands; and a processor for loading in the commands to perform the following steps: providing an image correction pattern; photographing the image correction pattern by the wide-angle camera to obtain a correction pattern image; and performing screen rotation correction, lens pitch angle correction, inverse perspective projection, and lens yaw angle correction on the wide-angle camera based on a correction formula to straighten a plurality of grids in the correction pattern image and convert a plurality of wide-angle image coordinates in the correction pattern image into a plurality of world coordinates, wherein the correction formula used in the correction step is as follows:

number

number

[0011] In some embodiments, the wide-angle camera is a fisheye lens camera.

[0012] In some embodiments, the image correction pattern comprises a black and white checkered pattern.

[0013] In some embodiments, when the image correction pattern is photographed by a wide-angle camera, the wide-angle camera is mounted on a vehicle and the image correction pattern is mounted on the ground beside the vehicle.

[0014] In some embodiments, the image correction pattern extends along the ground. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 2 is a schematic diagram showing an image captured by a wide-angle camera. [Figure 2] 1 is a flow diagram illustrating an intelligent identification method using a wide-angle camera according to an embodiment of the present invention; [Figure 3] 1 illustrates an image correction pattern according to an embodiment of the present invention. [Figure 4] 1 shows a photographing form of an image correction pattern according to an embodiment of the present invention. [Figure 5] 1 shows a straightened wide-angle image according to an embodiment of the present invention. [Figure 6] 10 shows a grid width based restoration of a wide-angle image according to an embodiment of the present invention; [Figure 7] 10 illustrates a grid length based restoration of a wide-angle image according to an embodiment of the present invention. [Figure 8] 10 illustrates roll angle rotation of a wide-angle image according to an embodiment of the present invention. [Figure 9] 1 illustrates an intelligent identification system using a wide-angle camera according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] Please refer to FIG. 1, which is a schematic diagram showing an image 100 captured by a wide-angle camera. The wide-angle camera is installed on a driver's vehicle and captures images of the road conditions around the driver's vehicle. The captured image 100 is transmitted to an onboard computer, which then determines whether the driver's vehicle is approaching another vehicle / object. Specifically, the onboard computer detects an object 110 in the image 100, determines whether the object 110 is approaching the driver's vehicle, and then determines whether to issue a warning. However, because the image 100 captured by the wide-angle camera is distorted, it is difficult for the onboard computer to accurately calculate the distance between the driver's vehicle and the surrounding object 110 and issue a warning accurately.

[0017] Please refer to FIG. 2, which is a flow diagram illustrating an intelligent identification method 200 using a wide-angle camera according to an embodiment of the present invention. The intelligent identification method 200 using a wide-angle camera is used to correct image data of an image 100 captured by the wide-angle camera. For example, the coordinate data in the image 100 can be corrected to accurately correspond to the camera coordinates, and the onboard computer can accurately calculate the distance between the driver's vehicle and surrounding objects. The intelligent identification method 200 using a wide-angle camera first performs step 210 of providing an image correction pattern 300, as shown in FIG. 3. In this embodiment, the image correction pattern 300 includes a black and white checkered pattern. Specifically, black checks 312 and white checks 314 are arranged at intervals in a matrix form to form the image correction pattern 300, although embodiments of the present invention are not limited thereto. In some embodiments, the image correction pattern 300 is formed as a corrected version, although embodiments of the present invention are not limited thereto.

[0018] Returning to FIG. 2 , after step 210, step 220 is performed in which a wide-angle camera photographs the image correction pattern 300 on the correction version, as shown in FIG. 4 . In this embodiment, the wide-angle camera 410 is mounted on the side of the vehicle 420, for example, on the rearview mirror. The wide-angle camera 410 may be, for example, a fisheye lens, but the present invention is not limited thereto. The correction version is mounted on the ground on the side of the vehicle 420 so that the image correction pattern 300 and the vehicle 420 are aligned and extend along the ground. In this embodiment, in order for the wide-angle camera 410 to accurately photograph the image correction pattern 300, the center line 310C of the image correction pattern 300 and the center line 420C of the vehicle 420 are parallel to each other.

[0019] In some embodiments, the wide-angle camera 410 is mounted behind the vehicle 420, but the calibrated version is mounted on the ground behind the vehicle 420 so that the image correction pattern 300 and the vehicle 420 are aligned and extend along the ground so that the wide-angle camera 410 can accurately photograph the image correction pattern 300.

[0020] Returning to FIG. 2 , after step 220, step 230 is performed to correct the correction pattern image captured in step 220. In the correction step, the internal and external parameters of the wide-angle camera 410 are calculated based on a correction formula, and a corresponding coordinate transformation matrix is ​​obtained. Specifically, the correction step of step 230 performs screen rotation correction, lens pitch angle correction, reverse perspective projection, and lens yaw angle correction on the wide-angle camera 410 based on the correction formula, thereby straightening multiple grids in the correction pattern image and converting the wide-angle image coordinates in the correction pattern image into world coordinates. Since the world coordinates have the camera as their origin, the distance between the driver's vehicle and surrounding objects can be calculated. The correction formula used in the embodiment of the present application is as follows:

number

number

[0021] However, θ yaw is the lens yaw angle, and θ pitch is the lens pitch angle, and θ roll is the lens roll angle, H is the height of the wide-angle camera, and Z c is the camera depth and K x is the proportionality constant between the pixel coordinate in the X-axis direction of the image and the world coordinate, and K y is the proportionality constant between pixel coordinates and world coordinates in the Y-axis direction of the image, w is the width of the image (e.g., in pixels), and h is the height of the image (e.g., in pixels).

[0022] The design of the above matrices A, B, C, and D (hereinafter collectively referred to as transformation matrices) takes into account internal parameters (camera distortion correction) and external parameters (pitch / yaw / roll angle calculation) to correct a distorted wide-angle image. For example, as shown in FIG. 5, a distorted wide-angle image is corrected and straightened when processed in step 230 of the present embodiment. Specifically, the wide-angle image of FIG. 5 has grids that represent the distortion of the wide-angle image. When processed in step 230 of the present embodiment, these grids are straightened. Also, as shown in FIG. 6, for example, the width of the visually deformed grid is restored by the yaw angle correction in step 230 of the present embodiment. Also, as shown in FIG. 7, for example, the length of the visually deformed grid is restored by the pitch angle correction in step 230 of the present embodiment. Also, as shown in FIG. 8, for example, step 230 of the present embodiment performs roll angle rotation on the entire screen.

[0023] By inputting the coordinate values ​​(world coordinates / wide-angle image coordinates) of the known positions in the corrected version into the above correction formula, a transformation matrix is ​​obtained, and then X w , Y w , Z w (3D coordinates on the world coordinate axis) and X i , Y i The relational expression of the two-dimensional coordinates (wide-angle image coordinate axis) can be obtained. When a user takes a picture using the wide-angle camera 410, the in-vehicle computer can correct the wide-angle image taken by the wide-angle camera 410 using the above transformation matrix and calculate the distance between the driver's vehicle and surrounding objects.

[0024] For example, in some embodiments, after the wide-angle camera 410 is calibrated by the intelligent recognition method 200 using the wide-angle camera, a transformation matrix of the wide-angle camera 410 is obtained. The transformation matrix is ​​written to the on-board computer, and then, during a later vehicle driving stage, the on-board computer can calibrate the wide-angle image captured by the wide-angle camera 410 according to the transformation matrix stored in the storage device, and calculate the distance between the driver's vehicle and surrounding objects.

[0025] 9 illustrates an intelligent identification system 900 using a wide-angle camera according to an embodiment of the present invention. The intelligent identification system 900 using a wide-angle camera includes a memory 910 and a processor 920. The memory 910 stores a plurality of commands, and the processor 920 is electrically connected to the memory 910 so as to load the commands and perform the intelligent identification method 200 using a wide-angle camera. In some embodiments, the intelligent identification system 900 using a wide-angle camera is actually a computer device that receives wide-angle images captured by the wide-angle camera 410 and performs the intelligent identification method 200 using a wide-angle camera to correct the wide-angle images captured by the wide-angle camera 410 and obtain the transformation matrix.

[0026] In summary, the intelligent identification method 200 using a wide-angle camera and the intelligent identification system 900 using a wide-angle camera according to the embodiment of the present invention use a lens rotation correction matrix (Matrix A), a lens pitch correction and inverse perspective projection matrix (Matrix B), a lens yaw correction matrix (Matrix C), and a coordinate transformation matrix (Matrix D) to correct the image captured by the wide-angle camera, eliminate image distortion, and calculate the distance between each object in the image and the coordinate origin (camera). In this way, the accuracy of the onboard computer's determination of the distance between the driver's vehicle and surrounding objects can be improved, and the onboard computer can easily identify each object in the image, greatly improving the driver's driving safety.

[0027] Although the present invention has been disclosed above based on the embodiments, the above embodiments do not limit the present invention, and anyone skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined based on what is defined in the appended claims. [Explanation of symbols]

[0028] 100:Image 110: Object 200: Intelligent identification method using wide-angle camera 210~230: Process 300: Image correction pattern 312: Black check 314: White check 310C: Center line 410: Wide-angle camera 420: Vehicle 420C: Center line 900: Intelligent identification system using wide-angle camera 910:Memory 920: Processor

Claims

1. In an intelligent identification method using a wide-angle camera that is preferably used in an image correction system for a wide-angle camera, providing an image correction pattern; photographing the image correction pattern with the wide-angle camera to obtain a correction pattern image; a correction process for performing screen rotation correction, lens pitch angle correction, inverse perspective projection, and lens yaw angle correction on the wide-angle camera based on a correction formula to straighten multiple grids in the correction pattern image and convert multiple wide-angle image coordinates in the correction pattern image into multiple world coordinates; Equipped with The correction formula used in the correction step is as follows: [Equation 1] However, X ｗ , Y ｗ , Z ｗ represents the three-dimensional coordinate of the world coordinate axis, and X ｉ , Y ｉ represents the two-dimensional coordinates of the wide-angle image coordinate axis, matrix A is the lens rotation correction matrix, matrix B is the lens pitch correction and inverse perspective projection matrix, matrix C is the lens yaw correction matrix, and matrix D is the coordinate transformation matrix, which are expressed as follows: [Equation 2] However, θ ｙａｗ is the lens yaw angle, and θ ｐｉｔｃｈ is the lens pitch angle, and θ ｒｏｌｌ is the lens roll angle, H is the height of the wide-angle camera, and Z ｃ is the camera depth, and K ｘ is a proportionality constant between pixel coordinates in the X-axis direction of the image and world coordinates, and K ｙ is the proportional constant between the pixel coordinate in the Y-axis direction of the image and the world coordinate, w is the width of the image, and h is the height of the image.

2. The intelligent identification method using a wide-angle camera according to claim 1 , wherein the wide-angle camera is a fisheye lens camera.

3. The intelligent identification method using a wide-angle camera according to claim 1 , wherein the image correction pattern includes a black and white checkered pattern.

4. 2. The intelligent recognition method using a wide-angle camera according to claim 1, wherein when the image correction pattern is photographed by the wide-angle camera, the wide-angle camera is installed on a vehicle and the image correction pattern is installed on the ground near the vehicle.

5. The intelligent identification method using a wide-angle camera according to claim 4 , wherein the image correction pattern extends along the ground.

6. memory for storing multiple commands; a processor that loads in the commands and is electrically connected to the memory to perform the following steps: providing an image correction pattern; photographing the image correction pattern with a wide-angle camera to obtain a correction pattern image; and performing screen rotation correction, lens pitch angle correction, reverse perspective projection, and lens yaw angle correction on the wide-angle camera based on a correction formula to straighten multiple grids in the correction pattern image and convert multiple wide-angle image coordinates in the correction pattern image into multiple world coordinates; Equipped with The correction formula used in the correction step is as follows: [Equation 3] However, X ｗ , Y ｗ , Z ｗ represents the three-dimensional coordinate of the world coordinate axis, and X ｉ , Y ｉ represents the two-dimensional coordinates of the wide-angle image coordinate axis, matrix A is the lens rotation correction matrix, matrix B is the lens pitch correction and inverse perspective projection matrix, matrix C is the lens yaw correction matrix, and matrix D is the coordinate transformation matrix, which are expressed as follows: [Equation 4] However, θ ｙａｗ is the lens yaw angle, and θ ｐｉｔｃｈ is the lens pitch angle, and θ ｒｏｌｌ is the lens roll angle, H is the height of the wide-angle camera, and Z ｃ is the camera depth, and K ｘ is a proportionality constant between pixel coordinates in the X-axis direction of the image and world coordinates, and K ｙ is the proportional constant between the pixel coordinate in the Y-axis direction of the image and the world coordinate, w is the width of the image, and h is the height of the image.

7. The intelligent identification system using a wide-angle camera according to claim 6 , wherein the wide-angle camera is a fisheye lens camera.

8. The intelligent identification system using a wide-angle camera according to claim 6 , wherein the image correction pattern includes a black and white checkered pattern.

9. 7. The intelligent identification system using a wide-angle camera according to claim 6, wherein when the wide-angle camera photographs the image correction pattern, the wide-angle camera is installed on a vehicle and the image correction pattern is installed on the ground near the vehicle.

10. The intelligent identification system using a wide-angle camera according to claim 9 , wherein the image correction pattern extends along the ground.

Images