Corner point detection method and device based on checkerboard image and medium

Abstract

The invention relates to a checkerboard image-based corner detection method and device, and a medium. The method comprises the steps of collecting a checkerboard image which is shot by a vehicle-mounted camera and is used for carrying out camera external parameter calibration; extracting all inner corner points in the checkerboard image and directions of all edge straight lines connected with the inner corner points; on the basis of the sizes of checkerboard units forming the checkerboard image, angular point searching is carried out on the basis of the directions of all edge straight lines connected with all the inner angular points; and for each searched angular point, extracting all edge straight lines connected with the angular point, judging whether the gray value difference value of two sides of each edge straight line is greater than a difference value threshold value or not, and if so, judging that the angular point belongs to the angular point of the checkerboard image. According to the invention, robustness extraction of angular point features in an illumination shadow interference environment can be realized, and the accuracy of angular point detection is ensured; detection of internal and external angular points of checkerboard images in various scenes can be realized, the number of angular point detection is increased, and the precision of external parameter calibration is further improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a corner point detection method, device and medium based on a chessboard image. Background Art

[0002] At present, computer vision technology has been widely used in various advanced driver assistance systems. As an important function, the panoramic view system can help drivers eliminate visual blind spots to the greatest extent and improve driving safety.

[0003] External parameter calibration is an important part of panoramic surround image generation. It uses the corner detection algorithm to extract various characteristic corner points in the calibration site for modeling, and uses the calibration algorithm to solve the camera's pose parameters for stitching to generate panoramic images. The robustness and detection accuracy of the corner detection algorithm determine the success rate and accuracy of the external parameter calibration. The existing corner detection algorithm has the following technical defects:

[0004] (1) Extracting corner points by segmentation or line detection methods is easily affected by light and shadow (such as Figure 1 As shown), the success rate of calibration will be reduced;

[0005] (2) The more corner points detected, the less error in external parameter calibration. Existing corner point detection methods (such as OpenCV, etc.) can only detect the internal corner points of the calibration block (such as Figure 2 As shown in Figure 3, the number of corner point detections is small in some scenes. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a corner point detection method, device and medium based on a chessboard image.

[0007] The specific plan is as follows:

[0008] A method for detecting corner points based on a chessboard image comprises the following steps:

[0009] S1: Collecting chessboard images taken by the vehicle camera for camera extrinsic calibration;

[0010] S2: extract all the inner corner points in the checkerboard image and the directions of all the edge lines connecting the inner corner points;

[0011] S3: searching for corner points based on the size of the checkerboard cells constituting the checkerboard image and the directions of all edge lines connecting the inner corner points;

[0012] S4: For each corner point searched, extract all the edge lines connected to it, and determine whether the gray value difference on both sides of each edge line is greater than the difference threshold. If so, determine that the corner point belongs to the corner point of the checkerboard image.

[0013] Furthermore, the vehicle-mounted camera adopts a fisheye camera.

[0014] Furthermore, the method for acquiring the checkerboard image is as follows: pre-configuring ROI area data of the checkerboard image in the image captured by the vehicle-mounted camera; when the vehicle-mounted camera completes image capture, extracting the checkerboard image from the captured image based on the configured ROI area data.

[0015] Furthermore, step S2 also includes filtering each extracted inner corner point, using the method of: judging whether the gray value difference on both sides of each edge straight line connected by the inner corner point is greater than the difference threshold; if so, the inner corner point is judged to be reliable and retained; otherwise, the inner corner point is judged to be unreliable and deleted.

[0016] Furthermore, when searching for corner points in step S3, movement is performed along a direction of an edge line connecting the inner corner points, and the step length during movement is the side length of the chessboard unit.

[0017] Furthermore, when searching for corner points in step S3, movement is performed simultaneously along the directions of two adjacent edge lines connecting the inner corner points, and the step length of movement in each direction is the side length of the chessboard unit.

[0018] A terminal device for detecting corner points based on chessboard images comprises a processor, a memory and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above in the embodiment of the present invention are implemented.

[0019] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above in an embodiment of the present invention are implemented.

[0020] The present invention adopts the above technical solution, which can realize the robust extraction of corner point features in an environment with light and shadow interference, and ensure the accuracy of corner point detection; it can realize the detection of internal and external corner points of checkerboard images in various scenes, increase the number of corner point detection, and thus improve the accuracy of external parameter calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure shows a schematic diagram showing how corner point detection is affected by uneven site illumination.

[0022] Figure 2 Shown is a schematic diagram of the inner corner points in the chessboard.

[0023] Figure 3 Shown is a flow chart of a method according to a first embodiment of the present invention.

[0024] Figure 4 FIG. 4 is a schematic diagram of the inner corner points and their edges detected in this embodiment.

[0025] Figure 5 FIG. 4 is a schematic diagram of the corner point search direction in this embodiment.

[0026] Figure 6 Shown is a schematic diagram of the corner point positions in this embodiment. DETAILED DESCRIPTION

[0027] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0028] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0029] Embodiment 1:

[0030] The embodiment of the present invention provides a corner point detection method based on a chessboard image. Figure 3 As shown, the method comprises the following steps:

[0031] S1: Collect the checkerboard image taken by the vehicle camera for camera extrinsic calibration.

[0032] In this embodiment, the vehicle camera adopts a fisheye camera. Since the checkerboard image is only in a part of the image captured by the camera, in order to facilitate subsequent better detection, in this embodiment, the checkerboard image is pre-configured in the ROI (region of interest) data in the image captured by the vehicle camera. When the vehicle camera completes the image capture, the checkerboard image is extracted from the captured image based on the configured ROI region data.

[0033] S2: Extract all inner corner points in the checkerboard image and the directions of all edge lines connecting the inner corner points.

[0034] The inner corner point is the corner point inside the chessboard image, such as Figure 2 As shown, the relative outer corner points are corner points located on the border of the chessboard image. The detection method of the inner corner points can use a commonly used template matching method, etc., which is not limited here.

[0035] The edge straight line connecting the inner corner points can be detected by edge detection technology, that is, the gradient of each pixel in the neighborhood of the inner corner point is calculated, and then the direction of the edge straight line is obtained according to the gradient direction, such as Figure 4 shown.

[0036] In addition, since adjacent edge lines in the chessboard image are perpendicular to each other, the perpendicularity can be used as a constraint condition to screen the edge lines.

[0037] Furthermore, when detecting inner corner points, some interference points that do not belong to the inner corner points may be detected, so these interference points need to be removed. In this embodiment, it is determined whether the gray value difference on both sides of each edge straight line connected to the inner corner point is greater than the difference threshold. If so, the inner corner point is determined to be reliable (i.e., not an interference point) and is retained; otherwise, the inner corner point is determined to be unreliable (i.e., an interference point) and is deleted. The difference threshold can be set by those skilled in the art based on experience or experimental results, and is not limited here.

[0038] S3: Based on the size of the checkerboard cells constituting the checkerboard image, a corner point search is performed along the direction of all edge lines connecting the inner corner points.

[0039] The checkerboard unit is the smallest unit that makes up the checkerboard image, for example Figure 5 A small black block in the middle.

[0040] S4: For each corner point searched, extract all the edge lines connected to it, and determine whether the gray value difference on both sides of each edge line is greater than the difference threshold. If so, determine that the corner point belongs to the corner point of the checkerboard image.

[0041] like Figure 6 As shown in the figure, for a 2*2 chessboard image, there is only one inner corner point, namely the center point, and there are four connected edge lines, which are oriented in the four directions of up, down, left, and right. At this time, the search is divided into three cases: (1) Searching only along the horizontal (x) direction (left and right), that is, only reducing or increasing the x coordinate value of the inner corner by Δx, where Δx represents the side length of the chessboard unit in the x direction, then the search can be obtained Figure 6 (2) Search only along the vertical (y) direction (upward and downward), that is, only increase or decrease the y coordinate value of the inner corner by Δy, where Δy represents the side length of the chessboard unit in the y direction, then the search can be obtained Figure 6 (3) Searching along the horizontal (x) and vertical (y) directions at the same time, including four situations: the x-coordinate value decreases by Δx and the y-coordinate value increases by Δy; the x-coordinate value increases by Δx and the y-coordinate value increases by Δy; the x-coordinate value decreases by Δx and the y-coordinate value decreases by Δy; the x-coordinate value increases by Δx and the y-coordinate value decreases by Δy, then the search can be obtained Figure 6 Corner point 5 and corner point 6 are shown.

[0042] It should be noted that when the checkerboard image includes more than one inner corner point, if the above three situations corresponding to the four directions of up, down, left and right are searched for each inner corner point, it is possible to repeatedly search for existing inner corner points. Therefore, in one embodiment, all corner points of the obtained checkerboard image can be deduplicated; in another embodiment, the search method of each inner corner point can be limited according to its position in the checkerboard image (i.e., the search is limited to the direction toward other inner corner points).

[0043] In the above search process, the radius corresponding to the search range should not exceed the side length of the chessboard unit to avoid repeatedly searching the same corner point.

[0044] The embodiment of the present invention uses the edge gradient information of the inner corner point to search for the outer corner point, combines the outer corner point features of different regions to judge and filter the outer corner points, realizes the inner and outer corner point detection of the checkerboard image, and improves the success rate and accuracy of the camera external parameter calibration.

[0045] Embodiment 2:

[0046] The present invention also provides a corner point detection terminal device based on a checkerboard image, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned method embodiment of the first embodiment of the present invention when executing the computer program.

[0047] Further, as an executable solution, the corner point detection terminal device based on chessboard images can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The corner point detection terminal device based on chessboard images can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the composition structure of the corner point detection terminal device based on chessboard images is only an example of a corner point detection terminal device based on chessboard images, and does not constitute a limitation on the corner point detection terminal device based on chessboard images. It can include more or less components than the above, or combine certain components, or different components. For example, the corner point detection terminal device based on chessboard images can also include input and output devices, network access devices, buses, etc., and the embodiments of the present invention do not limit this.

[0048] Further, as an executable solution, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the corner point detection terminal device based on chessboard images, and uses various interfaces and lines to connect various parts of the entire corner point detection terminal device based on chessboard images.

[0049] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the corner point detection terminal device based on chessboard image by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0050] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.

[0051] If the module / unit integrated in the corner point detection terminal device based on chessboard images is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc.

[0052] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A corner point detection method based on a chessboard image, characterized in that: The following steps are involved: S1: Collecting chessboard images taken by the vehicle camera for camera extrinsic calibration; S2: extract all the inner corner points in the checkerboard image and the directions of all the edge lines connecting the inner corner points; S3: searching for corner points based on the size of the checkerboard cells constituting the checkerboard image and the directions of all edge lines connecting the inner corner points; S4: For each corner point searched, extract all the edge lines connected to it, and determine whether the gray value difference on both sides of each edge line is greater than the difference threshold. If so, determine that the corner point belongs to the corner point of the checkerboard image.

2. The method for detecting corner points based on a checkerboard image according to claim 1, characterized in that: The on-board camera uses a fisheye camera.

3. The method for detecting corner points based on a checkerboard image according to claim 1, characterized in that: The acquisition method of the checkerboard image is as follows: pre-configuring the ROI area data of the checkerboard image in the image captured by the vehicle-mounted camera; when the vehicle-mounted camera completes the image capture, extracting the checkerboard image from the captured image based on the configured ROI area data.

4. The method for detecting corner points based on a checkerboard image according to claim 1, characterized in that: Step S2 also includes filtering each extracted inner corner point, using the method of: judging whether the gray value difference on both sides of each edge straight line connected by the inner corner point is greater than the difference threshold; if so, the inner corner point is judged to be reliable and retained; otherwise, the inner corner point is judged to be unreliable and deleted.

5. The method for detecting corner points based on a checkerboard image according to claim 1, characterized in that: When searching for corner points in step S3, movement is performed along a direction of an edge line connecting the inner corner points, and the step length during movement is the side length of the chessboard unit.

6. The method for detecting corner points based on a checkerboard image according to claim 1, characterized in that: When searching for corner points in step S3, movement is performed simultaneously along the directions of two adjacent edge lines connecting the inner corner points, and the step length of movement in each direction is the side length of the chessboard unit.

7. A terminal device for detecting corner points based on a chessboard image, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images