Checkerboard angular point extraction method and device for perspective deformation image, equipment and medium
By calculating the homography matrix to transform and detect corner points on the perspective-deformed image, the accuracy problem of chessboard corner points extraction in the perspective-deformed image is solved, and the corner points extraction without perspective-deformed image is achieved.
Patent Information
- Application Number
- CN202510741415.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-04
- Publication Date
- 2025-08-29
Smart Images
Figure CN120563548A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, equipment and medium for extracting checkerboard corner points of a perspective-distorted image. Background Art
[0002] Perspective causes the checkerboard grid in a camera image to bend and deform due to the relative proportions of near and far features. This results in large variations in the grid size within the image. Accurate corner extraction is difficult in areas with coarse grid sizes. Therefore, accurate corner extraction is a technical challenge that needs to be addressed in related technologies. Summary of the Invention
[0003] In view of this, the present disclosure provides a method, apparatus, device and medium for extracting checkerboard corner points of a perspective-distorted image.
[0004] One aspect of the present disclosure provides a checkerboard corner point extraction method for a perspective-distorted image, comprising: estimating vertex coordinates of a target quadrilateral in an ideal image space based on a target quadrilateral in a perspective-distorted image captured by a camera; calculating a homography transformation matrix based on the vertex coordinates of the target quadrilateral in the perspective-distorted image and the vertex coordinates in the ideal image space; transforming the perspective-distorted image based on the homography transformation matrix to generate a non-perspective-distorted image in the ideal image space, wherein the perspective-distorted image includes a first checkerboard, and the non-perspective-distorted image includes a checkerboard corresponding to the first checkerboard. The target quadrilateral is located in the second chessboard corresponding to the first chessboard, and the ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second chessboard; corner point detection is performed on the second chessboard in the ideal image space to obtain the corner point coordinates of the second chessboard in the non-perspective deformation image; according to the inverse matrix of the homography transformation matrix, the corner point coordinates of the second chessboard in the non-perspective deformation image are mapped back to the perspective deformation image to obtain the corner point coordinates of the first chessboard in the perspective deformation image, so as to realize corner point extraction of the perspective deformation image.
[0005] According to an embodiment of the present disclosure, the above-mentioned homography transformation matrix is calculated based on the vertex coordinates of the above-mentioned target quadrilateral in the above-mentioned perspective deformed image and the vertex coordinates in the ideal image space, including: determining the coordinates of each vertex of the above-mentioned target quadrilateral relative to the preset vertices of the above-mentioned perspective deformed image, wherein the vertices of the above-mentioned target quadrilateral are one-to-one matched with the vertices of the ideal quadrilateral of the above-mentioned target quadrilateral in the ideal image space; for each group of matching points between the above-mentioned target quadrilateral and the above-mentioned ideal quadrilateral, according to the coordinates of the above-mentioned each group of matching points, constructing a transformation equation for the above-mentioned each group of matching points; based on the transformation equation for the above-mentioned each group of matching points, the above-mentioned homography transformation matrix is calculated.
[0006] According to an embodiment of the present disclosure, the perspective deformed image is transformed according to the homography transformation matrix to generate an image without perspective deformation in an ideal image space, including: multiplying the coordinates of each pixel in the perspective deformed image by the homography transformation matrix to generate the image without perspective deformation in the ideal image space.
[0007] According to an embodiment of the present disclosure, the size of the ideal quadrilateral is a preset size, and the coordinates of each vertex of the ideal quadrilateral are the coordinates of the preset vertex relative to the image without perspective deformation.
[0008] According to an embodiment of the present disclosure, the above-mentioned corner point detection is performed on the above-mentioned second chessboard in the ideal image space to obtain the corner point coordinates of the above-mentioned second chessboard in the above-mentioned image without perspective deformation, including: determining the first coordinates of the preset vertices of the chessboard area relative to the preset vertices of the above-mentioned image without perspective deformation, wherein the above-mentioned chessboard area includes the above-mentioned second chessboard; extracting multiple corner points in the above-mentioned second chessboard; determining the second coordinates of each corner point relative to the preset vertices of the above-mentioned chessboard area; and obtaining the coordinates of each corner point in the above-mentioned image without perspective deformation based on the above-mentioned first coordinates and the second coordinates for each corner point.
[0009] According to an embodiment of the present disclosure, the coordinates of the corner points of the second chessboard in the non-perspective deformation image are mapped back to the perspective deformed image according to the inverse matrix of the homography transformation matrix to obtain the coordinates of the corner points of the first chessboard in the perspective deformed image, including: obtaining the coordinates of each corner point in the first chessboard in the perspective deformed image according to the coordinates of each corner point in the second chessboard in the non-perspective deformation image and the inverse matrix of the homography transformation matrix.
[0010] According to an embodiment of the present disclosure, the above-mentioned checkerboard corner point extraction method also includes: obtaining the original image taken by the above-mentioned camera; dedistorting the above-mentioned original image to obtain a dedistorted image; preprocessing the above-mentioned dedistorted image to obtain a preprocessed image; and segmenting the checkerboard and background areas in the above-mentioned preprocessed image to obtain the above-mentioned perspective deformed image.
[0011] Another aspect of the present disclosure provides a checkerboard corner point extraction device for perspective-distorted images, comprising: an estimation module for estimating the vertex coordinates of the target quadrilateral in the perspective-distorted image captured by a camera in an ideal image space; a calculation module for calculating a homography transformation matrix based on the vertex coordinates of the target quadrilateral in the perspective-distorted image and the vertex coordinates in the ideal image space; a generation module for transforming the perspective-distorted image according to the homography transformation matrix to generate a non-perspective-distorted image in the ideal image space, wherein the perspective-distorted image includes a first checkerboard, and the non-perspective-distorted image The image processing method comprises a second chessboard corresponding to the first chessboard, the target quadrilateral is located in the first chessboard, and the ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second chessboard; a detection module is used to perform corner point detection on the second chessboard in the ideal image space to obtain the corner point coordinates of the second chessboard in the non-perspective deformation image; an acquisition module is used to map the corner point coordinates of the second chessboard in the non-perspective deformation image back to the perspective deformation image according to the inverse matrix of the homography transformation matrix to obtain the corner point coordinates of the first chessboard in the perspective deformation image, so as to realize corner point extraction of the perspective deformation image.
[0012] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.
[0013] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.
[0014] Another aspect of the present disclosure provides a computer program product comprising computer executable instructions, which are used to implement the method described above when the instructions are executed.
[0015] According to a checkerboard corner point extraction method, device, equipment and medium for perspective-distorted images provided by the present disclosure, by estimating the vertex coordinates of a target quadrilateral in an ideal image space, a homography transformation matrix can be calculated based on the vertex coordinates of the target quadrilateral in the perspective-distorted image and the vertex coordinates in the ideal image space. Therefore, the perspective-distorted image can be transformed using the homography transformation matrix to generate an image without perspective distortion in the ideal image space, and corner point detection is performed on the image without perspective distortion to obtain the corner point coordinates of the second checkerboard in the image without perspective distortion. Finally, the corner point coordinates of the second checkerboard in the image without perspective distortion are mapped back to the perspective-distorted image using the inverse matrix of the homography transformation matrix to obtain the corner point coordinates of the first checkerboard in the perspective-distorted image, thereby completing the corner point extraction of the perspective-distorted image captured by the camera, improving the accuracy of corner point detection and extraction in the perspective-distorted image, and preventing redundant and missing corner points. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0017] Figure 1 A diagram schematically illustrates an application scenario of a checkerboard corner point extraction method for a perspective-distorted image according to an embodiment of the present disclosure;
[0018] Figure 2 The flowchart of the checkerboard corner point extraction method for perspective-distorted images according to an embodiment of the present disclosure is schematically shown;
[0019] Figure 3 A schematic diagram schematically illustrates a perspective-distorted image captured by a camera according to an embodiment of the present disclosure;
[0020] Figure 4 A schematic diagram of a target quadrilateral in a perspective-distorted image according to an embodiment of the present disclosure is schematically shown;
[0021] Figure 5 A schematic diagram of an image without perspective deformation in an ideal image space according to an embodiment of the present disclosure is schematically shown;
[0022] Figure 6 A schematic diagram schematically shows an image after preprocessing according to an embodiment of the present disclosure;
[0023] Figure 7 A schematic diagram schematically illustrates a checkerboard area according to an embodiment of the present disclosure;
[0024] Figure 8 Schematically shows a block diagram of a checkerboard corner point extraction device for a perspective-distorted image according to an embodiment of the present disclosure;
[0025] Figure 9 A block diagram of an electronic device suitable for implementing a checkerboard corner point extraction method for a perspective-distorted image according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.
[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0029] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0030] In the embodiments of this disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of all data involved (including, but not limited to, user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard the security of user personal information, network security, and national security.
[0031] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.
[0032] Figure 1The application scenario diagram 100 of the checkerboard corner point extraction method for perspective-distorted images according to an embodiment of the present disclosure is schematically shown. It should be noted that, Figure 1 The application scenario diagram 100 shown is only an example to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0034] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).
[0035] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0036] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.
[0037] For example, the server 105 can estimate the vertex coordinates of the target quadrilateral in the ideal image space based on the target quadrilateral in the perspective deformed image captured by the camera, and calculate the homography transformation matrix based on the vertex coordinates of the target quadrilateral in the perspective deformed image and the vertex coordinates in the ideal image space. Therefore, the perspective deformed image can be transformed according to the homography transformation matrix to generate an image without perspective deformation in the ideal image space, wherein the perspective deformed image includes a first chessboard, and the image without perspective deformation includes a second chessboard corresponding to the first chessboard, the target quadrilateral is located in the first chessboard, and the ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second chessboard; by performing corner point detection on the second chessboard in the ideal image space, the corner point coordinates of the second chessboard in the image without perspective deformation are obtained, and according to the inverse matrix of the homography transformation matrix, the corner point coordinates of the second chessboard in the image without perspective deformation are mapped back to the perspective deformed image to obtain the corner point coordinates of the first chessboard in the perspective deformed image, so as to realize corner point extraction of the perspective deformed image.
[0038] It should be noted that the checkerboard corner point extraction method for perspective-distorted images provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the checkerboard corner point extraction device for perspective-distorted images provided in the embodiments of the present disclosure can generally be set in the server 105. The checkerboard corner point extraction method for perspective-distorted images provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the checkerboard corner point extraction device for perspective-distorted images provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the checkerboard corner point extraction method for perspective-distorted images provided in the embodiments of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by a terminal device other than the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the checkerboard corner point extraction apparatus for perspective-distorted images provided in the embodiments of the present disclosure may also be provided in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may be provided in a terminal device other than the first terminal device 101, the second terminal device 102, or the third terminal device 103.
[0039] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.
[0040] The following will be based on Figure 1 The scene described by Figures 2 to 7 The checkerboard corner point extraction method for perspective-distorted images according to an embodiment of the present disclosure is described in detail.
[0041] Figure 2 The flowchart of the checkerboard corner point extraction method for perspective-distorted images according to an embodiment of the present disclosure is schematically shown.
[0042] like Figure 2 As shown, the method 200 includes operations S210 to S250.
[0043] In operation S210 , vertex coordinates of the target quadrilateral in an ideal image space are estimated based on the target quadrilateral in the perspective-distorted image captured by the camera.
[0044] According to an embodiment of the present disclosure, due to the perspective phenomenon, the checkerboard in the image captured by the camera may be bent and deformed.
[0045] Figure 3 The figure schematically shows a perspective-distorted image captured by a camera according to an embodiment of the present disclosure.
[0046] like Figure 3 As shown, this is a perspective-distorted image captured by a camera, that is, the checkerboard in the image is bent and deformed when the relative proportions of near and far features change.
[0047] by Figure 3 Taking the perspective-distorted image shown in as an example, a target quadrilateral in the checkerboard of the perspective-distorted image is selected. The target quadrilateral can be any quadrilateral region in the checkerboard.
[0048] Figure 4 A schematic diagram of a target quadrilateral in a perspective-distorted image according to an embodiment of the present disclosure is schematically shown.
[0049] like Figure 4 As shown, the point , , , The formed quadrilateral area A1 may be the selected target quadrilateral.
[0050] According to an embodiment of the present disclosure, assuming that there is an ideal image space without perspective distortion, the vertex coordinates of the target quadrilateral in the ideal image space are estimated, that is, the quadrilateral enclosed by the vertex coordinates of the target quadrilateral in the ideal image space does not have perspective distortion.
[0051] In operation S220 , a homography transformation matrix is calculated based on vertex coordinates of the target quadrilateral in the perspective-distorted image and vertex coordinates in the ideal image space.
[0052] According to an embodiment of the present disclosure, based on the vertex coordinates of the target quadrilateral in the perspective transformed image and the vertex coordinates in the ideal image space, a homography transformation matrix that can reflect the transformation between the pixel coordinates in the perspective transformed image and the coordinates in the ideal image space can be calculated.
[0053] In operation S230 , the perspective-distorted image is transformed according to the homography transformation matrix to generate an image without perspective distortion in an ideal image space.
[0054] The perspective deformed image includes a first checkerboard, the non-perspective deformed image includes a second checkerboard corresponding to the first checkerboard, the target quadrilateral is located in the first checkerboard, and the ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second checkerboard.
[0055] According to an embodiment of the present disclosure, the perspective-distorted image captured by the camera may include a first checkerboard, and the first checkerboard may be multiple, such as Figure 3 As shown, the perspective deformed image includes two first checkerboard grids.
[0056] According to embodiments of the present disclosure, a perspective-distorted image can be transformed based on a homography transformation matrix to generate an image without perspective distortion in an ideal image space. Accordingly, the image without perspective distortion can also have multiple second checkerboard squares, and the number of second checkerboard squares in the image without perspective distortion is the same as the number of first checkerboard squares in the image without perspective distortion.
[0057] Figure 5 A schematic diagram of an image without perspective deformation in an ideal image space according to an embodiment of the present disclosure is schematically shown.
[0058] like Figure 5 As shown, according to the homography transformation matrix, Figure 3 By transforming the perspective deformed image shown in Figure 5 The image shown has no perspective deformation, that is, there is no bending and deformation problem in the image.
[0059] According to an embodiment of the present disclosure, when a perspective-distorted image is transformed into an image without perspective distortion based on a homography transformation matrix, the ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second checkerboard of the image without perspective distortion. Figure 4 In the case of the quadrilateral area A1 shown in , the ideal quadrilateral of the target quadrilateral in the ideal image space should be Figure 5The quadrilateral area A2 shown in .
[0060] In operation S240 , corner point detection is performed on the second chessboard in the ideal image space to obtain the coordinates of the corner points of the second chessboard in the image without perspective deformation.
[0061] According to an embodiment of the present disclosure, when performing corner point detection on the second chessboard, the corner points in the second chessboard may be determined first, and then the coordinates of the corner points in the image without perspective deformation may be determined.
[0062] In operation S250, the coordinates of the corner points of the second checkerboard in the non-perspective-distorted image are mapped back to the perspective-distorted image based on the inverse matrix of the homography transformation matrix, thereby obtaining the coordinates of the corner points of the first checkerboard in the perspective-distorted image, thereby extracting the corner points of the perspective-distorted image. According to an embodiment of the present disclosure, once the coordinates of the corner points of the second checkerboard in the non-perspective-distorted image are obtained, the coordinates of the corner points of the second checkerboard in the non-perspective-distorted image can be mapped back to the perspective-distorted image based on the inverse matrix of the homography transformation matrix, thereby obtaining the coordinates of the corner points of the first checkerboard in the perspective-distorted image, thereby extracting the corner points of the perspective-distorted image.
[0063] According to an embodiment of the present disclosure, by estimating the vertex coordinates of the target quadrilateral in the ideal image space, the homography transformation matrix can be calculated based on the vertex coordinates of the target quadrilateral in the perspective deformed image and the vertex coordinates in the ideal image space, so that the perspective deformed image can be transformed through the homography transformation matrix to generate an image without perspective deformation in the ideal image space, and corner point detection is performed on the image without perspective deformation to obtain the corner point coordinates of the second chessboard in the image without perspective deformation. Finally, the corner point coordinates of the second chessboard in the image without perspective deformation are mapped back to the perspective deformed image through the inverse matrix of the homography transformation matrix to obtain the corner point coordinates of the first chessboard in the perspective deformed image, thereby completing the corner point extraction of the perspective deformed image taken by the camera, improving the accuracy of corner point detection and extraction in the perspective deformed image, and no corner points will be redundant or missing.
[0064] According to an embodiment of the present disclosure, the above-mentioned checkerboard corner point extraction method also includes: obtaining the original image taken by the camera; dedistorting the original image to obtain a dedistorted image; preprocessing the dedistorted image to obtain a preprocessed image; and segmenting the checkerboard and background areas in the preprocessed image to obtain a perspective deformed image.
[0065] According to an embodiment of the present disclosure, a perspective-distorted image is obtained by processing an original image captured by a camera. If the original image captured by the camera has distortion, after obtaining the original image captured by the camera, the original image is first subjected to dedistortion processing to obtain a dedistorted image.
[0066] According to an embodiment of the present disclosure, the dedistorted image is further preprocessed to obtain a preprocessed image, where the preprocessing may include denoising, contrast enhancement, etc. The preprocessed image may also be segmented to separate the checkerboard area and background area in the preprocessed image to obtain a perspective-distorted image.
[0067] Figure 6 A schematic diagram schematically shows a preprocessed image according to an embodiment of the present disclosure.
[0068] like Figure 6 As shown, the pre-processed image includes a checkerboard and a background area. The background area may refer to an area other than the checkerboard in the image, such as an area where a vehicle is located in the image.
[0069] According to the embodiment of the present disclosure, Figure 6 The pre-processed image shown in the figure is segmented, that is, the checkerboard and background areas are separated, and the following can be obtained: Figure 3 The perspective-distorted image shown.
[0070] According to an embodiment of the present disclosure, in order to correct image distortion caused by the camera lens, the original image taken by the camera is first dedistorted; then the dedistorted image is preprocessed, such as denoising, contrast enhancement, etc., to reduce image noise and improve image quality; image segmentation can also be used to separate the checkerboard and background areas in the preprocessed image, thereby reducing the interference of the background area on corner point detection and extraction, that is, by excluding interference from other objects in the image, such as wall edges, vehicle contours, etc., only corner point detection is performed on the checkerboard, so as to further improve the accuracy of subsequent corner point detection and extraction.
[0071] According to an embodiment of the present disclosure, a homography transformation matrix is calculated based on the vertex coordinates of the target quadrilateral in the perspective deformed image and the vertex coordinates in the ideal image space, including: determining the coordinates of each vertex of the target quadrilateral relative to the preset vertex of the perspective deformed image; for each group of matching points between the target quadrilateral and the ideal quadrilateral, constructing a transformation equation for each group of matching points based on the coordinates of each group of matching points; and calculating the homography transformation matrix based on the transformation equation for each group of matching points.
[0072] The vertices of the target quadrilateral are matched one-to-one with the vertices of the ideal quadrilateral of the target quadrilateral in the ideal image space.
[0073] The preset vertex may be any vertex of the perspective-deformed image. For example, the preset vertex may be the upper left corner vertex of the image. The following description will be made by taking the upper left corner as an example.
[0074] According to an embodiment of the present disclosure, when the preset vertex is the upper left corner, the coordinates of the four vertices of the target quadrilateral relative to the upper left corner of the perspective deformed image are determined. Figure 4 As shown in , determine the four vertices of the target quadrilateral A1 relative to Figure 3 The coordinates of the upper left corner of the perspective deformed image shown in can be , , , , where the x-axis can represent the top edge of the image and the y-axis can represent the left edge of the image, i.e. are the values along the x-axis, are the values along the y-axis.
[0075] by Figure 3 As an example, the first chessboard C on the left side of the perspective deformed image shown in FIG. Figure 3 The image shown is partially enlarged and focused on the chessboard C. A quadrilateral in the chessboard C is selected, such as Figure 4 , and record the coordinates of the four vertices of the target quadrilateral A1 relative to the upper left corner of the perspective-distorted image.
[0076] According to an embodiment of the present disclosure, the estimated coordinates of each vertex of the ideal quadrilateral of the target quadrilateral in the ideal image space are the coordinates of the vertices of the ideal quadrilateral relative to the upper left corner of the generated image without perspective deformation.
[0077] by Figure 4 As an example, the target quadrilateral A1 shown in FIG is assumed to be an ideal quadrilateral in the ideal image space. Figure 5 The quadrilateral A2 in the generated image without perspective deformation is shown. The vertex coordinates of the ideal quadrilateral can be respectively , , , .
[0078] Among them, the vertices in the target quadrilateral can form a set of matching points with the corresponding vertices in the ideal quadrilateral, that is, and A set of matching points can be formed. .
[0079] According to an embodiment of the present disclosure, a transformation equation for each group of matching points may be constructed based on the coordinates of each group of matching points. The transformation equation may be as shown in the following formula (1).
[0080] (1)
[0081] in, Can be a point The homogeneous coordinates of Can be a point The homogeneous coordinates of It can be the scaling factor between the target quadrilateral and the ideal quadrilateral; is the homography transformation matrix.
[0082] exist In the case of , the above formula (1) can be further expressed as the following formula (2).
[0083] (2)
[0084] The above formula (2) can be converted into the following formulas (3) and (4).
[0085] (3)
[0086] (4)
[0087] By writing the above formula (3) and formula (4) in matrix form, we can get the following formula (5).
[0088] (5)
[0089] According to an embodiment of the present disclosure, since the homography transformation matrix There are eight degrees of freedom. A set of matching points can provide two sets of equations, so four sets of non-collinear matching points are required to determine the unique , then the quadrilateral area is selected, so the homography transformation matrix can be solved based on the transformation equation of the four sets of matching points .
[0090] According to an embodiment of the present disclosure, a preset vertex, such as the upper left corner, is fixed, and the coordinates of each vertex of the target quadrilateral relative to the preset vertex of the perspective-distorted image are determined. The vertices of the target quadrilateral and the ideal quadrilateral are matched one-to-one, that is, the corresponding vertices form a set of matching points. Thus, based on the coordinates of the set of matching points, a transformation equation can be constructed between the target quadrilateral and the ideal quadrilateral. Based on the transformation equations of each of the four sets of matching points, a homography transformation matrix can be calculated for subsequent corner point detection and extraction.
[0091] According to an embodiment of the present disclosure, the size of the ideal quadrilateral is a preset size, and the coordinates of each vertex of the ideal quadrilateral are the coordinates of the preset vertex relative to the image without perspective deformation.
[0092] According to an embodiment of the present disclosure, the size of the ideal quadrilateral may be a preset size, that is, the coordinates of each vertex of the ideal quadrilateral may be determined based on the set size of the ideal quadrilateral.
[0093] In the case where each grid in the chessboard is a square, the ideal quadrilateral can be a square. In the case where the preset size is 30 mm in length, the coordinates of the four vertices of the ideal quadrilateral can be estimated to be , , , The coordinates of the four vertices of the estimated ideal quadrilateral are relative to the coordinates of the upper left corner of the subsequently generated image without perspective deformation.
[0094] Among them, the preset size can be set according to needs based on the shape and side length relationship of the grid in the actual chessboard. The preset size only affects the scaling scale of the generated non-perspective deformation image relative to the perspective deformation image. That is, when the preset size is large, the generated non-perspective deformation image is enlarged in size relative to the perspective deformation image, but does not affect the subsequent corner detection and corner extraction.
[0095] The coordinates of the four vertices of the target quadrilateral A1 relative to the upper left corner of the perspective deformed image are , , , , the coordinates of the four vertices of the ideal quadrilateral relative to the upper left corner of the image without perspective deformation are , , , For example, for the coordinates of the above four sets of matching points, the homography transformation matrix can be calculated by the above formulas (1) to (5): .
[0096] According to an embodiment of the present disclosure, based on a preset size, the coordinates of each vertex of the target quadrilateral in the ideal image space are estimated, so as to calculate the homography transformation matrix between the target quadrilateral and the ideal quadrilateral according to the vertex coordinates of the target quadrilateral in the perspective deformed image, and in the generated image without perspective deformation, the coordinates of each vertex of the ideal quadrilateral are the coordinates of the preset vertices relative to the image without perspective deformation, so as to facilitate the subsequent conversion of the perspective deformed image to the ideal image space.
[0097] According to an embodiment of the present disclosure, a perspective deformed image is transformed according to a homography transformation matrix to generate an image without perspective deformation in an ideal image space, including: multiplying the coordinates of each pixel in the perspective deformed image by the homography transformation matrix to generate an image without perspective deformation in the ideal image space.
[0098] According to an embodiment of the present disclosure, by multiplying the coordinates of each pixel in the perspective-distorted image by the homography transformation matrix, an image without perspective distortion in an ideal image space can be generated.
[0099] For example, Figure 3 The coordinates of each pixel in the perspective deformed image shown are multiplied by the homography transformation matrix to obtain Figure 5 The image without perspective distortion is shown.
[0100] According to an embodiment of the present disclosure, after obtaining the homography transformation matrix, the pixel coordinates in the perspective deformed image are multiplied by the homography transformation matrix to generate an image without perspective deformation, so that corner points can be subsequently extracted from the image without perspective deformation, which is beneficial to improving the accuracy of corner point extraction in the perspective deformed image.
[0101] According to an embodiment of the present disclosure, corner point detection is performed on the second chessboard in the ideal image space to obtain the coordinates of the corner points of the second chessboard in the image without perspective deformation, including: determining the first coordinates of the preset vertices of the chessboard area relative to the preset vertices of the image without perspective deformation; extracting multiple corner points in the second chessboard; determining the second coordinates of each corner point relative to the preset vertices of the chessboard area; and obtaining the coordinates of each corner point in the image without perspective deformation based on the first coordinates and the second coordinates for each corner point.
[0102] The checkerboard area includes a second checkerboard.
[0103] According to an embodiment of the present disclosure, in order to facilitate corner point detection for an image without perspective deformation, a second checkerboard in the image without perspective deformation may be firstly cut out, and corner point detection may be performed on the second checkerboard one by one through the checkerboard area including the second checkerboard.
[0104] According to an embodiment of the present disclosure, for each second checkerboard, a first coordinate of the upper left corner of the checkerboard area relative to the upper left corner of the image without perspective deformation may be determined.
[0105] Figure 7 A schematic diagram of a checkerboard area according to an embodiment of the present disclosure is schematically shown.
[0106] like Figure 7 As shown, a checkerboard area including the second checkerboard is intercepted from the image without perspective deformation. Figure 7 The upper left corner of the checkerboard area shown is relative to Figure 5 The first coordinate of the top left corner of the non-perspective-distorted image shown.
[0107] According to an embodiment of the present disclosure, multiple corner points in the second chessboard are extracted, and the extracted multiple corner points can be as follows: Figure 7 As shown in the circle marked in the middle.
[0108] According to an embodiment of the present disclosure, the second coordinate of each corner point relative to the upper left corner of the checkerboard area is determined to obtain the coordinates of each corner point in the image without perspective deformation based on the first coordinate and the second coordinate for each corner point, that is, the second coordinate of each corner point relative to the upper left corner of the checkerboard area is superimposed with the first coordinate of the upper left corner of the checkerboard area relative to the upper left corner of the image without perspective deformation to obtain the coordinates of each corner point relative to the upper left corner of the image without perspective deformation.
[0109] If the first coordinate of the upper left corner of the checkerboard area relative to the upper left corner of the image without perspective deformation is , and a corner point extracted on the second chessboard The second coordinate relative to the upper left corner of the checkerboard area is , then the coordinates of the corner point relative to the upper left corner of the image without perspective deformation can be ,in, is an integer greater than or equal to 1.
[0110] According to an embodiment of the present disclosure, since the coordinates of the corner points detected in the second checkerboard need to be subsequently converted to coordinates relative to the upper left corner of the perspective-distorted image based on the homography transformation matrix between the perspective-distorted image and the non-perspective-distorted image, it is necessary to determine the coordinates of the corner points in the second checkerboard relative to the preset vertices of the non-perspective-distorted image. Thus, multiple corner points in the second checkerboard are extracted, and the second coordinates of each corner point relative to the preset vertices of the checkerboard area are determined. These coordinates are then superimposed on the first coordinates of the preset vertices of the checkerboard area relative to the preset vertices of the non-perspective-distorted image to obtain the coordinates of the corner points in the non-perspective-distorted image. This facilitates the subsequent conversion of the corner point coordinates. Furthermore, because the influence of perspective distortion is eliminated, the accuracy of corner point detection is significantly improved.
[0111] According to an embodiment of the present disclosure, the coordinates of the corner points of the second chessboard in the non-perspective deformation image are mapped back to the perspective deformed image according to the inverse matrix of the homography transformation matrix to obtain the coordinates of the corner points of the first chessboard in the perspective deformed image, including: obtaining the coordinates of each corner point in the first chessboard in the perspective deformed image according to the coordinates of each corner point in the second chessboard in the non-perspective deformation image and the inverse matrix of the homography transformation matrix.
[0112] According to an embodiment of the present disclosure, the coordinates of the corner points in the first chessboard in the perspective-distorted image are obtained by multiplying the coordinates of the corner points in the second chessboard in the non-perspective-distorted image by the inverse matrix of the homography transformation matrix.
[0113] Corner points in the image without perspective deformation For example, the coordinates of the corner points in the non-perspective deformation image can be mapped back to the perspective deformation image by the following formula (6) to obtain the coordinates of the corner points in the perspective deformation image.
[0114] (6)
[0115] in, The corner point in the first chessboard after the inverse matrix transformation based on the homography transformation matrix is The coordinates are relative to the upper left corner of the perspective-distorted image.
[0116] against Figure 7 The extracted corner points, through the above formula (6), can be obtained by taking the coordinates of the perspective deformed image as (582.36, 544.47), (648.96, 513.43), (696.01, 490.46), (731.87, 473.86), (758.72, 460.99), (781.23, 450.95), (522.11, 541.10), (599.72, 510.91), (651.63, 488.78), (693.38, 472.01), (724.74, 459.3 3), (750.52, 449.19), (465.10, 537.37), (549.32, 508.28), (610.30, 486.53), (656.10, 469.82), (690.72, 457.45), (719.56, 447.57), (401.33, 534.88), (498.98, 504.34), (568.12, 484.08), (620.73, 468.07), (657.96, 455.83), (691.10, 446.91).
[0117] According to an embodiment of the present disclosure, after obtaining the coordinates of the corner points in the second chessboard in the non-perspective deformation image, the coordinates of the corner points in the non-perspective deformation image can be mapped back to the perspective deformed image based on the inverse matrix of the homography transformation matrix to obtain the coordinates of the corner points in the first chessboard in the perspective deformed image, thereby realizing the corner point extraction of the perspective deformed image and improving the accuracy of corner point detection and extraction.
[0118] Based on the foregoing, the disclosed checkerboard corner point extraction method for perspective-distorted images not only improves the accuracy of checkerboard corner point detection and extraction in perspective-distorted images, but also exhibits strong robustness to dimensional variations. It can adapt to the complex perspective distortion caused by the curvature and turning of car connections in multi-carriage train surround view systems. Precise corner point detection and extraction facilitates more accurate image correction and splicing, providing drivers with clearer and more coherent surround view images and enhancing driving safety and comfort. Furthermore, to increase the speed of corner point detection and extraction, parallel computing technology can be employed, utilizing multiple processors to simultaneously handle the detection and extraction of multiple corner points.
[0119] Based on the above-mentioned checkerboard corner point extraction method for perspective-distorted images, the present disclosure also provides a checkerboard corner point extraction device for perspective-distorted images. Figure 8 The device is described in detail.
[0120] Figure 8 The block diagram of a checkerboard corner point extraction device for a perspective-distorted image according to an embodiment of the present disclosure is schematically shown.
[0121] like Figure 8 As shown, the apparatus 800 for extracting checkerboard corner points from a perspective-distorted image in this embodiment includes an estimation module 810 , a calculation module 820 , a generation module 830 , a detection module 840 and an acquisition module 850 .
[0122] The estimation module 810 is configured to estimate the vertex coordinates of the target quadrilateral in the ideal image space based on the target quadrilateral region image in the perspective-distorted image captured by the camera. In one embodiment, the estimation module 810 may be configured to perform the operation S210 described above, which will not be described in detail here.
[0123] The calculation module 820 is configured to calculate a homography transformation matrix between the target quadrilateral and the ideal quadrilateral based on the target quadrilateral in the perspective-distorted image captured by the camera and the ideal quadrilateral corresponding to the target quadrilateral. In one embodiment, the calculation module 820 can be configured to perform operation S220 described above, which will not be further described here.
[0124] Generation module 830 is configured to transform the perspective-distorted image according to the homography transformation matrix to generate an image without perspective distortion, wherein the perspective-distorted image includes a first checkerboard grid, the image without perspective distortion includes a second checkerboard grid corresponding to the first checkerboard grid, the target quadrilateral is located in the first checkerboard grid, and the ideal quadrilateral is located in the second checkerboard grid. In one embodiment, generation module 830 may be configured to perform operation S230 described above, and will not be further described here.
[0125] The detection module 840 is configured to perform corner detection on the second checkerboard to obtain the coordinates of the corners in the image without perspective distortion. In one embodiment, the detection module 840 may be configured to perform the operation S240 described above, which will not be described in detail here.
[0126] Obtaining module 850 is configured to map the coordinates of the corner points in the non-perspective-distorted image back to the perspective-distorted image based on the homography transformation matrix, thereby obtaining the coordinates of the corner points in the perspective-distorted image. In one embodiment, obtaining module 850 may be configured to perform operation S250 described above, and will not be further described herein.
[0127] According to an embodiment of the present disclosure, the calculation module 820 includes a first determining unit, a constructing unit, and a calculating unit.
[0128] The first determining unit is configured to determine the coordinates of each vertex of the target quadrilateral relative to a preset vertex of the perspective deformed image, wherein the vertices of the target quadrilateral are matched one-to-one with the vertices of an ideal quadrilateral of the target quadrilateral in the ideal image space.
[0129] The construction unit is used to construct a transformation equation for each set of matching points between the target quadrilateral and the ideal quadrilateral according to the coordinates of each set of matching points.
[0130] The calculation unit is used to calculate the homography transformation matrix based on the transformation equation for each group of matching points.
[0131] According to an embodiment of the present disclosure, the generation module 830 includes a generation unit.
[0132] The generation unit is used to multiply the coordinates of each pixel in the perspective-distorted image by the homography transformation matrix to generate an image without perspective distortion in an ideal image space.
[0133] According to an embodiment of the present disclosure, the detection module 840 includes a second determination unit, an extraction unit, a third determination unit, and a first obtaining unit.
[0134] The second determining unit is configured to determine first coordinates of a preset vertex of a checkerboard area relative to a preset vertex of the image without perspective deformation, wherein the checkerboard area includes a second checkerboard.
[0135] The extraction unit is used to extract multiple corner points in the second chessboard.
[0136] The third determining unit is configured to determine a second coordinate of each corner point relative to a preset vertex of the checkerboard area.
[0137] The first obtaining unit is configured to obtain the coordinates of each corner point in the image without perspective deformation according to the first coordinates and the second coordinates for each corner point.
[0138] According to an embodiment of the present disclosure, the obtaining module 850 includes a second obtaining unit.
[0139] The second obtaining unit is used to obtain the coordinates of each corner point in the first chessboard in the perspective deformed image according to the coordinates of each corner point in the second chessboard in the non-perspective deformed image and the inverse matrix of the homography transformation matrix.
[0140] According to an embodiment of the present disclosure, the checkerboard corner point extraction device 800 further includes an acquisition module, a dedistortion module, a preprocessing module and a segmentation module.
[0141] The acquisition module is used to obtain the original image taken by the camera.
[0142] The dedistortion module is used to dedistort the original image to obtain a dedistorted image.
[0143] The preprocessing module is used to preprocess the dedistorted image to obtain a preprocessed image.
[0144] The segmentation module is used to segment the checkerboard and background areas in the preprocessed image to obtain a perspective deformed image.
[0145] According to the embodiments of the present invention, any number of modules, sub-modules, units, and sub-units, or at least part of the functions of any number of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a computer program module, which can perform the corresponding functions when the computer program module is executed.
[0146] For example, any number of the estimation module 810, the calculation module 820, the generation module 830, the detection module 840, and the acquisition module 850 can be combined into a single module / unit / sub-unit, or any one of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the estimation module 810, the calculation module 820, the generation module 830, the detection module 840, and the acquisition module 850 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the estimation module 810 , the calculation module 820 , the generation module 830 , the detection module 840 and the acquisition module 850 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.
[0147] It should be noted that the checkerboard corner point extraction device part for perspective deformed images in the embodiments of the present disclosure corresponds to the checkerboard corner point extraction method part for perspective deformed images in the embodiments of the present disclosure. The description of the checkerboard corner point extraction device part for perspective deformed images specifically refers to the checkerboard corner point extraction method part for perspective deformed images, which will not be repeated here.
[0148] Figure 9 A block diagram of an electronic device suitable for implementing a checkerboard corner point extraction method for a perspective-distorted image according to an embodiment of the present disclosure is schematically shown. Figure 9 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.
[0149] like Figure 9As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0150] Various programs and data required for the operation of the electronic device 900 are stored in the RAM 903. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the programs may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.
[0151] According to an embodiment of the present disclosure, electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to bus 904. Electronic device 900 may also include one or more of the following components connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 908 including a hard disk; and a communication section 909 including a network interface card such as a LAN card or modem. Communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 910 as needed, so that computer programs read from the removable media can be installed into storage section 908 as needed.
[0152] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0153] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.
[0154] According to embodiments of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0155] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .
[0156] An embodiment of the present disclosure also includes a computer program product, which includes a computer program containing program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the checkerboard corner point extraction method for perspective deformed images provided by the embodiment of the present disclosure.
[0157] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0158] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.
[0159] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).
[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, and all of these combinations and / or couplings fall within the scope of the present disclosure.
[0161] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A checkerboard corner point extraction method for perspective-distorted images, comprising: According to a target quadrilateral in a perspective-distorted image captured by a camera, estimating vertex coordinates of the target quadrilateral in an ideal image space; Calculating a homography transformation matrix according to vertex coordinates of the target quadrilateral in the perspective deformed image and vertex coordinates in an ideal image space; transforming the perspective-distorted image according to the homography transformation matrix to generate an image without perspective distortion in an ideal image space, wherein the perspective-distorted image includes a first checkerboard, the image without perspective distortion includes a second checkerboard corresponding to the first checkerboard, the target quadrilateral is located in the first checkerboard, and an ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second checkerboard; Performing corner point detection on the second chessboard in the ideal image space to obtain the coordinates of the corner points of the second chessboard in the image without perspective deformation; According to the inverse matrix of the homography transformation matrix, the coordinates of the corner points of the second chessboard in the non-perspective deformation image are mapped back to the perspective deformation image to obtain the coordinates of the corner points of the first chessboard in the perspective deformation image, so as to realize corner point extraction of the perspective deformation image.
2. The method according to claim 1, wherein The step of calculating a homography transformation matrix according to vertex coordinates of the target quadrilateral in the perspective deformed image and vertex coordinates in an ideal image space includes: Determining the coordinates of each vertex of the target quadrilateral relative to a predetermined vertex of the perspective deformed image, wherein the vertices of the target quadrilateral are matched one-to-one with the vertices of an ideal quadrilateral of the target quadrilateral in the ideal image space; For each set of matching points between the target quadrilateral and the ideal quadrilateral, constructing a transformation equation for each set of matching points according to the coordinates of the each set of matching points; The homography transformation matrix is calculated based on the transformation equation for each group of matching points.
3. The method according to claim 1, wherein The step of transforming the perspective-distorted image according to the homography transformation matrix to generate an image without perspective distortion in an ideal image space includes: The coordinates of each pixel in the perspective-distorted image are multiplied by the homography transformation matrix to generate the image without perspective distortion in an ideal image space.
4. The method according to claim 2, wherein: The size of the ideal quadrilateral is a preset size, and the coordinates of each vertex of the ideal quadrilateral are the coordinates of the preset vertex relative to the image without perspective deformation.
5. The method according to claim 4, wherein The performing corner point detection on the second chessboard in the ideal image space to obtain the coordinates of the corner points of the second chessboard in the image without perspective deformation includes: Determining first coordinates of a predetermined vertex of a checkerboard area relative to a predetermined vertex of the image without perspective distortion, wherein the checkerboard area includes the second checkerboard; extracting a plurality of corner points in the second chessboard; Determining a second coordinate of each corner point relative to a predetermined vertex of the checkerboard area; The coordinates of each corner point in the image without perspective deformation are obtained according to the first coordinates and the second coordinates for each corner point.
6. The method according to claim 1, wherein Mapping the coordinates of the corner points of the second checkerboard in the non-perspective-distorted image back to the perspective-distorted image according to the inverse matrix of the homography transformation matrix to obtain the coordinates of the corner points of the first checkerboard in the perspective-distorted image includes: The coordinates of each corner point in the first chessboard in the perspective-distorted image are obtained according to the coordinates of each corner point in the second chessboard in the non-perspective-distorted image and the inverse matrix of the homography transformation matrix.
7. The method according to claim 1, further comprising: Obtaining the original image captured by the camera; Dedistorting the original image to obtain a dedistorted image; Preprocessing the dedistorted image to obtain a preprocessed image; The checkerboard and background areas in the pre-processed image are segmented to obtain the perspective deformed image.
8. A device for extracting checkerboard corner points from a perspective-distorted image, comprising: an estimation module, configured to estimate vertex coordinates of a target quadrilateral in an ideal image space based on the target quadrilateral in the perspective-distorted image captured by the camera; a calculation module, configured to calculate a homography transformation matrix according to vertex coordinates of the target quadrilateral in the perspective deformed image and vertex coordinates in an ideal image space; a generating module, configured to transform the perspective-distorted image according to the homography transformation matrix to generate an image without perspective distortion in an ideal image space, wherein the perspective-distorted image includes a first checkerboard, the image without perspective distortion includes a second checkerboard corresponding to the first checkerboard, the target quadrilateral is located in the first checkerboard, and an ideal quadrilateral of the target quadrilateral in the ideal image space is located in the second checkerboard; a detection module, configured to perform corner detection on the second checkerboard in an ideal image space to obtain the coordinates of the corners of the second checkerboard in the image without perspective deformation; and An acquisition module is used to map the corner point coordinates of the second chessboard in the non-perspective deformation image back to the perspective deformation image according to the inverse matrix of the homography transformation matrix, and obtain the corner point coordinates of the first chessboard in the perspective deformation image, so as to realize corner point extraction of the perspective deformation image.
9. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 7.