Checkerboard corner detection method, electronic equipment and computer readable medium

Abstract

The invention belongs to the technical field of camera equipment calibration, and provides a checkerboard corner detection method, which comprises the following steps of S1, preprocessing an original image acquired by a vehicle camera; s2, carrying out edge detection on the image preprocessed in the step S1; s3, based on the edge detection result obtained in the step S2, carrying out peak detection to obtain a local peak point; and S4, angular point calculation: based on a peak value detection result, obtaining row and column peak value point vectors, calculating all angular point coordinates, and obtaining final angular point information. Through a space and channel image fusion method, checkerboard corner detection in a vehicle sensing calibration scene is realized. Under the environmental conditions of shadow, reflection, uneven brightness and the like, the edge of the checkerboard image can be effectively extracted, and the corner detection accuracy is improved. By fusing space and channel information, the adaptive capacity of the algorithm to interference factors is remarkably improved, and the false detection rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of camera equipment calibration technology, and in particular to a checkerboard corner detection method, electronic device, and computer-readable medium. Background Technology

[0002] The planar checkerboard calibration method is a camera parameter calibration method based on a two-dimensional planar checkerboard pattern. It involves taking multiple (usually 15-20) checkerboard images from different angles, extracting the pixel coordinates of the checkerboard corners using a corner detection algorithm, and combining the known physical dimensions of the checkerboard (setting a Z=0 plane coordinate system) to calculate the homography matrix H between the image plane and the checkerboard plane. Then, it uses orthogonal constraints and the least squares method to solve for the camera's intrinsic parameters (focal length, principal point coordinates, etc.), extrinsic parameters (rotation and translation), and lens distortion coefficients.

[0003] The goal of vehicle surround view system calibration is to acquire the intrinsic and extrinsic parameters of the cameras, thereby ensuring that the in-vehicle surround view system can accurately fuse the images captured by each camera and generate a seamless 360-degree panoramic image, helping the driver to better understand the surrounding environment. The core purpose of this process is to calculate and correct the relative relationships between the cameras based on their position, orientation, and distortion characteristics, enabling the system to accurately synthesize and display images from different angles and in different scenarios.

[0004] After surround-view calibration, a point mapping relationship needs to be established between the vehicle coordinate system and the pixel coordinate system for subsequent perception model input. During this mapping process, edge detection of the checkerboard pattern is required to obtain the corner coordinates of the calibration checkerboard and acquire point information in the pixel coordinate system. Traditional edge detection methods obtain target edges by calculating horizontal and vertical gradients in the image. These methods are sensitive to image noise and struggle to accurately distinguish semantic edges in complex scenes. In recent years, deep learning methods have gradually become the mainstream choice for edge detection. Therefore, this invention provides a checkerboard corner detection method based on spatial and channel image fusion in vehicle perception calibration scenarios. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art, and proposes a chessboard corner detection method, an electronic device, and a computer-readable medium.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a method for detecting corner points on a chessboard, comprising the following steps:

[0007] S1. Preprocess the raw images captured by the vehicle camera;

[0008] S2. Perform edge detection on the image preprocessed in step S1;

[0009] S3. Based on the edge detection results obtained in step S2, perform peak detection to detect local peak points;

[0010] S4. Corner point calculation;

[0011] Based on the peak detection results, the peak point vectors of rows and columns are obtained, the coordinates of all corner points are calculated, and the final corner point information is obtained.

[0012] Furthermore, step S1 includes:

[0013] S1.1 Perform grayscale processing on the images captured by the vehicle camera;

[0014] The formula for calculating grayscale value is:

[0015]

[0016] In the formula, It is a grayscale value; , , These are the values ​​for the red, green, and blue color channels of each pixel;

[0017] S1.2 The grayscale image is blurred using Gaussian blur;

[0018] The formula for the Gaussian function is:

[0019]

[0020] In the formula, These are the coordinates of a pixel in the image; It is the standard deviation; It is the value of a two-dimensional Gaussian function, representing the weight of a given pixel.

[0021] Furthermore, in step S2, the EdgeNAT model is used for edge detection, which includes an encoder DiNAT and a decoder SCAFM.

[0022] Furthermore, the decoder SCAFM includes a spatial attention module and a channel attention module.

[0023] Furthermore, the image processing performed by the encoder DiNAT includes the following:

[0024] I. By using the tokenizer in the DiNAT encoder, which consists of two 3x3 convolutional layers with a stride of 2, a feature map with a resolution of one-quarter of the input image is obtained;

[0025] II. Downsampling is performed between layers using a single 3x3 convolutional layer with a stride of 2 in the DiNAT encoder to obtain sizes of... , , , Feature map;

[0026] III. Calculation ;

[0027]

[0028]

[0029]

[0030] In the formula, Indicates the first A token in a neighborhood of size The expansion rate is Under the condition of "the output features after the expansion of the domain attention calculation; Indicates the scaling parameter; Indicates the embedding dimension; Indicates the size of the neighborhood; Indicates the expansion rate; Indicates the first Given a token, and assuming the input image is... , Input image There is A dimensional token vector; Let m be a real matrix with m rows and d columns; Indicates the neighborhood size as The Attention weight of each token; , It is the input image Linear transformation; The relative positional deviation between any two tokens; Indicates the first The token inflation rate is The The nearest neighbor; Indicates the first A token in a neighborhood of size The expansion rate is Under the given conditions, a matrix is ​​formed by concatenating the "value features" of neighboring tokens; Indicates the first The token inflation rate is The The transpose of the "value feature" of the nearest neighbor token;

[0031] The process of extracting the checkerboard edges involves repeatedly calculating each pixel in the image. .

[0032] Furthermore, the SCAFM decoder processes the image as follows:

[0033] I. Weights of the computation space Weights of the channel ;

[0034] calculate and Four feature maps are generated from the mean and maximum values ​​of the channel dimension, and then processed through convolution and... The operation processes the four feature maps to obtain... ;

[0035]

[0036]

[0037]

[0038] In the formula, Represents spatial attention weights. This represents the Sigmoid activation function. This indicates convolution processing. This represents the concatenation processing function. This indicates that the mean function is calculated along the channel dimension. This represents the function for calculating the maximum value in the channel dimension. High-level features of the checkerboard image representing the calibration cloth; The low-level features of the checkerboard image represent the calibration cloth. Indicates to The result of performing a convolution operation; Indicates to The results of upsampling;

[0039] Will and Perform average pooling and max pooling to generate four dimensions. The feature maps are then concatenated, and then processed by convolution and... The operation yields the result. ;

[0040]

[0041] In the formula, Indicates the channel attention weights. This represents the average pooling function. This represents the max pooling function;

[0042] II. Obtain fusion features;

[0043] The input features are combined with the generated weights through matrix multiplication and addition. and Fusion, generating features and ;Will and Concatenate and perform convolution to generate ;exist Perform convolution on top, then upsample to generate a size of of Then, , and Connect them and convolve them to obtain fused features. ;

[0044]

[0045]

[0046]

[0047]

[0048] .

[0049] Furthermore, during the image processing process of the decoder SCAFM, Upsampling with bilinear interpolation algorithm Same size.

[0050] Furthermore, step S3 includes the following sub-steps:

[0051] S3.1 calculates pixel intensity;

[0052] Based on the edge detection results obtained in step S2, pixel accumulation is performed on each row or column of the results. The accumulated result is a pixel intensity vector, which is then used... express:

[0053]

[0054] In the formula, For the first The cumulative pixel value of a row or column;

[0055] S3.2 First-order difference;

[0056] right Perform first-order difference to obtain the pixel intensity difference for each row or column;

[0057]

[0058] In the formula, This is the result after the first-order difference;

[0059] S3.3 Perform the second-order difference again to obtain the second-order difference result. ;

[0060]

[0061] Among them, if ,but As an index for local peak points, a distance limit is set during the search for local peak points. When the range is Local peak points are detected in real time.

[0062] To address the aforementioned technical problems, a second aspect of this application provides an electronic device, comprising:

[0063] One or more processors;

[0064] Memory, used to store one or more programs;

[0065] When the one or more programs are executed by the one or more processors, the one or more processors implement the chessboard corner detection method as provided in the first aspect above.

[0066] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the chessboard corner detection method provided in the first aspect above.

[0067] The beneficial effects of this application are: This invention achieves checkerboard corner detection in vehicle perception calibration scenarios through a spatial and channel image fusion method. Under environmental conditions such as shadows, reflections, and uneven brightness, it can effectively extract the edges of the checkerboard image, improving the accuracy of corner detection. By fusing spatial and channel information, the algorithm's adaptability to interference factors is significantly improved, reducing the false detection rate. Attached Figure Description

[0068] Figure 1 This is a structural diagram of the EdgeNAT network in an embodiment of the present invention;

[0069] Figure 2 This is a structural diagram of the SCAFM module in an embodiment of the present invention;

[0070] Figure 3 This is a flowchart of peak detection in an embodiment of the present invention;

[0071] Figure 4 A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0072] Figure 5 A schematic block diagram of the structure of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0073] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0074] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0075] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0076] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0077] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0078] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0079] The prior art related to this invention: CN114494279A discloses a calibration board corner detection method, device, and computer storage medium. The main technical method includes: converting an image into a YCbCr image and extracting the brightness information of each pixel; selecting a neighborhood range coordinate system, with the determination of calibration board corners based on the brightness values ​​of the neighborhood points; determining candidate points for calibration board corners, based on the brightness characteristics of the neighborhood points surrounding the corner point, where the candidate point should meet the following conditions: the two quadrants with larger brightness values ​​are diagonally opposite each other, and the two quadrants with smaller brightness values ​​are diagonally opposite each other; the difference in brightness between the candidate point and the two middle quadrants should be much greater than the difference between the two quadrants with larger brightness values, and also much greater than the difference between the two quadrants with smaller brightness values; determining the final corner coordinates from the candidate corner points based on the calibration board corner feature; and finally, verifying the detected corner coordinates to confirm their validity. This invention is not limited to the presence of distortion in the image and is not limited by the image shooting angle, has a wide range of applications, is simple to operate, and greatly reduces costs.

[0080] However, existing technologies have the following drawbacks: 1. Poor adaptability to different image conditions. Existing technologies rely on fixed brightness feature rules to detect corner points, which are prone to failure in situations with shadows, reflections, or uneven brightness. 2. Lack of utilization of channel and spatial fusion information. Existing technologies rely on the relationship between local neighborhood brightness values, utilizing only local image information. It is difficult to combine global channel and spatial fusion information to determine the authenticity of corner points, which may lead to false corner points or missed detections.

[0081] To address the aforementioned problems, this invention provides a checkerboard corner detection method for vehicle perception calibration scenarios based on spatial and channel image fusion. This invention achieves checkerboard corner detection in vehicle perception calibration scenarios through spatial and channel image fusion, solving the problem of edge extraction of checkerboard images under conditions of shadows, reflections, and uneven brightness. By combining channel and spatial fusion information, corner coordinate information is obtained, thereby improving corner detection efficiency.

[0082] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a method for detecting corner points of a chessboard grid. This embodiment includes the following steps:

[0083] S1. Preprocess the raw images captured by the vehicle camera;

[0084] S1.1 Perform grayscale processing on the images captured by the vehicle camera;

[0085] The formula for calculating grayscale value is:

[0086]

[0087] In the formula, It is a grayscale value; , , These are the values ​​for the red, green, and blue color channels of each pixel;

[0088] S1.2 Gaussian blurring is used to blur the grayscale image. The purpose of blurring is to reduce image noise and make subsequent corner detection more stable.

[0089] The formula for the Gaussian function is:

[0090]

[0091] In the formula, It represents the coordinates of a pixel in the image, relative to the center of the current pixel. It is the standard deviation, which controls the width of the Gaussian function and affects the intensity of the blur. It is the value of a two-dimensional Gaussian function, representing the weight of a given pixel.

[0092] S2. Perform edge detection on the image preprocessed in step S1;

[0093] Edge NAT is used to perform edge detection on the preprocessed image. The network employs the Review Dilated Neighborhood Attention Transformer (DiNAT) as the encoder, which preserves local information and expands the receptive field to capture long-range feature dependencies; the decoder uses a spatial and channel information fusion approach to effectively combine spatial and channel features. The EdgeNAT network structure in this embodiment is as follows: Figure 1 As shown. The EdgeNAT model includes an encoder DiNAT and a decoder SCAFM; the decoder SCAFM includes a spatial attention module SAM and a channel attention module CAM. The structure of the decoder SCAFM in this embodiment is as follows. Figure 2 As shown.

[0094] The DiNAT encoder processes images in the following ways:

[0095] I. By using the tokenizer in the DiNAT encoder, which consists of two 3x3 convolutional layers with a stride of 2, a feature map with a resolution of one-quarter of the input image is obtained;

[0096] Tokens: A set of discrete, processable regional units in an image;

[0097] Ttokenizer: Used to refer to tools or algorithms that divide input data such as text or images into smaller units;

[0098] II. By using a single 3x3 convolutional layer with a stride of 2 in the DiNAT encoder, downsampling is performed between layers, reducing the number of channels while halving the spatial resolution, ultimately yielding the following sizes: , , , Feature map;

[0099] III. Calculation ;

[0100]

[0101]

[0102]

[0103] In the formula, Indicates the first A token in a neighborhood of size The expansion rate is Under the condition of "the output features after the expansion of the domain attention calculation; Indicates the scaling parameter; Indicates the embedding dimension; Indicates the size of the neighborhood; Indicates the expansion rate; Indicates the first Given a token, and assuming the input image is... , Input image There is A dimensional token vector; Let m be a real matrix with m rows and d columns; Indicates the neighborhood size as The Attention weight of each token; , It is the input image Linear transformation; The relative positional deviation between any two tokens; Indicates the first The token inflation rate is The The nearest neighbor; Indicates the first A token in a neighborhood of size The expansion rate is Under the given conditions, a matrix is ​​formed by concatenating the "value features" of neighboring tokens; Indicates the first The token inflation rate is The The transpose of the "value feature" of the nearest neighbor token;

[0104] The process of extracting the checkerboard edges involves repeatedly calculating each pixel in the image. .

[0105] The SCAFM decoder processes images in the following ways:

[0106] I. Weights of the computation space Weights of the channel ;

[0107] calculate and Four feature maps are generated from the mean and maximum values ​​of the channel dimension, and then processed through convolution and... The operation processes the four feature maps to obtain... ;

[0108]

[0109]

[0110]

[0111] In the formula, Represents spatial attention weights. This represents the Sigmoid activation function. This indicates convolution processing. This represents the concatenation processing function. This indicates that the mean function is calculated along the channel dimension. This represents the function for calculating the maximum value in the channel dimension. High-level features of the checkerboard image representing the calibration cloth; The low-level features of the checkerboard image represent the calibration cloth. Indicates to The result of performing a convolution operation; Indicates to The result of upsampling; whereby, during the image processing by the decoder SCAFM, Upsampling with bilinear interpolation algorithm Same size;

[0112] Will and Perform average pooling and max pooling to generate four dimensions. The feature maps are then concatenated, and then processed by convolution and... The operation yields the result. ;

[0113]

[0114] In the formula, Indicates the channel attention weights. This represents the average pooling function. This represents the max pooling function;

[0115] II. Obtain fusion features;

[0116] The input features are combined with the generated weights through matrix multiplication and addition. and Fusion, generating features and ;Will and Concatenate and perform convolution to generate ;exist Perform convolution on top, then upsample to generate a size of of Then, , and Connect them and convolve them to obtain fused features. ;

[0117]

[0118]

[0119]

[0120]

[0121] .

[0122] S3. Peak Detection: Based on the edge detection results obtained in step S2, peak detection is performed to detect local peak points; the peak detection process in this embodiment of the invention is as follows: Figure 3 As shown;

[0123] S3.1 calculates pixel intensity;

[0124] Based on the edge detection results obtained in step S2, pixel accumulation is performed on each row or column of the results. The accumulated result is a pixel intensity vector, which is then used... express:

[0125]

[0126] In the formula, For the first The cumulative pixel value of a row or column;

[0127] S3.2 First-order difference;

[0128] right Perform first-order difference to obtain the pixel intensity difference for each row or column;

[0129]

[0130] In the formula, This is the result after the first-order difference;

[0131] S3.3 Perform the second-order difference again to obtain the second-order difference result. ;

[0132]

[0133] Among them, if ,but As an index for local peak points, a distance limit is set during the search for local peak points. When the range is Local peak points are detected in real time.

[0134] S4. Corner Calculation: Based on the results of peak detection, obtain the peak point vectors of rows and columns, calculate the coordinates of all corner points, and obtain the final corner point information.

[0135] By combining the indices of the "row peak point" and the "column peak point," and pairwise combining them to obtain (row index, column index), the corner coordinates are obtained. Essentially, a corner point is the intersection of points with significant peak values ​​in both the row and column directions. By filtering through the peak values ​​of each row and column, the corner point is ultimately located.

[0136] 1. The essential characteristics of corner points

[0137] Corner points are points in an image where both the row and column directions change significantly (such as the intersection of a chessboard or the corner of an object's edge).

[0138] Row direction: The pixel intensity change rate (difference) of the row where this point is located satisfies the "peak value in the same area" condition;

[0139] Column direction: The pixel intensity change rate (difference) of the column where the point is located also satisfies the "peak value in the same area" condition.

[0140] 2. Meaning of row / column peak point vectors

[0141] Row peak point vector Records the row indexes of all rows that "satisfy local peaks" (i.e., which rows may have corners);

[0142] Column peak vector: Records the column indexes of all columns that satisfy local peaks (i.e., which columns may have corners).

[0143] 3. "Combined Intersection" generates corner coordinates

[0144] Corner points must simultaneously satisfy the condition that "the row peak index and the column peak index are both high-value points". Therefore, the row peak index and the column peak index are combined pairwise to obtain the coordinates. That is: the set of corner coordinates .

[0145] Imagine the row peaks as "possible horizontal line candidates" and the column peaks as "possible vertical line candidates." Their intersection is the corner point where "both horizontal and vertical lines exist." This filters out points where "only the row changes but the column does not" (false positive row peaks) or "only the column changes but the row does not" (false positive column peaks), ensuring the bidirectional significance of the corner points.

[0146] 4. Provide examples

[0147] Assumption:

[0148] Row peak vector (Significant changes are observed in rows 2, 5, and 8);

[0149] Column peak vector (Significant changes were observed in columns 3, 6, and 9);

[0150] The corner coordinates are all combinations: (2,3), (2,6), (2,9), (5,3), (5,6), (5,9), (8,3), (8,6), (8,9). These correspond to the intersections of the horizontal lines in rows 2, 5, and 8 and the vertical lines in columns 3, 6, and 9 on the chessboard, which are the actual corner points.

[0151] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 4As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the chessboard corner detection methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0152] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0153] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0154] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0155] By fusing spatial and channel information, this invention enables a more comprehensive analysis of image features. The fusion strategy enhances the algorithm's adaptability to interference factors such as lighting variations, object reflections, and shadows. Traditional algorithms may exhibit false positives or false negatives; this invention, by combining multi-dimensional information, effectively reduces these problems, ensuring the accuracy and reliability of corner detection. This improvement not only enhances the algorithm's stability but also provides a more solid foundation for subsequent image processing and analysis.

[0156] In complex environments with shadows, reflections, and uneven brightness, traditional corner detection algorithms often struggle to identify sharp edges, leading to a significant drop in corner detection rates. This invention combines edge detection and peak detection, utilizing richer image features to effectively extract checkerboard edges even under these challenging conditions.

[0157] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the chessboard corner detection methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0158] Figure 5 This is a schematic block diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 200 stores program data 210. When the program data 210 is executed, it implements the steps of the external parameter verification methods for cameras and lidar as described in the above embodiments.

[0159] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described checkerboard corner detection method.

[0160] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0161] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0162] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0163] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0164] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0165] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0166] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0169] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for detecting corner points of a chessboard grid, characterized in that, Includes the following steps: S1. Preprocess the raw images captured by the vehicle camera; S2. Perform edge detection on the image preprocessed in step S1; S3. Based on the edge detection results obtained in step S2, perform peak detection to detect local peak points; S4. Corner Calculation: Based on the results of peak detection, obtain the peak point vectors of rows and columns, calculate the coordinates of all corner points, and obtain the final corner point information.

2. The method according to claim 1, characterized in that, Step S1 includes: S1.1 Perform grayscale processing on the images captured by the vehicle camera; The formula for calculating grayscale value is: ； In the formula, It is a grayscale value; , , These are the values ​​for the red, green, and blue color channels of each pixel; S1.2 The grayscale image is blurred using Gaussian blur; The formula for the Gaussian function is: ； In the formula, These are the coordinates of a pixel in the image; It is the standard deviation; It is the value of a two-dimensional Gaussian function, representing the weight of a given pixel.

3. The method according to claim 2, characterized in that, In step S2, the EdgeNAT model is used for edge detection. The EdgeNAT model includes an encoder DiNAT and a decoder SCAFM.

4. The method according to claim 3, characterized in that, The decoder SCAFM includes a spatial attention module and a channel attention module.

5. The method according to claim 4, characterized in that, The image processing performed by the encoder DiNAT includes the following: I. By using the tokenizer in the DiNAT encoder, which consists of two 3x3 convolutional layers with a stride of 2, a feature map with a resolution of one-quarter of the input image is obtained; II. Downsampling is performed between layers using a single 3x3 convolutional layer with a stride of 2 in the DiNAT encoder to obtain sizes of... , , , Feature map; III. Calculation ; ； ； ； In the formula, Indicates the first A token in a neighborhood of size The expansion rate is Under the condition of "the output features after the expansion of the domain attention calculation; Indicates the scaling parameter; Indicates the embedding dimension; Indicates the size of the neighborhood; Indicates the expansion rate; Indicates the first Given a token, and assuming the input image is... , Input image There is A dimensional token vector; Let m be a real matrix with m rows and d columns; Indicates the neighborhood size as The Attention weight of each token; , It is the input image Linear transformation; The relative positional deviation between any two tokens; Indicates the first The token inflation rate is The The nearest neighbor; Indicates the first A token in a neighborhood of size The expansion rate is Under the given conditions, a matrix is ​​formed by concatenating the "value features" of neighboring tokens; Indicates the first The inflation rate of each token is... The The transpose of the "value feature" of the nearest neighbor token; The process of extracting the checkerboard edges involves repeatedly calculating each pixel in the image. .

6. The method according to claim 5, characterized in that, The SCAFM decoder processes images in the following ways: I. Weights of the computation space Weights of the channel ; calculate and Four feature maps are generated from the mean and maximum values ​​of the channel dimension, and then processed through convolution and... The operation processes the four feature maps to obtain... ; ； ； ； In the formula, Represents spatial attention weights. This represents the Sigmoid activation function. This indicates convolution processing. This represents the concatenation processing function. This indicates that the mean function is calculated along the channel dimension. This represents the function for calculating the maximum value in the channel dimension. High-level features of the checkerboard image representing the calibration cloth; The low-level features of the checkerboard image represent the calibration cloth. Indicates to The result of performing a convolution operation; Indicates to The result of upsampling; Will and Perform average pooling and max pooling to generate four dimensions. The feature maps are then concatenated, and then subjected to convolution and... The operation yields the result. ; ； In the formula, Indicates the channel attention weights. This represents the average pooling function. This represents the max pooling function; II. Obtain fusion features; The input features are combined with the generated weights through matrix multiplication and addition. and Fusion, generating features and ;Will and Concatenate and perform convolution to generate ;exist Perform convolution on top, then upsample to generate a size of of Then, , and Connect them and convolve them to obtain fused features. ; ； ； ； ； 。 7. The method according to claim 6, characterized in that, During the image processing process of the decoder SCAFM Upsampling with bilinear interpolation algorithm Same size.

8. The method according to claim 7, characterized in that, Step S3 includes the following sub-steps: S3.1 calculates pixel intensity; Based on the edge detection results obtained in step S2, pixel accumulation is performed on each row or column of the results. The accumulated result is a pixel intensity vector, which is then used... express: ； In the formula, For the first The cumulative pixel value of a row or column; S3.2 First-order difference; right Perform first-order difference to obtain the pixel intensity difference for each row or column; ； In the formula, This is the result after the first-order difference; S3.3 Perform the second-order difference again to obtain the second-order difference result. ; ； Among them, if ,but As an index for local peak points, a distance limit is set during the search for local peak points. When the range is Detect local peak points in real time.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 8.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images