Corner point detection method and device, computer equipment and program product

Abstract

The invention provides a corner detection method and device, computer equipment and a program product, and the method comprises the steps: detecting each corner in a checkerboard image through employing a pre-trained corner detection network, and obtaining a first detection frame corresponding to each corner; performing image optimization operation on the detection frame images corresponding to the first detection frames to obtain optimized images; according to the gradient of each pixel point in the optimized image, screening out candidate saddle points conforming to saddle point features from each pixel point; and according to the first coordinate of each candidate saddle point in the optimized image, screening out a target corner point in the first detection frame from each candidate saddle point.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a corner point detection method, device, computer equipment and program product. Background Art

[0002] Checkerboard corner point recognition technology is one of the key technologies used in computer vision for camera calibration and geometric correction. This technology obtains feature points in the image by detecting the corner points of the checkerboard pattern, thereby achieving calibration or correction.

[0003] However, the current checkerboard corner point recognition technology has the disadvantages of low recognition accuracy and poor recognition effect, especially when the checkerboard image quality is low (such as image blur, ghosting, noise, checkerboard missing), this disadvantage will be more obvious. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a corner point detection method, apparatus, computer equipment and program product.

[0005] In a first aspect, an embodiment of the present disclosure provides a corner point detection method, including:

[0006] Using a pre-trained corner point detection network, detect each corner point in the checkerboard image to obtain a first detection frame corresponding to each corner point;

[0007] performing an image optimization operation on the detection frame images respectively corresponding to the first detection frames to obtain optimized images;

[0008] According to the gradient of each pixel point in the optimized image, a candidate saddle point that meets the saddle point characteristics is selected from each pixel point;

[0009] According to the first coordinate of each candidate saddle point in the optimized image, the target corner point in the first detection frame is screened out from each candidate saddle point.

[0010] In a possible implementation, selecting candidate saddle points that meet saddle point characteristics from each pixel point according to the gradient of each pixel point in the optimized image includes:

[0011] According to the gradient of each pixel point in the optimized image, two eigenvalues ​​corresponding to each pixel point in the optimized image under the characteristic equation are determined; the eigenvalues ​​are used to characterize the curvature characteristics of the pixel point;

[0012] According to the positive and negative signs of the two eigenvalues, candidate saddle points that meet the saddle point characteristics are screened out from the pixel points.

[0013] In a possible implementation, determining, according to the gradient of each pixel point in the optimized image, the eigenvalue corresponding to each pixel point in the optimized image under the characteristic equation includes:

[0014] Determine, according to the gradient of each pixel point in the optimized image, a preset matrix representation corresponding to each pixel point;

[0015] According to the preset matrix representation corresponding to each pixel point, construct the characteristic equation corresponding to each pixel point respectively;

[0016] According to the solution of the characteristic equation, two eigenvalues ​​corresponding to each pixel point under the characteristic equation are determined.

[0017] In a possible implementation manner, performing an image optimization operation on the detection frame images corresponding to the first detection frames to obtain an optimized image includes:

[0018] Determining a plurality of filtering scales for performing image filtering on the detection frame image;

[0019] According to the plurality of filtering scales, the detection frame images are respectively subjected to image filtering processing to obtain a plurality of first filtering images, and a second filtering image is determined based on the plurality of first filtering images;

[0020] determining an adaptive threshold of the second filtered image;

[0021] An image enhancement parameter is determined according to the adaptive threshold, and the image contrast of the second filtered image is adjusted using the image enhancement parameter to obtain an optimized image.

[0022] In a possible implementation, the filtering out target corner points in the first detection frame from each of the candidate saddle points according to the first coordinate of each of the candidate saddle points in the optimized image includes:

[0023] Performing sub-pixel optimization on the first coordinate of the candidate saddle point by interpolation to obtain a second coordinate of the candidate saddle point;

[0024] The target corner point is screened out from the candidate saddle points by using the second coordinate of each candidate saddle point and adopting a non-maximum suppression method.

[0025] In a possible implementation manner, after selecting the target corner point in the first detection frame from the candidate saddle points, the method further includes:

[0026] Taking each target corner point in the first detection frame as a node, and using the third coordinate of each node in the checkerboard image to calculate the distance between every two nodes;

[0027] According to the distance between every two nodes, a minimum spanning tree composed of each of the nodes is generated, and according to the minimum spanning tree, a node feature matrix and a node adjacency matrix are determined; the node feature matrix is ​​used to characterize the initial node features of each of the nodes, and the node adjacency matrix is ​​used to characterize the location features and neighbor node information of each of the nodes;

[0028] According to the node feature matrix and the node adjacency matrix, the tree structure corresponding to the minimum spanning tree is optimized, and according to the optimized minimum spanning tree, the target corner points corresponding to each of the nodes are sorted to obtain the sorting order of each of the target corner points.

[0029] In a possible implementation, optimizing the tree structure corresponding to the minimum spanning tree according to the node feature matrix and the node adjacency matrix includes:

[0030] Using a pre-trained graph neural network, according to the node feature matrix and the node adjacency matrix, an embedded feature of each of the nodes is determined; the embedded feature is used to characterize the target node feature of the node and the position and structure in the minimum spanning tree;

[0031] Using a pre-trained reinforcement learning network, according to the embedded features of each of the nodes, determine an optimized feature matrix corresponding to the node feature matrix and an optimized adjacency matrix corresponding to the node adjacency matrix;

[0032] An optimized minimum spanning tree is determined according to the optimized feature matrix and the optimized adjacency matrix.

[0033] In a possible implementation, the using of a pre-trained graph neural network to determine the embedding feature of each of the nodes according to the node feature matrix and the node adjacency matrix includes:

[0034] Using the graph convolutional network module of the graph neural network, the initial node feature of each node is updated according to the position feature and neighbor node information of each node to obtain a first updated feature;

[0035] Using the graph attention network module of the graph neural network, assigning an attention weight to a neighbor node of each of the nodes according to the first updated feature, and updating the first updated feature of the node based on the attention weight to obtain a second updated feature;

[0036] The graph isomorphism network module of the graph neural network is used to linearly combine the second updated features of each node and the second updated features of the neighboring nodes of the node to obtain the embedded features of each node.

[0037] In a possible implementation, the method of using a pre-trained reinforcement learning network to determine, according to the embedded features of each of the nodes, an optimized feature matrix corresponding to the node feature matrix and an optimized adjacency matrix corresponding to the node adjacency matrix includes:

[0038] Using the reinforcement learning network, according to the embedded features of each of the nodes, determining a current node to be processed;

[0039] Determine the action to be performed on the current node according to the state information corresponding to the current node; the state information includes at least one of the global structure information corresponding to the current minimum spanning tree, the processing order of the current node, the current operation progress and the historical action; wherein, when the current node is the first node, the current minimum spanning tree is the minimum spanning tree determined by the embedded features of each node;

[0040] Optimizing the embedding feature of the current node according to the action to obtain an optimized embedding feature;

[0041] Returning to the step of determining the current node to be processed according to the embedding features of each of the nodes, until the optimized embedding features of each of the nodes are obtained;

[0042] The optimized feature matrix and the optimized adjacency matrix are determined according to the optimized embedded features.

[0043] In a possible implementation, the action includes at least one of the following:

[0044] Select the next neighbor node of the current node, modify the node information of the current node, modify or reallocate the neighbor node information of the current node, backtrack one step forward, or initialize the current node.

[0045] In a possible implementation, the checkerboard image includes a target checkerboard with a target mark set thereon; the recognition result of the corner point detection network also includes a second detection frame corresponding to the target checkerboard; and the position of the second detection frame is obtained according to the target mark recognition in the target checkerboard;

[0046] According to the optimized minimum spanning tree, the target corner points corresponding to each node are sorted to obtain the sorting order of each target corner point, including:

[0047] Taking any corner point of the four target corner points corresponding to the second detection frame as a starting node, and assigning a starting grid coordinate to the starting node;

[0048] Starting from the starting node, traversing each other node in turn according to the neighbor nodes of each node indicated by the optimized minimum spanning tree;

[0049] For any of the other nodes, assign a target grid coordinate to the other node according to the third coordinate of the other node in the checkerboard image, the third coordinate of the first traversed node among the neighboring nodes of the other node, and a preset coordinate deviation angle;

[0050] The sorting order of each of the target corner points is determined according to the starting grid coordinates and the target grid coordinates.

[0051] In a second aspect, an embodiment of the present disclosure further provides a corner point detection device, including:

[0052] A detection module, used to detect each corner point in the checkerboard image using a pre-trained corner point detection network, and obtain a first detection frame corresponding to each corner point;

[0053] an optimization module, configured to perform an image optimization operation on the detection frame images respectively corresponding to the first detection frames to obtain optimized images;

[0054] A first screening module, used for screening out candidate saddle points that meet the saddle point characteristics from each pixel point in the optimized image according to the gradient of each pixel point;

[0055] The second screening module is used to screen out target corner points in the first detection frame from each of the candidate saddle points according to the first coordinate of each of the candidate saddle points in the optimized image.

[0056] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0057] In a fourth aspect, an optional implementation of the present disclosure further provides a computer program product, including a computer program, which, when executed, implements the above-mentioned first aspect, or the steps in any possible implementation of the first aspect.

[0058] The corner point detection method, apparatus, computer equipment and program product provided by the embodiments of the present disclosure can first detect the approximate position corresponding to each checkerboard corner point in the checkerboard image, that is, the position of the first detection frame, by using a trained corner point detection network. Then, by performing a variety of image optimization operations on the detection frame image, the image quality can be effectively improved and image noise can be eliminated. Since the pixel gradient can reflect edge information, texture information, detail information and noise information, the gradient of the pixel point in the optimized image obtained after optimization is used to identify the saddle point, which can effectively screen out candidate saddle points that may be corner points. Combined with the first coordinate of the candidate saddle point, the target corner point that best meets the corner point characteristics can be selected from multiple candidate saddle points, thereby realizing the accurate identification of the corner point position in the detection frame image and completing the precise positioning of the corner point.

[0059] Furthermore, the corner point detection method, apparatus, computer equipment and program product provided by the embodiments of the present disclosure can sort each target corner point based on the minimum spanning tree, optimize the connection order between the target corner points, and assign reasonable grid coordinates to each target corner point, thereby achieving reasonable sorting of the target corner points.

[0060] Furthermore, the corner point detection method, apparatus, computer equipment and program product provided by the embodiments of the present invention optimize the tree structure of the minimum spanning tree by combining a graph neural network and a reinforcement learning network to deal with conflicts and loops that may arise when constructing a minimum spanning tree for some complex checkerboards, resulting in unreasonable situations in the generated minimum spanning tree. This can effectively eliminate conflicts and loops in the tree structure, ensure the rationality of the tree structure, and thus facilitate the subsequent allocation of accurate grid coordinates to nodes.

[0061] For a description of the effects of the above-mentioned corner point detection device, computer equipment, and computer program product, please refer to the description of the above-mentioned corner point detection method, which will not be repeated here.

[0062] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0064] Figure 1 A flowchart of a corner point detection method provided by an embodiment of the present disclosure is shown;

[0065] Figure 2 A schematic diagram of an original chessboard picture provided by an embodiment of the present disclosure is shown;

[0066] Figure 3 A schematic diagram of a labeling result provided by an embodiment of the present disclosure is shown;

[0067] Figure 4 A schematic diagram of a recognition result of a chessboard image provided by an embodiment of the present disclosure is shown;

[0068] Figure 5 A schematic diagram showing another recognition result of a chessboard image provided by an embodiment of the present disclosure is shown;

[0069] Figure 6 A schematic diagram showing another recognition result of a chessboard image provided by an embodiment of the present disclosure is shown;

[0070] Figure 7 A schematic diagram showing another recognition result of a chessboard image provided by an embodiment of the present disclosure is shown;

[0071] Figure 8 A schematic diagram showing a saddle shape of a local image intensity provided by an embodiment of the present disclosure;

[0072] Fig. 9 A schematic diagram of a three-dimensional saddle point drawing provided by an embodiment of the present disclosure is shown;

[0073] Fig.10 A schematic diagram showing an ideal chessboard boundary provided by an embodiment of the present disclosure is shown;

[0074] Fig.11 A schematic diagram showing an actual chessboard boundary provided by an embodiment of the present disclosure is shown;

[0075] Fig.12 A schematic diagram of optimizing candidate saddle points and target corner points in an image provided by an embodiment of the present disclosure is shown;

[0076] Fig.13 A schematic diagram of a minimum spanning tree provided by an embodiment of the present disclosure is shown;

[0077] Fig.14 A schematic diagram of grid coordinate allocation provided by an embodiment of the present disclosure is shown;

[0078] Fig.15 A schematic diagram of grid coordinates of each target corner point in a chessboard image provided by an embodiment of the present disclosure is shown;

[0079] Fig.16 The figure shows a schematic diagram of a corner detection device provided by an embodiment of the present disclosure;

[0080] Fig.17 The figure shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. Detailed implementation manners

[0081] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only some of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed, but merely represents selected embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative efforts fall within the scope of protection of the present disclosure.

[0082] In addition, terms such as "first" and "second" in the description and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that shown or described herein.

[0083] As used herein, "a plurality of or several" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0084] It has been found through research that traditional corner recognition methods represented by opencv can identify corners from checkerboard images. However, this method can often only process clear and complete high-quality images. For some images with poor quality, its recognition effect will be greatly reduced, and the problem of being unable to identify all corners will occur, and the accuracy cannot be guaranteed. To address the corner recognition of images with poor quality, some methods using deep learning technology for recognition have emerged. Although this method can identify corners from images with poor quality, it cannot accurately locate the corners, and the accuracy is poor.

[0085] Based on the above research, the present disclosure provides a corner detection method, apparatus, computer device, and program product. By using a trained corner detection network, it is first possible to detect the approximate positions corresponding to the respective checkerboard corners in the checkerboard image, that is, the positions where the first detection frames are located. Then, by performing various image optimization operations on the detection frame images, the image quality can be effectively improved and image noise can be eliminated. Since the pixel point gradients can reflect edge information, texture information, detail information, and noise information, using the gradients of the pixel points in the optimized image obtained after optimization for saddle point recognition can effectively screen out candidate saddle points that may be used as corners. Combining with the first coordinates of the candidate saddle points, the target corner that most conforms to the corner characteristics can be selected from multiple candidate saddle points, thereby realizing the identification of the accurate corner positions in the detection frame image and completing the precise positioning of the corners.

[0086] Regarding the defects existing in the above solutions, they are all the results obtained by the inventors through practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by the present disclosure for the above problems in the following text should both be the contributions made by the inventors to the present disclosure during the process of the present disclosure.

[0087] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0088] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users and user authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0089] To facilitate the understanding of this embodiment, first, a corner detection method disclosed in the embodiments of the present disclosure will be introduced in detail. The execution subject of the corner detection method provided by the embodiments of the present disclosure is generally a terminal device or other processing device with certain computing capabilities. Among them, the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant device (PDA), a handheld device, a computer device, etc.; in some possible implementation manners, the corner detection method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0090] Next, the corner detection method provided by the embodiments of the present disclosure will be described by taking the execution subject as a computer device as an example.

[0091] As Figure 1 shown, it is a flowchart of a corner detection method provided by the embodiments of the present disclosure, which may include the following steps:

[0092] S101: Detect each corner point in the checkerboard image by using a pre-trained corner detection network, and obtain a first detection box corresponding to each corner point.

[0093] Here, the corner detection network can be a neural network based on YOLOv8. As an efficient object detection and image segmentation model in computer vision, YOLOv8 is the latest version of the You Only Look Once (YOLO) series. In application, a large number of sample checkerboard images and labeled image tags can be used to iteratively train the YOLOv8 network to obtain a trained corner detection network.

[0094] Among them, the sample checkerboard images can present regular shapes and irregular shapes. Among them, the regular shapes correspond to various high-quality checkerboard images, and the irregular shapes correspond to various low-quality checkerboard images. High-quality checkerboard images can be, for example, images including complete and clear checkerboards; low-quality checkerboard images can be, for example, checkerboard images taken in various noise environments, images with partial checkerboards missing, images with checkerboards covering the entire picture, etc. Various noise environments can be, for example, noises such as different brightness, different contrast, and partial occlusion. The image tags can be the positions and types of each corner point in the image. Among them, the corner position tags can be marked with sample detection boxes, and the corner types can include ordinary corner types and starting corner types; among them, the starting corner point can be any corner point corresponding to the detection box containing the target mark in the checkerboard image; the ordinary corner point is each corner point other than the starting corner point. The role of the starting corner point will be described in detail when introducing corner sorting later.

[0095] The sample checkerboard images can be obtained by photographing the original checkerboard pictures. In specific applications, the checkerboard images used for detection are also obtained by photographing the original checkerboard pictures. A target mark can be set on the target checkerboard in the original checkerboard picture. Among them, the target checkerboard can be any checkerboard, such as the central checkerboard; the target mark can have any shape, such as a circle, a hexagon, a triangle, etc.; the color of the target mark is different from the color of the target checkerboard. For example, the target checkerboard is black and the target mark is white. As Figure 2 shown, it is a schematic diagram of an original checkerboard picture provided by an embodiment of the present disclosure. Among them, in the target checkerboard at the central position of the original checkerboard picture, a circular target mark is set. The original checkerboard picture is placed in the actual application environment and photographed at various angles and various brightnesses to obtain various sample checkerboard images. When enough sample checkerboard images are collected, the positions and types of the checkerboard corner points in the sample checkerboard images are labeled. As Figure 3As shown in the figure, it is a schematic diagram of the annotation result provided by an embodiment of the present disclosure. Among them, each corner detection box is used to identify the corner position, where "small" indicates that the corner is an ordinary corner, and "big" indicates that the corner is a starting corner. The target checkerboard containing the target marker is also marked with a detection box, and the color of this detection box can be different from that of the corner detection box. The detection box of the target checkerboard usually corresponds to four ordinary corners, and any one of the corners can be marked as the starting corner. In Figure 3 the ordinary corner corresponding to the upper left corner of the detection box of the target checkerboard is marked as the starting corner.

[0096] For each sample checkerboard image, the image label corresponding to this image can be marked in a manual annotation manner as shown in Figure 3 the figure. Then, the YOLOv8 network can be iteratively trained using the sample checkerboard image and the image label until the pre-set training cut-off condition is met, and a trained corner detection network is obtained. Among them, the trained corner detection network can perform corner recognition on checkerboard images of any quality collected in any environment, and output the first detection box corresponding to each corner and its confidence, the second detection box corresponding to the target checkerboard and its confidence, and the corner type of each corner. Among them, the first detection box can be used to indicate the approximate position of the corner in the checkerboard image, and the second detection box is used to indicate the position of the target checkerboard. The specific use of the second detection box and the corner type will be described in detail when introducing corner sorting later.

[0097] As Figure 4 shown in the figure, it is a schematic diagram of the recognition result of a checkerboard image provided by an embodiment of the present disclosure. Among them, Figure 4 it is a schematic diagram of the recognition result of corner recognition for a checkerboard image taken in a darker environment. As Figure 5 shown in the figure, it is another schematic diagram of the recognition result of a checkerboard image provided by an embodiment of the present disclosure. Among them, Figure 5 it is a schematic diagram of the recognition result of corner recognition for a checkerboard image taken in a darker and noisy (the noise is specifically horizontal stripe noise) environment. As Figure 6 shown in the figure, it is another schematic diagram of the recognition result of a checkerboard image provided by an embodiment of the present disclosure. Among them, Figure 6 it is a schematic diagram of the recognition result of corner recognition for a checkerboard image with part of the checkerboard occluded. As Figure 7 shown in the figure, it is another schematic diagram of the recognition result of a checkerboard image provided by an embodiment of the present disclosure. Among them, Figure 7 it is a schematic diagram of the recognition result of corner recognition for a checkerboard image with ghosting (specifically, there is ghosting on the left side of the image) and corners covering the entire area of the image.

[0098] Based on Figures 4 to 7It can be seen that for the corner detection network provided by the embodiments of the present disclosure, regardless of the brightness or darkness of the checkerboard image to be recognized, whether it is affected by horizontal or vertical stripe noise, whether there is partial occlusion of some checkerboards, whether there is a ghosting problem, whether the checkerboard is complete, or whether the checkerboard fills the entire image, it can accurately identify each corner existing in the checkerboard image.

[0099] Exemplarily, for any checkerboard image captured in an actual environment, the pre-trained corner detection network can be used to perform corner recognition on the checkerboard image to obtain the first detection box corresponding to each corner, the second detection box corresponding to the target marker, and the corner type of each corner. Among them, the position of the first detection box or the second detection box can be indicated by the coordinates of the points corresponding to the four corners of the detection box in the checkerboard image.

[0100] After obtaining the first detection box, the steps of S102 to S104 described later can be used to further determine the corner position within the detection box to obtain an accurate corner position. In the checkerboard image, the local image intensity around each corner can present a saddle shape as shown in Figure 8 shown, and this shape can be characterized by a three-dimensional drawing method. As shown in Fig. 9 shown, it is a schematic diagram of a three-dimensional drawn saddle point provided by the embodiments of the present disclosure. However, due to the fact that in the actual application process, the boundaries of each checkerboard in the checkerboard image may not be particularly clear and distinct, and will not present the ideal boundary situation as shown in Fig.10 shown, but will be affected by the ambient light and picture blur degree of the actual shooting, and may present the situation as shown in Fig.11 shown. Therefore, to enhance the corner recognition accuracy under blurred boundaries, the embodiments of the present disclosure provide an enhanced saddle point algorithm as follows in S102 to S105 to achieve accurate detection of corners. Among them, in Figure 8 to Figure 11 , x and y respectively represent the horizontal axis and the vertical axis.

[0101] S102: Perform image optimization operations on the detection box images respectively corresponding to each first detection box to obtain optimized images.

[0102] According to various situations that may occur when actually shooting checkerboard images, various image optimization operations are preset. For example, various image filtering operations and image enhancement operations are preset to eliminate the influence of image brightness, various noises in the image, etc. on the recognition of the corner position.

[0103] During specific implementation, detection box images corresponding to each first detection box can be segmented from the checkerboard image according to the positions of the first detection boxes corresponding to each corner point. For any detection box image, one or more image optimization operations can be performed on the image to obtain an optimized image. Among them, the image optimization operations required for each detection box image can include various preset optimization operations, or one or more optimization operations can be selected from the preset optimization operations according to the actual situation of each detection box image for optimization processing. For example, if the brightness of the detection box image meets the preset brightness, the brightness optimization operation may not be performed; if it does not meet the preset brightness, the brightness optimization operation is performed.

[0104] S103: According to the gradients of the pixel points in the optimized image, candidate saddle points that meet the saddle point characteristics are screened out from the pixel points.

[0105] Here, the physical essence of the pixel gradient is the spatio-temporal representation of the brightness or color change in the image. It reveals key features such as object boundaries, texture details, and illumination changes in the image. As the basis for feature extraction, it plays an important role in object detection. In the embodiments of the present disclosure, by using the pixel gradient to assist in corner detection, saddle points that may be corners can be effectively detected from the pixel points.

[0106] A saddle point is a special point in the two-dimensional space of the image intensity (brightness or color) function, and its mathematical essence comes from the extreme value theory of multivariate functions. The physical meaning of the saddle point is closely related to the local geometric structure of the image and usually appears in areas with complex texture intersections or multi-directional gradient changes. Therefore, by first screening out candidate saddle points that meet the saddle point characteristics, pixel points with alternating horizontal and vertical gradient changes can be effectively screened out, and such pixel points are often the positions of the corners in the checkerboard image.

[0107] Since in the checkerboard image, the saddle point is often the position where the black and white squares intersect, which is also the corner position, accurate corners can be detected from the detection box image by detecting the saddle point.

[0108] During specific implementation, the gradient of each pixel point in the optimized image can be calculated, and then it can be determined whether there is a complex texture intersection or multi-directional gradient change at the pixel point according to the gradient, so as to determine whether the pixel point can be used as a candidate saddle point. Among them, when the quality of the checkerboard image is high, there may be only one candidate saddle point detected from the optimized image, and this point is the corner; when the quality of the checkerboard image is low, there may be multiple candidate saddle points detected from the optimized image, and further methods described later need to be used to further screen out the corners.

[0109] S104: According to the first coordinates of each candidate saddle point in the optimized image, target corners in the first detection box are screened out from the candidate saddle points.

[0110] In specific implementation, if there is only one candidate saddle point detected from the optimized image, this saddle point can be directly used as the target corner point. If there are multiple candidate saddle points detected from the optimized image, then each candidate saddle point can be detected according to its first coordinate in the optimized image to determine a candidate saddle point that is most likely to be the target corner point.

[0111] Exemplarily, according to the first coordinate of each candidate saddle point in the optimized image, a candidate saddle point whose position is closest to the center point of the optimized image can be selected as the target corner point. Or, according to the first coordinates of each candidate saddle point in the optimized image, the mean coordinate can be determined, and the candidate saddle point closest to this mean coordinate can be used as the target corner point.

[0112] In this way, using the trained corner detection network, first, the approximate positions corresponding to each checkerboard corner point in the checkerboard image can be detected, that is, the positions where the first detection frames are located. Then, by performing various image optimization operations on the detection frame images, the image quality can be effectively improved and image noise can be eliminated. Since the pixel point gradient can reflect edge information, texture information, detail information, and noise information, using the gradient of the pixel points in the optimized image obtained after optimization for saddle point recognition can effectively screen out candidate saddle points that may be corner points. Combining with the first coordinates of the candidate saddle points, the target corner point that best conforms to the corner point characteristics can be selected from multiple candidate saddle points, so as to accurately identify the corner point position in the detection frame image and complete the precise positioning of the corner point.

[0113] In one embodiment, in image processing and computer vision, the Hessian matrix is a core tool for describing the local second-order derivative information of an image and can be used to analyze the curvature characteristics of the brightness or color change in the image. It plays an important role in feature point recognition (such as blobs, corner points, saddle points). Therefore, the embodiments of the present disclosure can, based on the gradient and the Hessian matrix, analyze the eigenvalues of pixel points under a specific characteristic equation, and then determine whether the pixel points conform to the saddle point characteristics according to the eigenvalues. Specifically, the above S103 can be implemented according to the following steps:

[0114] S103-1: Determine two eigenvalues corresponding to each pixel point in the optimized image under the characteristic equation according to the gradient of each pixel point in the optimized image; the eigenvalues are used to characterize the curvature characteristics of the pixel points.

[0115] Here, the characteristic equation can be a preset equation, and the solutions of this equation become the eigenvalues of the pixel points. The magnitude and sign of the eigenvalues can represent the curvature characteristics of the pixel points in the vertical and horizontal directions.

[0116] In specific implementation, for any pixel point, a characteristic equation corresponding to the pixel point can be constructed according to the gradient of the pixel point, and the equation can be solved to obtain two solutions of the equation, and these two solutions are used as two eigenvalues of the pixel point under the characteristic equation.

[0117] S103-2: According to the positive and negative signs of the two eigenvalues, candidate saddle points that meet the saddle point characteristics are screened out from each pixel point.

[0118] In specific implementation, according to the positive and negative signs of the two eigenvalues and the positive and negative signs that the eigenvalues of the pixel points meeting the saddle point characteristics should have, it can be determined whether the pixel points meet the saddle point characteristics.

[0119] In one embodiment, the above S103-1 can be implemented according to the following steps:

[0120] S103-1-1: According to the gradients of the pixel points in the optimized image, determine the corresponding preset matrix representations of each pixel point.

[0121] Here, the preset matrix representation can be the Hessian matrix, where the Hessian matrix can be determined according to the following formula 1:

[0122]

[0123] Among them, H represents the Hessian matrix, and f xx represents the second-order derivative of the pixel point in the x direction, and f xy represents the mixed second-order derivative of the pixel point in the x direction and the y direction, and f yy represents the second-order derivative of the pixel point in the y direction, where the above various second-order derivatives can be determined according to the change rate of the gradient of the pixel point.

[0124] S103-1-2: According to the preset matrix representations corresponding to each pixel point, construct the characteristic equations corresponding to each pixel point.

[0125] In specific implementation, the characteristic equation can be |H - λI| = 0, where λ represents the solution of the equation, that is, the eigenvalue to be solved later, and I represents the identity matrix.

[0126] S103-1-3: According to the solutions of the characteristic equation, determine the two eigenvalues corresponding to each pixel point under the characteristic equation.

[0127] In specific implementation, the solutions of the characteristic equation can be used to solve the two eigenvalues λ1 and λ2 corresponding to the characteristic equation based on the following formula 2:

[0128]

[0129] After obtaining the two eigenvalues, if λ1×λ2 < 0, it indicates that one eigenvalue is positive and one is negative, showing that the pixel conforms to the saddle point feature, and the pixel is taken as a candidate saddle point; if both λ1 and λ2 are positive, it means that the pixel is a local minimum value and it cannot be used as a candidate saddle point; if both λ1 and λ2 are negative, it means that the pixel is a local maximum value and it also cannot be used as a candidate saddle point. Therefore, for any pixel, it can be determined whether the pixel can be used as a saddle point according to the positive and negative signs of the two eigenvalues of the pixel under the corresponding characteristic equation.

[0130] In one embodiment, the above S102 can be implemented according to the following steps:

[0131] S102-1: Determine multiple filtering scales for image filtering of the detection frame image.

[0132] Here, the image filtering method in the embodiments of the present disclosure can be Gaussian filtering. The multiple filtering scales can be preset scales, or can be partial scales selected from multiple preset scales according to the image information of the detection frame image. For example, according to the pixel average value in the grayscale image of the detection frame image, partial filtering scales are selected from multiple preset scales. The filtering scales of different detection frame images can be the same or different. For example, if there are 5 preset scales, these 5 filtering scales can be used as the filtering scales of each detection frame image. Or, partial filtering scales can also be selected for the detection frame image from the 5 filtering scales according to the image information of each detection frame image. For example, 3 filtering scales are selected for detection frame image 1, 2 filtering scales are selected for detection frame image 2, and 4 filtering scales are selected for detection frame image 3.

[0133] Exemplarily, multiple filtering scales can be preset in advance and used as the filtering scales corresponding to the detection frame image.

[0134] S102-2: Perform image filtering processing on the detection frame image according to the multiple filtering scales respectively to obtain multiple first filtered images, and determine a second filtered image according to the multiple first filtered images.

[0135] Specifically, the form of the Gaussian function G can be as shown in the following formula three:

[0136]

[0137] Among them, x and y respectively represent the abscissa and ordinate of a pixel point in the detection box image, and σ is the standard deviation, which determines the scale or degree of blurring of the Gaussian filter. The present disclosure realizes multi-scale filtering processing of the detection box image by using different σ values. Among them, different σ values will produce different degrees of smoothing effects. For example, a small σ value retains more image details, while a large σ value blurs more details and only retains the general structure of the image. By filtering the detection box image at different scales, multi-level features of the image can be effectively extracted, thereby enhancing the flexibility and robustness of image analysis.

[0138] In specific implementation, based on the coordinates of each pixel point in each detection box image, different σ values can be used to perform different Gaussian filtering processes on the detection box image respectively, obtaining multiple first filtered images. Among them, the number of first filtered images is related to the number of σ values used. For example, if the σ values include σ1, σ2, and σ3, the detection box image can be filtered using σ1 to obtain a first filtered image, filtered using σ2 to obtain a first filtered image, and filtered using σ3 to obtain a first filtered image. In this way, through three filtering scales, three first filtered images can be obtained.

[0139] After obtaining multiple filtered images, a second filtered image can be determined according to the parameter mean values of each pixel point in the multiple first filtered images. Alternatively, a second filtered image can also be selected from the multiple first filtered images according to features such as the image grayscale map of each first filtered image.

[0140] S102-3: Determine the adaptive threshold of the second filtered image.

[0141] Here, the embodiments of the present disclosure can use the method of adaptive threshold to process the local contrast change of the image to eliminate the influence of different illumination conditions and contrast changes on corner point detection.

[0142] Among them, the adaptive threshold can be determined using the following formula four:

[0143] T(x, y) = μ(x, y) + k·σ(x, y); (Formula four)

[0144] Among them, T(x, y) represents the adaptive threshold of point (x, y), μ(x, y) represents the local mean of point (x, y), where the size of the local range can be set according to experience and will not be specifically limited here, σ(x, y) represents the standard deviation of point (x, y), and k is a preset adjustment parameter.

[0145] A pixel point in the second filtered image corresponds to an adaptive threshold. When the local range is the entire second filtered image, the local mean and the adaptive threshold of each pixel point can be the same. Therefore, a second filtered image can have only one adaptive threshold for adjusting the image contrast.

[0146] Specifically, the adaptive threshold of each pixel point can be determined according to the above formula four, and then the adaptive threshold of the second filtered image can be determined according to the adaptive threshold of each pixel point.

[0147] S102-4: Determine the image enhancement parameter according to the adaptive threshold, and use the image enhancement parameter to adjust the image contrast of the second filtered image to obtain an optimized image.

[0148] In specific implementation, the image enhancement method can be, for example, histogram equalization, histogram specification, gamma correction, logarithmic and exponential transformation, local contrast enhancement, etc.

[0149] In specific implementation, the image enhancement parameter matching the selected image enhancement method can be determined according to the adaptive threshold of the second filtered image. For example, the parameters of methods such as histogram equalization are determined according to the adaptive threshold. Then, the second filtered image can be processed by image enhancement using the image enhancement parameter to obtain an optimized image. In this way, through the adaptive threshold, the contrast of the image can be enhanced, making the saddle points in the image more obvious, providing support for subsequent accurate positioning of the saddle points.

[0150] In one embodiment, the above S104 can be implemented according to the following steps:

[0151] S104-1: Optimize the first coordinate of the candidate saddle point at the sub-pixel level through interpolation to obtain the second coordinate of the candidate saddle point.

[0152] In specific implementation, a sub-pixel accuracy optimization method can be used to optimize the detected candidate saddle point at the sub-pixel accuracy, so as to obtain a more accurate coordinate corresponding to the candidate saddle point, that is, the second coordinate.

[0153] Sub-pixel accuracy optimization, as an image processing technique, aims to determine the positions of feature points (such as corner points and saddle points) to a finer level than pixels through mathematical interpolation methods. Usually, the feature detection of digital images can only be carried out at the pixel level, that is, the positions of feature points are represented by integer pixel coordinates. However, in many applications, such as computer vision, image processing, or augmented reality, accurately positioning feature points is crucial. Therefore, in the embodiments of the present disclosure, by positioning the coordinates of candidate saddle points to the sub-pixel level, the accuracy of corner detection can be effectively improved. Among them, the pixel level can be understood as: each pixel point in the optimized image has an integer coordinate representing its position in the two-dimensional image matrix; the sub-pixel level can be understood as: the position of the pixel point is represented by a floating-point number, allowing it to be located between pixels, thereby providing higher positioning accuracy.

[0154] The following takes the quadratic interpolation method as an example to illustrate the process of sub-pixel level optimization of the first coordinate of candidate saddle points:

[0155] 1. Select a local window, specifically, from the optimized image, according to the first coordinate of the candidate saddle point, select a window of a set size centered on the candidate saddle point, such as a 3×3 or 5×5 local window;

[0156] 2. Fit a quadratic surface, specifically, within the selected local window, use the pixel intensities of pixel points to fit a quadratic surface. For a two-dimensional optimized image, the fitted surface can be represented by the following formula five:

[0157] f(x, y) = ax 2 + by 2 + cxy + dx + ey + f; (Formula Five)

[0158] Among them, (x, y) represents the coordinates of the pixel point, and a, b, c, d, e, f can be determined according to the first coordinates of each pixel point within the selected local window.

[0159] 3. Solve the extreme point, specifically, calculate the partial derivatives of the fitted quadratic surface to obtain the coordinates of the extreme point of the quadratic equation corresponding to the quadratic surface, and use the extreme point coordinates as the second coordinates of the candidate saddle point. Specifically, the extreme point coordinates can be determined according to the following formula six:

[0160]

[0161] where, x max represents the abscissa of the extreme point, and y max represents the ordinate of the extreme point.

[0162] S104-2: Use the second coordinates of each candidate saddle point and adopt the non-maximum suppression method to screen out the target corner points from the candidate saddle points.

[0163] In specific implementation, non-maximum suppression can be performed on each candidate saddle point according to the second coordinate of each candidate saddle point to determine the candidate saddle point corresponding to the local maximum value to be retained as the target corner point.

[0164] In this way, by combining multi-scale Gaussian filtering, feature extraction, adaptive thresholding and image enhancement, sub-pixel accuracy optimization and non-maximum suppression, the noise in the checkerboard image can be effectively eliminated to more accurately detect the target corner points in checkerboard images of various qualities.

[0165] As Fig.12 shown, it is a schematic diagram of optimizing candidate saddle points and target corner points in an image provided by an embodiment of the present disclosure. A total of 3 candidate saddle points appear in Fig.12 wherein the candidate saddle point in the middle position is determined as the target corner point.

[0166] In one embodiment, since the sorting order of the target corner points in the image is required during the camera calibration process, the embodiment of the present disclosure also provides a method for sorting target corner points. Specifically, after screening out the target corner points in the first detection frame from each candidate saddle point, the target corner points in the checkerboard image can be sorted according to the following steps A1 to A3:

[0167] A1: Take the target corner points in each first detection frame as nodes, and calculate the distance between every two nodes by using the third coordinate of each node in the checkerboard image.

[0168] To further implement the sorting of each corner point in the checkerboard image, the embodiment of the present disclosure also provides a method based on minimum spanning tree and path optimization. Specifically, the target corner points in each first detection frame can be taken as nodes respectively. According to the third coordinate corresponding to each node in the checkerboard image, the Euclidean distance between every two nodes is calculated respectively. Among them, the third coordinate of the target corner point can be determined according to the second coordinate of the target corner point in the optimized image and the relative position relationship between the detection frame image corresponding to the optimized image and the checkerboard image.

[0169] A2: Generate a minimum spanning tree composed of each node according to the distance between every two nodes, and determine a node feature matrix and a node adjacency matrix according to the minimum spanning tree; the node feature matrix is used to characterize the initial node features of each node, and the node adjacency matrix is used to characterize the position features of each node and the neighbor node information.

[0170] In this way, there is a unique minimum spanning tree formed by each node. This tree includes each node, and there are connection edges between some nodes. Neighbor node information is used to represent the adjacency relationship between nodes, the number of neighbor nodes and node features, and the distances between neighbor nodes, etc.

[0171] During specific implementation, the distance between every two nodes can be used as the weight between the two nodes. Then, using the Kruskal algorithm, for any node, the node with the minimum weight can be selected according to the weights between this node and other nodes. It is judged whether there is a constructed coordinate set for this node. If not, an edge between the two nodes is connected by connecting this node and the node with the minimum weight, and the third coordinates of the two nodes are put into the same coordinate set. If so, it can be judged whether the third coordinate of the node with the minimum weight already belongs to the coordinate set corresponding to this node. If so, this node and the node with the minimum weight are not connected to prevent forming a loop. If not, an edge between the two nodes is connected by connecting this node and the node with the minimum weight, and the third coordinate of the node with the minimum weight is put into the coordinate set corresponding to this node. In this way, edge construction is performed based on the distances between every two nodes until n - 1 edges are obtained, determining that the construction of the minimum spanning tree is completed. Here, the value of n is the number of target corner points and also the number of nodes. As Fig.13 shown, it is a schematic diagram of a minimum spanning tree provided by an embodiment of the present disclosure. In Fig.13 a total of 16 nodes forming the minimum spanning tree are exemplified. Regarding Figure 3 the starting node in, it is the target corner point with the corner point type of the starting corner point type.

[0172] After obtaining the minimum spanning tree, a node feature matrix X and a node adjacency matrix A can be generated according to the connection relationship between nodes, the third coordinate of each node, the pixel information, image information, and edge weights corresponding to each node. Among them, the node feature matrix is used to represent the initial node features of each node, and this feature can be determined according to the pixel information, image information, edge weights, etc. corresponding to each node. The node adjacency matrix A is used to represent the position features of each node and neighbor node information. Among them, the neighbor node information is the position features of each neighbor node connected to the node, the weights between neighbor nodes, and other information.

[0173] A3: Optimize the tree structure corresponding to the minimum spanning tree according to the node feature matrix and the node adjacency matrix, and sort the target corner points corresponding to each node according to the optimized minimum spanning tree to obtain the sorting order of each target corner point.

[0174] Here, when constructing the minimum spanning tree, due to the complex adjacency relationship or errors of nodes, it may lead to position conflicts of nodes and the construction of incorrect adjacency relationships, thus affecting the final sorting result. Therefore, after obtaining the minimum spanning tree, it is necessary to further optimize the tree structure to eliminate conflicts and loops in the tree structure and ensure the rationality of the tree structure.

[0175] Specifically, during implementation, the node feature matrix and the node adjacency matrix can be input into a pre-trained structure optimization network to obtain an optimized node feature matrix and an optimized node adjacency matrix, and the optimized minimum spanning tree can be determined using the two optimized matrices. Then, according to the positions of each node in the optimized minimum spanning tree, the target corner points corresponding to each node can be sorted to obtain the sorting order of each target corner point.

[0176] Among them, sorting the target corner points can be to assign coordinates to each target corner point according to a preset rule, and use the assigned coordinates as the sorting order of the target corner points.

[0177] In one embodiment, for the steps of optimizing the tree structure in A3 above, it can be implemented according to the following steps:

[0178] A3-1: Use a pre-trained graph neural network to determine the embedding feature of each node according to the node feature matrix and the node adjacency matrix; the embedding feature is used to represent the target node feature of the node and its position and structure in the minimum spanning tree.

[0179] Here, the graph neural network (Graph Neural Network, GNN) is a pre-trained neural network used to extract and discover features and patterns in graph-structured data, and then output the embedding feature of each node in each graph structure; among them, the graph structure is the tree structure in the above text.

[0180] One node corresponds to one embedding feature, and the embedding feature can represent the target node feature of the node and its position and structure in the minimum spanning tree. The target node feature can be understood as the feature obtained after optimizing the initial node feature.

[0181] Specifically, during implementation, the node feature matrix X and the node adjacency matrix A can be input into the pre-trained GNN network. Since the matrices X and A can reflect the connection information of the minimum spanning tree carried by each target corner point, including position, number of connections, and distance, the GNN network can be used to perform feature extraction and fusion according to the two matrices to obtain the accurate embedding feature of each node.

[0182] For each node, the node information of its neighboring nodes is aggregated using a GNN to update the state of the node. During the update process, weighted summation and / or a weighted method based on the attention mechanism can be used. After being processed by the GNN, a high-order feature representation of the tree is formed, and an embedding vector is formed and output for each node, thereby characterizing the structure, position, and feature information of each node in the graph.

[0183] A3-2: Using a pre-trained reinforcement learning network, based on the embedding features of each node, determine the optimized feature matrix corresponding to the node feature matrix and the optimized adjacency matrix corresponding to the node adjacency matrix.

[0184] Here, the reinforcement learning (RL) network is also a neural network that is not pre-trained. This network learns an optimal policy by trying different actions, maximizing the long-term cumulative reward, so as to optimize the node feature matrix and the node adjacency matrix, and obtain the optimized feature matrix and the optimized adjacency matrix.

[0185] Exemplarily, each embedding feature input to the GNN can be input into the RL network, and the RL network is used to perform reinforcement learning on the target node features of the node and its position and structure in the minimum spanning tree, so as to output the optimized feature matrix and the optimized adjacency matrix.

[0186] It can be understood that when the tree structure of the minimum spanning tree generated in step A2 is reasonable enough, the optimized feature matrix can be consistent with the node feature matrix, and the optimized adjacency matrix can be consistent with the node adjacency matrix.

[0187] A3-3: Determine the optimized minimum spanning tree according to the optimized feature matrix and the optimized adjacency matrix.

[0188] Specifically, during implementation, the tree structure of the optimized minimum spanning tree can be deduced based on the node features of each node characterized by the optimized feature matrix, as well as the position features of each node and the neighboring node information characterized by the optimized adjacency matrix.

[0189] In this way, by combining the GNN and the RL network, conflicts and loops in the tree structure can be eliminated, ensuring the rationality of the tree structure and the smooth progress of the task, and obtaining an optimized tree structure without conflicts.

[0190] In one embodiment, the GNN may specifically include a Graph Convolution Neural Network (GCN for short) module, a Graph Attention Network (GAT for short) module, and a Graph Isomorphism Network (GIN for short) module. Among them, the GCN module is used to capture the neighborhood information of nodes and implement information aggregation on the tree through convolution operations. The GAT module is used to assign different weights to neighbor nodes through an attention mechanism to more finely process neighbor nodes with different degrees of importance. As a powerful graph embedding model, the GIN module performs well in capturing the structural information of the tree. Specifically, it can be used to capture the structural information of the tree to ensure that the aggregation of node features and neighbor features can reflect the global structure of the tree. Based on this, the above A3-1 can be implemented according to the following steps:

[0191] A3-1-1: Use the graph convolution network module of the graph neural network to update the initial node features of each node according to the position features of each node and the neighbor node information, and obtain the first updated features.

[0192] In specific implementation, the GCN module in the GNN can be used to aggregate the information of the neighbor nodes of each node to each node through convolution operations according to the position features of each node and the neighbor node information represented by the node feature matrix and the node adjacency matrix, and obtain the first updated features of each node. For example, for each node, the GCN module updates the initial node feature of the node by performing a weighted average on the position feature, the initial node feature, and the information of its neighbor nodes (such as distance, number of connected nodes, weight, etc.) of the node to obtain the first updated feature of the node.

[0193] By aggregating this information of the neighbor nodes, the features of each node can be updated to reflect the environmental information of the node in the tree. The aggregation process of the GCN module allows the node to gradually obtain information from farther nodes, so as to better capture the global structure and attributes of the tree.

[0194] Optionally, only one GCN module can be set, or multiple GCN modules can be set. In the case of setting multiple GCN modules, using multiple GCNs to sequentially update the node features can effectively balance the update depth and the calculation efficiency.

[0195] A3-1-2: Use the graph attention network module of the graph neural network to assign attention weights to the neighbor nodes of each node according to the first updated features, and update the first updated features of the node based on the attention weights to obtain the second updated features.

[0196] During specific implementation, the output of the GCN module can be input into the GAT module of the GNN. The GAT module can, through the attention mechanism, assign attention weights to the neighbor nodes of each node according to the first updated features of each node. Among them, for each node, the GAT module can selectively focus on the neighbor nodes that are most useful to it. If the attention weight of a neighbor node is high, it indicates that the correlation between this node and this neighbor node is higher and the connection is more accurate; if the attention weight of a neighbor node is low, it indicates that the correlation between this node and this neighbor node is low and the probability of an incorrect connection is high.

[0197] Then, for each node, the GAT module can use the attention weights of the neighbor nodes of this node to perform a weighted sum of the first updated features of this node and the first updated features of each neighbor node, so as to obtain the second updated feature. In this way, through the weighted sum by the attention mechanism, the further update of the node features can be realized.

[0198] A3-1-3: Use the graph isomorphism network module of the graph neural network to perform a linear combination of the second updated features of each node and the second updated features of the neighbor nodes of the node to obtain the embedding feature of each node.

[0199] During specific implementation, the output of the GAT module can be input into the GIN module of the GNN. The GIN module can respectively perform a linear combination of the second updated features of each node and the second updated features of the neighbor nodes of this node to further update the feature information of each node and obtain the embedding feature of each node.

[0200] In this way, by using multiple modules in the GNN to propagate information, each node can capture information from multiple neighborhoods,

[0201] and enable the node information of each node to be propagated to more distant nodes.

[0202] In one embodiment, the above A3-2 can be implemented according to the following steps:

[0203] A3-2-1: Use the reinforcement learning network to determine the current node to be processed according to the embedding feature of each node.

[0204] Here, the input of the RL network is the embedding feature of each node output by the GNN. The structure of this network includes multiple fully connected layers, and the fully connected layers serve as hidden layers. Each fully connected layer can be followed by an activation function.

[0205] In a specific implementation, the RL network can sequentially determine the current node to be processed each time according to the node order represented by the embedding features of each node. Alternatively, the RL network can use the node corresponding to the starting corner point as the first node to be processed initially, and in combination with the node order represented by the embedding features of each node, determine the current node to be processed each time after the first node. For example, starting from the starting node, first process the starting node, then process the nodes adjacent to the starting node, and then process the further adjacent nodes, …, until all nodes are processed completely.

[0206] A3-2-2: Determine the action to be performed on the current node according to the state information corresponding to the current node; the state information includes at least one of the global structure information of the current minimum spanning tree, the processing order of the current node, the current operation progress, and the historical actions; where, when the current node is the first node, the current minimum spanning tree is the minimum spanning tree determined by the embedding features of each node.

[0207] Here, the current operation progress is used to indicate whether the sorting of the nodes has been modified currently and / or whether further sorting modification is required. One node corresponds to one state information. In this state information, the entire tree structure is regarded as a whole, and each node shares a global structure information. Since the RL network may modify the structure of the tree, the global structure information may be continuously updated as the tree structure is modified. When the current node is the first node, the current minimum spanning tree is the minimum spanning tree determined by the embedding features of each node. When the current node is not the first node, the current minimum spanning tree is the minimum spanning tree determined by the latest embedding features of each node.

[0208] In a specific implementation, the state information of the nodes can be determined according to the embedding features of each node, so as to convert the tree structure and node features into a form that can be processed by RL. Then, for each current node to be processed, the RL network can determine the action to be performed on the current node according to the state information corresponding to the current node.

[0209] Among them, the action can include at least one of the following:

[0210] Select the next neighbor node of the current node, modify the node information of the current node, modify or reassign the neighbor node information of the current node, backtrack one step forward, or initialize the current node.

[0211] Among them, selecting the next neighbor node of the current node is used to determine the adjacency relationship of the current node to reduce conflicts and errors.

[0212] Node information can, for example, include node features, location features, structure in the tree, node order, and other information. Modify the node information of the current node to adjust the position of the current node and / or the importance of the current node in the tree structure. Modify or reassign the neighbor node information of the current node to add or remove connections between nodes to correct incorrect adjacency relationships and adjust the connection method between nodes.

[0213] Taking a step back can be understood as undoing the operation performed in the previous step. Initializing the current node can be understood as initializing the node information of the current node. When the RL network detects an obvious error, it can choose to go back to a previous state or completely reset certain parts to reduce conflicts and errors in the tree structure.

[0214] A3-2-3: Optimize the embedding features of the current node according to the action to obtain optimized embedding features.

[0215] Exemplarily, for the current node, the action determined for the node can be executed for the current node to optimize the embedding features of the current node and obtain optimized embedding features. After the embedding features of any node are optimized, the tree structure of the minimum spanning tree is naturally optimized accordingly.

[0216] Moreover, when the RL network optimizes the embedding features of the current node, a reward mechanism can be set as the core driving force of reinforcement learning to guide the network to learn in the correct direction. Specifically, when an action successfully resolves a conflict or optimizes the sorting, a positive reward is given to the RL network to encourage the network to repeat such an action. When an action causes a new problem or fails to solve an existing problem, a negative reward is given as a punishment to prevent the RL network from repeating unfavorable behaviors.

[0217] A3-2-4: Return the step of determining the current node to be processed according to the embedding features of each node until the optimized embedding features of each node are obtained.

[0218] Exemplarily, after performing the corresponding action for the current node, the above A3-2-1 to A3-2-3 can be continued to determine the next current node to be processed and optimize the features of the node. Through continuous cycling, the feature optimization of each node can be achieved, and the optimized embedding features of each node can be obtained.

[0219] A3-2-5: Determine the optimized feature matrix and the optimized adjacency matrix according to the optimized embedding features.

[0220] In specific implementation, after the RL network obtains the optimized embedding features of each node, it can output the optimized feature matrix and the optimized adjacency matrix according to the tree structure represented by the optimized embedding features and the information such as the position and features of each node.

[0221] In one embodiment, the checkerboard image includes a target checkerboard provided with target markers; the recognition result of the corner detection network further includes a second detection box corresponding to the target checkerboard; the position of the second detection box is recognized according to the target markers in the target checkerboard. When sorting the target corners, it can be implemented in combination with the starting corner corresponding to the second detection box. Specifically, for the step of "sorting the target corners corresponding to each node according to the optimized minimum spanning tree to obtain the sorting order of each target corner" in A3 above, it can be implemented according to P1 to P4 as follows:

[0222] P1: Use any one of the four target corners corresponding to the second detection box as the starting node, and assign a starting grid coordinate to the starting node.

[0223] Here, the second detection box is a detection box covering the target checkerboard corresponding to the target marker, so the four corners of the second detection box respectively correspond to a target corner. Therefore, any one of these corresponding corners can be used as the starting node. Alternatively, in the case of a starting corner where the corner detection network has marked the type of the starting corner, the node corresponding to the corner marked by the corner detection network can be directly used as the starting node.

[0224] After determining the starting node, a preset starting network coordinate can be assigned to the starting node. For example, the starting network coordinate can be (0, 0).

[0225] P2: Starting from the starting node, traverse each other node in turn according to the neighbor nodes of each node indicated by the optimized minimum spanning tree.

[0226] P3: For any other node, assign a target grid coordinate to the other node according to the third coordinate of the other node in the checkerboard image, the third coordinate of the first traversed node among the neighbor nodes of the other node, and a preset coordinate deviation angle.

[0227] Here, the preset coordinate deviation angle can include the deviation angle on the horizontal axis and the deviation angle on the vertical axis. For example, the deviation angle on the horizontal axis and the deviation angle on the vertical axis can be the positive and negative deviation angles on the horizontal axis and the positive and negative deviation angles on the vertical axis.

[0228] During specific implementation, after allocating the starting network coordinates for the starting node, the neighbor nodes of each node indicated by the optimized minimum spanning tree can be traversed along any direction of the starting node to visit the next neighbor node of the starting node. For each traversed node, the deviation angle of the node from the first traversed node in the horizontal axis and the deviation angle in the vertical axis can be determined according to the third coordinate of the node in the checkerboard image and the third coordinate of the first traversed node among the neighbor nodes of other nodes. Then, according to the deviation angles on the two axes, the preset coordinate deviation angle, and the target network coordinates of the first traversed node, the target network coordinates can be allocated for the node.

[0229] For example, according to the adjacency relationship indicated by the optimized minimum spanning tree, starting from the starting node, the neighbor nodes are sequentially visited along the positive direction of the tree. Among them, the positive direction can be any direction of the starting node. For each visited node, the deviation angle of the node relative to the first traversed neighbor node is determined to allocate the target grid coordinates for the node. For example, from (0, 0) to (1, 0) is the east direction, and from (0, 0) to (0, 1) is the north direction. A tolerance range for a direction (i.e., the preset coordinate deviation angle) is set in the middle to allocate the starting grid coordinates for each node. Each node in the optimized minimum spanning tree is used to visit the adjacent nodes until all nodes are visited, completing the coordinate allocation for each node.

[0230] P4: Determine the sorting order of each target corner point according to the starting grid coordinates and the target grid coordinates.

[0231] During specific implementation, the starting grid coordinates of the starting corner point and the target grid coordinates of each ordinary corner point can be used as the sorting order of the corresponding corner points.

[0232] As Fig.14 shown, it is a schematic diagram of the allocation of grid coordinates provided by an embodiment of the present disclosure. Among them, Fig.14 the minimum spanning tree in Fig.15 includes 16 nodes. The grid coordinates of the starting node are (0, 0), and the grid coordinates of other nodes are as shown. As

[0233] shown, it is a schematic diagram of the grid coordinates of each target corner point in a checkerboard image provided by an embodiment of the present disclosure.

[0234] Those skilled in the art can understand that in the above methods of the specific embodiments, the writing order of each step does not mean a strict execution order and does not impose any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0235] Based on the same inventive concept, the embodiments of the present disclosure also provide a corner detection device corresponding to the corner detection method. Since the principle of solving problems by the device in the embodiments of the present disclosure is similar to the above corner detection method in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0236] As Fig.16 shown, it is a schematic diagram of a corner detection device provided by an embodiment of the present disclosure, including:

[0237] A detection module 1601, configured to detect each corner in the checkerboard image by using a pre-trained corner detection network, and obtain a first detection frame corresponding to each corner;

[0238] An optimization module 1602, configured to perform an image optimization operation on the detection frame image corresponding to each of the first detection frames to obtain an optimized image;

[0239] A first screening module 1603, configured to screen out candidate saddle points that conform to the saddle point feature from the pixels according to the gradients of the pixels in the optimized image;

[0240] A second screening module 1604, configured to screen out the target corners in the first detection frame from the candidate saddle points according to the first coordinates of each candidate saddle point in the optimized image.

[0241] In a possible implementation manner, when the first screening module 1603 screens out candidate saddle points that conform to the saddle point feature from the pixels according to the gradients of the pixels in the optimized image, it is configured to:

[0242] Determine two eigenvalues corresponding to each of the pixels in the optimized image under the characteristic equation according to the gradients of the pixels in the optimized image; the eigenvalues are used to characterize the curvature feature of the pixel;

[0243] Screen out candidate saddle points that conform to the saddle point feature from the pixels according to the positive and negative signs of the two eigenvalues.

[0244] In a possible implementation manner, when the first screening module 1603 determines the eigenvalues corresponding to each of the pixels in the optimized image under the characteristic equation according to the gradients of the pixels in the optimized image, it is configured to:

[0245] Determine the corresponding preset matrix representation for each pixel point according to the gradient of each pixel point in the optimized image;

[0246] Construct the corresponding characteristic equation for each pixel point according to the preset matrix representation corresponding to each pixel point;

[0247] Determine two eigenvalues corresponding to each pixel point under the characteristic equation according to the solution of the characteristic equation.

[0248] In a possible implementation manner, when the optimization module 1602 performs an image optimization operation on the detection box image corresponding to each first detection box to obtain an optimized image, it is used for:

[0249] Determine multiple filtering scales for performing image filtering on the detection box image;

[0250] Perform image filtering processing on the detection box image according to multiple filtering scales respectively to obtain multiple first filtered images, and determine a second filtered image according to the multiple first filtered images;

[0251] Determine the adaptive threshold of the second filtered image;

[0252] Determine an image enhancement parameter according to the adaptive threshold, and use the image enhancement parameter to adjust the image contrast of the second filtered image to obtain an optimized image.

[0253] In a possible implementation manner, when the second screening module 1604 screens out the target corner points in the first detection box from each candidate saddle point according to the first coordinates of each candidate saddle point in the optimized image, it is used for:

[0254] Perform sub-pixel level optimization on the first coordinates of the candidate saddle points through an interpolation method to obtain the second coordinates of the candidate saddle points;

[0255] Use the second coordinates of each candidate saddle point and adopt a non-maximum suppression method to screen out the target corner points from the candidate saddle points.

[0256] In a possible implementation manner, the device further includes:

[0257] A sorting module 1605, after screening out the target corner points in the first detection box from each candidate saddle point, is used for:

[0258] Take the target corner points in each first detection box as nodes, and calculate the distance between every two nodes by using the third coordinates of each node in the checkerboard image;

[0259] Generate a minimum spanning tree formed by each of the nodes according to the distance between every two nodes, and determine a node feature matrix and a node adjacency matrix according to the minimum spanning tree; the node feature matrix is used to characterize the initial node features of each of the nodes, and the node adjacency matrix is used to characterize the position features of each of the nodes and the neighbor node information;

[0260] Optimize the tree structure corresponding to the minimum spanning tree according to the node feature matrix and the node adjacency matrix, and sort the target corner points corresponding to each of the nodes according to the optimized minimum spanning tree to obtain the sorting order of each of the target corner points.

[0261] In a possible implementation manner, when optimizing the tree structure corresponding to the minimum spanning tree according to the node feature matrix and the node adjacency matrix, the sorting module 1605 is configured to:

[0262] Use a pre-trained graph neural network to determine the embedding features of each of the nodes according to the node feature matrix and the node adjacency matrix; the embedding features are used to characterize the target node features of the nodes and their positions and structures in the minimum spanning tree;

[0263] Use a pre-trained reinforcement learning network to determine an optimized feature matrix corresponding to the node feature matrix and an optimized adjacency matrix corresponding to the node adjacency matrix according to the embedding features of each of the nodes;

[0264] Determine an optimized minimum spanning tree according to the optimized feature matrix and the optimized adjacency matrix.

[0265] In a possible implementation manner, when using the pre-trained graph neural network to determine the embedding features of each of the nodes according to the node feature matrix and the node adjacency matrix, the sorting module 1605 is configured to:

[0266] Use the graph convolutional network module of the graph neural network to update the initial node features of each of the nodes according to the position features of each of the nodes and the neighbor node information to obtain first updated features;

[0267] Use the graph attention network module of the graph neural network to assign attention weights to the neighbor nodes of each of the nodes according to the first updated features, and update the first updated features of the nodes based on the attention weights to obtain second updated features;

[0268] Use the graph isomorphism network module of the graph neural network to perform a linear combination of the second updated features of each of the nodes and the second updated features of the neighbor nodes of the nodes to obtain the embedding features of each of the nodes.

[0269] In a possible implementation manner, when the sorting module 1605 determines the optimized feature matrix corresponding to the node feature matrix and the optimized adjacency matrix corresponding to the node adjacency matrix according to the embedding features of each node by using a pre-trained reinforcement learning network, it is used for:

[0270] Using the reinforcement learning network, according to the embedding features of each node, to determine the current node to be processed;

[0271] According to the state information corresponding to the current node, to determine the action to be performed on the current node; the state information includes at least one of the global structure information of the current minimum spanning tree, the processing order of the current node, the current operation progress, and the historical actions; wherein, when the current node is the first node, the current minimum spanning tree is the minimum spanning tree determined by the embedding features of each node;

[0272] Optimizing the embedding feature of the current node according to the action to obtain an optimized embedding feature;

[0273] Returning to the step of determining the current node to be processed according to the embedding features of each node until the optimized embedding features of all the nodes are obtained;

[0274] According to the optimized embedding features, to determine the optimized feature matrix and the optimized adjacency matrix.

[0275] In a possible implementation manner, the action includes at least one of the following:

[0276] Selecting the next neighbor node of the current node, modifying the node information of the current node, modifying or reassigning the neighbor node information of the current node, backtracking one step forward, or initializing the current node.

[0277] In a possible implementation manner, the checkerboard image includes a target checkerboard provided with a target marker; the recognition result of the corner detection network further includes a second detection frame corresponding to the target checkerboard; the position of the second detection frame is recognized according to the target marker in the target checkerboard;

[0278] When the sorting module 1605 sorts the target corners corresponding to each node according to the optimized minimum spanning tree to obtain the sorting order of each target corner, it is used for:

[0279] Taking any one of the four target corners corresponding to the second detection frame as the starting node and assigning starting grid coordinates to the starting node;

[0280] Starting from the starting node, traverse each other node in sequence according to the neighbor nodes of each node indicated by the optimized minimum spanning tree;

[0281] For any one of the other nodes, allocate target grid coordinates for the other node according to the third coordinate of the other node in the checkerboard image, the third coordinate of the first traversed node among the neighbor nodes of the other node, and a preset coordinate deviation angle;

[0282] Determine the sorting order of each of the target corner points according to the starting grid coordinates and the target grid coordinates.

[0283] Descriptions of the processing procedures of each module in the device and the interaction procedures between the modules can refer to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0284] Based on the same inventive concept, an embodiment of the present application further provides a computer device. Referring to Fig.17 As shown, it is a schematic structural diagram of a computer device provided by an embodiment of the present application, including:

[0285] A processor 1701, a memory 1702, and a bus 1703. Among them, the memory 1702 stores machine-readable instructions executable by the processor 1701, and the processor 1701 is used to execute the machine-readable instructions stored in the memory 1702. When the machine-readable instructions are executed by the processor 1701, the processor 1701 executes the following steps: S101: Use a pre-trained corner detection network to detect each corner point in the checkerboard image, and obtain a first detection frame corresponding to each corner point; S102: Perform an image optimization operation on the detection frame image corresponding to each first detection frame to obtain an optimized image; S103: According to the gradients of each pixel point in the optimized image, screen out candidate saddle points that meet the saddle point characteristics from each pixel point; and S104: According to the first coordinate of each candidate saddle point in the optimized image, screen out the target corner points in the first detection frame from each candidate saddle point.

[0286] The above memory 1702 includes a memory 171 and an external memory 1722; here, the memory 1721 is also called an internal memory, which is used to temporarily store the operation data in the processor 1701 and the data exchanged with the external memory 1722 such as a hard disk. The processor 1701 exchanges data with the external memory 1722 through the memory 1721. When the computer device is running, the processor 1701 communicates with the memory 1702 through the bus 1703, so that the processor 1701 executes the execution instructions mentioned in the above method embodiments.

[0287] Embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the corner detection method described in the foregoing method embodiments. Among them, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0288] Embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the software update method described in the foregoing method embodiments. For details, please refer to the foregoing method embodiments and will not be elaborated here.

[0289] The computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is specifically embodied as a computer storage medium. In another alternative embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0290] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the foregoing method embodiments and will not be elaborated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined, or some features can be ignored, or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0291] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0292] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0293] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0294] If the technical solution of this application involves personal information, before the product applying the technical solution of this application processes personal information, it has clearly informed the personal information processing rules and obtained the personal's independent consent. If the technical solution of this application involves sensitive personal information, before the product applying the technical solution of this application processes sensitive personal information, it has obtained the personal's separate consent and at the same time meets the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set to inform that the personal information collection range has been entered and personal information will be collected. If an individual voluntarily enters the collection range, it is regarded as consenting to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up messages or by asking the individual to upload their personal information by themselves, etc.; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0295] Finally, it should be noted that: the above-mentioned embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A corner point detection method, characterized in that: include: Using a pre-trained corner point detection network, detect each corner point in the checkerboard image to obtain a first detection frame corresponding to each corner point; performing an image optimization operation on the detection frame images respectively corresponding to the first detection frames to obtain optimized images; According to the gradient of each pixel point in the optimized image, a candidate saddle point that meets the saddle point characteristics is selected from each pixel point; According to the first coordinate of each candidate saddle point in the optimized image, the target corner point in the first detection frame is screened out from each candidate saddle point.

2. The method according to claim 1, characterized in that According to the gradient of each pixel point in the optimized image, a candidate saddle point that meets the saddle point characteristics is selected from each pixel point, including: According to the gradient of each pixel point in the optimized image, two eigenvalues ​​corresponding to each pixel point in the optimized image under the characteristic equation are determined; the eigenvalues ​​are used to characterize the curvature characteristics of the pixel point; According to the positive and negative signs of the two eigenvalues, candidate saddle points that meet the saddle point characteristics are screened out from the pixel points.

3. The method according to claim 2, characterized in that Determining the eigenvalues ​​corresponding to the pixel points in the optimized image under the characteristic equation according to the gradients of the pixel points in the optimized image includes: Determine, according to the gradient of each pixel point in the optimized image, a preset matrix representation corresponding to each pixel point; According to the preset matrix representation corresponding to each pixel point, construct the characteristic equation corresponding to each pixel point respectively; According to the solution of the characteristic equation, two eigenvalues ​​corresponding to each pixel point under the characteristic equation are determined.

4. The method according to claim 1, characterized in that: The performing an image optimization operation on the detection frame images respectively corresponding to the first detection frames to obtain an optimized image includes: Determining a plurality of filtering scales for performing image filtering on the detection frame image; According to the plurality of filtering scales, the detection frame images are respectively subjected to image filtering processing to obtain a plurality of first filtering images, and a second filtering image is determined based on the plurality of first filtering images; determining an adaptive threshold of the second filtered image; An image enhancement parameter is determined according to the adaptive threshold, and the image contrast of the second filtered image is adjusted using the image enhancement parameter to obtain an optimized image.

5. The method according to claim 1, characterized in that The step of selecting target corner points in the first detection frame from each of the candidate saddle points according to the first coordinate of each of the candidate saddle points in the optimized image comprises: Performing sub-pixel optimization on the first coordinate of the candidate saddle point by interpolation to obtain a second coordinate of the candidate saddle point; The target corner point is screened out from the candidate saddle points by using the second coordinate of each candidate saddle point and adopting a non-maximum suppression method.

6. The method according to claim 1, characterized in that After selecting the target corner points in the first detection frame from the candidate saddle points, the method further includes: Taking each target corner point in the first detection frame as a node, and using the third coordinate of each node in the checkerboard image to calculate the distance between every two nodes; According to the distance between every two nodes, a minimum spanning tree composed of each of the nodes is generated, and according to the minimum spanning tree, a node feature matrix and a node adjacency matrix are determined; the node feature matrix is ​​used to characterize the initial node features of each of the nodes, and the node adjacency matrix is ​​used to characterize the location features and neighbor node information of each of the nodes; According to the node feature matrix and the node adjacency matrix, the tree structure corresponding to the minimum spanning tree is optimized, and according to the optimized minimum spanning tree, the target corner points corresponding to each of the nodes are sorted to obtain the sorting order of each of the target corner points.

7. The method according to claim 6, characterized in that Optimizing the tree structure corresponding to the minimum spanning tree according to the node feature matrix and the node adjacency matrix, including: Using a pre-trained graph neural network, according to the node feature matrix and the node adjacency matrix, an embedded feature of each of the nodes is determined; the embedded feature is used to characterize the target node feature of the node and the position and structure in the minimum spanning tree; Using a pre-trained reinforcement learning network, according to the embedded features of each of the nodes, determine an optimized feature matrix corresponding to the node feature matrix and an optimized adjacency matrix corresponding to the node adjacency matrix; An optimized minimum spanning tree is determined according to the optimized feature matrix and the optimized adjacency matrix.

8. The method according to claim 7, characterized in that The using of the pre-trained graph neural network to determine the embedding feature of each of the nodes according to the node feature matrix and the node adjacency matrix includes: Using the graph convolutional network module of the graph neural network, the initial node feature of each node is updated according to the position feature and neighbor node information of each node to obtain a first updated feature; Using the graph attention network module of the graph neural network, assigning an attention weight to a neighbor node of each of the nodes according to the first updated feature, and updating the first updated feature of the node based on the attention weight to obtain a second updated feature; The graph isomorphism network module of the graph neural network is used to linearly combine the second updated features of each node and the second updated features of the neighboring nodes of the node to obtain the embedded features of each node.

9. The method according to claim 7, characterized in that: The method of using a pre-trained reinforcement learning network to determine an optimized feature matrix corresponding to the node feature matrix and an optimized adjacency matrix corresponding to the node adjacency matrix according to the embedded features of each node includes: Using the reinforcement learning network, according to the embedded features of each of the nodes, determining a current node to be processed; Determine the action to be performed on the current node according to the state information corresponding to the current node; the state information includes at least one of the global structure information corresponding to the current minimum spanning tree, the processing order of the current node, the current operation progress and the historical action; wherein, when the current node is the first node, the current minimum spanning tree is the minimum spanning tree determined by the embedded features of each node; Optimizing the embedding feature of the current node according to the action to obtain an optimized embedding feature; Returning to the step of determining the current node to be processed according to the embedding features of each of the nodes, until the optimized embedding features of each of the nodes are obtained; The optimized feature matrix and the optimized adjacency matrix are determined according to the optimized embedded features.

10. The method according to claim 9, characterized in that The actions include at least one of the following: Select the next neighbor node of the current node and modify the node information of the current node 、 Modify or reallocate the neighbor node information of the current node, backtrack one step, or initialize the current node.

11. The method according to claim 6, characterized in that The checkerboard image includes a target checkerboard with a target mark; the recognition result of the corner point detection network also includes a second detection frame corresponding to the target checkerboard; the position of the second detection frame is obtained according to the target mark in the target checkerboard; According to the optimized minimum spanning tree, the target corner points corresponding to each node are sorted to obtain the sorting order of each target corner point, including: Taking any corner point of the four target corner points corresponding to the second detection frame as a starting node, and assigning a starting grid coordinate to the starting node; Starting from the starting node, traversing each other node in turn according to the neighbor nodes of each node indicated by the optimized minimum spanning tree; For any of the other nodes, assign a target grid coordinate to the other node according to the third coordinate of the other node in the checkerboard image, the third coordinate of the first traversed node among the neighboring nodes of the other node, and a preset coordinate deviation angle; The sorting order of each of the target corner points is determined according to the starting grid coordinates and the target grid coordinates.

12. A corner point detection device, characterized in that: include: A detection module, used to detect each corner point in the checkerboard image using a pre-trained corner point detection network, and obtain a first detection frame corresponding to each corner point; an optimization module, configured to perform an image optimization operation on the detection frame images respectively corresponding to the first detection frames to obtain optimized images; A first screening module, used for screening out candidate saddle points that meet the saddle point characteristics from each pixel point in the optimized image according to the gradient of each pixel point; The second screening module is used to screen out target corner points in the first detection frame from each of the candidate saddle points according to the first coordinate of each of the candidate saddle points in the optimized image.

13. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the corner point detection method according to any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a computer device, the computer device executes the steps of the corner detection method according to any one of claims 1 to 11.

Images