Landmark decoding method and electronic device

Abstract

This application provides a marker point decoding method and electronic device. The method includes: generating a three-dimensional marker point set based on multiple randomly arranged circular marker points in a calibration field, and generating a two-dimensional marker point set based on images of each circular marker point captured by a large distortion lens to be calibrated; generating a first seed point set for the image center region based on the three-dimensional marker point set and the two-dimensional marker point set, wherein the first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates; performing iterative neighborhood diffusion on the first seed point set based on the first seed point set and a first preset radius to establish a correspondence between the three-dimensional spatial coordinates of all circular marker points and the two-dimensional image coordinates; and generating and outputting a marker point correspondence list based on the correspondence between the three-dimensional spatial coordinates of each circular marker point and the two-dimensional image coordinates, so as to improve the accuracy and efficiency of marker point decoding under large distortion lenses.

Description

Technical Field

[0001] This application relates to the field of three-dimensional optical projection technology, and more specifically, to a marker point decoding method and electronic device. Background Technology

[0002] Marker point decoding is an indispensable part of photogrammetry and has important application value in fields such as image recognition, feature matching, and camera calibration.

[0003] Currently, in photogrammetry, marker decoding generally relies on preset encoding rules and a rigorously set calibration field, requiring pre-designed encoding rules and arrangement of markers. However, industrial surveying typically requires a large calibration field, and setting up coded markers according to a predetermined arrangement is time-consuming and labor-intensive, and it is difficult to guarantee that the collision rate meets the requirements. Furthermore, the shape of pre-designed coded points will be distorted after being imaged by a lens with high distortion, and decoding according to the established rules at this time is very likely to result in decoding errors.

[0004] Therefore, the existing technology for decoding marker points in high-distortion lenses has certain limitations. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a marker point decoding method and electronic device, thereby solving the practical problem of the limitations of marker point decoding in high-distortion lenses in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a marker point decoding method, the method comprising: A three-dimensional marker set is generated based on multiple randomly placed circular markers in the calibration field, and a two-dimensional marker set is generated based on the images of each of the circular markers captured by the large distortion lens to be calibrated; wherein, the three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each of the circular markers, and the two-dimensional marker set includes the two-dimensional image coordinates of the ellipse center of each of the circular markers. Based on the three-dimensional marker point set and the two-dimensional marker point set, a first seed point set for the image center region is generated. The first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates. Based on the first seed point set and the first preset radius, iterative neighborhood diffusion is performed on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points. Based on the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each circular marker, a marker correspondence list is generated and output.

[0007] As an optional implementation, the step of generating a three-dimensional marker set based on multiple randomly distributed circular markers in the calibration field, and generating a two-dimensional marker set based on images of each of the circular markers captured by the large-distortion lens to be calibrated, includes: The calibration field is scanned and measured to determine the three-dimensional spatial coordinates of the center of each of the circular marker points in the calibration field, and the three-dimensional marker point set is generated based on the three-dimensional spatial coordinates of the center of each of the circular marker points. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, an ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point, and the two-dimensional marker point set is generated based on the two-dimensional image coordinates of the ellipse center of each circular marker point.

[0008] As an optional implementation, generating a first seed point set for the image center region based on the three-dimensional marker point set and the two-dimensional marker point set includes: Determine the center point of the image, determine the central region of the image based on the center point and the second preset radius, and generate a two-dimensional central region marker point set based on the central region of the image and the two-dimensional marker point set. Five-point coplanar projection invariant matching is performed on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region.

[0009] As an optional implementation, the step of performing five-point coplanar projection invariant matching on the three-dimensional marker point set and the two-dimensional center region marker point set to generate the first seed point set of the image center region includes: Construct multiple three-dimensional five-point combinations in the set of three-dimensional marker points, and determine the projection invariant pairs of each three-dimensional five-point combination based on the three-dimensional spatial coordinates of the five three-dimensional coplanar points in each three-dimensional five-point combination. The three-dimensional five-point combination includes five three-dimensional coplanar points, and any three of the five three-dimensional coplanar points are not collinear. Construct multiple two-dimensional five-point combinations in the set of two-dimensional central region marker points, and determine the projection invariant pairs of each two-dimensional five-point combination based on the two-dimensional image coordinates of the five two-dimensional coplanar points in each two-dimensional five-point combination. The two-dimensional five-point combination includes five two-dimensional coplanar points, and any three of the five two-dimensional coplanar points are not collinear. The first seed point set for the central region of the image is generated based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination.

[0010] As an optional implementation, generating the first seed point set for the image center region based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination includes: Determine the absolute value of the difference between the first and second projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination; According to a preset threshold, the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination are matched, and the three-dimensional five-point combination and the two-dimensional five-point combination whose absolute values ​​of the first projection invariant difference and the second projection invariant difference are both less than the preset threshold are taken as a candidate matching item in the candidate matching set. Based on the preset consistency constraints, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set. Based on the target matching set, multiple correspondences between three-dimensional spatial coordinates and two-dimensional image coordinates are established. The circular marker points corresponding to the three-dimensional spatial coordinates and two-dimensional image coordinates are used as first seed points, and a first seed point set containing multiple first seed points is generated for the image center region.

[0011] As an optional implementation, the step of iteratively expanding the neighborhood of the first seed point set based on the first seed point set and a preset radius to establish the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points includes: Based on the first seed point set and the first preset radius, generate a neighborhood point set for each first seed point in the first seed point set, and determine the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point. A second seed point set is generated based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set. Iteratively perform neighborhood search and local homography matrix determination for each second seed point in the second seed point set until the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points is established.

[0012] As an optional implementation, determining the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point includes: Determine the number of decoding markers in the neighborhood set of the current first seed point. The decoding markers are circular markers that establish a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates. If the number of decoded markers in the neighborhood set of the current first seed point is greater than or equal to a preset value, then the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker in the neighborhood set of the current first seed point. Otherwise, skip the current first seed point and determine the local homography matrix of the region where the next first seed point is located based on the neighborhood point set of the next first seed point.

[0013] As an optional implementation, determining the local homography matrix of the region where the current first seed point is located based on the decoding markers in the neighborhood point set of the current first seed point includes: If the number of decoded markers in the neighborhood of the current first seed point is the preset value, then the DLT algorithm is used to determine the local homography matrix of the region where the current first seed point is located. If the number of decoded markers in the neighborhood of the current first seed point is greater than the preset value, then the RANSAC algorithm or the least squares estimation algorithm is used to determine the local homography matrix of the region where the current first seed point is located.

[0014] As an optional implementation, generating a second seed point set based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set includes: Based on the local homography matrix of the region where each first seed point is located, the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point are back-projected and transformed to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point. In the set of three-dimensional marker points, determine the candidate three-dimensional spatial coordinates that are closest to the predicted three-dimensional spatial coordinates of each non-decoded marker point in terms of Euclidean distance. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker point and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, then the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker point. Establish the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of each non-decoded marker point, update each non-decoded marker point as a second seed point, and add each second seed point to the first seed point set to obtain the second seed point set.

[0015] Secondly, embodiments of this application provide a marker point decoding device, the device comprising: The generation module is used to generate a three-dimensional marker set based on multiple randomly arranged circular markers in the calibration field, and to generate a two-dimensional marker set based on the images of each of the circular markers captured by the large distortion lens to be calibrated; wherein, the three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each of the circular markers, and the two-dimensional marker set includes the two-dimensional image coordinates of the ellipse center of each of the circular markers. The generation module is further configured to generate a first seed point set for the image center region based on the three-dimensional marker point set and the two-dimensional marker point set. The first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates. The diffusion module is used to perform iterative neighborhood diffusion on the first seed point set based on the first seed point set and the first preset radius, and to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points. The output module is used to generate a list of correspondences between the three-dimensional spatial coordinates and two-dimensional image coordinates of each circular marker point, and to output the list of correspondences between the marker points.

[0016] As an optional implementation, the generation module is specifically used for: The calibration field is scanned and measured to determine the three-dimensional spatial coordinates of the center of each of the circular marker points in the calibration field, and the three-dimensional marker point set is generated based on the three-dimensional spatial coordinates of the center of each of the circular marker points. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, an ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point, and the two-dimensional marker point set is generated based on the two-dimensional image coordinates of the ellipse center of each circular marker point.

[0017] As an optional implementation, the generation module is specifically used for: Determine the center point of the image, determine the central region of the image based on the center point and the second preset radius, and generate a two-dimensional central region marker point set based on the central region of the image and the two-dimensional marker point set. Five-point coplanar projection invariant matching is performed on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region.

[0018] As an optional implementation, the generation module is specifically used for: Construct multiple three-dimensional five-point combinations in the set of three-dimensional marker points, and determine the projection invariant pairs of each three-dimensional five-point combination based on the three-dimensional spatial coordinates of the five three-dimensional coplanar points in each three-dimensional five-point combination. The three-dimensional five-point combination includes five three-dimensional coplanar points, and any three of the five three-dimensional coplanar points are not collinear. Construct multiple two-dimensional five-point combinations in the set of two-dimensional central region marker points, and determine the projection invariant pairs of each two-dimensional five-point combination based on the two-dimensional image coordinates of the five two-dimensional coplanar points in each two-dimensional five-point combination. The two-dimensional five-point combination includes five two-dimensional coplanar points, and any three of the five two-dimensional coplanar points are not collinear. The first seed point set for the central region of the image is generated based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination.

[0019] As an optional implementation, the generation module is specifically used for: Determine the absolute value of the difference between the first and second projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination; According to a preset threshold, the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination are matched, and the three-dimensional five-point combination and the two-dimensional five-point combination whose absolute values ​​of the first projection invariant difference and the second projection invariant difference are both less than the preset threshold are taken as a candidate matching item in the candidate matching set. Based on the preset consistency constraints, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set. Based on the target matching set, multiple correspondences between three-dimensional spatial coordinates and two-dimensional image coordinates are established. The circular marker points corresponding to the three-dimensional spatial coordinates and two-dimensional image coordinates are used as first seed points, and a first seed point set containing multiple first seed points is generated for the image center region.

[0020] As an optional implementation, the diffusion module is specifically used for: Based on the first seed point set and the first preset radius, generate a neighborhood point set for each first seed point in the first seed point set, and determine the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point. A second seed point set is generated based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set. Iteratively perform neighborhood search and local homography matrix determination for each second seed point in the second seed point set until the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points is established.

[0021] As an optional implementation, the diffusion module is specifically used for: Determine the number of decoding markers in the neighborhood set of the current first seed point. The decoding markers are circular markers that establish a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates. If the number of decoded markers in the neighborhood set of the current first seed point is greater than or equal to a preset value, then the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker in the neighborhood set of the current first seed point. Otherwise, skip the current first seed point and determine the local homography matrix of the region where the next first seed point is located based on the neighborhood point set of the next first seed point.

[0022] As an optional implementation, the diffusion module is specifically used for: If the number of decoded markers in the neighborhood of the current first seed point is the preset value, then the DLT algorithm is used to determine the local homography matrix of the region where the current first seed point is located. If the number of decoded markers in the neighborhood of the current first seed point is greater than the preset value, then the RANSAC algorithm or the least squares estimation algorithm is used to determine the local homography matrix of the region where the current first seed point is located.

[0023] As an optional implementation, the diffusion module is specifically used for: Based on the local homography matrix of the region where each first seed point is located, the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point are back-projected and transformed to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point. In the set of three-dimensional marker points, determine the candidate three-dimensional spatial coordinates that are closest to the predicted three-dimensional spatial coordinates of each non-decoded marker point in terms of Euclidean distance. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker point and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, then the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker point. Establish the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of each non-decoded marker point, update each non-decoded marker point as a second seed point, and add each second seed point to the first seed point set to obtain the second seed point set.

[0024] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the marker point decoding method described in the first aspect above.

[0025] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the marker point decoding method described in the first aspect above.

[0026] The beneficial effects of this application are: This application provides a marker point decoding method and electronic device. A three-dimensional marker point set is generated based on multiple randomly placed circular marker points in a calibration field. A two-dimensional marker point set is generated based on images of each circular marker point captured by a large-distortion lens to be calibrated. Based on the three-dimensional and two-dimensional marker point sets, a first seed point set including multiple first seed points in the image center region is generated, with each first seed point's three-dimensional spatial coordinates corresponding one-to-one with the two-dimensional image coordinates. Based on the first seed point set and a first preset radius, iterative neighborhood diffusion is performed on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates of all circular marker points and the two-dimensional image coordinates. Based on this correspondence, a marker point correspondence list is generated and output. This significantly reduces the difficulty, time cost, and manpower cost of setting up the calibration field, and ensures that marker point decoding starts from the two-dimensional image through iterative neighborhood diffusion. The process progresses layer by layer from the image center region with less lens distortion to the image edge region with greater lens distortion, avoiding the problem of error accumulation and matching failure caused by large distortion when performing point matching across the entire two-dimensional image range. This greatly improves the success rate of landmark point decoding under conditions of large distortion. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 1 ; Figure 2 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 2 ; Figure 3 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 3 ; Figure 4A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 4 ; Figure 5 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 5 ; Figure 6 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 6 ; Figure 7 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 7 ; Figure 8 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 8 ; Figure 9 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 9 ; Figure 10 A module structure diagram of the marker decoding device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0030] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0031] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0032] Marker point decoding is an indispensable part of photogrammetry, with significant applications in image recognition, feature matching, and camera calibration. Currently, in photogrammetry, marker point decoding generally relies on pre-set encoding rules and a rigorously arranged calibration field, requiring prior design of the marker point encoding rules and arrangement. However, industrial measurement typically requires large-area calibration fields, and arranging coded marker points according to a predetermined pattern is time-consuming and labor-intensive, and it is difficult to guarantee that the collision rate meets requirements. Furthermore, the shape of pre-designed coded points will be distorted after being imaged by a lens with high distortion, and decoding according to the established rules at this time is likely to result in decoding errors. In other words, existing marker point decoding technologies using high distortion lenses have certain limitations.

[0033] Based on the above-mentioned problems, this application provides a marker decoding method to reduce the difficulty and cost of setting up the calibration field and improve the success rate and efficiency of marker decoding under large distortion imaging conditions.

[0034] Figure 1 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 1 The subject executing this method can be any electronic device with computing power. For example... Figure 1 As shown, the method includes: S101. Generate a three-dimensional set of marker points based on multiple randomly arranged circular marker points in the calibration field, and generate a two-dimensional set of marker points based on the images of each circular marker point taken by the large distortion lens to be calibrated.

[0035] The three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each circular marker, while the two-dimensional marker set includes the two-dimensional image coordinates of the ellipse center of each circular marker.

[0036] Alternatively, when setting up the calibration field, paste the materials at a uniform density on a flat wall. The calibration field consists of circular markers without coded information, spaced apart to avoid occlusion and ensure uniform spatial distribution. Since there is no need to design marker patterns according to specific coding rules or arrange markers in a predetermined manner, the difficulty, time, and labor costs of setting up the calibration field are significantly reduced.

[0037] According to the random arrangement in the calibration field 1 circular marker point, generate a 3D marker point set. The three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each circular marker, i.e., the three-dimensional marker set. , This represents the total number of circular markers.

[0038] The calibration field was photographed using a large-distortion camera to be calibrated, resulting in images containing... Two-dimensional image of a circular marker. According to the two-dimensional image Generate a set of two-dimensional marker points Among them, the set of two-dimensional marker points The image includes the two-dimensional coordinates of the ellipse centers of each circular marker point, i.e. , For two-dimensional images The number of circular markers detected in the image. It is worth noting that due to factors such as occlusion and changes in lighting, the number of circular markers detected in the two-dimensional image may vary. Number of circular markers detected This may be related to the total number of circular markers deployed in the calibration field. Not exactly equal.

[0039] S102. Generate the first seed point set for the central region of the image based on the three-dimensional marker point set and the two-dimensional marker point set.

[0040] The first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates.

[0041] Optionally, in the central region of the image where lens distortion is less. It satisfies the ideal perspective projection model, that is, the transformation from three-dimensional space to two-dimensional image. Image center area The mapping can be represented by a homography matrix. To characterize.

[0042] Based on this, according to the three-dimensional marker point set and two-dimensional marker set For the image center area with less lens distortion Each circular marker within the image is decoded with high reliability to establish the image center region. The correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each circular marker point within the image center region. Each circular marker within the image is used as a first seed point to generate an image center region comprising multiple first seed points. First seed point set .

[0043] Among them, the first seed point set Each first seed point simultaneously records its three-dimensional spatial coordinates. and corresponding two-dimensional image coordinates .

[0044] S103. Based on the first seed point set and the first preset radius, perform iterative neighborhood diffusion on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points.

[0045] Optionally, the central region of the image First seed point set Each seed point is used as an anchor point, and the first preset radius is used as the anchor point. From two-dimensional images Image center area Iterative neighborhood diffusion is performed towards the image edge region with large lens distortion. Through at least one round of iterative neighborhood diffusion, the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular markers is established. The circular markers with the established correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates are used as decoding markers.

[0046] Due to the central region of the image The circular markers with relatively small lens distortion serve as anchor points, from the two-dimensional image. Image center area Iterative neighborhood diffusion is performed towards the edge regions of the image with significant lens distortion. This iterative neighborhood diffusion ensures that the marker points are decoded from the two-dimensional image. Image center area Progressing layer by layer towards the image edge region avoids obscuring the entire two-dimensional image. This significantly improves the success rate of marker point decoding under conditions of large distortion, addressing the issues of error accumulation and matching failure caused by large distortion when performing point matching within a certain range.

[0047] S104. Based on the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each circular marker, generate a marker correspondence list and output the marker correspondence list.

[0048] Optionally, based on the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of each decoded marker point, a marker point correspondence list containing multiple correspondence items is generated, wherein each correspondence item contains the three-dimensional spatial coordinates of the decoded marker point. and its application in two-dimensional images Two-dimensional image coordinates .

[0049] Output a list of marker point correspondences to the user for subsequent tasks such as camera calibration, extrinsic parameter estimation, or 3D reconstruction.

[0050] In this embodiment, a three-dimensional marker set is generated based on multiple randomly placed circular markers in the calibration field, and a two-dimensional marker set is generated based on images of each circular marker taken by the large-distortion lens to be calibrated. Based on the three-dimensional and two-dimensional marker set, a first seed point set including multiple first seed points in the image center region is generated, with each first seed point's three-dimensional spatial coordinate corresponding to a one-to-one two-dimensional image coordinate. Based on the first seed point set and a first preset radius, iterative neighborhood diffusion is performed on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates of all circular markers and the two-dimensional image coordinates. Based on this correspondence, a marker point correspondence list is generated and output. This significantly reduces the difficulty, time cost, and manpower cost of setting up the calibration field, and ensures that marker point decoding is performed from the two-dimensional image... The process progresses layer by layer from the image center region with less lens distortion to the image edge region with greater lens distortion, avoiding the problem of error accumulation and matching failure caused by large distortion when performing point matching across the entire two-dimensional image range. This greatly improves the success rate of landmark point decoding under conditions of large distortion.

[0051] It is worth noting that if the lens to be calibrated is a low-distortion lens, then the two-dimensional image acquired under low-distortion imaging conditions... The image center area with less lens distortion It can be expanded to the entire image area, but only through the central region of the image with minimal lens distortion. By performing high-reliability decoding on each circular marker within the image, the central region of the image can be established. The circular markers within (i.e., the two-dimensional image) This establishes the correspondence between the three-dimensional spatial coordinates of all center markers (points within the circle) and the coordinates of the two-dimensional image, enabling efficient processing of two-dimensional images. High-reliability decoding of each circular marker point.

[0052] Furthermore, it is also possible to capture two-dimensional images acquired under conditions of low distortion. The image center area with less lens distortion Expanding to almost the entire image area, by focusing on the central region of the image where lens distortion is minimal. Each circular marker within the image is decoded with high reliability to establish the image center region. The circular markers within (i.e., the two-dimensional image) The correspondence between the three-dimensional spatial coordinates of most of the center marker points and the coordinates of the two-dimensional image is established, enabling efficient processing of two-dimensional images. High-reliability decoding of most circular markers in the image. Then, focusing on the central region of the image. Each circular marker within the image serves as an anchor point, from which the two-dimensional image... Image center area Perform a neighborhood diffusion to the image edge region to ensure that the marker decoding is performed from the two-dimensional image. Image center area The algorithm advances to the image edge region, improving the decoding accuracy of markers under conditions of low distortion while ensuring efficient decoding of markers.

[0053] In other words, the marker decoding method provided in this application is applicable not only to large distortion imaging scenarios but also to small distortion imaging scenarios, thus improving the versatility of marker decoding.

[0054] Figure 2 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 2 ,like Figure 2 As shown, in step S101 above, a three-dimensional marker set is generated based on multiple randomly placed circular markers in the calibration field, and a two-dimensional marker set is generated based on the images of each circular marker taken by the large distortion lens to be calibrated, including: S201. Scan and measure the calibration field to determine the three-dimensional spatial coordinates of the center of each circular marker point in the calibration field, and generate a three-dimensional marker point set based on the three-dimensional spatial coordinates of the center of each circular marker point.

[0055] Optionally, high-precision metrology methods are used to scan and measure the calibration field to obtain the three-dimensional spatial coordinates of the center of each circular marker point in the calibration field. And based on the three-dimensional spatial coordinates of the center of each circular marker point. Generate a set of three-dimensional marker points 3D marker set ,in, This represents the total number of circular markers.

[0056] For example, high-precision metrology methods can be used for Digital Photogrammetric Analysis (DPA) to ensure the accuracy of the three-dimensional spatial coordinates of the centers of each circular marker point in the calibration field, thereby improving the accuracy of the three-dimensional marker point set. The accuracy.

[0057] S202. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, the ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point, and a two-dimensional marker point set is generated based on the two-dimensional image coordinates of the ellipse center of each circular marker point.

[0058] Optionally, when a planar circular marker in three-dimensional space is projected onto a two-dimensional plane through the perspective projection of the large-distortion lens to be calibrated, the circle will be projected as an ellipse. This results in two-dimensional images of each circular marker in the calibration field captured by the large-distortion lens to be calibrated. In this study, an ellipse detection algorithm is used to detect and locate each circular marker point, and the two-dimensional image coordinates of the ellipse center of each circular marker point are extracted. And based on the two-dimensional image coordinates of the ellipse center of each circular marker point. Generate a set of two-dimensional marker points , ,in, For two-dimensional images The number of circular markers detected.

[0059] For example, the ellipse detection algorithm can be an edge detection algorithm based on arc segment extraction and a least-squares ellipse fitting algorithm to ensure the extraction accuracy of the two-dimensional image coordinates of the ellipse center of each circular marker point, thereby improving the accuracy of the two-dimensional marker point set. The accuracy.

[0060] In this embodiment, the calibration field is scanned and measured to determine the three-dimensional spatial coordinates of the center of each circular marker point in the calibration field. Based on the three-dimensional spatial coordinates of the center of each circular marker point, a three-dimensional marker point set is generated. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, an ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point. Based on the two-dimensional image coordinates of the ellipse center of each circular marker point, a two-dimensional marker point set is generated. This provides an accurate data foundation for marker point decoding.

[0061] Figure 3 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 3 ,like Figure 3 As shown, in step S102 above, generating the first seed point set for the image center region based on the three-dimensional marker point set and the two-dimensional marker point set includes: S301. Determine the center point of the image. Based on the center point of the image and the second preset radius, determine the central region of the image. Based on the central region of the image and the set of two-dimensional marker points, generate a set of two-dimensional central region marker points.

[0062] Optionally, determine the two-dimensional image. center point , in two-dimensional images center point Centered on the circle, the radius is selected as the second preset radius. circular regions as two-dimensional images Image center area Among them, the second preset radius It is necessary to ensure that the selected image center area The lens distortion is relatively small, and it approximately satisfies the ideal perspective projection model.

[0063] Based on two-dimensional marker set Located in the central region of the image The two-dimensional image coordinates of the elliptical centers of each circular marker point within the region are used to generate a set of marker points in the two-dimensional central region. .

[0064] S302. Perform five-point coplanar projection invariant matching on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region.

[0065] Alternatively, since the center markers in the calibration field are randomly arranged, the relative positions of the center markers do not have special geometric structures such as collinearity, and it is impossible to directly use the cross ratio of collinearity to establish the correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates.

[0066] Based on this, the five-point coplanar projection invariant is used to identify the central region of the image. Each of the first seed points within the plane. Specifically, for any five points on the same plane (where any three points are not collinear), there are two independent projection invariants under perspective projection transformation. Theoretically, the values ​​of these two projection invariants calculated in three-dimensional space and two-dimensional image are equal.

[0067] In other words, for the set of three-dimensional marker points and the set of two-dimensional central region markers Perform five-point coplanar projection invariant matching for the image center region with minimal lens distortion. Each circular marker within the image is decoded with high reliability to establish the image center region. The correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each circular marker point within the image center region. Each circular marker within the image is used as a first seed point to generate an image center region comprising multiple first seed points. First seed point set .

[0068] In this embodiment, the center point of the image is determined. Based on the center point and a second preset radius, the central region of the image is determined. A two-dimensional central region marker point set is generated based on the central region and a set of two-dimensional marker points. Five-point coplanar projection invariant matching is performed on the three-dimensional marker point set and the two-dimensional central region marker point set to generate a first seed point set for the central region of the image. Through five-point coplanar projection invariant matching, highly reliable decoding of each circular marker point within the central region of the image, where lens distortion is minimal, is achieved, ensuring a one-to-one correspondence between the three-dimensional spatial coordinates of each first seed point in the first seed point set and the two-dimensional image coordinates.

[0069] Figure 4 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 4 ,like Figure 4 As shown, in step S302 above, five-point coplanar projection invariant matching is performed on the three-dimensional marker point set and the two-dimensional center region marker point set to generate the first seed point set of the image center region, including: S401. Construct multiple three-dimensional five-point combinations in the three-dimensional marker point set, and determine the projection invariant pairs of each three-dimensional five-point combination based on the three-dimensional spatial coordinates of the five three-dimensional coplanar points in each three-dimensional five-point combination.

[0070] The three-dimensional five-point combination includes five three-dimensional coplanar points, and any three of these five three-dimensional coplanar points are not collinear.

[0071] Optionally, in the three-dimensional set of marker points In this process, all combinations of five points are enumerated, and degenerate combinations where any three points are collinear are removed, resulting in a set of three-dimensional marker points. Multiple three-dimensional five-point combinations are formed such that the three-dimensional five-point combination includes five three-dimensional coplanar points, and any three of the five three-dimensional coplanar points are not collinear.

[0072] Based on the three-dimensional spatial coordinates of the five coplanar points in each three-dimensional five-point combination, determine the projection invariant pairs of each three-dimensional five-point combination. ,in, The first projection invariant of the three-dimensional five-point combination, It is the second projection invariant of the three-dimensional five-point combination.

[0073] For example, the five coplanar points in the three-dimensional five-point combination are respectively , , , , ,but , ,in, Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed.

[0074] If any three points in a three-dimensional five-point combination are collinear, then the directed area of ​​the corresponding triangle is 0, which leads to the first projection invariant of the three-dimensional five-point combination. Or the second projection invariant Values ​​of 0, infinity, or unstable values ​​indicate degenerate combinations with extremely poor matching robustness. Therefore, it is necessary to eliminate all degenerate combinations in advance where any three points are collinear.

[0075] in, It is a point The comparison of the line bundles at the vertices, specifically by point. For common vertices, by , , , The cross ratio of the common point bundle formed by these four straight lines. It is a point The comparison of the line bundles at the vertices, specifically by point. For common vertices, by , , , The cross ratio of the common point bundle formed by these four straight lines.

[0076] Based on the projection invariants of each three-dimensional five-point combination The projection invariants of each three-dimensional five-point combination are used as a basis for... Using the index combination of five coplanar points in each three-dimensional five-point combination as the key, a three-dimensional invariant index hash table is established.

[0077] S402. Construct multiple two-dimensional five-point combinations in the set of two-dimensional central region marker points, and determine the projection invariant pairs of each two-dimensional five-point combination based on the two-dimensional image coordinates of the five two-dimensional coplanar points in each two-dimensional five-point combination.

[0078] The five-point combination in two dimensions includes five coplanar points in two dimensions, and any three of these five coplanar points are not collinear.

[0079] Optionally, in the set of marker points in the two-dimensional central region In this process, all combinations of five points are enumerated, and degenerate combinations where any three points are collinear are removed, resulting in the set of marker points for the two-dimensional central region. Multiple two-dimensional five-point combinations are formed such that the combination includes five two-dimensional coplanar points, and any three of these five coplanar points are not collinear.

[0080] Based on the two-dimensional image coordinates of the five coplanar points in each two-dimensional five-point combination, determine the projection invariant pairs of each two-dimensional five-point combination. ,in, The first projection invariant of the two-dimensional five-point combination, It is the second projection invariant of the two-dimensional five-point combination.

[0081] For example, the five coplanar points in the two-dimensional five-point combination are respectively , , , , ,but , ,in, Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed. Indicates the origin of the point , and The directed area of ​​the triangle formed.

[0082] If any three points in a two-dimensional five-point combination are collinear, then the directed area of ​​the corresponding triangle is 0, which leads to the first projection invariant of the two-dimensional five-point combination. Or the second projection invariant Values ​​of 0, infinity, or unstable values ​​indicate degenerate combinations with extremely poor matching robustness. Therefore, it is necessary to eliminate all degenerate combinations in advance where any three points are collinear.

[0083] in, It is a point The comparison of the line bundles at the vertices, specifically by point. For common vertices, by , , , The cross ratio of the common point bundle formed by these four straight lines. It is a point The comparison of the line bundles at the vertices, specifically by point. For common vertices, by , , , The cross ratio of the common point bundle formed by these four straight lines.

[0084] S403. Generate the first seed point set of the image center region based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination.

[0085] Alternatively, by utilizing the property of cross-ratio invariance, the first projection invariant of the three-dimensional five-point combination under perspective projection transformation can be made. Second projection invariant The value of this invariant is theoretically related to the first projection invariant of a two-dimensional five-point combination. Second projection invariant The values ​​remain equal.

[0086] Based on the cross-ratio invariant property, the projection invariants of each three-dimensional five-point combination are... The projection invariant pairs of each two-dimensional five-point combination Perform five-point coplanar projection invariant matching, and establish the image center region based on the matching results. The correspondence between the three-dimensional spatial coordinates of each circular marker point and the two-dimensional image coordinates is used to achieve a central image region with minimal lens distortion. High-reliability decoding of each circular marker point within the image. This involves decoding the central region of the image. Each circular marker within the image is used as a first seed point to generate an image center region comprising multiple first seed points. First seed point set .

[0087] In this embodiment, multiple three-dimensional five-point combinations are constructed from a set of three-dimensional marker points. Based on the three-dimensional spatial coordinates of the five coplanar points in each three-dimensional five-point combination, the projection invariant pairs of each three-dimensional five-point combination are determined. Similarly, multiple two-dimensional five-point combinations are constructed from a set of marker points in the two-dimensional central region. Based on the two-dimensional image coordinates of the five coplanar points in each two-dimensional five-point combination, the projection invariant pairs of each two-dimensional five-point combination are determined. Based on the projection invariant pairs of both the three-dimensional and two-dimensional five-point combinations, a first seed point set for the image central region is generated. Based on the cross-ratio invariant property, five-point coplanar projection invariant matching is performed on the projection invariant pairs of both the three-dimensional and two-dimensional five-point combinations. This achieves highly reliable decoding of the circular marker points within the image central region with minimal lens distortion, ensuring the accuracy of the first seed point set.

[0088] Figure 5 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 5 ,like Figure 5 As shown, in step S403 above, the first seed point set for the image center region is generated based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination, including: S501. Determine the absolute value of the difference between the first and second projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination.

[0089] Optionally, calculate the projection invariant pairs for each three-dimensional five-point combination. The projection invariant pairs of each two-dimensional five-point combination The absolute value of the first projection invariant difference and the absolute value of the difference of the second projection invariants .

[0090] S502. According to the preset threshold, the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination are matched, and the three-dimensional five-point combination and the two-dimensional five-point combination whose absolute values ​​of the first projection invariant difference and the second projection invariant difference are both less than the preset threshold are taken as a candidate matching item in the candidate matching set.

[0091] Optionally, if the absolute value of the difference between the first projection invariant of a three-dimensional five-point combination and a two-dimensional five-point combination... The absolute value of the difference between the second projection invariant All are less than the preset threshold Then the three-dimensional five-point combination and the two-dimensional five-point combination are taken as a candidate matching item in the candidate matching set.

[0092] According to the preset threshold The projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination are matched to generate a candidate matching set. A preset threshold is used. It is based on a two-dimensional image. Detection accuracy of the ellipse center, 2D image The threshold is set based on factors such as noise and lens distortion. If the threshold is too small, it will lead to missed matches, and correct candidate matches will be excluded due to detection errors; if the threshold is set too low, it will cause missed matches. If the value is too large, it will introduce a large number of incorrect candidate matches.

[0093] S503. Based on the preset consistency constraints, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set. Based on the target matching set, multiple correspondences between three-dimensional spatial coordinates and two-dimensional image coordinates are established. The circular marker points corresponding to the three-dimensional spatial coordinates and two-dimensional image coordinates are used as first seed points, and a first seed point set containing multiple first seed points in the image center region is generated.

[0094] Optionally, by utilizing the pre-defined consistency constraints among multiple candidate matching items in the candidate matching set (such as a 3D point should not be matched with multiple 2D points at the same time, and different matching combinations should satisfy geometric compatibility, etc.), a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set.

[0095] Based on the target matching set, establish multiple three-dimensional spatial coordinates and the image center region. A reliable correspondence between multiple two-dimensional image coordinates within the image center region. A circular marker point whose internal three-dimensional spatial coordinates correspond to two-dimensional image coordinates is used as a first seed point, and an image center region containing multiple first seed points is generated. First seed point set .

[0096] In this embodiment, the absolute values ​​of the first and second projection invariant differences between the projection invariant pairs of each 3D five-point combination and the projection invariant pairs of each 2D five-point combination are determined. Based on a preset threshold, the projection invariant pairs of each 3D five-point combination and each 2D five-point combination are matched. Three-dimensional and two-dimensional five-point combinations whose absolute values ​​of both the first and second projection invariant differences are less than the preset threshold are selected as candidate matching items in the candidate matching set. According to a preset consistency constraint, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set, resulting in a target matching set. Based on the target matching set, multiple correspondences between 3D spatial coordinates and 2D image coordinates are established. Circular marker points corresponding to the 3D spatial coordinates and 2D image coordinates are used as first seed points, and a first seed point set containing multiple first seed points is generated for the image center region. This ensures the accuracy of the five-point coplanar projection invariant matching, thereby improving the accuracy of the first seed point set.

[0097] Figure 6 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 6 ,like Figure 6 As shown, in step S103 above, based on the first seed point set and a preset radius, iterative neighborhood diffusion is performed on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points, including: S601. Based on the first seed point set and the first preset radius, generate the neighborhood point set of each first seed point in the first seed point set, and determine the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point.

[0098] Optionally, for the first seed point set First seed point in each Around its two-dimensional image coordinates with a first preset radius Perform a neighborhood search to generate the first seed point set. First seed point in each neighborhood point set .

[0099] Based on each first seed point neighborhood point set Local homography matrix estimation is performed to obtain the first seed points. Local homography matrix of the region .

[0100] The accuracy of the homography matrix decreases as the region expands, through the first seed point. Fit the first seed point in the neighborhood. Local homography matrix of the region It can effectively resist radial distortion of lenses with high distortion.

[0101] S602. Generate a second seed point set based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set.

[0102] Optionally, based on each first seed point neighborhood point set Local homography matrix of the region where each first seed point is located and the first seed point set In the region where each first seed point is located, the marker points are decoded to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates in the region where each first seed point is located, and a second seed point set is generated. .

[0103] Among them, the second seed point set The number of seed points in the first seed point set is greater than that in the second seed point set. The number of seed points in the dataset.

[0104] S603. Iteratively perform neighborhood search and local homography matrix determination for each second seed point in the second seed point set until the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points is established.

[0105] Optionally, with a second set of seed points Each of the two seed points is used as an anchor point, and the two points are arranged around their two-dimensional image coordinates with a first preset radius. Continue the neighborhood search to generate a second set of seed points. The set of neighborhood points for each second seed point is used, and local homography matrix estimation is performed based on the neighborhood points of each second seed point to obtain the homography matrix of the region where each second seed point is located. Based on the neighborhood points of each second seed point and the local homography matrix of the region where each second seed point is located, marker point decoding is performed in the region where each second seed point is located, and the correspondence between each three-dimensional spatial coordinate and each two-dimensional image coordinate in the region where each second seed point is located is established, and the set of second seed points is continuously expanded. This causes the marker decoding to continuously iterate and spread to the neighborhood of the image edge region where the lens distortion is large.

[0106] After a certain round of iterative neighborhood diffusion, the second seed point set... No new seed points were added, indicating that the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular markers has been established, meaning that all circular markers have been successfully decoded, and the iteration stops at this point.

[0107] In this embodiment, based on the first seed point set and the first preset radius, a neighborhood point set is generated for each first seed point in the first seed point set. Then, based on the neighborhood point sets of each first seed point, the local homography matrix of the region where each first seed point is located is determined. Based on the neighborhood point sets of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set, a second seed point set is generated. The neighborhood search and local homography matrix determination for each second seed point in the second seed point set are iteratively performed until the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points is established. Iterative neighborhood diffusion ensures that marker point decoding progresses layer by layer from the image center region to the image edge region of the two-dimensional image, improving the success rate of marker point decoding under conditions of large distortion imaging.

[0108] Figure 7 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 7 ,like Figure 7 As shown, in step S601 above, determining the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point includes: S701. Determine the number of decoding marker points in the neighborhood point set of the current first seed point.

[0109] Among them, the decoding marker is a circular marker that establishes a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates.

[0110] Optionally, with the current first seed point For example, to determine the current first seed point Local response matrix of the region The process will be explained.

[0111] Statistics on the current first seed point neighborhood point set The number of circular markers that have been successfully decoded in the Chinese database That is, to count the current first seed point neighborhood point set Number of decoding markers Among them, the decoding marker is a circular marker that establishes a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates.

[0112] S702. If the number of decoded markers in the neighborhood set of the current first seed point is greater than or equal to a preset value, then the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker in the neighborhood set of the current first seed point.

[0113] Optionally, since the homography matrix is ​​a 3×3 homogeneous matrix with 8 degrees of freedom, the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of a decoding marker can provide 2 linear equations. Therefore, at least 4 decoding markers are needed to uniquely solve the homography matrix. In other words, the minimum condition number for homography matrix estimation is 4, which is the preset value of 4.

[0114] If the current first seed point neighborhood point set Number of decoding markers If the minimum condition number for homography matrix estimation is greater than or equal to 4, then based on the current first seed point... neighborhood point set At least 4 decoding markers are needed to determine the current first seed point. Local homography matrix of the region .

[0115] S703. Otherwise, skip the current first seed point and determine the local homography matrix of the region where the next first seed point is located based on the neighborhood point set of the next first seed point of the current first seed point.

[0116] Optionally, if the current first seed point neighborhood point set Number of decoding markers If the number is less than 4, the minimum condition number for homography matrix estimation is not met, so the current first seed point is temporarily skipped. Process the current first seed point The next first seed point .

[0117] Specifically, based on the next first seed point neighborhood point set Determine the next first seed point Local homography matrix of the region .

[0118] The current first seed point in subsequent iterations neighborhood point set The number of decoding markers gradually increases, at the current first seed point neighborhood point set When the number of decoding markers increases to meet the minimum condition number, it can be based on the current first seed point. neighborhood point set At least 4 decoding markers are needed to determine the current first seed point. Local homography matrix of the region .

[0119] In this embodiment, the number of decoded marker points in the neighborhood set of the current first seed point is determined. If the number of decoded marker points in the neighborhood set of the current first seed point is greater than or equal to a preset value, the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker point in the neighborhood set of the current first seed point. Otherwise, the current first seed point is skipped, and the local homography matrix of the region where the next first seed point is located is determined based on the neighborhood set of the next first seed point. This ensures the accuracy of the local homography matrix of the region where each first seed point is located.

[0120] Figure 8 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 8 ,like Figure 8 As shown, in step S702 above, determining the local homography matrix of the region where the current first seed point is located based on the decoding markers in the neighborhood point set of the current first seed point includes: S801. If the number of decoded markers in the neighborhood of the current first seed point is a preset value, then the DLT algorithm is used to determine the local homography matrix of the region where the current first seed point is located.

[0121] Optionally, if the current first seed point neighborhood point set Number of decoding markers If the value is equal to the preset value of 4, then the Direct Linear Transformation (DLT) algorithm is used to estimate the current first seed point. Local homography matrix of the region .

[0122] S802. If the number of decoded markers in the neighborhood of the current first seed point is greater than a preset value, the RANSAC algorithm or the least squares estimation algorithm is used to determine the local homography matrix of the region where the current first seed point is located.

[0123] Optionally, if the current first seed point neighborhood point set Number of decoding markers If the value is greater than the preset value of 4, the Random Sampling Consensus (RANSAC) algorithm is used for robust estimation to determine the current first seed point. Local homography matrix of the region .

[0124] Specifically, randomly select the current first seed point. neighborhood point set Calculate all candidate local homography matrices using the four decoding markers, and use the current first seed point. neighborhood point set The remaining decoding markers evaluate the number of interior points in each candidate local homography matrix, and the candidate local homography matrix with the most interior points is selected as the current first seed point. Local homography matrix of the region To improve the current first seed point Local homography matrix of the region The accuracy.

[0125] Or, if the current first seed point neighborhood point set Number of decoding markers If the value is greater than the preset value of 4, then the current first seed point will be used directly. neighborhood point set The first seed point is obtained by performing least squares estimation on all decoded marker points. Local homography matrix of the region To improve the current first seed point Local homography matrix of the region The accuracy.

[0126] In this embodiment, if the number of decoded markers in the neighborhood set of the current first seed point is a preset value, the DLT algorithm is used to determine the local homography matrix of the region where the current first seed point is located; if the number of decoded markers in the neighborhood set of the current first seed point is greater than the preset value, the RANSAC algorithm or the least squares estimation algorithm is used to determine the local homography matrix of the region where the current first seed point is located. This maximizes the accuracy of the local homography matrix.

[0127] Figure 9 A flowchart illustrating the marker decoding method provided in this application embodiment. Figure 9 ,like Figure 9 As shown, in step S602 above, the generation of the second seed point set based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set includes: S901. Based on the local homography matrix of the region where each first seed point is located, perform back projection transformation on the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point.

[0128] Optionally, for the current first seed point neighborhood point set For non-decoded markers in the data, the current first seed point is used. Local homography matrix of the region Two-dimensional image coordinates of undecoded markers Perform a back projection transformation to obtain the current first seed point. neighborhood point set Predicted 3D spatial coordinates of decoding markers in China and Africa .

[0129] Among them, the predicted three-dimensional spatial coordinates of the undecoded markers .

[0130] S902. In the set of three-dimensional marker points, determine the candidate three-dimensional spatial coordinates that have the closest Euclidean distance to the predicted three-dimensional spatial coordinates of each non-decoded marker point. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker point and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, then the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker point.

[0131] Optionally, in the three-dimensional set of marker points Searching for and predicting the 3D spatial coordinates of undecoded markers The closest candidate 3D space coordinates in Euclidean distance If not the predicted three-dimensional spatial coordinates of the decoded markers With candidate three-dimensional spatial coordinates The Euclidean distance is less than the preset distance threshold. Then the candidate three-dimensional spatial coordinates The three-dimensional spatial coordinates of the undecoded marker points.

[0132] S903. Establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each non-decoded marker point, update each non-decoded marker point to a second seed point, and add each second seed point to the first seed point set to obtain the second seed point set.

[0133] Optionally, establish the three-dimensional spatial coordinates of the undecoded markers. with two-dimensional image coordinates The correspondence between them makes the non-decoded marker point become the newly decoded marker point, and the newly decoded marker point is updated as the second seed point and added to the first seed point set. In the middle, the set of first seed points with multiple second seed points will be added. As the second seed point set .

[0134] By iteratively expanding the neighborhood, the set of second seed points is gradually increased. The number of seed points in the image ensures that the marker points are decoded from the 2D image. Image center area Progress layer by layer towards the edge region of the image.

[0135] In this embodiment, based on the local homography matrix of the region where each first seed point is located, the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point are back-projected to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point. In the three-dimensional marker point set, the candidate three-dimensional spatial coordinates with the closest Euclidean distance to the predicted three-dimensional spatial coordinates of each non-decoded marker point are determined. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker point and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker point. A correspondence is established between the three-dimensional spatial coordinates of each non-decoded marker point and the two-dimensional image coordinates. Each non-decoded marker point is then updated as a second seed point, and each second seed point is added to the first seed point set to obtain the second seed point set. Through iterative neighborhood diffusion, the number of seed points in the second seed point set is gradually increased to ensure that the marker point decoding is performed from the two-dimensional image... The image progresses layer by layer from the image center region to the image edge region.

[0136] Based on the same inventive concept, this application also provides a marker point decoding device corresponding to the marker point decoding method. Since the principle of the device in this application is similar to the marker point decoding method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0137] Figure 10 This is a module structure diagram of the marker decoding device provided in the embodiments of this application, as shown below. Figure 10 As shown, the device includes: The generation module 1001 is used to generate a three-dimensional marker set based on multiple randomly arranged circular markers in the calibration field, and to generate a two-dimensional marker set based on the images of each circular marker taken by the large distortion lens to be calibrated; wherein, the three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each circular marker, and the two-dimensional marker set includes the two-dimensional image coordinates of the ellipse center of each circular marker. The generation module 1001 is also used to generate a first seed point set for the central region of the image based on the three-dimensional marker point set and the two-dimensional marker point set. The first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates. The diffusion module 1002 is used to perform iterative neighborhood diffusion on the first seed point set according to the first seed point set and the first preset radius, and to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points. The output module 1003 is used to generate a list of correspondences between circular markers based on the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each marker, and to output the list of correspondences between markers.

[0138] As an optional implementation, the generation module 1001 is specifically used for: The calibration field is scanned and measured to determine the three-dimensional spatial coordinates of the center of each circular marker point in the calibration field, and a three-dimensional marker point set is generated based on the three-dimensional spatial coordinates of the center of each circular marker point. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, an ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point, and a two-dimensional marker point set is generated based on the two-dimensional image coordinates of the ellipse center of each circular marker point.

[0139] As an optional implementation, the generation module 1001 is specifically used for: Determine the center point of the image, determine the central region of the image based on the center point and the second preset radius, and generate a set of two-dimensional central region marker points based on the central region of the image and the set of two-dimensional marker points. Perform five-point coplanar projection invariant matching on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region.

[0140] As an optional implementation, the generation module 1001 is specifically used for: Construct multiple 3D five-point combinations in the set of 3D marker points, and determine the projection invariant pairs of each 3D five-point combination based on the 3D spatial coordinates of the five coplanar 3D points in each 3D five-point combination. The 3D five-point combination includes five coplanar 3D points, and any three of the five coplanar 3D points are not collinear. Construct multiple two-dimensional five-point combinations in the set of two-dimensional central region marker points, and determine the projection invariant pairs of each two-dimensional five-point combination based on the two-dimensional image coordinates of the five two-dimensional coplanar points in each two-dimensional five-point combination. The two-dimensional five-point combination includes five two-dimensional coplanar points, and any three of the five two-dimensional coplanar points are not collinear. Based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination, the first seed point set of the image center region is generated.

[0141] As an optional implementation, the generation module 1001 is specifically used for: Determine the absolute value of the difference between the first and second projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination; According to the preset threshold, the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination are matched, and the three-dimensional five-point combination and the two-dimensional five-point combination whose absolute values ​​of the first projection invariant difference and the second projection invariant difference are both less than the preset threshold are taken as a candidate matching item in the candidate matching set. Based on the preset consistency constraints, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set. Based on the target matching set, multiple correspondences between three-dimensional spatial coordinates and two-dimensional image coordinates are established. The circular marker points corresponding to the three-dimensional spatial coordinates and two-dimensional image coordinates are used as first seed points, and a first seed point set containing multiple first seed points in the image center region is generated.

[0142] As an optional implementation, the diffusion module 1002 is specifically used for: Based on the first seed point set and the first preset radius, generate the neighborhood point set of each first seed point in the first seed point set, and determine the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point. A second seed point set is generated based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the set of first seed points. Iteratively perform neighborhood search and local homography matrix determination for each second seed point in the second seed point set until the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points is established.

[0143] As an optional implementation, the diffusion module 1002 is specifically used for: Determine the number of decoding markers in the neighborhood of the current first seed point. The decoding markers are circular markers that establish a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates. If the number of decoded markers in the neighborhood set of the current first seed point is greater than or equal to a preset value, then the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker in the neighborhood set of the current first seed point. Otherwise, skip the current first seed point and determine the local homography matrix of the region where the next first seed point is located based on the neighborhood point set of the next first seed point.

[0144] As an optional implementation, the diffusion module 1002 is specifically used for: If the number of decoded markers in the neighborhood of the current first seed point is a preset value, then the DLT algorithm is used to determine the local homography matrix of the region where the current first seed point is located. If the number of decoded markers in the neighborhood of the current first seed point is greater than a preset value, then the RANSAC algorithm or the least squares estimation algorithm is used to determine the local homography matrix of the region where the current first seed point is located.

[0145] As an optional implementation, the diffusion module 1002 is specifically used for: Based on the local homography matrix of the region where each first seed point is located, the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point are back-projected and transformed to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point. In the set of three-dimensional markers, determine the candidate three-dimensional spatial coordinates that are closest to the predicted three-dimensional spatial coordinates of each non-decoded marker in terms of Euclidean distance. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, then the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker. Establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each non-decoded marker point, update each non-decoded marker point as a second seed point, and add each second seed point to the first seed point set to obtain the second seed point set.

[0146] This application also provides an electronic device, such as... Figure 11 The diagram shown is a schematic representation of the structure of an electronic device provided in an embodiment of this application, including a processor 111, a memory 112, and a bus 113. The memory 112 stores machine-readable instructions executable by the processor 111. When the electronic device is running, the processor 111 communicates with the memory 112 via the bus 113, and the processor 111 executes the machine-readable instructions to perform the steps of the marker point decoding method in the aforementioned embodiment.

[0147] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the marker point decoding method described in the foregoing embodiments.

[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0150] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A marker point decoding method, characterized in that, include: A three-dimensional marker set is generated based on multiple randomly placed circular markers in the calibration field, and a two-dimensional marker set is generated based on the images of each of the circular markers captured by the large distortion lens to be calibrated; wherein, the three-dimensional marker set includes the three-dimensional spatial coordinates of the center of each of the circular markers, and the two-dimensional marker set includes the two-dimensional image coordinates of the ellipse center of each of the circular markers. Based on the three-dimensional marker point set and the two-dimensional marker point set, a first seed point set for the image center region is generated. The first seed point set includes multiple first seed points, and the three-dimensional spatial coordinates of each first seed point correspond one-to-one with the two-dimensional image coordinates. Based on the first seed point set and the first preset radius, iterative neighborhood diffusion is performed on the first seed point set to establish the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of all circular marker points. Based on the correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates of each circular marker, a marker correspondence list is generated and output.

2. The method according to claim 1, characterized in that, The process of generating a three-dimensional marker set based on multiple randomly placed circular markers in the calibration field, and generating a two-dimensional marker set based on images of each of the circular markers captured by the large-distortion lens to be calibrated, includes: The calibration field is scanned and measured to determine the three-dimensional spatial coordinates of the center of each of the circular marker points in the calibration field, and the three-dimensional marker point set is generated based on the three-dimensional spatial coordinates of the center of each of the circular marker points. In the images of each circular marker point in the calibration field captured by the large distortion lens to be calibrated, an ellipse detection algorithm is used to determine the two-dimensional image coordinates of the ellipse center of each circular marker point, and the two-dimensional marker point set is generated based on the two-dimensional image coordinates of the ellipse center of each circular marker point.

3. The method according to claim 1, characterized in that, The step of generating a first seed point set for the image center region based on the three-dimensional marker point set and the two-dimensional marker point set includes: Determine the center point of the image, determine the central region of the image based on the center point and the second preset radius, and generate a two-dimensional central region marker point set based on the central region of the image and the two-dimensional marker point set. Five-point coplanar projection invariant matching is performed on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region.

4. The method according to claim 3, characterized in that, The step of performing five-point coplanar projection invariant matching on the three-dimensional marker point set and the two-dimensional central region marker point set to generate the first seed point set of the image central region includes: Construct multiple three-dimensional five-point combinations in the set of three-dimensional marker points, and determine the projection invariant pairs of each three-dimensional five-point combination based on the three-dimensional spatial coordinates of the five three-dimensional coplanar points in each three-dimensional five-point combination. The three-dimensional five-point combination includes five three-dimensional coplanar points, and any three of the five three-dimensional coplanar points are not collinear. Construct multiple two-dimensional five-point combinations in the set of two-dimensional central region marker points, and determine the projection invariant pairs of each two-dimensional five-point combination based on the two-dimensional image coordinates of the five two-dimensional coplanar points in each two-dimensional five-point combination. The two-dimensional five-point combination includes five two-dimensional coplanar points, and any three of the five two-dimensional coplanar points are not collinear. The first seed point set for the central region of the image is generated based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination.

5. The method according to claim 4, characterized in that, The step of generating the first seed point set for the image center region based on the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination includes: Determine the absolute value of the difference between the first and second projection invariants of each three-dimensional five-point combination and each two-dimensional five-point combination; According to a preset threshold, the projection invariant pairs of each three-dimensional five-point combination and the projection invariant pairs of each two-dimensional five-point combination are matched, and the three-dimensional five-point combination and the two-dimensional five-point combination whose absolute values ​​of the first projection invariant difference and the second projection invariant difference are both less than the preset threshold are taken as a candidate matching item in the candidate matching set. Based on the preset consistency constraints, a voting mechanism or graph matching algorithm is used to eliminate contradictory candidate matching items in the candidate matching set to obtain the target matching set. Based on the target matching set, multiple correspondences between three-dimensional spatial coordinates and two-dimensional image coordinates are established. The circular marker points corresponding to the three-dimensional spatial coordinates and two-dimensional image coordinates are used as first seed points, and a first seed point set containing multiple first seed points is generated for the image center region.

6. The method according to claim 1, characterized in that, The step of iteratively expanding the neighborhood of the first seed point set based on the first seed point set and a preset radius to establish the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points includes: Based on the first seed point set and the first preset radius, generate a neighborhood point set for each first seed point in the first seed point set, and determine the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point. A second seed point set is generated based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set. Iteratively perform neighborhood search and local homography matrix determination for each second seed point in the second seed point set until the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of all circular marker points is established.

7. The method according to claim 6, characterized in that, The step of determining the local homography matrix of the region where each first seed point is located based on the neighborhood point set of each first seed point includes: Determine the number of decoding markers in the neighborhood set of the current first seed point. The decoding markers are circular markers that establish a correspondence between three-dimensional spatial coordinates and two-dimensional image coordinates. If the number of decoded markers in the neighborhood set of the current first seed point is greater than or equal to a preset value, then the local homography matrix of the region where the current first seed point is located is determined based on each decoded marker in the neighborhood set of the current first seed point. Otherwise, skip the current first seed point and determine the local homography matrix of the region where the next first seed point is located based on the neighborhood point set of the next first seed point.

8. The method according to claim 7, characterized in that, The step of determining the local homography matrix of the region where the current first seed point is located based on the decoding markers in the neighborhood point set of the current first seed point includes: If the number of decoded markers in the neighborhood of the current first seed point is the preset value, then the Direct Linear Transform (DLT) algorithm is used to determine the local homography matrix of the region where the current first seed point is located. If the number of decoded markers in the neighborhood of the current first seed point is greater than the preset value, the local homography matrix of the region where the current first seed point is located is determined by the Random Sample Consensus (RANSAC) algorithm or the least squares estimation algorithm.

9. The method according to claim 6, characterized in that, The step of generating a second seed point set based on the neighborhood point set of each first seed point, the local homography matrix of the region where each first seed point is located, and the first seed point set includes: Based on the local homography matrix of the region where each first seed point is located, the two-dimensional image coordinates of the non-decoded marker points in the neighborhood point set of each first seed point are back-projected and transformed to obtain the predicted three-dimensional spatial coordinates of the non-decoded marker points in the neighborhood point set of each first seed point. In the set of three-dimensional marker points, determine the candidate three-dimensional spatial coordinates that are closest to the predicted three-dimensional spatial coordinates of each non-decoded marker point in terms of Euclidean distance. If the Euclidean distance between the predicted three-dimensional spatial coordinates of the non-decoded marker point and the candidate three-dimensional spatial coordinates is less than a preset distance threshold, then the candidate three-dimensional spatial coordinates are used as the three-dimensional spatial coordinates of the non-decoded marker point. Establish the correspondence between the three-dimensional spatial coordinates and two-dimensional image coordinates of each non-decoded marker point, update each non-decoded marker point as a second seed point, and add each second seed point to the first seed point set to obtain the second seed point set.

10. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the marker decoding method as described in any one of claims 1 to 9.

Images