Landmark point matching method, splicing method, device and storage medium

Abstract

This application relates to a marker point matching method, splicing method, apparatus, device, and storage medium. The method includes: obtaining a current frame marker point set and a buffer frame marker point set; the current frame marker point set includes current frame marker points; the buffer frame marker points in the buffer frame marker point set are selected from global marker points; obtaining a first distance matrix corresponding to the current frame marker points and a second distance matrix corresponding to the buffer frame marker points; based on the first and second distance matrices, obtaining a pair of matching points with the same name for the current frame marker points and the buffer frame marker points; the matching point pair includes the same-name points of the current frame marker points; updating the buffer frame marker points based on the same-name points of the current frame marker points to obtain updated buffer frame marker points; the updated buffer frame marker points are used for matching with marker points in the next frame. This method can shorten the marker point matching time, thereby improving real-time performance.

Description

Technical Field

[0001] This application relates to the field of three-dimensional measurement technology, and in particular to a marker point matching method, marker point stitching method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Three-dimensional measurement can quickly obtain three-dimensional surface data of a target surface and has been widely used in fields such as industrial inspection and three-dimensional reconstruction.

[0003] Limited by the camera's limited field of view and the common occlusion issues during actual measurements, a single measurement cannot achieve complete coverage of the target object's surface. Therefore, when using a handheld scanner for dynamic measurement, it is necessary to rapidly acquire 3D data of various regions of the target object's surface from different perspectives. Subsequently, the local 3D data obtained from multiple measurements are precisely stitched together in the same coordinate system to construct a complete 3D model. Currently, multi-view... Figure 3 There are three methods for stitching dimensional data: (1) stitching method based on high-precision mobile devices; (2) stitching method based on the inherent features of the object itself; and (3) stitching method based on marker points.

[0004] However, current marker-based splicing methods suffer from a huge number of markers to match in large-scale scenarios with thousands or tens of thousands of markers, resulting in long matching times and failing to meet real-time requirements. Summary of the Invention

[0005] Therefore, it is necessary to provide a marker matching method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can reduce marker matching time and improve real-time performance in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a marker point matching method, the method comprising:

[0007] Obtain the current frame marker set and the buffer frame marker set; wherein, the current frame marker set includes current frame markers; the buffer frame markers in the buffer frame marker set are selected from global markers; the intersection of the current frame marker set and the buffer frame marker set is not an empty set; and the number of current frame markers is less than the number of buffer frame markers.

[0008] Obtain the first distance matrix corresponding to the current frame marker and the second distance matrix corresponding to the buffer frame marker;

[0009] Based on the first distance matrix and the second distance matrix, a pair of matching points with the same name between the current frame marker and the buffer frame marker is obtained; the pair of matching points with the same name includes the same name as the current frame marker.

[0010] The buffer frame marker is updated based on the corresponding point of the current frame marker to obtain the updated buffer frame marker; the updated buffer frame marker is used for matching with the marker of the next frame; wherein, the marker of the next frame is the marker corresponding to the next frame of the current frame.

[0011] In one embodiment, updating the buffer frame marker based on the corresponding point of the current frame marker to obtain the updated buffer frame marker includes:

[0012] Write the corresponding point of the current frame marker into the buffer frame, and record the frame index corresponding to the corresponding point written into the buffer frame.

[0013] Obtain a first seed point from the current frame marker points, use the first seed point as a reference point to obtain a first number of target marker points from the global marker points, write them into the buffer frame, and record the frame index corresponding to the target marker point;

[0014] If the number of markers stored in the buffer frame is greater than the quantity threshold, then the marker corresponding to the smallest frame index in the buffer frame is deleted to obtain the updated buffer frame markers; wherein, the number of the first target is less than the quantity threshold.

[0015] In one embodiment, the method further includes:

[0016] If the number of markers stored in the buffer frame is less than or equal to the quantity threshold, then a second seed point is obtained from the current frame markers, and a second target number of target markers is obtained from the global markers using the second seed point as a reference point, and written into the buffer frame to obtain updated buffer frame markers; wherein, the second target number is determined based on the number of markers stored in the buffer frame and the quantity threshold; the markers corresponding to the first seed point and the second seed point are different.

[0017] In one embodiment, obtaining the same-name matching point pair between the current frame marker and the buffer frame marker based on the first distance matrix and the second distance matrix includes:

[0018] Based on the first distance matrix and the second distance matrix, a set of potential matching point pairs is determined; wherein, the set of potential matching point pairs is a set of potential matching point pairs with the same name between the current frame marker and the buffer frame marker;

[0019] Based on the set of potential matching point pairs, determine the baseline matching point pairs;

[0020] Based on the benchmark matching point pairs and the set of potential matching point pairs, a set of target congruent triangles is determined;

[0021] Based on the set of congruent target triangles, the same-name matching point pairs of the current frame marker point and the buffer frame marker point are obtained.

[0022] In one embodiment, the potential matching point pair set includes a plurality of potential matching point pairs; determining the benchmark matching point pair based on the potential matching point pair set includes:

[0023] Each potential matching point pair is taken as a target potential matching point pair, and the distance between the marker point in the target potential matching point pair and the corresponding marker point in the candidate potential matching point pair is determined; wherein, the candidate potential matching point pair represents potential matching point pairs other than the target potential matching point pairs in the potential matching point pair set;

[0024] Count the number of identical distances for each potential matching point pair; wherein, the identical distances represent the distances that satisfy a distance threshold;

[0025] The baseline matching point pair is determined based on the potential matching point pairs with the largest number of points of equal distance.

[0026] In one embodiment, the potential matching point pair set includes multiple potential matching point pairs; determining a set of target congruent triangles based on the benchmark matching point pairs and the potential matching point pair set includes:

[0027] Based on the benchmark matching point and the set of potential matching point pairs, any two potential matching point pairs are selected to construct a triangle;

[0028] For each set of triangles, if the three sides of two triangles in the set are all equal, the set of triangles is a set of congruent triangles.

[0029] For each set of congruent triangles, determine the initial pose of the set of congruent triangles, and based on the initial pose, transform the current frame marker point and the buffer frame marker point to the same coordinate system;

[0030] In the same coordinate system, obtain the same matching point pairs of each set of congruent triangles in the initial pose;

[0031] Count the number of identical matching point pairs for each set of congruent triangles in the initial pose;

[0032] Based on the set of congruent triangles with the largest number of identical matching point pairs, determine the set of target congruent triangles.

[0033] Secondly, this application also provides a method for splicing marker points, the method comprising:

[0034] Obtain the same-name matching point pairs of the current frame marker and the buffer frame marker as described above; the same-name matching point pairs include the same-name point of the current frame marker and the same-name point of the buffer frame marker corresponding to the same-name point of the current frame marker.

[0035] The global marker is obtained by concatenating the corresponding points of the current frame marker and the corresponding points of the buffer frame marker.

[0036] Thirdly, this application also provides a marker matching device, the device comprising:

[0037] The first acquisition module is used to acquire a current frame marker set and a buffer frame marker set; wherein, the current frame marker set includes current frame markers; the buffer frame markers in the buffer frame marker set are selected from global markers; the intersection of the current frame marker set and the buffer frame marker set is not an empty set; and the number of current frame markers is less than the number of buffer frame markers.

[0038] The second acquisition module is used to acquire the first distance matrix corresponding to the current frame marker and the second distance matrix corresponding to the buffer frame marker;

[0039] The determining module is configured to obtain a pair of matching points with the same name between the current frame marker and the buffer frame marker based on the first distance matrix and the second distance matrix; the pair of matching points with the same name includes the same point as the current frame marker;

[0040] An update module is used to update the buffer frame marker based on the same point of the current frame marker to obtain the updated buffer frame marker; the updated buffer frame marker is used for matching with the marker of the next frame; wherein, the marker of the next frame is the marker corresponding to the next frame of the current frame.

[0041] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0042] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0043] The aforementioned marker matching method, apparatus, computer device, computer-readable storage medium, and computer program product first acquire the current frame marker set and the buffer frame marker set. The buffer frame markers in the buffer frame marker set are selected from global markers. A buffer frame mechanism is introduced during the marker matching process. Since the number of buffer frame markers is much smaller than the number of global markers, the number of markers involved in subsequent calculations can be reduced. Second, corresponding matching point pairs are determined by comparing the first distance matrix of the current frame markers with the second distance matrix of the buffer frame markers. This significantly reduces the calculation of matching redundant and invalid markers with the current marker, saving matching time. Finally, the buffer frame markers are updated based on the corresponding points of the current frame markers, resulting in updated buffer frame markers, which are then used for matching with markers in the next frame. Based on the continuous spatial characteristics of scanner-acquired data, a buffer frame mechanism is proposed to limit the number of global markers involved in the matching process, reducing the matching time of the current marker and thus improving real-time performance.

[0044] The aforementioned marker stitching method, by introducing a buffer frame mechanism to determine the corresponding points of the current frame marker and the corresponding points of the buffer frame marker, stitches together the corresponding points of the current frame marker and the corresponding points of the buffer frame marker to obtain the global marker point stitched with the current frame marker. The buffer frame mechanism reduces redundant calculations, significantly improving stitching efficiency and fully meeting the real-time requirements of scanners in practical applications. It avoids the problem of traditional methods, which, when dealing with large-scale scenes with thousands or tens of thousands of marker points, consume a lot of time and cannot achieve the goal of fast and real-time stitching. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating a marker matching method in one embodiment;

[0047] Figure 2 This is a schematic diagram of the current frame marker set and the buffer frame marker set in one embodiment;

[0048] Figure 3 This is a schematic diagram of the process for updating buffer frame flags in one embodiment;

[0049] Figure 4 This is a schematic diagram of the process of obtaining the same-name matching point pairs of the current frame marker point and the buffer frame marker point based on the first distance matrix and the second distance matrix in one embodiment;

[0050] Figure 5 This is a schematic diagram of the process for determining a baseline matching point pair based on a set of potential matching point pairs in one embodiment;

[0051] Figure 6 This is a flowchart illustrating the process of determining a set of target congruent triangles based on a set of baseline matching point pairs and a set of potential matching point pairs in one embodiment.

[0052] Figure 7 This is a schematic diagram of the concatenation of the current frame marker and the buffer frame marker in one embodiment;

[0053] Figure 8 This is a structural block diagram of a marker matching device in one embodiment;

[0054] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] In one embodiment, such as Figure 1 As shown, a marker point matching method is provided. This embodiment illustrates the application of this method to a scanner, including but not limited to handheld scanners. This embodiment includes the following steps S102 to S108. Wherein:

[0057] Step S102: Obtain the current frame marker set and the buffer frame marker set; wherein, the current frame marker set includes the current frame markers; the buffer frame markers in the buffer frame marker set are selected from the global markers.

[0058] The current frame is a complete dataset or snapshot of data captured in a single scan by the scanner. Current frame markers are specific markers identified within the current frame. A buffer frame is a temporary dataset that stores recently acquired point cloud frames or keyframe data. Buffer frame markers are specific markers identified within the buffer frame. Global markers differ at different times.

[0059] The buffer frame marker is a part of the global marker, therefore it is selected from the global marker. The method for determining the buffer frame marker corresponding to the current frame includes: updating the buffer frame marker corresponding to the previous frame marker based on the previous frame marker and the corresponding global marker, obtaining the updated buffer frame marker, and matching the updated buffer frame marker with the current frame marker.

[0060] Further, the corresponding points of the previous frame's markers are written into the buffer frame corresponding to the previous frame's markers, and the frame index corresponding to the corresponding points written into the buffer frame is recorded. A first seed point is obtained from the previous frame's markers, and a first number of target markers are obtained from the global markers corresponding to the previous frame using the first seed point as a reference point, and written into the buffer frame, with the frame index corresponding to the target marker recorded. If the number of markers stored in the buffer frame is greater than a quantity threshold, the marker corresponding to the smallest frame index in the buffer frame is deleted, resulting in an updated buffer frame marker. The first number of targets is less than the quantity threshold. If the number of markers stored in the buffer frame corresponding to the previous frame's markers is less than or equal to the quantity threshold, a second seed point is obtained from the previous frame's markers, and a second number of target markers are obtained from the global markers corresponding to the previous frame's markers using the second seed point as a reference point, and written into the buffer frame, resulting in an updated buffer frame marker. The second number of targets is determined based on the number of markers stored in the buffer frame and the quantity threshold. The markers corresponding to the first seed point and the second seed point are different.

[0061] Similarly, the method for determining the buffer frame marker corresponding to the next frame of the current frame includes: the current frame marker and the global marker corresponding to the current frame, updating the buffer frame marker corresponding to the current frame marker to obtain the updated buffer frame marker, and matching the updated buffer frame marker with the next frame marker of the next frame.

[0062] For example, if both the current frame and the buffer frame are point cloud frames, and the current frame is the point cloud frame with frame index 100, then the global marker points are determined by point cloud frames with frame indices 1 to 99. The buffer frame can include point cloud frames with frame indices 99 and 98, or it can be point cloud frames with frame indices 99, 98, and 97. In this case, the current frame marker point is the marker point in the point cloud frame with frame index 99; the buffer frame marker points could be, for example, the marker points in the point cloud frames with frame indices 99 and 98. The buffer frame records the marker points based on the previous... A subset of global markers determined by the spatial location of the frame, wherein the number of markers does not exceed Because of the continuous spatial nature of scanner data acquisition, the marker points in adjacent frames are not significantly different. Therefore, there must be some matching points between the marker points in the buffer frame and those in the current frame. Thus, the intersection of the current frame marker set and the buffer frame marker set is not empty. The number of marker points in the current frame is less than the number of marker points in the buffer frame, such as... Figure 2 As shown.

[0063] Optionally, the scanner acquires the set of marker points for the current frame. and the set of buffer frame markers ; Each represents a current frame marker point; Each represents a buffer frame marker. m is less than n.

[0064] In practical applications, the number of markers extracted per frame is limited due to camera field of view constraints. Global markers are only related to the size of the target object, and the size of the global marker set is primarily determined by the spatial scale of the target object. When processing large-scale scenes, the number of global markers often exceeds several thousand or even tens of thousands. Let n represent the global set of marker points, where n is much larger than m. Then, the current frame's set of marker points P and the global set of marker points are combined. The time complexity for matching is O(n log n). As the scanning range increases, the number of detectable markers in the scene increases significantly, leading to a sharp rise in the computation time required for marker stitching operations, making it difficult to meet real-time processing requirements.

[0065] Step S104: Obtain the first distance matrix corresponding to the current frame marker and the second distance matrix corresponding to the buffer frame marker.

[0066] Optionally, the scanner constructs a first distance matrix corresponding to the marker points in the current frame. Construct the second distance matrix corresponding to the buffer frame markers. First distance matrix Second distance matrix As shown in formula (1).

[0067] Formula (1)

[0068] in, It is the Euclidean distance between two current frame markers. j is less than m, and the j in the first distance matrix is ​​the Euclidean distance between two current frame markers. The row vector represents the first row vector in the current frame's set of marker points. The distance from one current frame marker to all other current frame markers in the set of current frame markers. For example... This represents the Euclidean distance from the first current frame marker to the second current frame marker. The other elements in the first distance matrix are similarly calculated. The elements in the second distance matrix are similarly calculated, such as... This represents the Euclidean distance between the first current buffer frame marker and the second buffer frame marker.

[0069] Step S106: Based on the first distance matrix and the second distance matrix, obtain the same-name matching point pairs of the current frame marker point and the buffer frame marker point; the same-name matching point pairs include the same-name points of the current frame marker point.

[0070] Optionally, the scanner uses a first distance matrix. Second distance matrix This yields a set of potential matching pairs. Where k is greater than m and k is greater than n. In the potential matching point pair set Select a baseline matching point pair Based on benchmark matching point pairs Get all the matching pairs of the same name between the current frame marker and the buffer frame marker.

[0071] Step S108: Update the buffer frame marker based on the same point of the current frame marker to obtain the updated buffer frame marker.

[0072] The updated buffer frame marker is used to match the marker in the next frame; the marker in the next frame is the marker in the frame following the current frame.

[0073] Optionally, the scanner updates the buffer frame marker corresponding to the current frame based on all the corresponding points of the current frame marker, and obtains the updated buffer frame marker. The updated buffer frame marker is then used as the buffer frame marker for matching with the marker in the next frame.

[0074] In the above marker matching method, firstly, the current frame marker set and the buffer frame marker set are obtained. The buffer frame markers in the buffer frame marker set are selected from the global markers. A buffer frame mechanism is introduced in the marker matching process. Since the number of buffer frame markers is much smaller than the number of global markers, the number of markers involved in the calculation can be reduced in subsequent calculations. Secondly, corresponding matching point pairs are determined by the first distance matrix of the current frame markers and the second distance matrix of the buffer frame markers. That is, when matching the current frame markers with the buffer frame markers to determine corresponding matching point pairs, redundant and invalid markers are matched with the current markers, saving matching time. Finally, the buffer frame markers are updated based on the corresponding points of the current frame markers to obtain updated buffer frame markers, which are used for matching with the markers of the next frame. Based on the characteristic of the spatial continuity of the scanner-acquired data, a buffer frame mechanism is proposed to limit the number of global markers involved in the matching process, reduce the matching time of the current markers, and thus improve real-time performance.

[0075] In one exemplary embodiment, such as Figure 3 As shown, the buffer frame flag point is updated based on the same point as the current frame flag point to obtain the updated buffer frame flag point, including steps S302 to S306. Wherein:

[0076] Step S302: Write the corresponding point of the current frame marker into the buffer frame and record the frame index corresponding to the corresponding point written into the buffer frame.

[0077] Optionally, the scanner writes all corresponding points of the current frame marker into a buffer frame and records the frame index corresponding to the corresponding points written into the buffer frame, for example, 100.

[0078] Step S304: Obtain the first seed point from the current frame marker points, use the first seed point as the reference point to obtain the target marker points of the first target number from the global marker points, write them into the buffer frame, and record the frame index corresponding to the target marker point.

[0079] Optionally, the scanner selects one of multiple current frame markers as the first seed point, and uses the first seed point as a reference point to obtain the first target count from the global markers corresponding to the current frame. The target marker is recorded and written to the buffer frame, and the frame index corresponding to the target marker is recorded. For example, the frame index corresponding to the target marker is 99, 98 and 97.

[0080] Step S306: If the number of markers stored in the buffer frame is greater than the quantity threshold, delete the marker corresponding to the smallest frame index in the buffer frame to obtain the updated buffer frame markers.

[0081] Where the number of the first target is less than the quantity threshold, such as the quantity threshold being... The number of first targets is .

[0082] Optionally, if the number of markers stored in the buffer frame is greater than the quantity threshold, the scanner deletes the marker corresponding to the smallest frame index in the buffer frame, such as the marker corresponding to the point cloud frame with frame index 97, and obtains updated buffer frame markers that meet the quantity threshold.

[0083] In this embodiment, by controlling the number of markers in the buffer frame, the matching time can be reduced while ensuring the matching success rate, thereby meeting the real-time requirements.

[0084] In an exemplary embodiment, the marker matching method further includes: if the number of markers stored in the buffer frame is less than or equal to a quantity threshold, then obtaining a second seed point from the current frame markers, obtaining a second target number of target markers from the global markers using the second seed point as a reference point, and writing them into the buffer frame to obtain updated buffer frame markers.

[0085] The second target number is determined based on the number of marker points stored in the buffer frame and a quantity threshold; such as the quantity threshold. The difference between the number of seed points and the number of stored marker points. The first seed point and the second seed point correspond to different marker points.

[0086] Optionally, the number of markers stored in the buffer frame is less than a quantity threshold. In this case, the scanner obtains a second seed point from the current frame markers, and uses the second seed point as a reference point to obtain a quantity threshold from the global markers. The target markers are selected from the global markers in order of increasing distance, with the number of markers differing from the number of stored markers. These target markers are then written into the buffer frame to obtain the updated buffer frame markers.

[0087] Optionally, if the number of markers stored in the buffer frame equals the quantity threshold, it indicates that the buffer frame is full, and the updated buffer frame markers are obtained. That is, the quantity threshold. The difference between the number of stored and non-stored markers is 0. The number of target markers written to the buffer frame is 0.

[0088] In this embodiment, by controlling the number of markers in the buffer frame, the matching time can be reduced while ensuring the matching success rate, thereby meeting the real-time requirements.

[0089] In one exemplary embodiment, such as Figure 4As shown, based on the first distance matrix and the second distance matrix, matching point pairs with the same name between the current frame marker and the buffer frame marker are obtained, including steps S402 to S408. Wherein:

[0090] Step S402: Based on the first distance matrix and the second distance matrix, determine the potential matching point pair set.

[0091] The potential matching point pair set is the set of potential matching point pairs with the same name between the current frame marker and the buffer frame marker.

[0092] Optionally, the scanner traverses the first distance matrix. Row vectors and second distance matrix For row vectors, the number of elements with equal Euclidean distance between two row vectors is calculated using formula (2). When the number of elements with equal Euclidean distance between two row vectors is 3 (to determine whether triangles are congruent, the minimum number of points required to form a triangle is 3), the marker points corresponding to these two row vectors are considered a potential matching point pair and will be added to the potential matching point pair set, thus obtaining the final potential matching point pair set. , where k is greater than m and k is greater than n. Where p is the current frame marker in the current frame marker set; q is the buffer frame marker in the buffer frame marker set. In practice, considering the influence of noise and calculation error, formula (2) is used to determine whether the Euclidean distances are equal.

[0093] Formula (2)

[0094] in express The distance in the middle, express The distance in the middle, This represents the distance threshold. Considering measurement error, the distance difference between two Euclidean distances is less than... They believed that the two Euclidean distances were equal.

[0095] It should be noted that in the potential matching point pair set midpoint set and Each point may have multiple potential matching point pairs. For example, the first current frame marker point has 4 potential matching point pairs, namely (p1, q1), (p1, q2), (p1, q3), and (p1, q4).

[0096] Step S404: Determine the baseline matching point pair based on the potential matching point pair set.

[0097] Optionally, the scanner scans the potential matching point pairs. Select a baseline matching point pair .in The current frame marker is from the current frame marker set; Buffer frame markers from the set of buffer frame markers.

[0098] Step S406: Based on the benchmark matching point pairs and the potential matching point pairs set, determine a set of target congruent triangles.

[0099] Optionally, the scanner is based on a reference matching point pair. and potential matching point pairs The search yields a set of congruent target triangles. This set consists of two congruent target triangles: one is determined based on the current frame markers in the current frame marker set, and the other is determined based on the buffer frame markers in the buffer frame marker set.

[0100] Step S408: Based on a set of congruent target triangles, obtain the same-name matching point pairs between the current frame marker point and the buffer frame marker point.

[0101] Optionally, the scanner obtains a set of matching point pairs of the target congruent triangles in the initial pose, based on the target congruent triangles determined by the current frame markers in the current frame marker set and another target congruent triangle determined by the buffer frame markers in the buffer frame marker set. An intermediate pose, i.e., a rotation matrix and a translation vector, is then calculated based on the matching point pairs. The current frame marker set is then adjusted according to the intermediate pose. and the set of buffer frame markers Transform to the same coordinate system. Compare the set of marker points in the current frame under the same coordinate system. and the set of buffer frame markers The positional relationship between the markers, such as Does it satisfy formula (2)? If it does, then it is considered that... These are pairs of matching points with the same name. Find all pairs of matching points with the same name in the intermediate pose. At this time, in order to find the matching points with the same name that have large errors due to noise and measurement errors, the distance threshold in formula (2) is adjusted. Zoom in, use a larger distance threshold To find all matching pairs of the same name between the current frame marker and the buffer frame marker.

[0102] In this embodiment, a set of potential matching point pairs is determined based on the first distance matrix and the second distance matrix. At this time, the number of potential matching point pairs is much smaller than the set of potential matching point pairs determined by the first distance matrix and the matrix corresponding to the global marker points. This can successfully overcome the problem of mismatch caused by structurally similar marker points, find the vast majority of matching point pairs, and significantly improve the stability of subsequent 3D data stitching.

[0103] In one exemplary embodiment, such as Figure 5 As shown, the potential matching point pair set includes multiple potential matching point pairs; based on the potential matching point pair set, the baseline matching point pair is determined, including steps S502 to S506. Wherein:

[0104] Step S502: Take each potential matching point pair as the target potential matching point pair, and determine the distance between the marker point in the target potential matching point pair and the corresponding marker point in the candidate potential matching point pair.

[0105] The potential matching point pair set includes multiple potential matching point pairs, defined as point pairs that may be corresponding points in two different coordinate systems. Based on spatial feature invariance constraints, the distance between marker points is used to determine whether marker points in two different coordinate systems are corresponding points. Candidate potential matching point pairs represent potential matching point pairs in the potential matching point pair set other than the target potential matching point pair.

[0106] In practical applications, a vast number of potential matching point pairs are typically obtained after searching all of them. Directly searching for corresponding points on all these pairs would be extremely complex and computationally intensive. Therefore, to simplify the search, this paper searches for a single, absolutely matching baseline pair from the potential matching point pairs and uses this as the basis for searching for corresponding points.

[0107] Optionally, the scanner will set a potential matching point pair. Each potential matching point pair in the data is as follows: As a target potential matching point pair, determine the marker point and candidate potential matching point pair in the target potential matching point pair. Where j can be any number from 1 to m, such as The distance to the corresponding marker point in the middle.

[0108] Step S504: Count the number of identical distances for each potential matching point pair.

[0109] Where, the same distance represents the distance that satisfies the distance threshold, denoted as . ,express and The distance difference between corresponding points is calculated as follows: If the distance is determined by formula (3) Does it meet the distance threshold? If they are equal, then they are considered to be at the same distance.

[0110] Formula (3)

[0111] Optionally, the scanner counts the number of identical distances for each potential matching point pair.

[0112] Step S506: Determine the baseline matching point pair based on the largest number of potential matching point pairs with the same distance.

[0113] Optionally, if there exists a pair of potential matching points with the largest number of points at the same distance, the scanner will determine the pair of potential matching points with the largest number of points at the same distance as the baseline matching point pair. The success of the splicing depends directly on whether the reference matching point pairs are selected correctly.

[0114] Optionally, if there are multiple potential matching point pairs with the largest number of identical distances, the scanner calculates the average distance difference of identical distances among all potential matching point pairs and selects the potential matching point pair with the smallest average distance as the baseline matching point pair. .

[0115] In this embodiment, determining the baseline matching point pair from the potential matching point pair set using a distance threshold can improve the accuracy of the baseline matching point pair determination. Certain accuracy.

[0116] In one exemplary embodiment, such as Figure 6 As shown, the potential matching point pair set includes multiple potential matching point pairs; based on the baseline matching point pairs and the potential matching point pair set, a set of target congruent triangles is determined, including steps S602 to S610. Wherein:

[0117] Step S602: Based on the benchmark matching point and the set of potential matching point pairs, arbitrarily select two potential matching point pairs to construct a set of triangles.

[0118] Optionally, the scanner will use a reference matching point. As a vertex, then sequentially from the set of potential matching points Extract two point pairs as the other two vertices of the triangle. This leads to the current frame's set of marker points. and the set of buffer frame markers Two triangles were constructed in the middle. and .

[0119] Step S604: For each set of triangles, if the three sides of two triangles in a set of triangles are all equal, then the set of triangles is a set of congruent triangles.

[0120] Optionally, for each set of triangles, the scanner compares the two triangles using formula (2). and Are the corresponding side lengths equal? There are three edges, namely , and . There are three edges, namely , and .calculate and Are they equal? and Are they equal? and Are they equal? ​​If all three corresponding side lengths are equal, then... and It is a set of congruent triangles.

[0121] Step S606: For each set of congruent triangles, determine the initial pose of the congruent triangles, and transform the current frame marker point and the buffer frame marker point to the same coordinate system based on the initial pose; under the same coordinate system, obtain the same-name matching point pairs of each set of congruent triangles under the initial pose.

[0122] Optionally, for each set of congruent triangles, the scanner calculates an initial pose for that set of congruent triangles, i.e., a rotation matrix and a translation vector. Based on the initial pose, the current frame's set of marker points is then... and the set of buffer frame markers Transform to the same coordinate system. Compare the set of marker points in the current frame under the same coordinate system. and the set of buffer frame markers The positional relationship between the markers, such as Does it satisfy formula (2)? If it does, then it is considered that... It is a pair of matching points with the same name. Find all matching points with the same name under the initial pose.

[0123] In practical applications, due to noise and measurement errors, the initial pose determined by congruent triangles often fails to find all matching point pairs. Therefore, it is necessary to further increase the distance threshold. Use a larger distance threshold To find all matching pairs of the same name between the current frame marker and the buffer frame marker.

[0124] Step S608: Count the number of identical matching point pairs for each set of congruent triangles in the initial pose.

[0125] Optionally, the scanner counts the number of identical matching point pairs for each set of congruent triangles in the initial pose.

[0126] Step S610: Based on the set of congruent triangles with the largest number of matching point pairs, determine a set of target congruent triangles.

[0127] Optionally, the scanner determines a set of target congruent triangles based on the set of congruent triangles with the most pairs of matching points of the same name.

[0128] In this embodiment, since the placement of markers is done manually, it is inevitable that markers with similar structural features, such as isosceles triangles, may appear. For those based on... All congruent triangles were calculated, and only one congruent triangle was found that could match multiple pairs of points with the same name, thus eliminating mismatches caused by similar structural feature labels.

[0129] In an exemplary embodiment, a marker concatenation method is provided, wherein the same-name matching point pairs of the current frame marker point and the buffer frame marker point are obtained as determined by the above method; the same-name matching point pairs include the same-name point of the current frame marker point and the same-name point of the buffer frame marker point corresponding to the same-name point of the current frame marker point; the same-name point of the current frame marker point and the same-name point of the corresponding buffer frame marker point are concatenated to obtain a global marker point.

[0130] Optionally, the scanner acquires the same-name matching point pairs of the current frame marker and the buffer frame marker determined by the above method, such as... Figure 7 As shown, there are 5 current frame markers that match the buffer frame markers. Each line corresponds to a current frame marker and a buffer frame marker, which form a matching point pair. Since the buffer frame markers are selected from the global markers, there must be a buffer frame marker among the global markers. The scanner splices the corresponding points of the current frame markers and the corresponding points of the buffer frame markers to obtain the global markers spliced ​​with the current frame markers.

[0131] The aforementioned marker stitching method, by introducing a buffer frame mechanism to determine the corresponding points of the current frame marker and the corresponding points of the buffer frame marker, stitches together the corresponding points of the current frame marker and the corresponding points of the buffer frame marker to obtain the global marker point stitched with the current frame marker. The buffer frame mechanism reduces redundant calculations, significantly improving stitching efficiency and fully meeting the real-time requirements of scanners in practical applications. It avoids the problem of traditional methods, which, when dealing with large-scale scenes with thousands or tens of thousands of marker points, consume a lot of time and cannot achieve the goal of fast and real-time stitching.

[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0133] Based on the same inventive concept, this application also provides a marker matching device for implementing the marker matching method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more marker matching device embodiments provided below can be found in the limitations of the marker matching method described above, and will not be repeated here.

[0134] In one exemplary embodiment, such as Figure 8 As shown, a marker matching device is provided, comprising: a first acquisition module 801, a second acquisition module 802, a determination module 803, and an update module 804, wherein:

[0135] The first acquisition module 801 is used to acquire the current frame marker set and the buffer frame marker set; wherein, the current frame marker set includes the current frame markers; the buffer frame markers in the buffer frame marker set are selected from the global markers; the intersection of the current frame marker set and the buffer frame marker set is not an empty set; and the number of current frame markers is less than the number of buffer frame markers.

[0136] The second acquisition module 802 is used to acquire the first distance matrix corresponding to the current frame marker and the second distance matrix corresponding to the buffer frame marker.

[0137] The determination module 803 is used to obtain the same-name matching point pairs of the current frame marker point and the buffer frame marker point based on the first distance matrix and the second distance matrix; the same-name matching point pairs include the same-name points of the current frame marker point.

[0138] The update module 804 is used to update the buffer frame markers based on the same-named markers of the current frame markers to obtain the updated buffer frame markers; the updated buffer frame markers are used for matching with the markers of the next frame; wherein, the marker of the next frame is the marker corresponding to the next frame of the current frame.

[0139] In an exemplary embodiment, the update module 804 is further configured to write the corresponding points of the current frame markers into the buffer frame and record the frame index corresponding to the corresponding points written into the buffer frame; obtain a first seed point from the current frame markers, obtain a first number of target markers from the global markers using the first seed point as a reference point, write them into the buffer frame, and record the frame index corresponding to the target markers; if the number of markers stored in the buffer frame is greater than the quantity threshold, delete the marker corresponding to the smallest frame index in the buffer frame to obtain the updated buffer frame markers; wherein, the first number of targets is less than the quantity threshold.

[0140] In an exemplary embodiment, the update module 804 is further configured to, if the number of markers stored in the buffer frame is less than or equal to a quantity threshold, obtain a second seed point from the current frame markers, obtain a second target number of target markers from the global markers using the second seed point as a reference point, and write them into the buffer frame to obtain updated buffer frame markers; wherein, the second target number is determined based on the number of markers stored in the buffer frame and the quantity threshold; the markers corresponding to the first seed point and the second seed point are different.

[0141] In an exemplary embodiment, the determining module 803 is configured to determine a set of potential matching point pairs based on a first distance matrix and a second distance matrix; wherein the set of potential matching point pairs is a set of potential matching point pairs with the same name between the current frame marker and the buffer frame marker; determine a reference matching point pair based on the set of potential matching point pairs; determine a set of target congruent triangles based on the reference matching point pairs and the set of potential matching point pairs; and obtain the matching point pairs with the same name between the current frame marker and the buffer frame marker based on the set of target congruent triangles.

[0142] In an exemplary embodiment, the potential matching point pair set includes multiple potential matching point pairs; the determining module 803 is further configured to use each potential matching point pair as a target potential matching point pair, and determine the distance between the marker point in the target potential matching point pair and the corresponding marker point in the candidate potential matching point pair; wherein, the candidate potential matching point pair represents potential matching point pairs other than the target potential matching point pair in the potential matching point pair set; count the number of identical distances in each potential matching point pair; wherein, identical distances represent distances that satisfy a distance threshold; and determine a benchmark matching point pair based on the potential matching point pair with the largest number of identical distances.

[0143] In an exemplary embodiment, the potential matching point pair set includes multiple potential matching point pairs; the determining module 803 is further configured to construct a set of triangles by arbitrarily selecting two potential matching point pairs based on the reference matching point and the potential matching point pair set; for each set of triangles, if the three sides of the two triangles in the set are all equal, the set of triangles is a set of congruent triangles; for each set of congruent triangles, the initial pose of the set of congruent triangles is determined, and the current frame marker point and the buffer frame marker point are transformed to the same coordinate system based on the initial pose; in the same coordinate system, the same-name matching point pairs of each set of congruent triangles in the initial pose are obtained; the number of same-name matching point pairs of each set of congruent triangles in the initial pose is counted; and a set of target congruent triangles is determined based on the set of congruent triangles with the largest number of same-name matching point pairs.

[0144] In one exemplary embodiment, a marker stitching device is provided, comprising: a third acquisition module and a stitching module, wherein:

[0145] The third acquisition module is used to acquire the same-name matching point pair of the current frame marker and the buffer frame marker as described in any of the above methods; the same-name matching point pair includes the same-name point of the current frame marker and the same-name point of the buffer frame marker corresponding to the same-name point of the current frame marker.

[0146] The splicing module is used to splice the corresponding points of the current frame marker and the corresponding points of the buffer frame marker to obtain the global marker.

[0147] Each module in the aforementioned marker matching device and marker splicing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0148] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores marker points. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a marker point matching method and a marker point concatenation method.

[0149] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of landmark point matching, characterized by, The method comprises: obtaining a current frame landmark point set and a buffer frame landmark point set; wherein the current frame landmark point set comprises current frame landmark points; the buffer frame landmark points in the buffer frame landmark point set are selected from global landmark points; the intersection result of the current frame landmark point set and the buffer frame landmark point set is not an empty set; and the number of the current frame landmark points is less than the number of the buffer frame landmark points; obtaining a first distance matrix corresponding to the current frame landmark points and a second distance matrix corresponding to the buffer frame landmark points; based on the first distance matrix and the second distance matrix, obtaining a same-name matching point pair of the current frame landmark points and the buffer frame landmark points; the same-name matching point pair comprises a same-name point of the current frame landmark points; writing the same-name points of the current frame landmark points into a buffer frame and recording the frame index corresponding to the same-name points written into the buffer frame; obtaining a first seed point from the current frame landmark points, obtaining a target number of target landmark points in the global landmark points based on the first seed point as a reference point, and writing the target landmark points into the buffer frame and recording the frame index corresponding to the target landmark points; in the case that the number of landmark points stored in the buffer frame is greater than a number threshold, deleting the landmark points corresponding to the minimum frame index in the buffer frame to obtain updated buffer frame landmark points; wherein the target number is less than the number threshold; the updated buffer frame landmark points are used for matching with next frame landmark points; wherein the next frame landmark points are landmark points corresponding to the next frame of the current frame.

2. The method of claim 1, wherein, The method further comprises: in the case that the number of landmark points stored in the buffer frame is less than or equal to the number threshold, obtaining a second seed point from the current frame landmark points, obtaining a target number of target landmark points in the global landmark points based on the second seed point as a reference point, and writing the target landmark points into the buffer frame to obtain updated buffer frame landmark points; wherein the target number is determined based on the number of landmark points stored in the buffer frame and the number threshold; the landmark points corresponding to the first seed point and the second seed point are different.

3. The method of claim 1, wherein, The method further comprises: based on the first distance matrix and the second distance matrix, determining a set of potential matching point pairs; wherein the set of potential matching point pairs is a set of potential same-name matching point pairs of the current frame landmark points and the buffer frame landmark points; determining a reference matching point pair according to the set of potential matching point pairs; based on the reference matching point pair and the set of potential matching point pairs, determining a set of target congruent triangles; based on the set of target congruent triangles, obtaining the same-name matching point pair of the current frame landmark points and the buffer frame landmark points.

4. The method of claim 3, wherein, The set of potential matching point pairs comprises a plurality of potential matching point pairs; the method further comprises: determining distances between a landmark point in the target potential matching point pair and corresponding landmark points in candidate potential matching point pairs, wherein the candidate potential matching point pairs represent potential matching point pairs in the set of potential matching point pairs other than the target potential matching point pair; counting a number of same distances of each of the potential matching point pairs, wherein the same distance represents the distance satisfying a distance threshold; determining the reference matching point pair based on the potential matching point pair with the largest number of same distances.

5. The method of claim 3, wherein, The set of potential matching point pairs includes a plurality of potential matching point pairs; and determining a group of target congruent triangles based on the reference matching point pair and the set of potential matching point pairs includes: constructing a group of triangles based on the reference matching point and any two potential matching point pairs in the set of potential matching point pairs; for each group of triangles, the group of triangles is a group of congruent triangles in a case where two triangles in the group of triangles have three equal sides; for each group of congruent triangles, determining an initial pose of the group of congruent triangles, and converting the current frame landmark points and the buffer frame landmark points to a same coordinate system based on the initial pose; in the same coordinate system, obtaining a same-name matching point pair of each group of congruent triangles under the initial pose; counting a number of same-name matching point pairs of each group of congruent triangles under the initial pose; determining the group of target congruent triangles based on the group of congruent triangles with the largest number of same-name matching point pairs.

6. A method of stitching together landmark points, the method comprising: The method includes: obtaining a same-name matching point pair of the current frame landmark point and the buffer frame landmark point; the same-name matching point pair includes a same-name point of the current frame landmark point and a same-name point of the buffer frame landmark point corresponding to the same-name point of the current frame landmark point; stitching the same-name point of the current frame landmark point and the same-name point of the corresponding buffer frame landmark point to obtain a global landmark point.

7. A landmark point matching apparatus characterized by comprising: The apparatus includes: a first obtaining module configured to obtain a current frame landmark point set and a buffer frame landmark point set; wherein the current frame landmark point set includes a current frame landmark point; a buffer frame landmark point in the buffer frame landmark point set is selected from global landmark points; an intersection result of the current frame landmark point set and the buffer frame landmark point set is not an empty set; and a number of the current frame landmark points is less than a number of the buffer frame landmark points; a second obtaining module configured to obtain a first distance matrix corresponding to the current frame landmark point and a second distance matrix corresponding to the buffer frame landmark point; a determining module configured to obtain a same-name matching point pair of the current frame landmark point and the buffer frame landmark point based on the first distance matrix and the second distance matrix; the same-name matching point pair includes a same-name point of the current frame landmark point. An updating module is configured to write the same-named points of the current frame landmark points into a buffer frame and record the frame indexes corresponding to the same-named points written into the buffer frame; acquire a first seed point from the current frame landmark points, acquire a first target number of target landmark points from the global landmark points based on the first seed point as a reference point, and write the target landmark points into the buffer frame and record the frame indexes corresponding to the target landmark points; in a case where the number of landmark points stored in the buffer frame is greater than a quantity threshold, delete the landmark points corresponding to the smallest frame index in the buffer frame to obtain updated buffer frame landmark points; the first target number is less than the quantity threshold; the updated buffer frame landmark points are used for matching with next frame landmark points; the next frame landmark points are landmark points corresponding to a next frame of the current frame.

8. The apparatus of claim 7, wherein, The updating module is further configured to, in a case where the number of landmark points stored in the buffer frame is less than or equal to the quantity threshold, acquire a second seed point from the current frame landmark points, acquire a second target number of target landmark points from the global landmark points based on the second seed point as a reference point, and write the target landmark points into the buffer frame to obtain updated buffer frame landmark points; the second target number is determined based on the number of landmark points stored in the buffer frame and the quantity threshold; the first seed point and the second seed point correspond to different landmark points. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor, when executing the computer program, implements the steps of the method in any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method in any one of claims 1 to 6.

Images