Mark point processing method, loopback detection method, scanning device and storage medium

Abstract

The embodiment of the invention provides a mark point processing method, a loopback detection method, scanning equipment and a storage medium, and the mark point processing method comprises the steps: determining at least one mark point pair with staggered layers under the condition that the layers of mark point data of a plurality of scanning frames are staggered; each mark point pair comprises two specified mark points; and according to the at least one mark point pair, splicing the mark point data of the plurality of scanning frames frame by frame to obtain a mark point splicing result. According to the method and the device, the aim of repairing the staggered layers of the mark points can be achieved, loopback detection is executed again through the mark point splicing result obtained by executing the mark point processing method under the conditions that loopback detection fails and the staggered layers of the mark point data of the multiple scanning frames occur, the loopback detection effect can be effectively improved, and especially in a large-range scanning scene, the detection efficiency is improved. And the re-scanning frequency and the operation cost can be effectively reduced.

Description

Technical Field

[0001] The present application relates to the technical field of scanning devices, and in particular to a marker point processing method, a loop detection method, a scanning device, and a storage medium. Background Art

[0002] Markers are typically special patterns or dots manually affixed to the surface of an object. They aid in registration and stitching scan frames together during 3D scanning. During 3D scanning, objects can be large or complex in shape, making it difficult to capture complete marker data in a single scan.

[0003] In related technologies, it is easy for marking points to be misplaced in large-scale scanning scenarios. In the event of misplaced marking points, users need to repeat the scan and adjust, resulting in a sharp increase in workload and uncontrollable costs. Summary of the Invention

[0004] The purpose of this application is to provide a marking point processing method, a loop detection method, a scanning device and a storage medium, which can effectively alleviate the problem of marking point misalignment in large-scale scanning scenarios and reduce the number of manual rescanning times and operating costs.

[0005] The purpose of this application is achieved by the following technical solutions:

[0006] In the first aspect, an embodiment of the present application provides a marker point processing method, the method comprising: in the event that marker point data of multiple scanning frames are misplaced, determining at least one marker point pair where the misplacement occurs; each marker point pair includes two specified marker points located in different scanning frames; and based on at least one marker point pair, splicing the marker point data of multiple scanning frames frame by frame to obtain a marker point splicing result.

[0007] In some embodiments, the at least one marker point pair is determined according to a received user operation.

[0008] In some embodiments, the marker point data includes coordinate information of the marker point, and the frame-by-frame splicing of the marker point data of multiple scan frames includes: when the marker point data of the currently spliced ​​scan frame contains a specified marker point and the other specified marker point in the marker point pair to which the specified marker point belongs is located in the spliced ​​marker point set, based on the corresponding extended matching distance of the marker point pair, matching at least one marker point around the specified marker point with the marker points in the spliced ​​marker point set; adding the coordinate information of the marker point that failed to match to the spliced ​​marker point set to update the spliced ​​marker point set; wherein the extended matching distance is determined based on the distance between the two specified marker points in the marker point pair and is greater than a preset matching distance.

[0009] In some embodiments, the frame-by-frame splicing of the marker point data of the plurality of scan frames further includes: updating the coordinate information of the corresponding marker point in the spliced ​​marker point set based on the coordinate information of the successfully matched marker point.

[0010] In some embodiments, the spliced ​​marker point set includes an identifier for each marker point, and the identifiers of any two marker points in the spliced ​​marker point set are different; before splicing the marker point data of multiple scanned frames frame by frame, the method further includes: assigning the same or associated identifiers to two designated marker points in the same marker point pair; wherein, the process of determining whether the other designated marker point in the marker point pair where the designated marker point is located is located in the spliced ​​marker point set includes: if there is a marker point in the spliced ​​marker point set that has the same or associated identifier as the designated marker point, then determining that the other designated marker point is located in the spliced ​​marker point set.

[0011] In some embodiments, the identifier of the marking point is a numerical type; two designated marking points in the same marking point pair have the same numerical identifier, and the numerical identifier is a negative number or a positive number greater than a preset value;

[0012] The numerical identifiers of the other marking points in the spliced ​​marking point set except the designated marking point are determined according to the order in which they are added to the spliced ​​marking point set.

[0013] In some embodiments, the method further includes: displaying the marker point splicing result through a visual object.

[0014] In a second aspect, an embodiment of the present application provides a loop detection method, the method comprising: when loop detection fails and the marker point data of multiple scanning frames are misplaced, determining at least one marker point pair where the misplacement occurs; each marker point pair includes two specified marker points located in different scanning frames; based on at least one marker point pair, the marker point data of multiple scanning frames are spliced ​​frame by frame to obtain a marker point splicing result; and re-executing loop detection based on the marker point splicing result.

[0015] In a third aspect, an embodiment of the present application provides a computing module, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the marking point processing method provided in the first aspect or the loop detection method provided in the second aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a scanning device, which includes a scanning module and the computing module provided in the third aspect, and the scanning module is used to collect marker point data of multiple scanning frames.

[0017] In some embodiments, the scanning device is a handheld laser scanning device or a tracking scanning device.

[0018] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the marking point processing method provided in the first aspect or the loop detection method provided in the second aspect.

[0019] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the marker point processing method provided in the first aspect or the loop detection method provided in the second aspect.

[0020] The embodiments of the present application provide a marker point processing method, a loop detection method, a scanning device and a storage medium. The marker point processing method determines at least one marker point pair where a mismatch occurs in the marker point data of multiple scanning frames; and splices the marker point data of multiple scanning frames frame by frame based on the at least one marker point pair to obtain a marker point splicing result. In the case that a mismatch occurs in the marker point data of multiple scanning frames, by determining at least one marker point pair where a mismatch occurs, it is possible to force the matching relationship between two specified marker points in different scanning frames for each marker point pair where a mismatch occurs, and splice the marker point data of multiple scanning frames frame by frame. This allows the mismatched data in the scanning frames to be adjusted frame by frame, so that the mismatched marker point data in different scanning frames remain spatially consistent, thereby achieving the purpose of repairing the mismatched marker points. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present application is further described below with reference to the accompanying drawings and specific implementation methods.

[0022] Figure 1 This is a flow chart of a marking point processing method provided in an embodiment of the present application.

[0023] Figure 2 This is a flow chart of a process for splicing mark point data of multiple scan frames frame by frame, provided in an embodiment of the present application.

[0024] Figure 3 This is a flow chart of a loop detection method provided in an embodiment of the present application.

[0025] Figure 4 This is a structural block diagram of a computing module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0027] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly indicate the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0028] Loop closure detection is a key algorithm in SLAM (Simultaneous Localization and Mapping) systems. It determines when a mobile device (such as a scanner) or robot has returned to a previously visited location, thereby correcting accumulated errors and improving map consistency and accuracy. Markers serve as a crucial reference for map construction and loop closure detection, and their accuracy and consistency are crucial to the success of loop closure detection.

[0029] In related technologies, loop detection solutions often rely on the threshold of the marker point splicing distance. If the threshold is too small, it is easy to cause loop detection failure and marker point delamination, while if the threshold is too large, it is easy to cause splicing errors. In addition, the phenomenon of marker point misalignment may be caused by the accuracy of the scanning equipment, the shaking or deformation of the scanned object. Using the coding point solution to fix the unique coding can only alleviate this problem to a certain extent. In the event of marker point misalignment, the user still needs to adjust and scan again, and the workload of re-scanning is very large, making it difficult to effectively control the operating cost.

[0030] See also Figure 1 , Figure 1 This is a flow chart of a marking point processing method provided in an embodiment of the present application.

[0031] To solve the problem of mismatched marker data, users are allowed to specify the marker pairs that failed to match by post-repair and automatically stitch the marker results. Figure 1 As shown, an embodiment of the present application provides a marking point processing method, which includes steps S101 to S102.

[0032] Step S101: when a mismatch occurs in the marking point data of a plurality of scanning frames, determining at least one marking point pair where the mismatch occurs; each marking point pair includes two designated marking points located in different scanning frames.

[0033] Step S102: splicing the marker point data of multiple scan frames frame by frame according to at least one marker point pair to obtain a marker point splicing result.

[0034] In some embodiments, a scan frame may include point cloud data acquired by a single scan operation of a target object (e.g., a vehicle) by a handheld laser scanner or a tracking scanner. The marker point data may be obtained by processing the point cloud data.

[0035] In some embodiments, the marker point data may include coordinate information of the marker point. In addition to the coordinate information, the marker point data may also include normal information of the marker point. The coordinate information may correspond to a global coordinate system. For example, the coordinate information may be represented by the three-dimensional position (X, Y, Z) of the marker point in the global coordinate system, and the normal information may be represented by the normal vector of the plane on which the marker point is located.

[0036] Here, the coordinate information of the marker points in different scanning frames can be converted from the local coordinate system to the global coordinate system using the pose transformation matrix.

[0037] In some embodiments, misalignment of marker point data across multiple scan frames may occur due to spatial offset or inconsistency in the marker point data across different scan frames. For example, if the coordinate information corresponding to the same marker point in different scan frames differs significantly, resulting in a mismatch, the marker point data across multiple scan frames may be misaligned.

[0038] In some embodiments, when marker point data from multiple scan frames is misaligned, at least one marker point pair with the misaligned layer can be determined based on a user-specified method. For example, the at least one marker point pair specified by the user can be determined from among the marker point pairs with misaligned layers from different scan frames through a marker point repair interface.

[0039] In some embodiments, two designated markers in the same marker pair may be assigned the same identifier or associated identifiers.

[0040] In some embodiments, during the frame-by-frame stitching process, the marker point data of multiple scanning frames can be spliced ​​frame by frame according to at least one marker point pair and the frame order of the scanning frames, wherein the frame order of the scanning frames can be determined, for example, based on the acquisition time of the scanning frames.

[0041] In some embodiments, during the frame-by-frame splicing process, two designated marker points belonging to the same marker point pair in different scanning frames may be fused and spliced ​​into one marker point.

[0042] In some embodiments, the marker point stitching result may include at least one of the following: a unique identifier of the marker point (eg, a numerical identifier), and marker point data (eg, coordinate information and normal information of the marker point).

[0043] After determining at least one pair of marker points where a mismatch occurs, these marker point pairs can be used as a basis for splicing, and the marker point data in different scanning frames can be spliced ​​frame by frame to obtain the marker point splicing results, and the marker point splicing results can be added to the globally consistent set of spliced ​​marker points.

[0044] It is understandable that the number of marking points of any scanning frame may be one or more, and the multiple marking points of the scanning frame may include marking points where the scanning frame has a layer mismatch relative to other scanning frames and marking points where no layer mismatch has occurred.

[0045] In some embodiments, the identifier of a designated marker point can be a special numerical identifier (for example, a negative number or a positive number greater than a preset value). When the designated marker point is added to the set of spliced ​​marker points, its special numerical identifier can remain unchanged. The special numerical identifier can be isolated from the regular numerical identifier (such as an increasing serial number with relatively few digits) automatically generated by the system for the undesignated marker points added to the set of spliced ​​marker points. The advantage of doing this is that it is convenient to set non-repetitive, simple and intuitive identifiers for the marker points added to the set of spliced ​​marker points. In addition, if the marker point repair interface uses the method of user input of marker point identifiers to specify marker point pairs, and the undesignated marker points use regular numerical identifiers generated in an incremental manner, then when the user needs to edit again, the user can conveniently input a relatively simple numerical identifier, which is simple to operate and provides a better user experience.

[0046] In the above embodiment, in the case where the mark point data of multiple scan frames are misplaced, at least one mark point pair where the misplacement occurs is determined; based on the at least one mark point pair, the mark point data of the multiple scan frames are spliced ​​frame by frame to obtain a mark point splicing result. In the case where the mark point data of multiple scan frames are misplaced, by determining at least one mark point pair where the misplacement occurs, the matching relationship between the two specified mark points of each mark point pair where the misplacement occurs in different scan frames can be forced to be specified, and the mark point data of the multiple scan frames are spliced ​​frame by frame, thereby adjusting the misplaced data in the scan frames frame by frame, so that the mark point data where the misplacement occurs in different scan frames are spatially consistent, thereby achieving the purpose of repairing the misplacement of the mark points. In some embodiments, at least one mark point pair can be determined based on the received user operation. In this embodiment, in the case where the mark point data of multiple scan frames are misplaced, a mark point repair interface can be provided to the user to support the user to manually specify the mark point pairs where the misplacement occurs across the scan frames.

[0047] For example, in the case where the marker data of multiple scan frames are misplaced, the marker points where the misplacement occurs can be displayed on the marker point repair interface; according to the specified operation performed by the user on the marker point repair interface for two marker points that are located in different scan frames and have misplacement, the two specified marker points are used as specified marker points to form a marker point pair. The specified operation can indicate that the same identifier or associated identifier is assigned to the two specified marker points. The specified operation can, for example, be inputting the same identifier or associated identifier in the input box for the two selected marker points. For example, the user needs to specify that a marker point pair where the misplacement occurs includes marker point #4 of the first scan frame and marker point #10 of the third scan frame. After selecting marker point #4 and marker point #10, the user can enter the same negative number as the identifier of the two marker points in the input box, such as -5959. For another example, the user enters the associated identifier #4_10 for the two selected marker points in the input box.

[0048] It is understandable that the marker point repair interface can also support users to specify multiple marker point pairs between different scanning frames, and based on the multiple marker point pairs, the marker point data of multiple scanning frames can be spliced ​​frame by frame to eliminate marker point misalignment frame by frame.

[0049] In some embodiments, during the process of scanning marker points, the marker point data of each scan frame can be saved in a preset database. When the scan is completed, if there is a marker point mismatch, the marker point pair specified by the user is determined through the marker point repair interface. The marker point data of multiple scan frames in the preset database are spliced ​​frame by frame based on the marker point pairs to repair the marker point mismatch frame by frame, and whether to continue repairing or output the final marker point splicing result is decided based on user feedback. In this way, the interactive repair method allows the user to specify or modify the marker point pairs that have mismatches, so that the user can intervene to repair the marker point mismatches, and the iterative repair mechanism can gradually eliminate the marker point mismatches, thereby improving the repair effect of the marker point mismatches, and can effectively reduce the number of rescans and operating costs.

[0050] Exemplarily, the specific implementation process may include: Step 1, determining the two designated marker points specified by the user on the marker point repair interface as a marker point pair (also referred to as a marker point pair with the same name), determining the coordinate information of the two designated marker points, and calculating and recording the distance between the two designated marker points based on the coordinate information of the two designated marker points in the global coordinate system. Step 2, based on at least one marker point pair, performing frame-by-frame splicing of the marker point data of different scanning frames to obtain a marker point splicing result, and displaying the marker point splicing result to the user. Step 3, determining whether there is still a marker point mismatch based on the user's feedback information on the marker point splicing result. If the feedback information indicates that there is still a marker point mismatch, repeat the above steps 1 to 2. If the feedback information indicates that there is no marker point mismatch, output the final marker point splicing result.

[0051] In some embodiments, after determining at least one pair of marker points where a misalignment occurs, the distance between each pair of marker points can be calculated based on the coordinate information of the two designated marker points in the global coordinate system, and information such as the distance between the two designated marker points and the identifiers of the two designated marker points can be recorded.

[0052] In some embodiments, as Figure 2 As shown, the marking point data includes coordinate information of the marking point, and the frame-by-frame splicing of the marking point data of multiple scan frames may include steps S201 to S202.

[0053] Step S201: When the marker point data of the currently stitched scan frame includes a designated marker point and another designated marker point in the marker point pair to which the designated marker point belongs is located in the stitched marker point set, at least one marker point surrounding the designated marker point is matched with marker points in the stitched marker point set based on an extended matching distance corresponding to the marker point pair.

[0054] Step S202: adding the coordinate information of the marker points that failed to match to the set of spliced ​​marker points to update the set of spliced ​​marker points.

[0055] The extended matching distance for a marker point pair is determined based on the distance between two designated marker points in the marker point pair and is greater than a preset matching distance. The preset matching distance can serve as a minimum distance threshold for matching marker point data from different scan frames. The extended matching distance can be set as the product of the distance between the two designated marker points in the marker point pair and a preset coefficient. The preset coefficient can be a value greater than or equal to 0.5 and less than or equal to 3.

[0056] For example, assuming that the distance between two designated marking points in a marking point pair is 10 mm and the preset matching distance is 3 mm, the value range of the extended matching distance can be 5 mm to 30 mm. Preferably, the value of the extended matching distance is 20 mm.

[0057] Assuming that the corresponding extended matching distance of the marker point pair is 20 mm, if the distance between a marker point around the specified marker point and a marker point in the spliced ​​marker point set is less than or equal to the extended matching distance, the matching is successful, otherwise the matching fails.

[0058] It is understandable that the value of the extended matching distance should not be too large, otherwise it may cause the marking points where no layer mismatch occurs to be incorrectly matched, nor should it be too small, otherwise it may miss the matching of the marking points where the layer mismatch actually occurs.

[0059] Exemplarily, the process of obtaining the corresponding extended matching distance of the marker point pair may include: when splicing the marker point data of the i-th scanning frame (i>1), the marker point set S of the i-th scanning frame may be determined. i Is there a specified mark point? If the mark point set S of the i-th scan frame i There is a specified marker point A1, and the other specified marker point A2 in the marker point pair where the specified marker point A1 is located already exists in the spliced ​​marker point set F. i-1 , the set of spliced ​​marker points F i-1 Based on the 1st to Sth i-1 The marker point data of the scan frames are spliced ​​together, and the distance ||A1-A2|| of the marker point pair (A1, A2) can be used as a reference to set the extended matching distance corresponding to the marker point pair (A1, A2) to D'=k·||A1-A2|| (k>1, k can be any number between 1.5 and 2.5, for example).

[0060] Among them, the judgment process of whether the marker point data of the currently spliced ​​scanning frame contains the specified marker point may include: for the marker point set corresponding to the currently spliced ​​scanning frame, if the marker point set contains the identifiers of one or more specified marker points, then it is determined that the marker point data of the scanning frame contains the specified marker point; otherwise, it is determined that the marker point data of the scanning frame does not contain the specified marker point.

[0061] Exemplarily, matching at least one marker point around the designated marker point with marker points in the spliced ​​marker point set based on the extended matching distance corresponding to the marker point pair may include: searching for at least one marker point B1 around the designated marker point A1 in the currently spliced ​​scan frame; for each searched marker point B1, taking the marker point B1 as the center and the extended matching distance D' corresponding to the marker point pair (A1, A2) as the radius, searching for at least one marker point B1 around the designated marker point A1 in the spliced ​​marker point set F1; i-1 Search whether there is a marker point B2 within the radius of the extended matching distance D' to select the marker point set F from the spliced ​​marker point set F. i-1 Determine whether there is a marker point B2 that matches the marker point B1.

[0062] In which, searching for at least one marker point around the designated marker point A1 in the currently spliced ​​scanning frame can be performed according to a preset search radius to search for at least one marker point around the designated marker point A1, or, alternatively, searching for a preset number of marker points closest to the designated marker point A1 among the marker points surrounding the designated marker point A1 in the currently spliced ​​scanning frame according to a preset number n (n is, for example, 2, 3, 5, 6, etc.).

[0063] In some embodiments, after searching for marker points that match at least one marker point around a specified marker point from the spliced ​​marker point set, each marker point in the at least one marker point around the specified marker point can be spliced ​​with its matching marker point in the spliced ​​marker point set to update the spliced ​​marker point set.

[0064] Exemplarily, for each of at least one marker point surrounding a designated marker point, performing a splicing process on the designated marker point and its matching marker point in the spliced ​​marker point set to update the spliced ​​marker point set may include: obtaining coordinate information for a marker point to be fused together with the two marker points based on at least one of the coordinate information of the two matching marker points, and updating the spliced ​​marker point set; wherein the fused marker point may inherit the identifiers of the two marker points in the spliced ​​marker point set. For example, coordinate information obtained by averaging or weighted fusion of the coordinate information of the two designated marker points in the global coordinate system may be used as the coordinate information of the fused marker point, or the coordinate information of any staggered coordinate point of the two designated marker points in the global coordinate system may be used as the coordinate information of the marker point.

[0065] In the above embodiment, at least one marker point around the designated marker point is matched with the marker points in the set of spliced ​​marker points based on the corresponding extended matching distance of the marker point pair. Since the corresponding extended matching distance is determined based on the distance between the two designated marker points in the marker point pair and is greater than the preset matching distance, the undesignated marker points around the designated marker point can be automatically matched with the marker points in the set of spliced ​​marker points, and the marker point pairs that may have mismatched layers are searched within the extended search range. The user does not need to specify the marker point pairs one by one, which can effectively reduce the workload of the user in manually specifying the marker point pairs and improve the efficiency of marker point processing.

[0066] It should be noted that, for the marker points in the marker point data of the currently spliced ​​scan frame that do not belong to the area around the specified marker point, the marker point can be matched with the marker points in the spliced ​​marker point set based on the preset matching distance, and the coordinate information of the marker points that failed to match can be added to the spliced ​​marker point set to update the spliced ​​marker point set.

[0067] In some embodiments, the method further includes: when the marker point data of the currently spliced ​​scan frame does not contain the designated marker point, adding the coordinate information of the undesignated marker point contained in the marker point data of the currently spliced ​​scan frame to the spliced ​​marker point set to update the spliced ​​marker point set. In the above embodiment, there may be one or more marker points in the currently spliced ​​scan frame that fail to match the marker points in the spliced ​​marker point set. The marker points that failed to match and their coordinate information can be added to the spliced ​​marker point set. For example, the coordinate information of the marker points that failed to match in the global coordinate system can be added to the spliced ​​marker point set to update the spliced ​​marker point set, thereby ensuring that the marker point splicing of subsequent scan frames can be effectively performed.

[0068] In some embodiments, the frame-by-frame splicing of the marker point data of the plurality of scan frames may further include: updating the coordinate information of the corresponding marker point in the spliced ​​marker point set based on the coordinate information of the successfully matched marker point.

[0069] Exemplarily, a successfully matched marker point can be used as a fused marker point, and the coordinate information of the fused marker point and the marker point paired with the fused marker point in the spliced ​​marker point set in the global coordinate system can be averaged or weighted to obtain a coordinate calculation result, and the coordinate calculation result can be used to update the coordinate information of the marker point paired with the fused marker point in the spliced ​​marker point set.

[0070] In the above embodiment, by dynamically updating the coordinate information of the corresponding marker points in the spliced ​​marker point set based on the coordinate information of the successfully matched marker points, the three-dimensional reconstruction effect of the subsequent scan frames can be effectively improved.

[0071] In some embodiments, the stitched marker point set may include an identifier for each marker point, and the identifiers of any two marker points in the stitched marker point set may be different; before stitching the marker point data of multiple scanned frames frame by frame, the method may further include: assigning the same or associated identifiers to two designated marker points in the same marker point pair; wherein, the process of determining whether the other designated marker point in the marker point pair where the designated marker point is located is located in the stitched marker point set may include: if there is a marker point in the stitched marker point set that has the same or associated identifier as the designated marker point, then determining that the other designated marker point is located in the stitched marker point set.

[0072] For example, when determining whether the other designated marker in a designated marker pair is located in the concatenated marker set, the identifier of the designated marker can be first obtained. Then, a search is performed in the concatenated marker set to determine whether there is a marker with the same or related identifier as the designated marker. If so, the designated marker is determined to be located in the concatenated marker set.

[0073] In the above embodiment, by assigning the same or associated identifiers to two designated marker points in the same marker point pair, and using the identifiers for judgment during the marker point splicing process across scanning frames, the marker points with staggered layers corresponding to the same physical position in different scanning frames can be quickly and accurately identified, thereby improving the marker point splicing efficiency.

[0074] In some embodiments, the identifier of the marking point can be a numerical type; the two designated marking points in the same marking point pair have the same numerical identifier, and the numerical identifier is a negative number or a positive number greater than a preset value; the numerical identifiers of other marking points in the spliced ​​marking point set except the designated marking points are determined according to the order in which they are added to the spliced ​​marking point set.

[0075] In the above embodiment, different markers in the spliced ​​marker set have different numerical identifiers. For any marker in the spliced ​​marker set, if the marker was a designated marker before being spliced, the numerical identifier of the marker in the spliced ​​marker set can be a special value, such as a negative number or a positive number greater than a preset value. If the marker was not a designated marker before being spliced, the numerical identifier of the marker in the spliced ​​marker set is determined according to the order in which it was added to the spliced ​​marker set. This can help to quickly distinguish different markers in the spliced ​​marker set in subsequent processing, and iteratively repair marker misalignment based on the markers with sequential numerical identifiers in the spliced ​​marker set, thereby further improving the repair effect of marker misalignment.

[0076] In the above embodiment, the same numerical identifier is used for both designated markers in the same marker pair, and this numerical identifier is a negative number or a positive number greater than a preset value. This unique identifier effectively distinguishes the designated marker from other markers. Furthermore, assigning corresponding identifiers to the other markers in the concatenated marker set, excluding the designated marker, in the order in which they were added, ensures the uniqueness and order of the marker identifiers and improves marker identifier processing efficiency.

[0077] In some embodiments, the method may further include: displaying the marker point splicing result through a visual object.

[0078] For example, when displaying the results of marker point stitching to the user, different marker points can be represented by different numerical identifiers in the visual object, allowing the user to clearly distinguish each marker point and check the effect of the marker point stitching. Detailed information about the marker point stitching results can also be displayed to the user, such as the marker point's coordinate information, posture information, and the scan frame to which it belongs. This allows the user to visually check the repair effect of the marker point misalignment in real time, improving the user interaction experience.

[0079] Next, the marking point processing method provided in the embodiment of the present application will be further explained with reference to specific application examples.

[0080] An embodiment of the present application provides a method for processing marker points. When marker point data of multiple scanning frames generated by a scanning device for a single scan of a target object are misaligned, the following steps are performed.

[0081] Marker point pair specification step: In the marker point repair interface, the user-specified marker point pair with mismatch is received as the same-name marker point pair. A same-name marker point pair contains two specified marker points corresponding to the same physical marker point and located in different scanning frames. The two specified marker points in the same-name marker point pair have user-specified numerical identifiers.

[0082] The original numerical identifiers of the designated marker point and another designated marker point that belongs to the same marker point pair with the designated marker point (for example, num1 and num2, respectively) are both set to num3, where num3 is a special numerical identifier. The special numerical identifier can be, for example, a negative number or a maximum value that exceeds the range of conventional numerical identifiers (for example, -5959 or 44433434); for marker points that do not belong to the marker point pair with the same name, their original numerical identifiers are eliminated. Here, the original numerical identifiers of the marker points that do not belong to the marker point pair with the same name can be eliminated, so that when the marker points with cleared numerical identifiers are added to the spliced ​​marker point set in the subsequent splicing process, they will be reassigned unique identifiers (that is, conventional numerical identifiers). In this way, when the user needs to edit again, the user can quickly and easily identify which marker points are designated marker points, thereby facilitating the re-designation of marker point pairs.

[0083] For example, if a user enters a numerical identifier of #44433434 for marker #3 of the first scan frame and marker #12 of the third scan frame on the marker point repair interface, the numerical identifiers of these two marker points will be forcibly designated as special numerical identifiers. The distance between the two marker points, for example, 10 mm, is calculated, and the coordinate information of marker #44433434 (e.g., a weighted average coordinate) is generated based on the coordinate information of the two marker points. The numerical identifier #44433434, the distance 10 mm, and the coordinate information of #44433434 are then recorded. Furthermore, since only marker #3 of the first scan frame and marker #12 of the third scan frame are designated as designated markers by the user, the numerical identifiers of all marker points generated in all scan frames except marker #3 and marker #12 are cleared.

[0084] Frame-by-frame stitching step: stitching the marker point data of different scan frames in the order of scan frames.

[0085] Specifically, the marker point set S1 of the first scan frame can be used as the initial marker point splicing set F0; the marker point set S i+1 To the spliced ​​marked point set Fi Perform splicing to generate an updated spliced ​​marker point set F i+1 , and so on, until the marker point data of all scan frames are spliced.

[0086] During the splicing process, the splicing process can be performed in the following ways: (1) automatically assigning global incremental identifiers to markers that do not carry identifiers; (2) inheriting the special identifier values ​​of markers that carry special identifier values; (3) when the marker set S of the i+1th scan frame is detected, i+1 The other specified marking point in the marking point pair where the specified marking point is located is located in the spliced ​​marking point set F i When the extended matching distance of the marker point pair is used, at least one marker point around the designated marker point on the i+1th scanning frame is matched with the spliced ​​marker point set F i In this way, the surrounding markers can also be spliced ​​together. In practice, this eliminates the staggered layers around the user-specified marker.

[0087] For example, since one of the reasons for misalignment is that the three-dimensional error of the marker point is too large, during the splicing process, if a user-specified marker point (such as -5959) appears, then the distance threshold of the N marker points around the user-specified marker point (such as -5959) after splicing can be set according to the distance of the same-name marker point pair specified by the user. Assuming that the distance of the same-name marker point pair actually selected by the user is 10mm, then during the splicing process, the distance threshold of the n marker points around the specified marker point in this same-name marker point pair is 20mm, while the distance threshold of the splicing under normal circumstances may be 3mm, that is, special treatment is performed near the marker point in the specified marker point pair. In this way, during the splicing process, the misalignment problem caused by large splicing errors can be more effectively reduced.

[0088] If the user repeatedly edits the same-named marker pairs during the above process, for example, a new set of same-named marker pairs is added, the distance between the two specified marker points will be calculated for the new same-named marker pairs based on the coordinate information of the two specified marker points contained in the marker pair in the global coordinate system, and the distance between the two specified marker points and the identifiers of the two specified marker points and other information will be recorded.

[0089] When performing frame-by-frame splicing of the marker point data based on the new synchronization marker point pair, the data will be reprocessed based on the newly edited marker point pair information with the same name and in accordance with the above frame-by-frame splicing process.

[0090] Through the above steps, after using a scanning device (for example, a handheld laser scanner) to perform a single scan of the target object to obtain multiple scan frames, if the marker point data of multiple scan frames are misplaced, the marker point data can be spliced ​​and fused frame by frame according to the user-specified marker point information with the same name to repair the misplaced marker points. This can effectively solve the marker point misplacement problem in three-dimensional scanning, reduce the number of manual rescanning times, and improve the quality and efficiency of three-dimensional modeling.

[0091] See also Figure 3 , Figure 3 This is a flow chart of a loop detection method provided in an embodiment of the present application.

[0092] In order to alleviate the problem of loop detection failure caused by the misalignment of marker points, the embodiment of the present application provides a marker point processing method, such as Figure 3 As shown, the method includes steps S301 to S303.

[0093] Step S301: when loop detection fails and marker point data of multiple scan frames are misplaced, at least one marker point pair where the misplaced layer occurs is determined; each marker point pair includes two designated marker points located in different scan frames.

[0094] Step S302: splicing the marker point data of multiple scan frames frame by frame according to at least one marker point pair to obtain a marker point splicing result.

[0095] Step S303: re-perform loop closure detection based on the marker point splicing result.

[0096] In some embodiments, the optional implementation of step S301 can be found in Figure 1 Optional implementation of step S101, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0097] In some embodiments, the optional implementation of step S302 can be found in Figure 1 Optional implementation of step S102, Figure 1 and Figure 2 Other related parts in the embodiments involved will not be described in detail here.

[0098] In some embodiments, re-performing loop detection based on the marker point splicing result may include: calculating the number of marker points with the same identification shared by the first scanning frame and the last scanning frame based on the marker point splicing result, and determining whether the loop closure is successful based on whether the number of marker points exceeds the loop detection threshold.

[0099] For example, taking the scanning of a car as an example, the user uses a handheld scanning device to scan around the car body in a circle, forming a scanning frame S1 at the starting point of the front of the car, and after completing a circle, passing the front of the car again to form a scanning frame S10. Due to factors such as unreasonable setting of the threshold value of the marker point splicing distance (for example, the threshold value is 3mm), the same physical marker point in the front area generates two marker points with inconsistent coordinate information in S1 and S10 respectively. In this way, the distance between their three-dimensional coordinates is 10mm, which makes the system fail to recognize them as the same marker point, resulting in the failure of loop detection. After the user specifies the two marker points with inconsistent coordinate information as a marker point pair in the interface, the marker point data of multiple scanning frames are spliced ​​frame by frame based on the marker point pair to obtain the marker point splicing result, and the loop detection is re-executed based on the marker point splicing result to determine whether the loop closure is successful.

[0100] In the above embodiment, the loop detection method determines at least one pair of marker points that have a mismatch when loop detection fails and the marker point data of multiple scan frames have a mismatch; based on the at least one marker point pair, the marker point data of the multiple scan frames are spliced ​​frame by frame to obtain a marker point splicing result. When loop detection fails and the marker point data of multiple scan frames have a mismatch, by determining at least one pair of marker points that have a mismatch, the marker point data of the multiple scan frames can be spliced ​​frame by frame using the correspondence between the two designated marker points of each pair of marker points that have a mismatch in different scan frames. This can adjust the mismatched data in the scan frames frame by frame, so that the mismatched marker point data in different scan frames remain consistent in space, thereby eliminating the mismatched marker points, thereby effectively improving the loop detection effect, especially in large-scale scanning scenarios, and effectively reducing the number of rescans and operating costs.

[0101] See also Figure 4 , Figure 4 This is a structural block diagram of a computing module provided in an embodiment of the present application.

[0102] An embodiment of the present application further provides a computing module, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the marker processing method or loop detection method provided in the aforementioned embodiment.

[0103] The computing module may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected via an internal connection path.

[0104] The memory 110 is used to store computer programs. In some implementations, the computer programs may include codes for implementing the methods of the embodiments of the present application.

[0105] The processor 120 is configured to execute the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information and output data such as operation results. In some implementations, when the solutions of the embodiments of the present application are implemented through software or firmware, the computer program for implementing the solutions of the embodiments of the present application may be stored in the processor 120 and executed by the processor 120.

[0106] The memory 110 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes a random access memory (RAM), a cache memory and a read-only memory (ROM). Among them, the memory 110 stores a computer program, and the computer program can be executed by the processor 120 so that the processor 120 implements the steps of any of the above methods.

[0107] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or the processor 120 may be any conventional processor.

[0108] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 120 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor 120. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0109] In some implementations, the computing module may include, in addition to the hardware units described above, a software module, where the software module may be, for example, an operating system, a basic input and output system (BIOS), application software, etc.

[0110] The operating system manages one or more of the computing module's hardware and software resources and is the core and cornerstone of the computing module. The operating system handles basic tasks such as managing and allocating memory, prioritizing system resource supply and demand, controlling input and output devices, operating the network, and managing the file system. To facilitate user operations, most operating systems provide an interface for user interaction with the system.

[0111] The BIOS is used to run hardware initialization during the power-on boot phase and provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display the processor temperature and execute functions such as adjusting temperature protection strategies.

[0112] Application software, also known as application program, can be understood as software written for a specific application purpose of the user. It is one of the main categories of computer software.

[0113] An embodiment of the present application further provides a scanning device, which includes a scanning module and the calculation module described in the above embodiment, and the scanning module is used to collect marker point data of multiple scanning frames.

[0114] In some implementations, the scanning device is a handheld laser scanning device or a tracking scanning device.

[0115] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the marking point processing method or the loop detection method provided in the aforementioned embodiment is implemented.

[0116] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the marker processing method or loop detection method provided in the aforementioned embodiment.

[0117] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the computer program product of the present application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0118] It should be understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation methods of this application, rather than to limit the scope of protection of this application.

[0119] It can be understood that in various implementations of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the present application.

[0120] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and this application is not limited to this.

[0121] Unless otherwise indicated, all technical and scientific terms used in this application have the same meaning as those generally understood by those skilled in the art in the technical field of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The term "one or more" used in this application includes any and all combinations of one or more related listed items. The singular forms "a", "above", and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0122] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0123] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described embodiments may refer to the corresponding processes in other embodiments and will not be repeated here.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed modules, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0125] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the technical solutions of this application.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computing module (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A marking point processing method, characterized in that: The method comprises: In the case where the marking point data of multiple scanning frames are misplaced, determining at least one marking point pair where the misplaced occurs; each marking point pair includes two designated marking points located in different scanning frames; According to at least one marker point pair, the marker point data of multiple scan frames are spliced ​​frame by frame to obtain a marker point splicing result.

2. The marking point processing method according to claim 1, characterized in that: The at least one marker point pair is determined according to a received user operation.

3. The marking point processing method according to claim 1, characterized in that: The marking point data includes coordinate information of the marking point, and the marking point data of the plurality of scan frames are spliced ​​frame by frame, including: When the marker point data of the currently stitched scan frame includes a designated marker point and another designated marker point in the marker point pair to which the designated marker point belongs is located in the stitched marker point set, matching at least one marker point around the designated marker point with marker points in the stitched marker point set based on the corresponding extended matching distances of the marker point pairs; Adding coordinate information of the marker points that failed to match to the set of spliced ​​marker points to update the set of spliced ​​marker points; The extended matching distance is determined based on the distance between two designated marking points in the marking point pair and is greater than a preset matching distance.

4. The marking point processing method according to claim 3, characterized in that: The step of splicing the marker point data of the plurality of scan frames frame by frame further includes: Based on the coordinate information of the successfully matched marking point, the coordinate information of the corresponding marking point in the spliced ​​marking point set is updated.

5. The marking point processing method according to claim 3, characterized in that: The set of spliced ​​marking points includes an identifier of each marking point, and the identifiers of any two marking points in the set of spliced ​​marking points are different; Before splicing the marker data of the plurality of scanned frames frame by frame, the method further comprises: assigning the same or associated identifiers to two designated markers in the same marker pair; The process of determining whether another designated marking point in the marking point pair in which the designated marking point is located is in the spliced ​​marking point set includes: if there is a marking point in the spliced ​​marking point set that has the same or associated identifier as the designated marking point, determining that the other designated marking point is in the spliced ​​marking point set.

6. The marking point processing method according to claim 5, characterized in that: The identifier of the marking point is a numerical type; The two designated marking points in the same marking point pair have the same numerical identifier, and the numerical identifier is a negative number or a positive number greater than a preset value; The numerical identifiers of the other marking points in the spliced ​​marking point set except the designated marking point are determined according to the order in which they are added to the spliced ​​marking point set.

7. The marking point processing method according to claim 1, characterized in that: The method further comprises: The marker point splicing result is displayed through a visual object.

8. A loop detection method, characterized in that: The method comprises: In the case where loop detection fails and the marker point data of multiple scan frames are misplaced, determining at least one marker point pair where the misplaced layer occurs; each marker point pair includes two designated marker points located in different scan frames; splicing the marker point data of the plurality of scan frames frame by frame according to at least one marker point pair to obtain a marker point splicing result; Re-perform loop closure detection based on the marker point splicing result.

9. A calculation module, characterized in that: The computing module includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A scanning device, characterized in that: The scanning device includes a scanning module and the calculation module according to claim 9, and the scanning module is used to collect marker point data of multiple scanning frames.

11. The scanning device according to claim 10, wherein: The scanning device is a handheld laser scanning device or a tracking scanning device.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images