Precise registration method and system based on 3D point cloud and model contour

Abstract

The present invention provides a precise registration method and system based on 3D point cloud and model contour. The method includes: obtaining a preset registration accuracy of a photographed material; obtaining original point cloud data of multiple target points of the photographed material; performing preliminary processing on the original point cloud data to obtain a target layer point cloud map; based on the target layer point cloud map, determining the inner contour data of the target layer point cloud map and multiple target feature points therein; based on the multiple target feature points, registering the photographed material with the preset model contour to determine a first registration result; if its point cloud registration accuracy is greater than or equal to the preset registration accuracy, determining the first registration result as a second registration result; if its point cloud registration accuracy is less than the preset registration accuracy, performing at least one correction iteration on the first registration result to obtain an iterative registration result; in response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, ending the correction iteration and determining the iterative registration result as the second registration result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for precise registration based on 3D point cloud and model contour. Background Art

[0002] With the advancement of 3D scanning technology, three-dimensional point cloud data has been widely used in engineering, construction, autonomous driving and other fields. However, high-precision registration of point cloud data still faces many technical challenges, especially when dealing with complex shapes and large-scale data. There are several major problems with current point cloud registration technology. First, the high dimensionality and complexity of point cloud data lead to huge computational complexity of the registration algorithm. Especially when dealing with large-scale point clouds, traditional methods often cannot complete the task within a reasonable time and are inefficient. Secondly, since point cloud data is affected by environmental noise and sensor errors, directly registering the original point cloud often cannot achieve high accuracy, especially when there are complex surfaces (such as depressions, irregular edges, etc.) on the surface of the object. Traditional algorithms are difficult to effectively capture these details, resulting in a decrease in registration accuracy.

[0003] Therefore, it is hoped to provide a precise registration method and system based on 3D point cloud and model contour, which can improve the registration accuracy while ensuring efficient calculation, especially when facing complex shapes, noise interference and sensor errors, and has stronger robustness and adaptability. Summary of the invention

[0004] One or more embodiments of the present specification provide a precise registration method based on 3D point cloud and model contour. The method includes: obtaining a preset registration accuracy of the photographed material; obtaining the original point cloud data of multiple target points of the photographed material; performing preliminary processing on the original point cloud data to obtain a target layer point cloud map; determining the inner contour data of the target layer point cloud map based on the target layer point cloud map; determining multiple target feature points in the inner contour data; based on the multiple target feature points, registering the photographed material with the preset model contour to determine a first registration result; in response to the point cloud registration accuracy of the first registration result being greater than or equal to the preset registration accuracy, determining the first registration result as a second registration result; in response to the point cloud registration accuracy of the first registration result being less than the preset registration accuracy, performing at least one correction iteration on the first registration result to obtain an iterative registration result; in response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, ending the correction iteration and determining the iterative registration result as the second registration result.

[0005] One or more embodiments of the present specification provide a precise registration system based on 3D point cloud and model contour, the system comprising a data acquisition module, configured to: obtain a preset registration accuracy of a photographed material; obtain original point cloud data of multiple target points of the photographed material; a data analysis module, configured to: perform preliminary processing on the original point cloud data to obtain a target layer point cloud map; determine inner contour data of the target layer point cloud map based on the target layer point cloud map; determine multiple target feature points in the inner contour data; and compare the photographed material with a preset registration accuracy based on the multiple target feature points. The model contour is aligned to determine a first alignment result; in response to the point cloud registration accuracy of the first alignment result being greater than or equal to the preset registration accuracy, the first alignment result is determined as the second alignment result; a result correction module is configured to: in response to the point cloud registration accuracy of the first alignment result being less than the preset registration accuracy, perform at least one correction iteration on the first alignment result to obtain an iterative registration result; in response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, end the correction iteration and determine the iterative registration result as the second registration result.

[0006] One or more embodiments of the present specification provide a precise registration device based on 3D point cloud and model contour, the device comprising at least one processor and at least one memory, the at least one storage medium being used to store computer instructions; the at least one processor being used to execute at least part of the computer instructions to implement the precise registration method based on 3D point cloud and model contour described in the above embodiments.

[0007] One or more embodiments of the present specification provide a computer-readable storage medium that stores computer instructions, and when the computer instructions are executed by a computer, the precise registration method based on 3D point cloud and model contour described in the above embodiment is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is an exemplary module diagram of a precise registration system based on 3D point cloud and model contour according to some embodiments of this specification;

[0010] Figure 2 is an exemplary flow chart of a precise registration method based on 3D point cloud and model contour according to some embodiments of this specification;

[0011] Figure 3is an exemplary schematic diagram of obtaining a preset registration accuracy according to some embodiments of this specification;

[0012] Figure 4 is an exemplary schematic diagram of obtaining original point cloud data according to some embodiments of this specification;

[0013] Figure 5 is an exemplary schematic diagram of updating a preset registration accuracy according to some embodiments of the present specification. DETAILED DESCRIPTION

[0014] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0016] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0018] Figure 1 It is an exemplary module diagram of a precise registration system based on 3D point cloud and model contour according to some embodiments of this specification.

[0019] Some embodiments of this specification provide a precise registration system based on 3D point cloud and model contour (hereinafter referred to as precise registration system). The precise registration system 100 may include a data acquisition module 110 , a data analysis module 120 and a result correction module 130 .

[0020] In some embodiments, the data acquisition module 110, the data analysis module 120 and the result correction module 130 are all connected in communication. Exemplary communication connection methods include but are not limited to Bluetooth, WIFI, 5G, etc.

[0021] In some embodiments, the data acquisition module 110, the data analysis module 120 and the result correction module 130 may have their own processors or may share a processor.

[0022] The processor may process data and / or information obtained from other devices or system components. The processor may execute program instructions based on these data, information and / or processing results to perform one or more functions described in this application. In some embodiments, the processor may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-core processing device). By way of example only, the processor may include a central processing unit (CPU), a controller, a microprocessor, etc., or any combination thereof.

[0023] The data acquisition module 110 may be configured to acquire a preset registration accuracy of the photographed material; and acquire original point cloud data of multiple target points of the photographed material.

[0024] In some embodiments, the data acquisition module 110 may include an acquisition device. Exemplary acquisition devices may include, but are not limited to, 3D cameras, etc. In some embodiments, the data acquisition module acquires raw point cloud data of multiple target points of the photographed material through its acquisition device. Among them, the raw point cloud data refers to the point cloud data directly acquired by the 3D camera. The target point refers to the point photographed by the 3D camera. In some embodiments, the target point can be preset based on experience. Preferably, the target point can be located directly above the photographed material to facilitate vertical photography of the photographed material.

[0025] The data acquisition module 110 may be further configured to determine the surface features and edge features to be extracted of the photographed material based on the geometric features of the photographed material, and determine the preset registration accuracy through an accuracy prediction model based on the surface features and edge features to be extracted. The accuracy prediction model is a machine learning model.

[0026] The data acquisition module 110 can be further configured to determine point cloud acquisition parameters based on the surface features to be extracted, the edge features to be extracted and the preset registration accuracy; and based on the point cloud acquisition parameters, acquire the original point cloud data of multiple target points of the photographed material.

[0027] The data analysis module 120 can be configured to perform preliminary processing on the original point cloud data to obtain a target layer point cloud map; determine the inner contour data of the target layer point cloud map based on the target layer point cloud map; determine multiple target feature points in the inner contour data; based on the multiple target feature points, align the photographed material with the preset model contour to determine a first alignment result; and in response to the point cloud alignment accuracy of the first alignment result being greater than or equal to the preset alignment accuracy, determine the first alignment result as the second alignment result.

[0028] The result correction module 130 can be configured to, in response to the point cloud registration accuracy of the first registration result being less than a preset registration accuracy, perform at least one correction iteration on the first registration result to obtain an iterative registration result; and in response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, end the correction iteration and determine the iterative registration result as the second registration result.

[0029] In some embodiments, the fine registration system 100 may further include a data updating module 140 .

[0030] In some embodiments, the data updating module 140 is in communication with the aforementioned data acquisition module 110, data analysis module 120, and result correction module 130. In some embodiments, the data updating module 140 may also have its own processor, or share a processor with other modules.

[0031] The data update module 140 can be configured to determine the point cloud registration accuracy of the second registration result in response to the completion of the second registration result of the photographed material, based on the number of corrections and the deviation data of each correction; and update the preset registration accuracy of the photographed material based on the point cloud registration accuracy of the second registration result.

[0032] For more information about the fine registration system and its modules, please refer to Figure 2-Figure 5 and its related description.

[0033] It should be noted that the above description of the precise registration system and its modules is only for convenience of description and does not limit the present specification to the scope of the embodiments. It is understandable that, after understanding the principle of the system, those skilled in the art may arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from the principle. In some embodiments, Figure 1The data acquisition module 110, data analysis module 120, result correction module 130 and data update module 140 disclosed in the specification can be different modules in a system, or one module can realize the functions of two or more modules. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the protection scope of this specification.

[0034] Figure 2 is an exemplary flow chart of a precise registration method based on 3D point cloud and model contour according to some embodiments of this specification. Figure 2 As shown, the process 200 includes the following steps. In some embodiments, the process 200 may be performed by a fine registration system.

[0035] Step S210, obtaining a preset registration accuracy of the photographed material. In some embodiments, step S210 may be performed by a data acquisition module.

[0036] Photographed materials refer to objects that need to be registered, such as mechanical parts on industrial production lines and structural components at construction sites that require 3D point cloud technology for precise positioning and registration.

[0037] The preset registration accuracy refers to the preset registration accuracy that needs to be achieved during registration.

[0038] In some embodiments, the data acquisition module can obtain the preset registration accuracy of the photographed material in a variety of ways. For example, the preset registration accuracy can be manually input into the storage module of the fine registration system in advance based on past experience, and the data acquisition module can read the preset registration accuracy from the storage module. For another example, the data acquisition module can match the corresponding preset registration accuracy in the preset table based on the type of photographed material. Among them, the preset table can be used to characterize the correlation between the type of photographed material and the preset registration accuracy, and each type of photographed material can correspond to a preset registration accuracy. In some embodiments, the preset table can be constructed according to the type of historical photographed material and its corresponding preset registration accuracy. For more information about historical photographed materials, please refer to Figure 3 and its related contents.

[0039] In some embodiments, the data acquisition module may also determine the surface features and edge features to be extracted of the photographed material based on the geometric features of the photographed material; and determine the preset alignment accuracy through an accuracy prediction model based on the surface features and edge features to be extracted.

[0040] For more information on how to determine the preset registration accuracy, see Figure 3 and its related description.

[0041] Step S220, obtaining original point cloud data of multiple target points of the photographed material. In some embodiments, step S220 can be performed by a data acquisition module.

[0042] The target point refers to the pre-set shooting point where the shooting is required.

[0043] In some embodiments, the target point may be preset based on past experience.

[0044] In some embodiments, the target point can be located directly above the photographed material, so that the acquisition device can vertically photograph the photographed material. For more information about the acquisition device, please refer to Figure 1 and its related description.

[0045] Original point cloud data refers to point cloud data directly acquired by the acquisition device, for example, point cloud data directly captured by a 3D camera, where point cloud data is a data structure representing a collection of points in a three-dimensional space, and each point contains at least three coordinate values ​​(X, Y, Z).

[0046] In some embodiments, the data acquisition module may acquire the original point cloud data of multiple target points of the photographed material in a variety of ways. For example, the data acquisition module may acquire the original point cloud data by laser scanning or the like.

[0047] In some embodiments, the data acquisition module can also determine point cloud acquisition parameters based on the surface features to be extracted, the edge features to be extracted and the preset alignment accuracy; and based on the point cloud acquisition parameters, acquire the original point cloud data of multiple target points of the photographed material.

[0048] For more information on how to obtain raw point cloud data, see Figure 4 and its related description.

[0049] Step S230, preliminarily process the original point cloud data to obtain a point cloud image of the target layer. In some embodiments, step S230 can be performed by a data analysis module.

[0050] Preliminary processing refers to preprocessing of the original point cloud data. In some embodiments, the preliminary processing may include filtering outliers to reduce noise interference, downsampling to increase processing speed, and using a plane segmentation algorithm to separate the plane where the photographed material is located and the background.

[0051] The target layer point cloud image refers to the point cloud data of the plane where the material is photographed.

[0052] Step S240, based on the target layer point cloud image, determine the inner contour data of the target layer point cloud image. In some embodiments, step S240 can be performed by a data analysis module.

[0053] Inner contour data refers to the set of contour lines inside the plane where the material is photographed. For example, when performing a 3D scan of a mechanical part, the inner contour data can be the edge line inside the cross section of the part.

[0054] In some embodiments, the data analysis module may determine the inner contour data of the target layer point cloud image based on the target layer point cloud image by a first preset algorithm.

[0055] The first preset algorithm refers to an algorithm for extracting inner contour data.

[0056] In some embodiments, the first preset algorithm may be set in advance based on past experience.

[0057] In some embodiments, the first preset algorithm may include a concave hull extraction algorithm.

[0058] In some embodiments, the data analysis module can use a convex hull extraction algorithm based on the target layer point cloud map to find a reference point in the target layer point cloud map, calculate the angles of other points relative to the reference point and sort them, and use a stack structure to process all points in sequence, and finally form a convex hull containing all points, and then extract the convex hull, that is, extract multiple inner contour lines inside the material photographed in the target layer point cloud map, and obtain the inner contour data of the target layer point cloud map.

[0059] Step S250, determining a plurality of target feature points in the inner contour data. In some embodiments, step S250 may be performed by a data analysis module.

[0060] The target feature points refer to the data points on the inner contour line with significant local shape characteristics.

[0061] In some embodiments, the data analysis module may determine a plurality of target feature points in the inner contour data based on the inner contour data by a second preset algorithm.

[0062] The second preset algorithm refers to an algorithm for extracting target feature points.

[0063] In some embodiments, the second preset algorithm may be set in advance based on past experience.

[0064] In some embodiments, the second preset algorithm may include an intrinsic shape signature (ISS) algorithm.

[0065] In some embodiments, the data analysis module can use an intrinsic shape feature algorithm based on the inner contour data to identify key points and local geometric properties on the object, generate a unique shape signature, and thus extract multiple target feature points from the inner contour line to obtain multiple target feature points in the inner contour data.

[0066] Step S260, based on the multiple target feature points, the photographed material is registered with the preset model contour to determine a first registration result. In some embodiments, step S260 can be performed by a data analysis module.

[0067] The first registration result refers to the registration result after the preliminary registration is performed.

[0068] In some embodiments, the data analysis module may align the photographed material with the preset model contour based on multiple target feature points through a third preset algorithm to determine a first alignment result.

[0069] The third preset algorithm refers to an algorithm used for point cloud registration. Point cloud registration refers to the mutual alignment of different point cloud data in terms of position.

[0070] In some embodiments, the third preset algorithm may be set in advance based on past experience.

[0071] In some embodiments, the third preset algorithm includes an iterative closest point (ICP) algorithm.

[0072] In some embodiments, the data analysis module can use the nearest point iteration algorithm based on multiple target feature points to iteratively match the nearest point pairs in the photographed material and the preset model contour and calculate the optimal transformation, so that the photographed material and the preset model contour can be aligned to obtain the transformation matrix of the point cloud.

[0073] Step S270, in response to the point cloud registration accuracy of the first registration result being greater than or equal to a preset registration accuracy, determining the first registration result as a second registration result. In some embodiments, step S270 may be performed by a data analysis module.

[0074] Point cloud registration accuracy refers to the measure of the positional deviation between corresponding points of two or more point cloud datasets after they are registered (i.e., spatially aligned or matched) in three-dimensional space.

[0075] In some embodiments, the data analysis module may use the average of the squares of the distances between the original point cloud data and the corresponding points of the reference point cloud data of the preset model contour after registration as the point cloud registration accuracy of the first registration result.

[0076] The second registration result refers to a registration result that meets a preset registration accuracy.

[0077] In some embodiments, when the point cloud registration accuracy of the first registration result is greater than or equal to a preset registration accuracy, the first registration result is the second registration result.

[0078] Step S280, in response to the point cloud registration accuracy of the first registration result being less than the preset registration accuracy, performing at least one correction iteration on the first registration result to obtain an iterative registration result. In some embodiments, step S280 may be performed by a result correction module.

[0079] The iterative registration result refers to the registration result after the correction iteration is performed based on the first registration result.

[0080] Correction iteration refers to the process of continuously adjusting and optimizing the registration result through iteration when the first registration result of point cloud registration does not reach the preset registration accuracy.

[0081] In some embodiments, the result correction module can find the nearest neighbor reference point cloud point in the preset model contour from each point of the original point cloud data, and calculate the best transformation matrix, so that each point of the original point cloud data is as close as possible to the nearest neighbor reference point cloud point in the preset model contour, and apply the calculated transformation matrix to the original point cloud data, update the position of each point in the original point cloud data, and then calculate the point cloud registration accuracy after correction. If the point cloud registration accuracy of the iterative registration result is less than the preset registration accuracy, repeat the above steps to perform correction iterations again until the point cloud registration accuracy of the iterative registration result is greater than or equal to the preset registration accuracy, and / or the number of iterations reaches the preset number of iterations, and the correction iteration ends.

[0082] Among them, the nearest neighbor reference point cloud point refers to the point with the closest distance, and the point cloud registration accuracy can be determined according to the average value of the square of the distance from each point in the original point cloud data to the nearest neighbor reference point cloud point in the preset model contour.

[0083] Step S290, in response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, the correction iteration is ended, and the iterative registration result is determined as the second registration result. In some embodiments, step S290 may be performed by a result correction module.

[0084] In some embodiments, the result correction module can perform correction iterations on the first registration result, calculate the point cloud registration accuracy of the iterative registration result, and terminate the correction iteration until the point cloud registration accuracy of the iterative registration result is greater than or equal to the preset registration accuracy, and determine the iterative registration result as the second registration result.

[0085] It should be understood that step S270 is not executed simultaneously with step S280 and step S290. That is, after the data analysis module executes step S270, the result correction module does not need to execute step S280 and step S290.

[0086] In some embodiments of the present specification, by presetting the registration accuracy and preliminary processing of the original point cloud data, the target feature points can be determined more accurately, thereby achieving more accurate material and model contour registration. Furthermore, the registration results are automatically adjusted and optimized through the correction iteration mechanism, ensuring the high quality and reliability of the registration results.

[0087] Figure 3 It is an exemplary schematic diagram of a method for obtaining a preset registration accuracy according to some embodiments of this specification.

[0088] In some embodiments, Figure 3 As shown, the data acquisition module can determine the surface features 320 and edge features 330 to be extracted of the photographed material based on the geometric features 310 of the photographed material; and determine the preset alignment accuracy 350 through the accuracy prediction model 340 based on the surface features 320 and edge features 330 to be extracted.

[0089] Geometric features are attributes that can be used to describe the shape, structure, and spatial relationship of the photographed material. For example, the shape, size, and number of edges of the photographed material.

[0090] For more information about shooting materials, please refer to Figure 2 and its related description.

[0091] In some embodiments, the geometric features of the photographed material can be acquired by an acquisition device of a data acquisition module. For example, the acquisition device can capture an image or three-dimensional data (such as point cloud data) of the photographed material, and then use an image processing algorithm or a three-dimensional reconstruction algorithm to extract the geometric features of the photographed material from the acquired image or point cloud data.

[0092] The surface features to be extracted refer to the relevant features extracted from the surface of the photographed material, for example, the shape, size, and concave-convex difference of the surface of the photographed material. Among them, the concave-convex difference refers to the difference between the highest point and the lowest point on the surface of the photographed material.

[0093] The edge features to be extracted refer to geometric information extracted from the edge of the photographed material, such as straight lines, curves, angles, etc. In some embodiments, the edge features to be extracted include the number of irregular edges. The number of irregular edges refers to the number of edges of the photographed material that do not conform to straight lines, circles, or other simple geometric shapes. For example, the number of curves and straight lines, broken line edges, wavy edges, etc., that exist on one edge of the photographed material at the same time.

[0094] In some embodiments, the data acquisition module may acquire the surface features and edge features to be extracted of the photographed material through an image recognition algorithm based on the geometric features of the photographed material. The image recognition algorithm may include a combination of one or more of YOLO (You Only Look Once), Support Vector Machine (SVM), or other custom algorithms.

[0095] The image recognition algorithm can extract and classify features of different regions in the image of the photographed material to identify the surface features and edge features to be extracted in the image. In some embodiments, the image recognition algorithm model can be obtained by training images of different photographed materials and their corresponding surface features and edge features to be extracted.

[0096] In some embodiments, the shape of the photographed material is fixed, and its surface features and edge features to be extracted can be provided by the material manufacturer and then manually input into the precise registration system.

[0097] In some embodiments, the accuracy prediction model may be a machine learning model. For example, the accuracy prediction model may include a combination of one or more of a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, or other custom models.

[0098] In some embodiments, Figure 3 As shown, the input of the accuracy prediction model 340 may include the surface features 320 and the edge features 330 to be extracted of the photographed material, and the output of the accuracy prediction model 340 may include the preset registration accuracy 350 of the photographed material.

[0099] For more information about preset registration accuracy, see Figure 2 and its related description.

[0100] In some embodiments, the accuracy prediction model can be obtained by training a large number of first training samples with first labels. The data acquisition module can input multiple first training samples with first labels into the initial accuracy prediction model, construct a loss function through the first label and the result of the initial accuracy prediction model, and iteratively update the initial accuracy prediction model based on the loss function. When the preset conditions are met, the model training is completed and a trained accuracy prediction model is obtained. Among them, the preset conditions can be that the loss function converges, the number of iterations reaches a threshold, etc.

[0101] In some embodiments, the first sample of the training accuracy prediction model may be the face features and edge features to be extracted of the historical photographed material. The historical photographed material refers to the photographed material in the historical data. The first label may be a preset registration accuracy with the highest registration efficiency when the historical photographed material is repeatedly registered with the point cloud, provided that the registration quality is met.

[0102] In some embodiments, satisfying the registration quality means that the overlap rate of the registered point cloud data is greater than a preset overlap rate threshold. The overlap rate can be used to characterize the proportion of the number of feature points that successfully match (accurately align) with the preset model contour. The preset overlap rate threshold can be preset based on experience.

[0103] In some embodiments, the preset registration accuracy with the highest registration efficiency refers to the preset registration accuracy with the least amount of data processing and / or the shortest registration time during the registration process. For more information about the amount of data processing, please refer to Figure 4 and its related description.

[0104] In some embodiments, if the historical photographed material corresponding to the first training sample has its preset registration accuracy updated after registration, the first label corresponding to the first training sample further includes: the updated preset registration accuracy of the historical photographed material.

[0105] For more information about updating the preset registration accuracy, see Figure 5 and its related description.

[0106] By further using the updated preset registration accuracy as the first label, the model training effect can be effectively improved, thereby improving the accuracy of model prediction.

[0107] In some embodiments of the present specification, the surface features and edge features to be extracted are determined by photographing the geometric features of the material, and the preset registration accuracy is determined using a trained accuracy prediction model, which can improve the accuracy and reliability of point cloud registration.

[0108] Figure 4 It is an exemplary schematic diagram of obtaining original point cloud data according to some embodiments of this specification.

[0109] In some embodiments, Figure 4 As shown, the data acquisition module can determine the point cloud acquisition parameters 410 based on the surface features 320 to be extracted, the edge features 330 to be extracted and the preset alignment accuracy 350; and based on the point cloud acquisition parameters 410, obtain the original point cloud data 420 of multiple target points of the photographed material.

[0110] For more information about the surface features to be extracted, the edge features to be extracted, and the preset registration accuracy, please refer to the previous article (such as Figure 2-Figure 3 ) in the relevant description.

[0111] Point cloud acquisition parameters refer to parameters related to the acquisition of point cloud data. In some embodiments, the point cloud acquisition parameters may include sampling density, sampling interval, and sampling mode. The sampling mode includes whether upsampling or downsampling is applied.

[0112] In some embodiments, the data acquisition module may determine the point cloud acquisition parameters through a first preset table based on the surface features to be extracted, the edge features to be extracted, and the preset registration accuracy. The first preset table may be used to characterize the correspondence between the surface features to be extracted, the edge features to be extracted, the preset registration accuracy, and the point cloud acquisition parameters of the photographed material. In some embodiments, the first preset table may be constructed based on the surface features to be extracted, the edge features to be extracted, the actual point cloud acquisition parameters, and the actual point cloud registration accuracy after registration of the photographed material in the historical data.

[0113] In some embodiments, the data acquisition module may use an acquisition device (such as a 3D camera) to capture multiple target points of the material to acquire original point cloud data based on the point cloud acquisition parameters. As an example, if the sampling interval of the point cloud acquisition parameters is 2s, the 3D camera may capture multiple target points of the material once every 2s to acquire original point cloud data.

[0114] In some embodiments, the data acquisition module may also acquire original point cloud data of multiple target points of the photographed material in the following manner.

[0115] S221: Clustering historical photographic materials based on the face features to be extracted and the edge features to be extracted. For more information about historical photographic materials, please refer to Figure 3 and its related contents.

[0116] In some embodiments, the data acquisition module may cluster the historical photographed materials based on the surface features to be extracted and the edge features to be extracted by using a clustering algorithm, wherein the clustering algorithm includes a K-means clustering algorithm and the like.

[0117] Exemplarily, the data acquisition module randomly selects k historical shooting materials from the historical shooting materials as initial clustering centers; constructs feature vectors based on the surface features to be extracted and the edge features to be extracted of the historical shooting materials, each feature vector corresponding to a historical shooting material; determines the vector distance between each historical shooting material and the initial clustering center based on multiple feature vectors, and assigns the historical shooting material to the cluster where the initial clustering center with the closest vector distance is located; calculates the vector mean of multiple feature vectors within the cluster where each initial clustering center is located, and uses it as the new clustering center; recalculates the vector distance between each historical shooting material and the new clustering center, and assigns the historical shooting material to the cluster where the new clustering center with the closest vector distance is located; repeats the above steps until the clustering center no longer changes, and k clusters are obtained.

[0118] S222: For each cluster obtained by clustering, based on the to-be-extracted surface features and to-be-extracted edge features of each historical photographed material in the cluster, a preset registration accuracy of each historical photographed material is generated through an accuracy prediction model.

[0119] In some embodiments, the accuracy prediction model can process the to-be-extracted surface features and to-be-extracted edge features of each historical photographic material in the cluster to obtain a preset registration accuracy corresponding to each historical photographic material. For more information on how to generate a preset registration accuracy through the accuracy prediction model, please refer to Figure 3 and its related contents.

[0120] S223: Generate multiple candidate point cloud acquisition parameters based on the surface features to be extracted, the edge features to be extracted and the preset registration accuracy of each historically photographed material.

[0121] The candidate point cloud acquisition parameters refer to parameters related to the acquisition of point cloud data for reference.

[0122] In some embodiments, the data acquisition module can generate multiple candidate point cloud acquisition parameters through a first preset table based on the surface features to be extracted, the edge features to be extracted and the preset registration accuracy of each historically photographed material. For more information about the first preset table, please refer to the relevant description in the previous text.

[0123] S224: Determine the optimal point cloud acquisition parameters corresponding to the cluster based on the data processing amount, preset registration accuracy and corresponding confidence of the plurality of candidate point cloud acquisition parameters.

[0124] The data processing amount refers to the amount of point cloud data involved in the registration. In some embodiments, the data acquisition module can obtain the data processing amount through the storage module. For example, the storage module can count and store the data processing amount when generating the candidate point cloud acquisition parameters, and the data acquisition module can directly obtain the data processing amount of the candidate point cloud acquisition parameters through the storage module.

[0125] The optimal point cloud acquisition parameters refer to the acquisition parameters with the best data quality among the point cloud acquisition parameters.

[0126] In some embodiments, the data acquisition module may determine the optimal point cloud acquisition parameters corresponding to the cluster by weighted summation based on the data processing volume of multiple candidate point cloud acquisition parameters, preset registration accuracy and their corresponding confidence levels.

[0127] For example, the preset registration accuracy and its corresponding confidence are multiplied to obtain the expected value of the preset registration accuracy, and then the expected value of the preset registration accuracy and the data processing amount of the candidate point cloud acquisition parameters are weighted and summed, and the candidate point cloud acquisition parameters with the largest weighted sum are used as the optimal point cloud acquisition parameters corresponding to the cluster. The weighted sum can be calculated by formula (1), which is as follows:

[0128] (1)

[0129] Where S is the weighted sum; m is the preset registration accuracy; a is the confidence level corresponding to the preset registration accuracy; is the data processing amount of the candidate point cloud acquisition parameters; k1 and k2 are weights, and k1 and k2 can be set based on experience.

[0130] In some embodiments, the size of k1 is related to the edge features to be extracted of the historical photographed material. For example, the more the number of irregular edges of the historical photographed material is, the larger k1 is.

[0131] For more information about special-shaped edges, please refer to other parts of this manual (such as Figure 5 and its related description).

[0132] In some embodiments of the present specification, the size of k1 is related to the edge features to be extracted of the historical photographed material, and the influence of the shape of the historical photographed material can be fully considered to improve the accuracy of the point cloud registration and ensure the registration effect. For example, the more irregular edges of the historical photographed material, the more complex its shape is, so it is necessary to improve the accuracy of the point cloud registration to ensure the registration effect.

[0133] In some embodiments, the confidence level corresponding to the preset registration accuracy may be determined based on the point cloud registration accuracy of the second registration result and the preset registration accuracy.

[0134] The point cloud registration accuracy of the second registration result refers to the point cloud registration accuracy after correction. In some embodiments, the point cloud registration accuracy of the second registration result can be determined based on the distance between each point in the original point cloud data after correction and the nearest neighbor reference point cloud point in the preset model contour. For example, it can be determined based on the average of the squares of the distances between each point in the original point cloud data after correction and the nearest neighbor reference point cloud point in the preset model contour. For more information about iterative correction of original point cloud data, please refer to Figure 2 and its related contents.

[0135] In some embodiments, the confidence level may be calculated using formula (2), which is as follows:

[0136] (2)

[0137] Among them, a is the confidence; m2 is the point cloud registration accuracy of the second registration result; and m is the preset registration accuracy.

[0138] By determining the above confidence level through the point cloud registration accuracy of the second registration result and the preset registration accuracy, the reliability of the registration accuracy evaluation can be improved and subsequent data processing and decision-making can be optimized.

[0139] In some embodiments of the present specification, by comprehensively considering the point cloud registration accuracy that can be achieved by the point cloud acquisition parameters and the amount of data that needs to be processed, the optimal point cloud acquisition parameters are determined, which can improve the point cloud registration efficiency while ensuring the registration accuracy.

[0140] S225: Acquire original point cloud data of multiple target points of the photographed material based on the optimal point cloud acquisition parameters corresponding to the cluster to which the photographed material belongs.

[0141] In some embodiments, the data acquisition module may acquire original point cloud data by photographing multiple target points of the photographed material based on the optimal point cloud acquisition parameters corresponding to the cluster to which the photographed material belongs.

[0142] In some embodiments of the present specification, the quality and applicability of point cloud data are ensured by accurately setting point cloud acquisition parameters, thereby improving the accuracy and efficiency of subsequent registration work.

[0143] Figure 5 is an exemplary schematic diagram of updating a preset registration accuracy according to some embodiments of the present specification.

[0144] In some embodiments, Figure 5As shown, the data update module can determine the point cloud registration accuracy 530 of the second registration result in response to the completion of the second registration result of the photographed material, based on the number of corrections 510 and the deviation data 520 of each correction; and update the preset registration accuracy 540 of the photographed material based on the point cloud registration accuracy 530 of the second registration result.

[0145] For more information about the captured material, preset registration accuracy, and the second registration result, see Figure 2 For more information about the point cloud registration accuracy of the second registration result, see Figure 4 and its related description.

[0146] The number of corrections refers to the number of correction iterations performed during the registration process. For more information about correction iterations, see Figure 2 and its related description.

[0147] The deviation data of each correction refers to the deviation data of each correction iteration in the alignment process. In some embodiments, the deviation data may include the deviation size and deviation position of the original point cloud data during correction. The deviation size refers to the data size deviation between each point of the original point cloud data after correction and each point of the original point cloud data after the last correction. The deviation position refers to the coordinate position deviation between each point of the original point cloud data after correction and each point of the original point cloud data after the last correction. For more information about original point cloud data, please refer to Figure 2 and its related description.

[0148] In some embodiments, the data update module can communicate and interact with the storage module to obtain the number of corrections and the deviation data of each correction. For example, the storage module can count and store the number of correction iterations performed by the result correction module and store the deviation data of each correction, and the data update module can obtain the number of corrections and the deviation data of each correction from the storage module.

[0149] In some embodiments, the data update module can determine the final coordinate position of the original point cloud data based on the number of corrections and the correction data of each correction, and compare the coordinate position with the preset model contour, so as to determine the actual point cloud registration accuracy after the correction, and use it as the point cloud registration accuracy of the second registration result. As an example only, for each correction, the data update module can determine the point to be corrected and the coordinate position of the point after correction based on the correction data of the correction; when all the correction data are traversed, the coordinate position of each point in the original point cloud data after correction can be obtained, and then the coordinate position of each point in the original point cloud data after correction is compared with each point of the preset model contour, and the actual point cloud registration accuracy after correction is obtained by calculating the average of the squares of the distances from each point in the original point cloud data after correction to the nearest reference point cloud point in the preset model contour.

[0150] In some embodiments, the data update module may end the correction iteration in response to the total data processing volume of the current correction and registration not exceeding a preset data volume threshold and the point cloud registration accuracy of the second registration result being greater than a preset registration accuracy, and determine the point cloud registration accuracy of the second registration result as the preset registration accuracy corresponding to the photographed material.

[0151] The total data processing volume refers to the sum of the point cloud data involved in each correction and registration. It should be understood that the method of obtaining the total data processing volume is similar to the method of obtaining the data processing volume. For details, please refer to Figure 4 and its related description.

[0152] In some embodiments, the data update module may also terminate the deflection correction iteration in response to the total data processing volume of the current deflection correction and registration not exceeding the preset data volume threshold and the number of iterations reaching the preset number of iterations, and determine the point cloud registration accuracy of the second registration result as the preset registration accuracy corresponding to the photographed material. The preset number of iterations may be preset based on experience.

[0153] In some embodiments, the preset data volume threshold may be determined based on empirical values.

[0154] In some embodiments, the data update module can also obtain a preset data volume threshold based on cluster analysis. Exemplarily, the objects to be clustered can be multiple cluster vectors and target vectors, wherein the cluster vectors can be constructed based on the edge features to be extracted, the surface features to be extracted, the preset registration accuracy and the total data processing volume of the historical photographed materials, and the target vectors can be constructed based on the edge features to be extracted, the surface features to be extracted and the preset registration accuracy of the current photographed materials. The clustering index can be the edge features to be extracted, the surface features to be extracted and the preset registration accuracy of the photographed materials.

[0155] In some embodiments, the data update module can perform clustering based on clustering indicators to obtain multiple cluster clusters, select the cluster cluster containing the target vector as the target cluster cluster; then calculate the average value of the total data processing volume in all cluster vectors in the target cluster cluster, and use it as the final preset data volume threshold.

[0156] It can be understood that when the actual point cloud registration accuracy of the second registration result is greater than the preset registration accuracy, and the total data processing volume of the actual point cloud registration accuracy is less than the total data processing volume of the preset registration accuracy, the preset registration accuracy of the photographed material is too small and there is a better solution, and the preset registration accuracy needs to be adjusted.

[0157] In some embodiments of the present specification, the preset registration accuracy is adjusted dynamically through deviation correction and the registration process is optimized, which can effectively improve the accuracy and adaptability and reduce future registration errors.

[0158] In some embodiments, the data acquisition module may update a preset registration accuracy in response to the presence of a preset number threshold of photographed materials after completing the determination of the second registration result, thereby determining at least one enhanced sample and performing enhanced training on the accuracy prediction model based on the at least one enhanced sample.

[0159] In some embodiments, the preset quantity threshold may be set in advance based on historical data.

[0160] In some embodiments, the preset quantity threshold is related to an average value of the proportion of irregular edges of a plurality of photographed materials.

[0161] In some embodiments, the multiple photographed materials may include one or more photographed materials that have been registered within a preset time, and / or one or more photographed materials that are about to be registered.

[0162] Irregular edges refer to the edges of the material that do not conform to straight lines, circles or other simple geometric shapes. The irregular edge ratio refers to the ratio of the number of irregular edges of the material to the total number of edges of the material. For more information about irregular edges, please refer to Figure 3 and its related description.

[0163] In some embodiments, the preset number threshold may be positively correlated with the average value of the proportion of irregular edges of the plurality of photographed materials. For example, the larger the average value of the proportion of irregular edges of the plurality of photographed materials, the larger the preset number threshold.

[0164] In some embodiments of the present specification, the larger the average value of the proportion of special-shaped edges of multiple photographed materials, the greater the irregularity of the multiple photographed materials, and the higher the difficulty of registration. By increasing the preset quantity threshold by considering the average value of the proportion of special-shaped edges, the frequency of enhanced training of the model can be reduced, thereby enhancing the system's adaptability to complex materials and registration accuracy.

[0165] Enhanced training means adding samples to the trained model to train it. Enhanced samples refer to samples used for enhanced training.

[0166] In some embodiments, the data acquisition module can update the photographed material with a preset registration accuracy after completing the determination of the second registration result, construct a sample using its surface features and edge features to be extracted, and use it as an enhanced sample; and use the preset registration accuracy of the above-mentioned photographed material after the update as the label corresponding to the enhanced sample.

[0167] In some embodiments, the data acquisition module may perform enhanced training on the accuracy prediction model based on at least one enhanced sample. For more information about the accuracy prediction model training, please refer to Figure 3 and its related description.

[0168] In some embodiments of the present specification, by enhancing training and continuously optimizing the accuracy prediction model using actual data, the registration accuracy and the intelligence level of the precise registration system can be effectively improved.

[0169] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

Claims

1. A precise registration method based on 3D point cloud and model contour, characterized in that: include: Get the preset registration accuracy of the captured material, including: Based on the geometric features of the photographed material, determining the surface features and edge features to be extracted of the photographed material; and Based on the surface features to be extracted and the edge features to be extracted, the preset registration accuracy is determined through an accuracy prediction model; the accuracy prediction model is a machine learning model; in response to the historical photographed material corresponding to the first training sample of the accuracy prediction model, the preset registration accuracy is updated after registration, and the first label corresponding to the first training sample also includes the preset registration accuracy of the historical photographed material after update; the preset registration accuracy refers to the registration accuracy that is preset to be achieved when performing registration; the registration accuracy refers to the measurement of the position deviation of corresponding points between two or more point cloud data sets in three-dimensional space after they are registered; Acquiring original point cloud data of multiple target points of the photographed material, including: Determining point cloud acquisition parameters based on the surface features to be extracted, the edge features to be extracted, and the preset registration accuracy; and Based on the point cloud acquisition parameters, the original point cloud data of multiple target points of the photographed material are acquired; the point cloud acquisition parameters include sampling density, sampling interval and sampling mode; Performing preliminary processing on the original point cloud data to obtain a point cloud image of a target layer; Based on the target layer point cloud image, determine the inner contour data of the target layer point cloud image; Determine a plurality of target feature points in the inner contour data; Based on the multiple target feature points, the photographed material is registered with a preset model contour to determine a first registration result; In response to the point cloud registration accuracy of the first registration result being greater than or equal to the preset registration accuracy, determining the first registration result as a second registration result; In response to the point cloud registration accuracy of the first registration result being less than the preset registration accuracy, performing at least one correction iteration on the first registration result to obtain an iterative registration result; In response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, the correction iteration is terminated, and the iterative registration result is determined as the second registration result.

2. The method according to claim 1, characterized in that Also includes: In response to the photographed material completing the determination of the second registration result, determining the point cloud registration accuracy of the second registration result based on the number of corrections and the deviation data of each correction; as well as Based on the point cloud registration accuracy of the second registration result, updating the preset registration accuracy of the photographed material; In response to the presence of the photographed materials exceeding a preset number threshold, after the second registration result is determined, the preset registration accuracy is updated, thereby determining at least one enhanced sample, and performing enhanced training on the accuracy prediction model based on the at least one enhanced sample; The preset quantity threshold is related to an average value of the proportion of irregular edges of the plurality of photographed materials.

3. A precise registration system based on 3D point cloud and model contour, characterized in that: include: The data acquisition module is configured as follows: Get the preset registration accuracy of the captured material, including: Based on the geometric features of the photographed material, determining the surface features and edge features to be extracted of the photographed material; and Based on the surface features to be extracted and the edge features to be extracted, the preset registration accuracy is determined through an accuracy prediction model; the accuracy prediction model is a machine learning model; in response to the historical photographed material corresponding to the first training sample of the accuracy prediction model, the preset registration accuracy is updated after registration, and the first label corresponding to the first training sample also includes the preset registration accuracy of the historical photographed material after update; the preset registration accuracy refers to the registration accuracy that is preset to be achieved when performing registration; the registration accuracy refers to the measurement of the position deviation of corresponding points between two or more point cloud data sets in three-dimensional space after they are registered; Acquiring original point cloud data of multiple target points of the photographed material, including: Determining point cloud acquisition parameters based on the surface features to be extracted, the edge features to be extracted, and the preset registration accuracy; and Based on the point cloud acquisition parameters, the original point cloud data of multiple target points of the photographed material are acquired; the point cloud acquisition parameters include sampling density, sampling interval and sampling mode; The data analysis module is configured as follows: Performing preliminary processing on the original point cloud data to obtain a point cloud image of a target layer; Based on the target layer point cloud image, determine the inner contour data of the target layer point cloud image; Determine a plurality of target feature points in the inner contour data; Based on the multiple target feature points, the photographed material is registered with a preset model contour to determine a first registration result; In response to the point cloud registration accuracy of the first registration result being greater than or equal to the preset registration accuracy, determining the first registration result as a second registration result; The result correction module is configured as follows: In response to the point cloud registration accuracy of the first registration result being less than the preset registration accuracy, performing at least one correction iteration on the first registration result to obtain an iterative registration result; In response to the point cloud registration accuracy of the iterative registration result being greater than or equal to the preset registration accuracy, the correction iteration is terminated, and the iterative registration result is determined as the second registration result.

4. The system according to claim 3, characterized in that Also includes: The data update module is configured as follows: In response to the photographed material completing the determination of the second registration result, determining the point cloud registration accuracy of the second registration result based on the number of corrections and the deviation data of each correction; as well as Based on the point cloud registration accuracy of the second registration result, updating the preset registration accuracy of the photographed material; In response to the presence of the photographed materials exceeding a preset number threshold, after the second registration result is determined, the preset registration accuracy is updated, thereby determining at least one enhanced sample, and performing enhanced training on the accuracy prediction model based on the at least one enhanced sample; The preset quantity threshold is related to an average value of the proportion of irregular edges of the plurality of photographed materials.

5. A precise registration device based on 3D point cloud and model contour, comprising a processor, characterized in that: The apparatus comprises at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is used to execute at least part of the computer instructions to implement the precise registration method based on 3D point cloud and model contour as described in any one of claims 1 to 2.

6. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the precise registration method based on 3D point cloud and model contour as described in any one of claims 1 to 2.

Images