Autonomous modeling implementation method and device and electronic equipment

Abstract

The invention provides an autonomous modeling implementation method and device and electronic equipment, and the method comprises the steps: sequentially shooting a target workpiece to obtain multiple pieces of point cloud data, carrying out the preprocessing of the point cloud data when obtaining new point cloud data, carrying out the point cloud modeling based on the preprocessed point cloud data, and if the modeling succeeds, carrying out the point cloud modeling. And splicing the modeling result to the existing modeling result to obtain an updated modeling result, and obtaining an initial modeling result. Comparing the initial modeling result with a preset welding seam database, if it is judged that supplementary building needs to be conducted, obtaining supplementary building point cloud data, executing point cloud supplementary building based on the supplementary building point cloud data to obtain a supplementary building result, and splicing the supplementary building result to the initial modeling result to obtain a final modeling result. In the scheme, the newly added point cloud data is modeled and then spliced to the existing modeling result, so that the problem of increment benefit reduction existing in global re-optimization can be avoided, and an efficient modeling effect can be obtained in the face of complex modeling requirements through supplementary building and continuous building based on the modeling result.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud processing technology, and more specifically, to an autonomous modeling implementation method, apparatus, and electronic device. Background Technology

[0002] Existing point cloud modeling methods mainly include delayed global optimization modeling, static incremental modeling, partitioned database modeling, and deep learning modeling.

[0003] The delayed global optimization modeling involves the operator collecting a series of photos or point cloud data. The software provides a real-time preview but does not perform any actual modeling or save any inheritable intermediate modeling progress. All collected data is stored in a designated queue or folder. After data collection is complete, the user clicks the "Start Reconstruction" button, and the software initiates a single, continuous computation task covering key aspects such as local registration, global optimization, and point cloud fusion. If the computation is successful, a complete 3D model will be output; if the computation terminates due to poor data quality or feature matching failure, the entire reconstruction task fails, and no intermediate results are available for subsequent use. The specific logic is divided into two parts: initial modeling: using the first batch of data (such as the first 10 images) to generate a preliminary sparse point cloud, cloud, or model; subsequent addition: the user can continue to add new data. This modeling method concentrates all computational pressure on the final "one-click reconstruction" stage, leading to two major flaws. First, the process is extremely fragile; any quality problem with a single point cloud or a matching failure occurring in the middle stage can cause the entire batch processing task to crash. Because there is no preservation and inheritance of intermediate results, any partial failure means that all previous data acquisition work and computing resources are completely wasted. Users must return to the first step for a global rescan, which is inefficient and results in a poor user experience. Secondly, this logic lacks an effective intermediate monitoring and correction mechanism. Users cannot know the validity of the modeling during the data acquisition stage and can only passively accept the result of "complete success" or "complete failure" in the final stage. They lack the ability to recover from the failure point or to make targeted reconstructions, which greatly reduces its practical value and reliability when dealing with complex workpieces or non-ideal scanning environments.

[0004] Static incremental modeling: In the initial modeling phase, the system uses the first batch of data (such as the first 10 images) to generate a preliminary sparse point cloud or basic model. In subsequent addition phases, users can continuously supplement new data. In the global re-optimization phase, after new data is added, the system typically triggers a global optimization process starting from scratch. This process covers all collected data, rather than performing local incremental expansions on the existing model. While this modeling approach allows for step-by-step data addition, its "global re-optimization" nature leads to two major drawbacks. First, its incremental benefits diminish drastically with increasing data volume. Each addition of new data requires recalculation of all data, resulting in massive redundant computations and significantly reducing system response speed and availability. Second, this logic cannot achieve precise model maintenance, as each optimization may fine-tune the entire model, causing previously stable structures to become unstable. This makes the model lack the version stability and determinism required for engineering benchmarks, unable to support true local "supplementation" or "renewal," making it rigid and inefficient when dealing with complex modeling needs.

[0005] Modeling by dividing the data into blocks involves manually dividing the complete scan sequence into several independent batches. First, intra-batch modeling is performed. The first batch (e.g., the first 20 point cloud images) undergoes a complete modeling process, generating "sub-model A". Then, the inter-batch alignment stage begins. When processing the second batch, the system treats this batch as a whole, registers and merges it with the already generated "sub-model A", resulting in "sub-model AB". This batch processing logic is then repeated, aligning and piecing together the corresponding sub-models of each batch like building blocks to form a complete model. Missing parts in the modeling process are treated as independent "supplementary batches", generating "sub-model C". Finally, "sub-model C" is manually or semi-automatically aligned and merged with the main model. This modeling method forcibly decomposes the originally continuous scan sequence into isolated batches, forcing the system to abandon a large number of high-precision continuous inter-frame constraints and instead rely on sparse feature matching between batches for alignment. This approach accumulates errors due to multiple alignments, ultimately compromising the accuracy and consistency of the overall model. Furthermore, its "supplementary" process is severely disconnected from the main modeling process. The supplementary scanned data is treated as a completely new project, generating an independent sub-model, which is then stitched together with the main model through secondary operations. This approach increases the complexity of the operation, making the entire process neither efficient nor flexible.

[0006] Deep learning-based modeling involves users uploading batches of randomly captured point clouds to a pre-trained deep learning model. These batch point clouds, powered by a pre-trained large neural network, automatically identify common features in the scene (such as edges and corners) and attempt to directly generate a relatively complete model. The drawbacks of this modeling approach lie in the unpredictability of its results and the opacity of its process. As a typical "black box" model, the geometric structures generated based on statistical patterns of big data may be "illusions," creating non-existent or inaccurate features out of thin air, which is unacceptable for industrial models requiring precise dimensions. Furthermore, deep learning methods completely lose engineering controllability and interpretability. Users cannot intervene or understand why the model is shaped in a certain way, and it is difficult to make targeted corrections when its output is flawed. It lacks a clear definition of "modeling failure" and cannot provide clear "re-building" or "re-building" paths. Ultimately, the system outputs only a probabilistic, "seemingly correct" visual approximation, rather than a rigorously tested, deterministic engineering model. Summary of the Invention

[0007] The purpose of this invention is to provide an autonomous modeling method, apparatus, and electronic device to improve modeling efficiency and ensure modeling accuracy.

[0008] In a first aspect, the present invention provides an autonomous modeling implementation method, the method comprising: Multiple point cloud data images are sequentially captured for the target workpiece, and each time a new point cloud data image is acquired, the point cloud data is preprocessed to obtain preprocessed point cloud data. Point cloud modeling is performed based on the preprocessed point cloud data. The modeling is then determined to be successful. If the modeling is unsuccessful, the modeling result is discarded. If the modeling is successful, the modeling result is stitched into the existing modeling result to obtain an updated modeling result. The initial modeling result is obtained after all the multiple point cloud data have completed point cloud modeling. Compare the initial modeling results with the preset weld database to determine whether additional modeling is needed; If it is determined that supplementary modeling is required, supplementary point cloud data is obtained, and point cloud supplementation is performed based on the supplementary point cloud data to obtain the supplementary modeling result. The supplementary modeling result is then stitched together with the initial modeling result to obtain the final modeling result.

[0009] In an optional implementation, the step of preprocessing the point cloud data to obtain preprocessed point cloud data includes: The ground in the point cloud data is fitted to obtain the fitted ground; Remove the fitted ground and points below the fitted ground from the point cloud data; The points in the point cloud data are clustered, and the point cloud blocks corresponding to the largest connected component after clustering are extracted.

[0010] In an optional implementation, each point cloud data is saved to a point cloud folder named sequentially according to the shooting order. In the point cloud modeling process, each point cloud data to be modeled is associated with an index number, which includes the current number, the previous number, and the previous number in reverse order. The current number, the previous number, and the previous number each include an index name and an index number. The index name is consistent with the number in the file name of the point cloud folder where the point cloud data is located, and the index number is numbered sequentially according to the storage order of each point cloud data to be modeled. In the point cloud modeling process, the index name and index number in the current numbering correspond to the latest point cloud data. If the previous modeling was successful, the previous number is assigned to the previous number, and the current number is assigned to the previous number. If the previous modeling was unsuccessful, the previous number remains unchanged, and the previous number is assigned to the previous number, so that the current modeling can reuse the modeling results generated by the previous successful modeling.

[0011] In an optional implementation, the step of determining whether the modeling was successful includes: Perform point cloud registration between the current point cloud data in this modeling and the existing point cloud data in the existing modeling results; Detect whether the current point cloud data after registration has any overlap with the existing point cloud data; If there is an intersection, the modeling is considered successful; otherwise, the modeling is considered unsuccessful.

[0012] In an optional implementation, the step of performing point cloud modeling based on the preprocessed point cloud data includes: Perform a surface lookup on the preprocessed point cloud data, where the surface includes planes and cylinders; Based on the found planes and cylinders, a surface structure is constructed, and surface structure information is obtained. A structure list containing surface structure information of all surface structures is constructed, and the surface structures include planar structures and cylindrical structures.

[0013] In an optional implementation, the step of stitching the current modeling result into the existing modeling result to obtain the updated modeling result includes: A merging operation is performed on the surface structures in the current modeling result and the existing modeling result, and the surface structure information of the merged surface structure is updated to the structure list. Read all surface structure information in the structure list, analyze the correlation between different surface structure information, parse multiple weld data based on the correlation, and update the weld data obtained by parsing to the weld database; The merged and stitched point cloud data, structure list, and weld database are saved to the results directory as updated modeling results.

[0014] In an optional implementation, the step of merging the surface structures in the current modeling result and the existing modeling result includes: Determine the surface structures in the same spatial location as the current modeling results and existing modeling results; The system detects whether surface structures in the same spatial location meet preset matching conditions. If the preset matching conditions are met, the surface structures in the same spatial location are merged.

[0015] In an optional implementation, the step of performing point cloud reconstruction based on the reconstructed point cloud data to obtain the reconstruction result, and then stitching the reconstruction result to the initial modeling result to obtain the final modeling result, includes: Determine whether this is the first time to supplement the model. If it is the first time to supplement the model, obtain the result directory corresponding to the initial modeling result, save the result directory back to the modeling directory, and set its index name as the initial index name. Set the index number and index name for this supplementary point cloud data based on the latest index number and index name; The surface structure is searched in the latest supplementary point cloud data, and the found surface structure is merged with the surface structure in the initial modeling result. The merged surface structure is updated to the structure list, and the weld database is updated based on the structure list. The result directory is updated based on the weld database, the structure list, and the spliced ​​merged point cloud data.

[0016] Secondly, the present invention provides an autonomous modeling implementation device, the device comprising: The processing module is used to sequentially capture multiple point cloud data images of the target workpiece, and to preprocess the point cloud data each time a new point cloud data is acquired to obtain preprocessed point cloud data. The modeling module is used to perform point cloud modeling based on preprocessed point cloud data, and to determine whether the modeling is successful. If the modeling is unsuccessful, the modeling result is discarded. If the modeling is successful, the modeling result is stitched into the existing modeling result to obtain the updated modeling result. The result acquisition module is used to obtain the initial modeling result until all the multiple point cloud data have completed point cloud modeling. The judgment module is used to compare the initial modeling results with the preset weld database to determine whether supplementary modeling is required. The supplementary modeling module is used to obtain supplementary point cloud data if it is determined that supplementary modeling is needed, perform point cloud supplementary modeling based on the supplementary point cloud data to obtain the supplementary modeling result, and stitch the supplementary modeling result to the initial modeling result to obtain the final modeling result.

[0017] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method described in any of the foregoing embodiments.

[0018] This invention provides an autonomous modeling method, apparatus, and electronic device. Multiple point cloud images are sequentially captured on a target workpiece. Each time new point cloud data is acquired, it is preprocessed to obtain preprocessed point cloud data. Point cloud modeling is performed based on the preprocessed point cloud data. The success of the modeling is determined; if unsuccessful, the result is discarded; if successful, it is appended to existing modeling results to obtain an updated result. This process continues until point cloud modeling is completed on all point cloud images, yielding an initial modeling result. The initial modeling result is compared with a preset weld database to determine if supplementary modeling is needed. If supplementary modeling is required, supplementary point cloud data is obtained. Point cloud supplementation is performed based on this supplementary data to obtain a supplementary result, which is then appended to the initial modeling result to obtain the final modeling result.

[0019] In this solution, by modeling the newly added point cloud data and then stitching it into the existing modeling results, the problem of reduced incremental benefits caused by global re-optimization can be avoided. Furthermore, by supplementing and continuing to build based on the modeling results, efficient modeling can be achieved when facing complex modeling needs. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the autonomous modeling implementation method provided in this embodiment of the invention; Figure 2 for Figure 1 A flowchart of the sub-steps included in S11; Figure 3 This is a schematic diagram illustrating the implementation logic of point cloud modeling in an embodiment of the present invention; Figure 4 for Figure 1A flowchart of the sub-steps included in S12; Figure 5 A schematic diagram illustrating the concavity and convexity between surfaces; Figure 6 This is a schematic diagram of a planar structure; Figure 7 This is a schematic diagram of a cylindrical structure; Figure 8 for Figure 1 A flowchart of the sub-steps included in S13; Figure 9 for Figure 1 A flowchart of the sub-steps included in S15; Figure 10 for Figure 1 A flowchart of the sub-steps included in S18; Figure 11 This is a schematic diagram illustrating the implementation logic of the supplementary construction process in an embodiment of the present invention; Figure 12 A functional block diagram of the autonomous modeling implementation device provided in the embodiments of the present invention; Figure 13 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0023] Please see Figure 1 The above is a flowchart of an autonomous modeling implementation method provided in an embodiment of the present invention. The autonomous modeling implementation method can be executed by an autonomous modeling implementation device, which can be implemented by software and / or hardware and can be configured in an electronic device, such as a computer device or a server. The detailed steps of the autonomous modeling implementation method are described below.

[0024] S11: Take multiple point cloud data images of the target workpiece in sequence, and preprocess the point cloud data each time a new point cloud data is obtained to obtain preprocessed point cloud data. S12, Perform point cloud modeling based on the preprocessed point cloud data; S13, Determine whether the modeling was successful. If the modeling was unsuccessful, proceed to step S14. If the modeling was successful, proceed to step S15. S14, Discard the results of this modeling; S15, the current modeling result is stitched into the existing modeling result to obtain the updated modeling result; S16, until point cloud modeling is completed for all point cloud data, and the initial modeling result is obtained; S17. Compare the initial modeling results with the weld database to determine whether additional modeling is required. If additional modeling is required, proceed to the following step S18. S18: Obtain the supplementary point cloud data, perform point cloud supplementation based on the supplementary point cloud data to obtain the supplementary result, and stitch the supplementary result to the initial modeling result to obtain the final modeling result.

[0025] The autonomous modeling method provided in this embodiment avoids the problem of reduced incremental benefits caused by global re-optimization by modeling newly added point cloud data and then stitching it into existing modeling results. Furthermore, by supplementing and continuing to build based on the modeling results, it can achieve efficient modeling when facing complex modeling needs.

[0026] The specific implementation methods of each of the above steps will be explained in detail below.

[0027] In this embodiment, a camera can be used to photograph the target workpiece. Multiple images of the target workpiece can be obtained from different shooting angles, and adjacent images need to have a certain overlap, following the shooting order. Each image is then processed to obtain corresponding point cloud data.

[0028] In this embodiment, each time a new point cloud data is obtained, the new point cloud data is processed, including preprocessing and modeling, and the modeling result of the new point cloud data is stitched into the existing modeling result to obtain a new modeling result.

[0029] Since the obtained point cloud data may contain interfering factors such as the ground and other objects, it is necessary to remove these interfering factors first to ensure the accuracy of subsequent modeling. Therefore, in this embodiment, each time new point cloud data is obtained, it is first preprocessed to obtain preprocessed point cloud data. Please refer to [link to relevant documentation]. Figure 2 This step can be achieved in the following way: S111, Fit the ground in the point cloud data to obtain the fitted ground; S112, Remove the fitted ground and points below the fitted ground from the point cloud data; S113, perform clustering processing on the points in the point cloud data, and extract the point cloud block corresponding to the largest connected component after clustering.

[0030] In this embodiment, the least squares method or RANSAC method can be used to fit the ground and obtain the ground equation of the fitted ground, which facilitates subsequent de-grounding processing of the local point cloud.

[0031] The least squares method seeks a plane such that the sum of the squares of the distances from all points in the point cloud to that plane is minimized. Mathematically, this can be expressed as: for a plane equation... Ax + By+ Cz + D =0 (where the normal vector) n = [ A , B , C ]and ||n|| = 1) The goal is to minimize the following function: Σ( Axi + Byi + Czi + D ) 2 .

[0032] RANSAC is a random sampling consensus algorithm that achieves ground fit in the following way: Random hypothesis: Randomly select 3 points from the point cloud to calculate an initial plane; Find consensus: Calculate how many other points are within a certain threshold of the plane (these points are called "interiors"); Optimize model: In all iterations, select the model with the most "interiors" as the final result.

[0033] In this embodiment, the point cloud data obtained from multiple shots of the target workpiece are saved in the order of shooting: for each point cloud data shot, the point cloud file name is sequentially incremented, for example, starting with "0.pcd" and saved in the original point cloud folder in an incrementing numerical manner.

[0034] Based on the above ground fitting, point cloud de-landing and maximum connected component extraction operations are performed on the point cloud data. For point cloud de-landing, the ground equation obtained from the fitted ground is used to remove the fitted ground and points below the fitted surface (below the ground equation) from the point cloud data. For maximum connected component extraction, Euclidean clustering can be used to obtain the point cloud block corresponding to the largest connected component (largest point cluster) after clustering, which is the point cloud block where the target workpiece is located.

[0035] Euclidean clustering is a point cloud segmentation algorithm based on spatial Euclidean distance. Its core idea is to classify points in space that are less than a distance threshold into the same class (and points that are greater than a distance threshold into different classes).

[0036] Clustering classification criteria: Two points are classified into the same cluster if the distance between them meets the following conditions: d ( Pi , Pj ) ≤d threshold in, d threshold This is the preset distance threshold.

[0037] Maximum Connectivity Component Selection: Let the cluster set be C = {C1, C2, ..., Cn}, where C k Let |C| represent the k-th cluster, which contains |C| points. k |, then the largest connected component C max satisfy: C max =argmax Ck∈C | C k | Based on the preprocessing of point cloud data using the above methods, point cloud modeling is performed on the preprocessed point cloud data.

[0038] Please refer to the following: Figure 3 In this embodiment, each point cloud data is saved to a point cloud folder named sequentially according to the shooting order. In the point cloud modeling process, each point cloud data to be modeled is associated with an index number. The index number includes the current number, the previous number, and the previous number in reverse order. The current number, the previous number, and the last number include the index name and the index number, respectively. The index name is consistent with the number in the file name of the point cloud folder where the point cloud data is located, and the index number is numbered sequentially according to the storage order of each point cloud data to be modeled. In the point cloud modeling process, the index name and index number in this numbering correspond to the latest point cloud data.

[0039] Taking a real-world scenario as an example, if there are 5 discontinuous point cloud data images in the current stitching path, with files named "0.pcd, 1.pcd, 3.pcd, 4.pcd, 5.pcd" (the point cloud data that should originally be 2.pcd may have been deleted due to insufficient shooting requirements or other reasons), the specific correspondence is as follows: Index name: The core identifier is directly used from the file name of the point cloud folder, that is, the index name in this scenario is "0, 1, 3, 4, 5".

[0040] Index number: Arranged sequentially according to the storage order of point cloud data under the path, the index numbers corresponding to the above index names are "0, 1, 2, 3, 4".

[0041] In point cloud processing, recording both the index number and index name is crucial. When modeling fails and needs to be reprocessed, or when additional modeling is required for a portion of the point cloud, the index number can be used to quickly locate the sequence, and the index name can be used to match the original file, significantly improving processing efficiency and accuracy. During the initialization phase of the index numbering system, the initial parameters of all point cloud data must be set uniformly: the initial index name is an empty string (""); the initial index number is set to -1.

[0042] If the current modeling is the first time, it is necessary to initialize the index number of the spliced ​​point cloud and process the relevant directories.

[0043] If the current modeling is not the first time modeling, and the previous modeling was successful, the previous number is assigned to the previous number, and the current number is assigned to the previous number. If the previous modeling was unsuccessful, the previous number remains unchanged, and the previous number is assigned to the previous number, so that the current modeling can reuse the modeling results generated by the previous successful modeling.

[0044] Each time modeling is performed, the existence of the results directory can be checked. If it exists, the internal files are cleared; otherwise, the results directory is created. The results directory includes: model files with index numbers generated after each round of modeling, a list of structures, surface structure information, and a generated weld database, etc.

[0045] In this embodiment, based on the aforementioned modeling rules, point cloud modeling is performed using preprocessed point cloud data. For details, please refer to [link to relevant documentation]. Figure 4 This can be achieved in the following ways: S121 performs a face lookup on the preprocessed point cloud data, where the face includes planes and cylinders; S122, construct a surface structure based on the found planes and cylinders, obtain surface structure information, and construct a structure list containing surface structure information of all surface structures. Surface structures include planar structures and cylindrical structures.

[0046] In this embodiment, a surface structure search is performed on the point cloud data. Specifically, surface structures can be categorized as follows: biconcave planar structure, triconcave planar structure, biconcave-convex planar structure, biconcave-concave planar structure, biconcave-convex planar structure, biconcave-convex planar structure, biconcave cylindrical structure (Type 1), biconcave cylindrical structure (Type 2), triconcave cylindrical structure, biconcave-convex cylindrical structure, biconcave-concave cylindrical structure, and tangent cylindrical structure. Surface structure filtering involves selecting from these surface structures. The definition of surface structure involves the following concepts: Surface structure: A structure composed of planes and cylinders, which can be divided into planar structures and cylindrical structures.

[0047] Concavity / convexity: The relationship between two surfaces based on a viewpoint, including planes and cylinders. Examples are shown below. Figure 5 As shown, the camera is on the extension of line HB, and the shooting angle is indicated by the arrow. In the figure, planes ABCD, ABEF, and BDFG form convex planar structures in pairs, while planes ACEH, CDGH, and EFGH form concave planar structures in pairs.

[0048] Surface structure type: such as Figure 6 and Figure 7As shown, based on the number, type, and concavity / convexity of the surfaces, surface structures can be divided into: two concave plane structures, three concave plane structures, two concave and one convex plane structures, two convex and one concave plane structures, two concave and one convex plane structures, two concave cylindrical structures (Type 1), two concave cylindrical structures (Type 2), three concave cylindrical structures, two concave and one convex cylindrical structures, two convex and one concave cylindrical structures, and tangent cylindrical structures.

[0049] After completing the modeling and obtaining the results, it is necessary to first determine whether the modeling was successful. For details, please refer to [link to relevant documentation]. Figure 8 The determination can be achieved in the following ways: S131, perform point cloud registration between the current point cloud data in this modeling and the existing point cloud data in the existing modeling results; S132, check whether the current point cloud data after registration has an intersection with the existing point cloud data. If there is an intersection, execute the following step S133; otherwise, execute the following step S134. S133, the modeling was successful. S134, the modeling attempt was unsuccessful.

[0050] In this embodiment, the current modeling result is registered with the existing modeling result using a registration method to perform point cloud registration. It is then checked whether there is any intersection between the current modeling result and the existing modeling result after registration. If there is, the current modeling is considered successful, and there is no need to change the modeling failure flag (default is True). If there is no intersection, the current modeling is considered unsuccessful, and the modeling failure flag needs to be modified (modified to False).

[0051] In this embodiment, any fine registration method can be used because point cloud modeling is completed in the same coordinate system. Therefore, there is no significant error between the local point cloud and the modeled point cloud, and fine registration methods such as ICP or NDT can be used to accomplish this function. The mathematical formulas involved in ICP fine registration are as follows: Point cloud to be registered: P= { p 1, p 2, p N},in p i = ( x i , y i , z i ) T ∈ R 3 (N is the number of points) 。

[0052] Target point cloud: Q={q1,q2,…,q N}, where q=(a i b i c i ) T ∈R 3 (Assume the number of corresponding points is the same as P).

[0053] Transformation matrix: derived from rotation matrix R ∈ R 3×3 (satisfies R) T R=1 and det(R)=1) and translation vector t∈R 3 The composition and transformation relationship of points are: p ’ i =Rp i +t (the point to be registered after transformation).

[0054] If the modeling attempt fails, the index number and name will be updated according to the aforementioned index number and name update rules, and the system will automatically activate the modeling failure flag. When the next round of modeling begins, if this flag is detected as activated, the index number assignment order of the stitched point cloud will be adjusted. The core purpose is to ensure that if the previous round of modeling failed, the system will reuse the successfully generated model from the previous round, thus preventing modeling process crashes and ensuring that each successful modeling result is the correct version. Possible reasons for modeling failure include: ① no intersection between the local point cloud and the existing model; ② empty point cloud data (i.e., zero points), etc.

[0055] If the modeling is deemed successful, the index number and index name are updated according to the index number and index name update rules described above.

[0056] Based on the above index number and index name update rules, when the modeling failure flag is not started: the previous update (including index number and file name): last_file_index_str , last_file_index The previous assignment (including index number and file name): last_last_file_index , last_last_file_index This (including index number and file name): lastest_file_index_str , lastest_file_index To assign the previous value and retrieve the latest index, see the example below: last_last_file_index_str=last_file_index_str ; last_last_file_index = last_file_index; last_file_index_str=lastest_file_index_str ; last_file_index =lastest_file_index; lastest_file_index = Current index number; lastest_file_index_str= The latest filename.

[0057] When the modeling failure flag is set: the previous value remains unchanged, the previous value is assigned the value from the previous time, or the latest index is retrieved. An example is shown below: last_last_file_index_str= Remain unchanged; last_last_file_index= Remain unchanged; last_file_index_str = last_last_file_index_str; last_file_index =last_last_file_index; lastest_file_index = Current index number; lastest_file_index_str= The latest filename.

[0058] If the modeling is successful, the results will be appended to existing modeling results to obtain an updated modeling result. For details, please refer to [link / reference]. Figure 9 This can be achieved in the following ways: S151, Perform a merging operation on the surface structures in the current modeling result and the existing modeling result, and update the surface structure information of the merged surface structure to the structure list; S152, Read all surface structure information in the structure list, analyze the correlation between different surface structure information, parse out multiple weld data based on the correlation, and update the weld data obtained by parsing to the weld database; S153 saves the merged point cloud data, structure list, and weld database to the results directory as the updated modeling results.

[0059] The step of merging the surface structures in the current modeling results with those in existing modeling results can be achieved in the following way: Identify the surface structures in the same spatial location as the current modeling result and the existing modeling results; check whether the surface structures in the same spatial location meet the preset matching conditions. If the preset matching conditions are met, perform a merging operation on the surface structures in the same spatial location.

[0060] Structure merging refers to integrating different structures located in the same position. Specifically, when two structures of different types are in the same spatial location and meet the core matching conditions, they can be identified as the same type of structure and merged. For example, if a three-sided concave structure and a two-sided concave structure exist in the same location, and their normal vectors are consistent and their positions are basically overlapping, then they meet the merging criteria. After merging, the relevant structural information will be synchronously updated to the structure list. Subsequently, the system will perform unified filtering and secondary merging of all structural information to ensure the uniqueness and integrity of the structural data.

[0061] The system first reads all structural information from the structure list, then analyzes the relationships between different structures, and based on these relationships, parses out multiple weld data, ultimately forming a complete weld database.

[0062] After modeling is complete, the configuration file will be saved to the results directory: After each round of modeling, the system will save an additional copy of the latest generated results to the results directory. The saved content includes the model point cloud, structure list, and surface structure information. This operation provides key data support for subsequent model reconstruction and is crucial for the smooth progress of the reconstruction process.

[0063] Using the above method, point cloud modeling is performed on all the captured point cloud data (e.g., 10 images) to obtain the initial modeling result. The weld database in the initial modeling result is compared with the preset weld database. If it is found that the weld database in the initial modeling result is missing welds from the preset weld database, the existing model needs to be supplemented.

[0064] The logic for supplementary modeling is relatively independent, and it's used when previous index information is unavailable. First, determine the workpiece name that needs supplementation. Then, retrieve the model point cloud, structure list, and surface structure information from the last modeling result (initial modeling result) from the results directory, manually setting the index to 0, and re-store them in the relevant modeling directory. Next, assign the newly acquired point cloud with the current index information and continue supplementary modeling.

[0065] In this embodiment, point cloud reconstruction is performed based on the reconstructed point cloud data to obtain the reconstruction result. The reconstruction result is then stitched together with the initial modeling result to obtain the final modeling result. For details, please refer to the relevant documentation. Figure 10 and Figure 11 This step can be achieved in the following way: S181, determine whether this supplementary construction is the first supplementary construction. If it is the first supplementary construction, obtain the result directory corresponding to the initial modeling result, save the result directory back to the modeling directory, and set its index name as the initial index name. S182, Set the index number and index name for this supplementary point cloud data based on the latest index number and index name; S183, perform surface structure lookup on the latest supplementary point cloud data, and merge the found surface structures with the surface structures in the initial modeling results; S184 updates the surface structure after the merging operation to the structure list, updates the weld database based on the structure list, and updates the result directory based on the weld database, structure list, and spliced ​​merged point cloud data.

[0066] In this embodiment, the above methods can support both "model continuation" and "model supplementation," combining these two logics to form a systematic solution. The focus of this invention is to construct a novel and flexible modeling process logic. Its core value lies in its application-oriented approach, which, through process control and state management, endows the point cloud modeling process with robustness and scalability, rather than being confined to the specific implementation of any module.

[0067] In this embodiment, a "modeling failure flag" is established, and the order of the stitched point cloud index numbers is dynamically changed accordingly to ensure continued modeling even after a local modeling failure. Furthermore, separating the complete structure list from the point cloud data for management lays the foundation for local remodeling. Moreover, a remodeling method centered on loading the initial modeling results and resetting the index is designed to ensure that the supplementary information is highly integrated with the original model.

[0068] Based on the same inventive concept, please refer to Figure 12 This invention also provides a functional module diagram of an autonomous modeling implementation device. This embodiment divides the autonomous modeling implementation device into functional modules based on the above method embodiments. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this invention embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0069] The autonomous modeling implementation device may include a processing module, a modeling module, a result acquisition module, a judgment module, and a supplementary module. The functions of each functional module of the autonomous modeling implementation device will be described in detail below.

[0070] The processing module is used to sequentially capture multiple point cloud data images of the target workpiece, and to preprocess the point cloud data each time a new point cloud data is acquired to obtain preprocessed point cloud data. The modeling module is used to perform point cloud modeling based on preprocessed point cloud data, and to determine whether the modeling is successful. If the modeling is unsuccessful, the modeling result is discarded. If the modeling is successful, the modeling result is stitched into the existing modeling result to obtain the updated modeling result. The result acquisition module is used to obtain the initial modeling result until point cloud modeling is completed for multiple point cloud data. The judgment module is used to compare the initial modeling results with the preset weld database to determine whether additional modeling is needed. The supplementary modeling module is used to obtain supplementary point cloud data if it is determined that supplementary modeling is needed, perform point cloud supplementary modeling based on the supplementary point cloud data to obtain the supplementary modeling result, and stitch the supplementary modeling result to the initial modeling result to obtain the final modeling result.

[0071] The autonomous modeling implementation device provided in this embodiment can be used to execute the autonomous modeling implementation method under any of the above embodiments. For details not covered in this embodiment, please refer to the corresponding descriptions in the above embodiments. This embodiment will not elaborate further here.

[0072] Please see Figure 13 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device can be, for example, a computer device or a server. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0073] The memory is used to store computer programs or data. Memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0074] The processor is used to read / write data or programs stored in the memory and execute the autonomous modeling implementation method provided in any embodiment of the present invention.

[0075] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0076] It should be understood that, Figure 13 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 13 The more or fewer components shown, or having the same Figure 13 The different configurations shown.

[0077] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when executed, implement the autonomous modeling implementation method provided in the above embodiments.

[0078] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is run, it can execute the above-described autonomous modeling implementation method. The processes involved in the execution of the executable instructions on the computer-readable storage medium can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0079] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0080] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] Furthermore, in the various embodiments of the present invention, the functional modules can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0082] It should be noted that if the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] The above are merely embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for implementing autonomous modeling, characterized in that, The method includes: Multiple point cloud data images are sequentially captured for the target workpiece, and each time a new point cloud data image is acquired, the point cloud data is preprocessed to obtain preprocessed point cloud data. Point cloud modeling is performed based on the preprocessed point cloud data. The modeling is then determined to be successful. If the modeling is unsuccessful, the modeling result is discarded. If the modeling is successful, the modeling result is stitched into the existing modeling result to obtain an updated modeling result. The initial modeling result is obtained after all the multiple point cloud data have completed point cloud modeling. Compare the initial modeling results with the preset weld database to determine whether additional modeling is needed; If it is determined that supplementary modeling is required, supplementary point cloud data is obtained, and point cloud supplementation is performed based on the supplementary point cloud data to obtain the supplementary modeling result. The supplementary modeling result is then stitched together with the initial modeling result to obtain the final modeling result.

2. The autonomous modeling implementation method according to claim 1, characterized in that, The step of preprocessing the point cloud data to obtain preprocessed point cloud data includes: The ground in the point cloud data is fitted to obtain the fitted ground; Remove the fitted ground and points below the fitted ground from the point cloud data; The points in the point cloud data are clustered, and the point cloud blocks corresponding to the largest connected component after clustering are extracted.

3. The autonomous modeling implementation method according to claim 1, characterized in that, Each point cloud data is saved to a point cloud folder named sequentially according to the shooting order. In the point cloud modeling process, each point cloud data to be modeled is associated with an index number, which includes the current number, the previous number, and the previous number in reverse order. The current number, the previous number, and the previous number each include an index name and an index number. The index name is consistent with the number in the file name of the point cloud folder where the point cloud data is located, and the index number is numbered sequentially according to the storage order of each point cloud data to be modeled. In the point cloud modeling process, the index name and index number in the current numbering correspond to the latest point cloud data. If the previous modeling was successful, the previous number is assigned to the previous number, and the current number is assigned to the previous number. If the previous modeling was unsuccessful, the previous number remains unchanged, and the previous number is assigned to the previous number, so that the current modeling can reuse the modeling results generated by the previous successful modeling.

4. The autonomous modeling implementation method according to claim 1, characterized in that, The steps for determining whether the modeling was successful include: Perform point cloud registration between the current point cloud data in this modeling and the existing point cloud data in the existing modeling results; Detect whether the current point cloud data after registration has any overlap with the existing point cloud data; If there is an intersection, the modeling is considered successful; otherwise, the modeling is considered unsuccessful.

5. The autonomous modeling implementation method according to claim 1, characterized in that, The step of performing point cloud modeling based on preprocessed point cloud data includes: Perform a surface lookup on the preprocessed point cloud data, where the surface includes planes and cylinders; Based on the found planes and cylinders, a surface structure is constructed, and surface structure information is obtained. A structure list containing surface structure information of all surface structures is constructed, and the surface structures include planar structures and cylindrical structures.

6. The autonomous modeling implementation method according to claim 5, characterized in that, The step of stitching the current modeling result into the existing modeling result to obtain the updated modeling result includes: A merging operation is performed on the surface structures in the current modeling result and the existing modeling result, and the surface structure information of the merged surface structure is updated to the structure list. Read all surface structure information in the structure list, analyze the correlation between different surface structure information, parse multiple weld data based on the correlation, and update the weld data obtained by parsing to the weld database; The merged and stitched point cloud data, structure list, and weld database are saved to the results directory as updated modeling results.

7. The autonomous modeling implementation method according to claim 6, characterized in that, The steps for merging the surface structures in the current modeling result and the existing modeling result include: Determine the surface structures in the same spatial location as the current modeling results and existing modeling results; The system detects whether surface structures in the same spatial location meet preset matching conditions. If the preset matching conditions are met, the surface structures in the same spatial location are merged.

8. The autonomous modeling implementation method according to claim 1, characterized in that, The step of performing point cloud reconstruction based on the reconstructed point cloud data to obtain the reconstruction result, and then stitching the reconstruction result to the initial modeling result to obtain the final modeling result, includes: Determine whether this is the first time to supplement the model. If it is the first time to supplement the model, obtain the result directory corresponding to the initial modeling result, save the result directory back to the modeling directory, and set its index name as the initial index name. Set the index number and index name for this supplementary point cloud data based on the latest index number and index name; The surface structure is searched in the latest supplementary point cloud data, and the found surface structure is merged with the surface structure in the initial modeling result. The merged surface structure is updated to the structure list, and the weld database is updated based on the structure list. The result directory is updated based on the weld database, the structure list, and the spliced ​​merged point cloud data.

9. An autonomous modeling implementation device, characterized in that, The device includes: The processing module is used to sequentially capture multiple point cloud data images of the target workpiece, and to preprocess the point cloud data each time a new point cloud data is acquired to obtain preprocessed point cloud data. The modeling module is used to perform point cloud modeling based on preprocessed point cloud data, and to determine whether the modeling is successful. If the modeling is unsuccessful, the modeling result is discarded. If the modeling is successful, the modeling result is stitched into the existing modeling result to obtain the updated modeling result. The result acquisition module is used to obtain the initial modeling result until all the multiple point cloud data have completed point cloud modeling. The judgment module is used to compare the initial modeling results with the preset weld database to determine whether supplementary modeling is required. The supplementary modeling module is used to obtain supplementary point cloud data if it is determined that supplementary modeling is needed, perform point cloud supplementary modeling based on the supplementary point cloud data to obtain the supplementary modeling result, and stitch the supplementary modeling result to the initial modeling result to obtain the final modeling result.

10. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 8.

Images