Deep learning based assembly geometry digital twin modeling and pose calculation method

Abstract

The application discloses an assembly geometry digital twin modeling and pose calculation method based on deep learning, and belongs to the technical field of precision mechanical system modeling. The method comprises the following steps: firstly, a small amount of product key contact surfaces are selected for point cloud measurement; secondly, the data is de-redundant, expanded and sorted by using upsampling / downsampling and data enhancement methods; then, assembly data set is established through assembly pose parameter calculation and data enhancement; finally, a neural network is trained by using deep learning technology, and a geometry distribution error and assembly pose mapping model is established. The application considers part surface geometry distribution error and assembly pose error in the assembly, establishes a geometry digital twin model with complete and accurate error information, provides a model basis for product precision prediction, product assembly process optimization, product assembly visualization and product quality improvement, and is suitable for precision mechanical products with high precision and large batch production requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of precision mechanical system modeling, and in particular to an assembly geometry digital twin modeling and pose calculation method based on deep learning. BACKGROUND

[0002] At present, the existing digital twin modeling method manually establishes and assembles an ideal three-dimensional CAD model to represent the geometric features and assembly relationship of the virtual model, and does not involve non-uniformly distributed surface geometric distribution error and non-ideal part assembly pose, and cannot represent the geometric information of the physical model with high precision, high fidelity and high efficiency. However, in the actual production of precision mechanical systems, the actual geometric distribution error of the parts directly affects the final assembly accuracy in the process of matching contact and assembly transmission, and finally affects the static and dynamic mechanical properties and running functions of the mechanical product in use. At the same time, for mechanical products with complex geometric structure and assembly process, the modeling of geometric distribution error and the calculation of assembly error transmission are difficult, time-consuming and laborious work, and are prone to model rough errors due to human operation errors. However, the geometric distribution error modeling method and the deep learning method can accurately, efficiently and intelligently predict and represent the actual assembly accuracy of the mechanical product. SUMMARY

[0003] The purpose of the present application is to provide an assembly geometry digital twin modeling and pose calculation method based on deep learning, which measures the geometric distribution error data of product samples, models part errors, calculates assembly accuracy, constructs assembly data sets, and trains deep learning neural networks to establish a mapping model of part geometric distribution error data and assembly accuracy parameters, further establish a mechanical product geometry digital twin model, and realize mechanical product assembly accuracy prediction.

[0004] To achieve the above purpose, the present application provides an assembly geometry digital twin modeling and pose calculation method based on deep learning, comprising the following steps:

[0005] S1, point cloud measurement, select a small number of product samples, select the key contact surface of the product assembly, and measure the geometric distribution error to obtain initial point cloud data;

[0006] S2, data processing, according to the point cloud quantity and arrangement requirements of modeling and deep learning, the initial point cloud is up-sampled or down-sampled to obtain processed point cloud data, and the data is enhanced to increase the data quantity;

[0007] S3, construct an assembly data set, select a pair of surface point cloud data to calculate the assembly pose parameters, obtain three contact points and the translation and rotation of the surface point cloud pose variation, and form an assembly data set. Multiple sets of assembly data form an assembly data set;

[0008] S4, train the neural network, use the assembly data set for deep learning neural network training, and the deep learning neural network obtained after training is the geometric distribution error and assembly pose mapping model;

[0009] S5, product measurement, measure the geometric distribution error of the product during production to obtain point cloud data;

[0010] S6, secondary data processing, the same upsampling or downsampling method as step S2 is used to process the point cloud data to obtain processed point cloud data;

[0011] S7, assembly pose error mapping, the processed point cloud data is input into the geometric distribution error and assembly pose mapping model of step S4 as an input quantity to obtain the predicted value of the assembly pose parameters;

[0012] S8, part geometric digital twin modeling, use the point cloud data of step S5 or the processed point cloud data of step S6 to perform three-dimensional geometric error CAD modeling to obtain a three-dimensional error model of the part with geometric distribution error on the surface;

[0013] S9, assembly body geometric digital twin modeling, according to the assembly pose parameters of step S7, the part three-dimensional error model of step S8 is digitally assembled to obtain a geometric digital twin model of the mechanical product assembly body, which contains the geometric distribution error of the part surface and the assembly pose error of the assembly body, and is used to accurately represent the real geometric state of the actual product.

[0014] Preferably, in step S2, for the geometric error topography of a single surface, the error topography of the unmeasured area is predicted through a small amount of point cloud measurement data and data upsampling, and a large amount of point cloud measurement data is used to retain the error topography features and delete redundant point clouds, thereby reducing the data amount and regularizing the point cloud distribution.

[0015] Preferably, in step S2, for the geometric error topography of a large number of products, a small amount of product surface point cloud measurement data is used for data enhancement to predict the geometric error topography features of a large number of other product surfaces.

[0016] Preferably, in step S3, selecting a pair of surface point cloud data to calculate the assembly pose parameters is to calculate the contact point condition of a pair of surface error point clouds and the assembly pose parameters.

[0017] Preferably, in step S4, a deep learning technique is used to train a deep learning neural network with point cloud and corresponding assembly pose parameters to establish a geometric distribution error and assembly pose mapping model. The mapping model adopts an "end-to-end" strategy to quickly calculate the assembly pose parameters.

[0018] Preferably, in steps S5, S6 and S7, a data processing method and a trained deep learning neural network are used to realize fast prediction of product assembly pose parameters from product measurement data.

[0019] Preferably, in steps S8 and S9, the surface point cloud data processed by the data processing method, the predicted assembly pose parameters and computer-aided design modeling techniques are used to establish a high-precision, high-fidelity and high-recovery geometric digital twin assembly model of the actual product assembly.

[0020] Therefore, the assembly geometry digital twin modeling and pose calculation method based on deep learning has the following beneficial effects:

[0021] (1) The present application can obtain a geometric distribution error and assembly pose mapping model by measuring the surface geometric distribution error of a small number of products, processing point cloud data, calculating assembly pose, and training a neural network. Then, in actual production, the surface geometric distribution error of the product is measured, the point cloud data is processed, the assembly pose is predicted, the three-dimensional geometric error is modeled, and the digital assembly is obtained to obtain the geometric digital twin modeling of the mechanical product assembly. And it can predict the assembly error and establish a visual digital model containing geometric distribution error and assembly pose error.

[0022] (2) The present application considers the part surface geometric distribution error and the assembly pose error in the assembly, and establishes a geometric digital twin model with complete and accurate error information. It provides a model basis for product precision prediction, product assembly process optimization, product assembly visualization and product quality improvement.

[0023] (3) The present application adopts a deep learning method in the process of geometric digital twin modeling, which is an end-to-end mapping method. Using the trained deep learning neural network can save the complex intermediate modeling and calculation links, simplify the manual operation, and reduce the technical threshold requirement for production line operators.

[0024] (4) The deep learning network of the present application only needs to be trained once to realize high-accuracy and high-reliability assembly pose parameter prediction. The response time is seconds, and the calculation speed is fast. Except for the part measurement link, automatic modeling can be realized through programming, meeting the use requirements of online prediction of product precision in mass production.

[0025] The technical solutions of the present application will be further described in detail below with reference to the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A modeling flowchart of a kind of assembly geometry digital twin modeling and pose calculation method based on deep learning of the application;

[0027] Figure 2 A circle surface point cloud upsampling schematic diagram of a kind of assembly geometry digital twin modeling and pose calculation method based on deep learning of the application;

[0028] Figure 3 A geometry digital twin model schematic diagram of the circular surface assembly structure of a kind of assembly geometry digital twin modeling and pose calculation method based on deep learning of the application. DETAILED DESCRIPTION

[0029] The technical solutions of the application are further described below by means of the accompanying drawings and examples.

[0030] Unless otherwise defined, technical terms or scientific terms used in the present application shall be understood as their ordinary meaning to those having ordinary skill in the art to which the present application pertains. The terms "first", "second" and similar words used in the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. The terms "connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, which may change accordingly when the absolute position of the described object changes.

[0031] EMBODIMENT

[0032] The purpose of the present application is to provide a kind of mechanical product assembly geometry digital twin modeling method based on deep learning for industrial production.According to the geometry digital twin model established by the method, the actual assembly error of mechanical product can be predicted and characterized concisely, efficiently and accurately. Figure 1 It is the modeling flowchart of the mechanical product assembly geometry digital twin modeling method based on deep learning of the application.The present application provides a kind of geometry digital twin of circular surface assembly structure as an embodiment for description, including the following steps:

[0033] S1, point cloud measurement:

[0034] The geometric distribution error and assembly pose mapping model is constructed. First, the target assembly is analyzed. The circular surface assembly structure has two circular surfaces that are contact mating surfaces. The measurement point distribution scheme is planned. 11 circles are measured, with 25 points per circle, for a total of 275 points. The OGP optical image instrument is selected to measure multiple circular contact surfaces to obtain multiple sets of point cloud XYZ coordinate values.

[0035] S2, data processing:

[0036] To meet the requirements of the point cloud quantity and arrangement of the deep learning neural network training data set, the initial measurement point cloud of each circular surface is upsampled using interpolation to obtain processed point cloud data. Each set of point cloud data has 30 circles, 74 points per circle, and a total of 2220 points, as shown in Figure 2 To improve the accuracy and reliability of the degree learning neural network, the point cloud data set is expanded according to the rules and mathematical calculation methods to obtain more sets of circular surface point clouds.

[0037] S3, construct assembly data set:

[0038] According to the surface matching relationship in the assembly, two circular surface point clouds are extracted from the processed point cloud data set for virtual assembly. The lower part is fixed and the difference surface contact algorithm and assembly error transmission method are used to calculate the assembly pose parameters T i (d x , d y , d z , θ x , θ y , θ z ) of the upper part after assembly, including the translation along the X, Y, Z axes and the rotation around the X, Y, Z axes. In addition to the difference surface method, the minimum potential energy method and the minimum adjacent distance method can also be selected. The assembly error transmission method can be selected from the second transformation matrix method and the Jacobian spinor method. The two sets of circular surface point clouds P 1i and P 2i and the corresponding assembly pose parameters T i of the upper part constitute an assembly data set. Repeatedly selecting different circular surface point clouds and calculating the assembly pose parameters can obtain multiple assembly data sets, i.e. an assembly data set.

[0039] S4, train neural network:

[0040] All circular surface point clouds P 1i and P 2i and all corresponding assembly pose parameters T i of a certain part in the assembly data set are used to train the neural network.x , d y , d z , θ x , θ y or θ z is taken as a single assembly pose parameter training set. A deep learning neural network is trained using the selected single assembly pose parameter training set to establish a single assembly pose parameter mapping model According to the product assembly accuracy requirements, multiple single assembly pose parameter mapping models from point cloud to any part and any degree of freedom can be obtained by training the neural network multiple times. Multiple single assembly pose parameter mapping models used in actual production are integrated into a geometric distribution error and assembly pose mapping model.

[0041] S5, product measurement:

[0042] In actual production, a geometric digital twin model of a circular surface assembly body is constructed. First, the geometric distribution error of all key surfaces of the product is measured to obtain initial point cloud data. The measurement method is the same as that of S1, and the initial point cloud set of the product is obtained.

[0043] S6, secondary data processing:

[0044] The initial point cloud data is processed using the same upsampling or downsampling method as S2 to obtain the processed point cloud data of the two key surfaces of the product.

[0045] S7, assembly pose error mapping:

[0046] According to the production process requirements, the processed point cloud data of the key surfaces is respectively imported into the trained deep learning neural network to obtain the predicted value of the assembly pose parameter of the corresponding part as the output. And output in the form of homogeneous transformation matrix:

[0047]

[0048] Where the translation d x = 0, d y = 0, d z = 2.1655, the rotation sinθ x = 0.0167, sinθ y = -0.0468, sinθ z = 0.

[0049] S8, part geometric digital twin modeling:

[0050] According to different requirements, different modeling techniques are used to model the post-processing point cloud data of S6 in terms of geometric distribution error CAD (Computer-Aided Design). The methods include but are not limited to the following: NURBS (NonUniform Rational B-Spline) surface model can be used to establish a part surface model, and stored in IGES (Initial Graphics Exchange Specification) format as a model file readable by a CAD modeling software system, which can be further combined with a three-dimensional ideal model of the part to establish a three-dimensional error model of the part containing geometric distribution error. The point cloud can also be triangulated and divided to establish a triangular mesh part model based on error point cloud, and stored in STL (Standard Triangle Language) format as a model file readable by a CAD modeling software system. The NURBS modeling method is used in this embodiment to establish a part geometric digital twin model.

[0051] S9, assembly geometric digital twin modeling:

[0052] In the CAD modeling software system, according to the multi-degree-of-freedom assembly pose parameters of each part of S7, the three-dimensional error model of each part of S8 is digitally assembled and pose registered to obtain the geometric digital twin model of the assembly, as shown in Figure 3 The model contains geometric distribution error of the part surface and assembly pose error of the assembly, and can accurately represent the real geometric state of the actual product. Further, the assembly accuracy of the mechanical product, such as concentricity, parallelism, perpendicularity, etc., can be obtained through assembly pose parameter calculation method or CAD modeling software geometric feature data extraction method.

[0053] Therefore, the assembly geometric digital twin modeling and pose calculation method based on deep learning is used to predict the assembly pose parameters of the mechanical product assembly in the actual batch production of the product, and the geometric digital twin model of the product part containing micro-geometric error is established based on the point cloud data. Finally, the mechanical product assembly geometric digital twin model containing geometric shape error and assembly error is established according to the assembly pose parameters and the geometric digital twin model of the part. The present application is suitable for precision mechanical products with high precision and large batch production requirements.

[0054] It should be pointed out finally that the above examples are only used to illustrate the technical solutions of the present application but not to limit it, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can still be modified or replaced equivalently, and these modifications or equivalent replacements should not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A deep learning-based assembly geometry digital twin modeling and pose calculation method, characterized in that, Comprise the following steps: S1, point cloud measurement, select a small amount of product sample, select product assembly key contact surface, and carry out geometric distribution error measurement, obtain initial point cloud data; S2, data processing, according to the point cloud quantity, arrangement requirement of modeling and deep learning, the initial point cloud is up sampled or down sampled, obtain the point cloud data after processing, and the point cloud data set after processing is enhanced, increase the data quantity; S3, construct the assembly data set, select a pair of surface point cloud data to calculate the assembly pose parameters, obtain three contact points and the translation and rotation of the surface point cloud pose variation, form a group of assembly data, select different surface point cloud data multiple times to calculate the corresponding assembly pose parameters, and multiple groups of assembly data form the assembly data set; S4, train the neural network, use the assembly data set to train the deep learning neural network, and the deep learning neural network obtained after training is the geometric distribution error and assembly pose mapping model; S5, production product measurement, carry out geometric distribution error measurement on the product in the production process, and obtain point cloud data; S6, secondary data processing, the point cloud data is processed by using the same up sampling or down sampling method as step S2, and the point cloud data after processing is obtained; S7, assembly pose error mapping, the point cloud data after processing is taken as the input quantity and is input into the geometric distribution error and assembly pose mapping model of step S4, and the predicted value of the assembly pose parameters is obtained; S8, part geometric digital twin modeling, using the point cloud data of step S5 or the point cloud data after processing of step S6, three-dimensional geometric error CAD modeling is carried out, and a three-dimensional error model of the part with surface geometric distribution error is obtained; S9, assembly body geometric digital twin modeling, according to the assembly pose parameters of step S7, the part three-dimensional error model of step S8 is digitally assembled to obtain the geometric digital twin model of the mechanical product assembly body, which contains the geometric distribution error of the part surface and the assembly pose error of the assembly body, and is used for accurately representing the real geometric state of the actual product.

2. The method of claim 1, wherein the method is based on deep learning. In step S2, for the geometric error topography of a single surface, the error topography of the unmeasured area is predicted through a small amount of point cloud measurement data and data up sampling, and the data quantity is reduced and the point cloud distribution is regularized by retaining the error topography characteristics and deleting redundant point clouds through a large amount of point cloud measurement data and down sampling.

3. The method of claim 1, wherein the method is based on deep learning. In step S2, for the geometric error topography of a large number of products, the geometric error topography characteristics of a large number of other product surfaces are predicted through a small amount of product surface point cloud measurement data and data enhancement.

4. The method of claim 1, wherein the method is based on deep learning. In step S3, a pair of surface point cloud data is selected for assembly pose parameter calculation to calculate a pair of surface error point cloud contact point conditions and assembly pose parameter set T (d x , d y , d z , θ x , θ y , θ z ).

5. The method of claim 1, wherein: In step S4, the deep learning technology is used to train the deep learning neural network with the point cloud and the corresponding assembly pose parameters, and the geometric distribution error and assembly pose mapping model is established. The mapping model adopts an "end-to-end" strategy to quickly calculate the assembly pose parameters.

6. The method of claim 1, wherein: In steps S5, S6 and S7, the data processing method and the trained deep learning neural network are used to realize the rapid prediction from the product measurement data to the product assembly pose parameters.

7. The method of claim 1, wherein the method is based on deep learning. In steps S8 and S9, the data-processed surface point cloud data, the predicted assembly pose parameters, and the computer-aided design modeling technology are used to establish a geometric digital twin assembly model of the actual product assembly body with high precision, high fidelity, and high restoration degree.

Images