An assembly state recognition method based on preferred viewing angle

Abstract

In order to solve the problems of existing product assembly state recognition method, such as recognition error, misjudgment and long time calculation, the application provides an assembly state recognition method based on preferred view angle. The application firstly establishes the circumscribed regular polyhedron of each assembly state of product digital model, obtains the point cloud data of digital model under different view angles through each vertex and face center of regular polyhedron, constructs shape vector descriptor of adjacent order sequence number model point cloud under different view angles and carries out cosine distance measurement calculation, and selects the preferred view angle of different adjacent order sequence; then the product point cloud descriptor and the number model point cloud hierarchical projection contour descriptor are constructed, and the local to global registration of product contour and number model contour is carried out; then the sampling view angle is transformed to the preferred view angle and resampling is carried out according to the registration relationship, the key section is judged by analyzing the coincidence degree difference of product point cloud and number model point cloud under the preferred view angle, and finally the assembly state recognition is completed by using the key section coincidence degree.

Description

Technical Field

[0001] This invention belongs to the field of computer-aided assembly, specifically relating to a method for identifying the preferred viewing angle of each adjacent sequence using a product digital model, and then identifying the product assembly status under the preferred viewing angle. Background Technology

[0002] Manual assembly plays a crucial role in the assembly of complex products due to its high flexibility and adaptability. With the rapid development of intelligent manufacturing, the assembly processes of complex products are becoming increasingly difficult, increasing the learning cost for workers to understand assembly progress and operations, thus becoming a major obstacle to improving enterprise production efficiency. Analyzing the assembly situation in real-time and promptly pushing assembly guidance information to operators to guide their work, adopting a human-machine integrated assembly model, is an effective way to solve this problem. The product being assembled is the target object for parts installation and contains process information about the product assembly process. Analyzing the real-time assembly progress by combining the process information of the product being assembled with the real-time assembly progress, and providing assembly guidance information to reduce the learning cost for operators and prevent errors, has positive engineering application significance for improving the efficiency and quality of manual assembly. Currently, several methods for assembly status recognition have emerged.

[0003] Patent document CN113111741A discloses an assembly state recognition method based on three-dimensional feature points. This method extracts feature points based on the local structural features of the assembly, compares the feature points between the digital model and the assembled product, realizes point cloud registration and similarity calculation, and thus completes the product assembly state recognition. However, when recognizing products in adjacent assembly stages, this method may lead to misjudgment of the recognition results due to the high model similarity.

[0004] A patent document with publication number CN114842221A discloses a method for identifying unmarked assembly states based on product depth image point clouds. This method constructs shape vector descriptors from the projected point clouds of a digital model, then acquires depth image point clouds of the assembled products in a real assembly scenario using a depth camera and constructs shape vector descriptors for those same objects. Finally, it calculates the similarity between the two descriptors using the cosine similarity method, and identifies the assembly state based on the similarity result. However, further research reveals that from certain perspectives, the shape vector descriptors of adjacent product point clouds lack sufficient distinguishability, potentially leading to errors in assembly state identification. Furthermore, this method involves a one-to-many model with indiscriminate traversal recognition, resulting in significant computational time consumption. Summary of the Invention

[0005] To address the issues of existing product assembly state recognition methods, which suffer from high similarity in assembly states between adjacent product sequence models, making it difficult to observe differences from certain perspectives and leading to recognition errors and misjudgments, as well as the high computational time required by existing methods, this invention proposes an assembly state recognition method, storage medium, and electronic device based on a preferred perspective.

[0006] The inventive concept of this invention is:

[0007] First, shape vector descriptors are constructed and cosine distance measures are calculated for the neighboring order sequence point clouds under different viewpoints, thereby quantifying the viewpoint recognition level and selecting the preferred viewpoint for different neighboring order sequences.

[0008] Then, a hierarchical projection contour of the in-installation product point cloud and the digital model point cloud is constructed. Local to global point cloud registration is performed based on the point-to-edge Hausdorf distance. Based on this, the spatial mapping relationship between the in-installation product point cloud and the digital model point cloud is analyzed.

[0009] Finally, the sampling points are guided to the preferred viewing point through the mapping relationship, and the assembly status is identified based on the overlap of the local cross-sectional contours of the key assembly parts under the preferred viewing angle.

[0010] The technical solution of this invention is:

[0011] An assembly state recognition method based on a preferred perspective, characterized by comprising the following steps:

[0012] Step 1: For each assembly state of the assembly, establish m coaxial circumscribed regular polyhedra in the digital model, where m ≥ 1;

[0013] Step 2: For each assembled digital model, the local point cloud of the circumscribed regular polyhedron is obtained from the vertices and the center point of each face as the viewpoints.

[0014] Step 3: Construct the adjacent sequence in the assembly process P i P refers to the i-th assembly state of the assembly. i+1 This refers to the (i+1)th assembly state of the assembly, and n refers to the total number of parts in the assembly.

[0015] Step 4: For each adjacent sequence, obtain the set of model differences between the two assembly state digital models involved in it under different perspectives, and take the perspective corresponding to the maximum value of model difference as the preferred perspective of the assembly state involved in it, so as to obtain the preferred perspective of each assembly state.

[0016] Step 5: For each assembly state, an assembly recognition template for the product in assembly is jointly constructed based on the local point cloud of the digital model under the preferred view and the preferred view.

[0017] Step 6: Using any free viewpoint as the initial viewpoint, obtain the point cloud of the assembled product in the current assembly state under this initial viewpoint, and transform it to the world coordinate system;

[0018] Step 7: Construct the adaptive hierarchical projection contour descriptor (LPC) for the point cloud of the assembled product in the current assembly state. R And the LPC (Layered Projection Contour Descriptor) of the digital model point cloud in the current assembly state. Q ;

[0019] Step 8: Perform local registration of each layer of the product outline and the digital model outline to obtain the optimal registration matrix for each layer of outline;

[0020] Step 9: Achieve overall registration based on contour overlap, and select the optimal overall registration matrix from the optimal registration matrix obtained in Step 8 based on the maximum value of the overall matching value:

[0021] Step 10: Obtain the coordinates of the preferred viewpoint corresponding to the current neighboring sequence from the assembly recognition template obtained in Step 5. Using the overall optimal registration matrix obtained in Step 9, transform the coordinates of the preferred viewpoint from the digital model coordinate system to the world coordinate system. Use the coordinates after transformation to the world coordinate system as the resampling viewpoint to resample the point cloud of the assembled product to obtain the point cloud of the assembled product under the preferred viewpoint.

[0022] Step 11: Analyze the overlap of the cross-sectional contours of the product point cloud obtained in Step 10 and the corresponding digital model point cloud searched from the assembly recognition template established in Step 5 under the preferred perspective, so as to determine the key cross-sections, and finally use the overlap of the key cross-sections to complete the recognition of the assembly status.

[0023] Furthermore, step 1 specifically includes:

[0024] 1.1) A circumscribed regular polyhedron is created for the digital model of each assembly state;

[0025] 1.2) Rotate the circumscribed regular polyhedra of each assembly state digital model established in step 1.1) around the same rotation axis by an angle α m-1 times, so that each assembly state digital model has m coaxial circumscribed regular polyhedra.

[0026] Furthermore, the angle α ≤ 360° / m.

[0027] Furthermore, for each neighboring sequence i takes the values ​​1, 2, ..., n respectively. Step 4 is as follows:

[0028] Step 4.1) For a certain viewpoint j, the digital model point cloud Q i Local point cloud Random sample x 2 Once, the Euclidean distance between sampling points in each sampling is calculated to obtain set L. l1 is the Euclidean distance between two sampling points obtained in the first sampling, ..., l x For the xth 2 The Euclidean distance between two sampling points obtained from the second sampling;

[0029] Step 4.2) Represent the distribution of the set L using an x-dimensional isometric histogram;

[0030] Step 4.3) Construct the digital model point cloud Q from the isometric histogram at viewpoint j. i Local point cloud Q i (j) Shape distribution vector SV i ;

[0031] Step 4.4) Perform point cloud analysis on the digital model Q from the same viewpoint j. i+1 Local point cloud Repeat steps 4.1), 4.2), and 4.3) to construct its shape distribution vector SV. i+1 ;

[0032] Step 4.5) Obtain the neighbor sequence under viewpoint j Shape distribution vector SV of digital model i and SV i+1 The included angle is Based on this included angle Calculate neighboring sequences The difference between the two digital models involved, Dif(SV) i ,SV i+1 );

[0033] Step 4.6) Repeat steps 4.1)-4.5) from the remaining perspectives to obtain the neighboring sequence. The model dissimilarity among the digital models involved is determined from various perspectives, with the perspective corresponding to the maximum model dissimilarity being used as the neighboring sequence. The preferred perspective of the two assembly states involved.

[0034] Furthermore, step 7 specifically includes:

[0035] 7.1) Using the positive Z-axis direction as the layering direction, analyze the point cloud R of the assembled product. i Construct a hierarchical projection contour projection point set pc k ={(x,y)|(x,y,z)∈Pc k};

[0036] In the formula:

[0037]

[0038]

[0039] Pc k This represents the k-th segment of the point cloud after adaptive layering.

[0040] h * This represents the maximum height of a point in the point cloud PC, i.e., h. * =z max ;

[0041] h * ×(k-1) / λ represents the starting ordinate of the k-th segment of the layered point cloud, i.e., the minimum ordinate.

[0042] h * ×k / λ represents the terminating ordinate of the k-th segment of the layered point cloud, i.e., the maximum ordinate.

[0043] This represents the floor function;

[0044] 7.2) The convex hull algorithm is used to project the two-dimensional point set pc obtained after layering. k Expressed as outline

[0045]

[0046] In the formula:

[0047] The outline enclosed by a convex bound in a two-dimensional projection;

[0048] p1, p2, ..., p n Represents outline The vertices are arranged counterclockwise;

[0049] Convex() represents the algorithm for solving the convex hull;

[0050] 7.3) Adaptive Layered Projection Contour Descriptor for Product Point Cloud for:

[0051]

[0052] Constructing a hierarchical projection contour descriptor for a digital model point cloud Methods and Construction The method is the same.

[0053] Further, step 8 specifically involves: using a contour registration method based on the Hausdorff distance from point to edge to achieve local registration between the contour of the assembled product and the contour of the digital model, thereby obtaining the optimal registration matrix for each contour layer.

[0054] Furthermore, step 9 specifically includes:

[0055] Each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered contour descriptor will be matched with the optimal registration matrix T1 of the first contour. * The transformation is performed, and the average overlap of the two contours after the transformation is calculated. Then, each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered descriptor are aligned using the optimal registration matrix T2 of the second contour. * Perform the transformation, calculate the average overlap of the two contours after the transformation, ..., and then pass the optimal registration matrix of the λ-th layer for each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered descriptor. Perform the transformation, calculate the average overlap of the two contours after the transformation, and select the optimal registration matrix of the layer corresponding to the maximum average overlap as the overall optimal registration matrix.

[0056] Furthermore, step 11 specifically includes:

[0057] 11.1) Obtaining the general cut-off L of the point cloud of the adjacent-order sequence digital model under the preferred viewpoint - and key cross-section L * As shown below:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] In the formula:

[0064] This represents the mean overlap of the contours in the r-th layer;

[0065] n * This indicates the number of digital model point clouds that need to be matched with the products in use;

[0066] This represents the r-th layer contour of the LPC descriptor of the m-th digital model point cloud under the preferred viewpoint.

[0067] This represents the average overlap of all corresponding contours;

[0068] COVER* Indicates the threshold range for cross-sectional profile overlap;

[0069] η represents the threshold range factor, which takes the value (0, 1);

[0070] 11.2) Sum the non-overlapping areas of each key section to obtain the total non-overlapping value of each state model.

[0071]

[0072]

[0073] In the formula:

[0074] This represents the total non-overlapping value of the key cross-sections between the m-th digital model point cloud and the product point cloud;

[0075] Let r represent the non-overlapping area between the m-th digital model point cloud and the product point cloud at the key section r, where r∈[1,2,…,r]. * ], r * Represents the set {L * The number of key cut-off layers;

[0076] This represents the overlapping contour area of ​​the key cross section at the r-th layer between the digital model point cloud and the product point cloud.

[0077] 11.3) Take the total non-overlapping degree value At the minimum value, the assembly state corresponding to the digital model is taken as the assembly state of the product being assembled.

[0078] The present invention also provides a storage medium storing a computer program; characterized in that: the computer program is executed by a processor to perform the above-described method.

[0079] The present invention also provides an electronic device, including a processor and a storage medium; the storage medium stores a computer program; the special feature is that the computer program is executed by the processor to perform the above-described method.

[0080] The beneficial effects of this invention are:

[0081] 1. This invention first establishes circumscribed regular polyhedra of the product's digital model for each assembly state. Point cloud data of the digital model from different viewpoints is obtained through the vertices and face centers of the regular polyhedra. Shape vector descriptors are constructed for the neighboring sequence digital model point clouds from different viewpoints, and cosine distance is calculated to quantify the viewpoint recognition level and select the preferred viewpoints for different neighboring sequences. Next, a point cloud descriptor for the assembled product and a layered projection contour descriptor for the digital model point cloud are constructed. A contour registration method based on Hausdorff distance from points to edges is used to achieve local-to-global registration between the assembled product contour and the digital model contour. Then, based on the registration relationship, the sampling viewpoint is transformed to the preferred viewpoint and resampled. The difference in the overlap of the cross-sectional contours between the product point cloud and the digital model point cloud from the preferred viewpoint is analyzed to determine the key cross-sections. Finally, the overlap of the key cross-sections is used to complete the identification of the assembly state. Because different viewpoints have different distinguishing abilities for neighboring sequence models, the difference between neighboring sequence models under the preferred viewpoint is high, which can avoid problems such as recognition errors and misjudgments due to high similarity between neighboring sequence models, thus improving the accuracy of product assembly state identification.

[0082] 2. The method of this invention can pre-construct an offline database based on the product digital model, including virtual multi-view digital model point cloud, shape vector descriptor, and preferred view information. In subsequent assembly recognition, it can guide the operator to sample the product at the preferred view, eliminating the need to perform indiscriminate matching and analysis of all virtual view point clouds, greatly reducing the number of parsing operations and improving the speed of product assembly status recognition. Attached Figure Description

[0083] Figure 1 These are schematic diagrams of the four sets of samples selected in specific embodiments of the present invention and their included assembly states.

[0084] Figure 2 This is a schematic diagram of the circumscribed icosahedron of the digital model Product1 in the assembly state P5 according to a specific embodiment of the present invention.

[0085] Figure 3 yes Figure 2 The diagram shows the various viewpoints after the circumscribed regular icosahedron of the digital model has been rotated.

[0086] Figure 4 This is a partial point cloud model diagram of the digital model Product1 from a partial viewpoint in a specific embodiment of the present invention.

[0087] Figure 5 This is a schematic diagram of the various coordinate systems involved in the assembly scene of this invention.

[0088] Figure 6 This is a schematic diagram of the adjacent sequence model of the digital model Product1 in a specific embodiment of the present invention.

[0089] Figure 7 This is a flowchart of the method for calculating the difference between two adjacent sequence models according to the present invention.

[0090] Figure 8 This is a histogram showing the shape distribution of the digital model Product1 from the perspective of five assembly states in a specific embodiment of the present invention.

[0091] Figure 9 It is the difference value of the adjacent sequence model of the digital model Product1 in a specific embodiment of the present invention from a partial perspective.

[0092] Figure 10 This is a point cloud diagram of the sample Product1 in a specific embodiment of the present invention.

[0093] Figure 11 This is a schematic diagram illustrating the process of constructing a layered projection contour descriptor in this invention.

[0094] Figure 12 This is a schematic diagram illustrating the process of calculating the optimal registration matrix of the layered projection contours of the product point cloud and the digital model point cloud in this invention.

[0095] Figure 13 This is a flowchart of the point cloud overall registration method based on contour overlap in this invention.

[0096] Figure 14 This is a schematic diagram of the optimal registration result between the point cloud of the sample Product1P5 and the point cloud of the digital model in a specific embodiment of the present invention.

[0097] Figure 15 This is a schematic diagram of the preferred sampling perspective based on point cloud registration in this invention.

[0098] Figure 16 This is a schematic diagram illustrating the analysis of the preferred viewpoint recognition capability and the initial viewpoint recognition capability in this invention.

[0099] Figure 17 This is a comparison of the assembly state recognition rates of each sample selected in a specific embodiment of the present invention under the preferred viewing angle and the initial viewing angle. Detailed Implementation

[0100] The present invention will be further described in detail below with reference to the accompanying drawings.

[0101] The assembly state recognition method based on the preferred perspective proposed in this invention specifically includes the following steps:

[0102] Step 1: For each assembly state of the assembly, create one or m coaxial circumscribed regular polyhedra in the digital model, where m ≥ 2.

[0103] Step 1.1: Let the assembly be Product = {P1, P2, ..., P}. i ,…,P n}, P i This represents the i-th assembly state of the assembly, where n is the total number of parts in the assembly. There are a total of n assembly states, and a new assembly state is obtained by assembling each part. For each assembly state of the assembly, a circumscribed regular polyhedron is built in the digital model, so that the vertices of the outer contour of the digital model are located at or close to the face of the regular polyhedron. The more complex the digital model, the more faces of the regular polyhedron are built to ensure the accuracy of recognition.

[0104] Step 1.2: Rotate the circumscribed regular polyhedron of each assembly state digital model established in Step 1.1 around the same rotation axis (generally the assembly plane is horizontal, rotating around the Z-axis of the world coordinate system makes the registration of virtual and real point clouds more convenient) by an angle α m-1 times, so that each assembly state digital model has m coaxial circumscribed regular polyhedra; preferably, α = 360° / m, at which time the obtained viewing angle is more uniform, and the partitioning of the projected point cloud obtained from each viewing angle is sufficiently obvious; if it is greater than this angle, there may be repeated viewing angles, and if it is less than this angle, the obtained viewing angle is not uniform.

[0105] Step 2: For each assembled digital model, the local point cloud Q of the circumscribed regular polyhedron is obtained from each viewpoint, using the vertices and the center point of each face as the perspective. i :

[0106] For each assembly state of the digital model, the point cloud of the digital model is sampled from each vertex and the center point of each face of its corresponding m coaxial circumscribed regular polyhedra (a total of s viewpoints), to obtain the local point cloud Q of the digital model point cloud of each assembly state under these different viewpoints. i Q i =(Q i (1) Q i (2) ,…,Q i (j) ,…,Q i (s) ), i takes values ​​of 1, 2, ..., n; s is the total number of viewpoints; Q i The digital model point cloud representing the i-th assembly state; Q i (j) The digital model point cloud Q represents the i-th assembly state. iThe local point cloud is obtained from the j-th viewpoint. The local point clouds obtained from each viewpoint are located in the sampling coordinate system (i.e., the coordinate system with the sampling point as the origin and the direction from the sampling point to the center of the digital model as the Z-axis). In order to register with the point cloud of the product in use, the sampled point cloud needs to be transformed into the digital model coordinate system (i.e., the coordinate system with the center of the digital model as the origin and the vertical direction as the Z-axis) through translation and rotation transformation.

[0107] Step 3: Construct the adjacent sequence in the assembly process

[0108] Constructing neighboring sequences Among them, P i The i-th assembly state of the assembly corresponds to the i-th assembly stage; P i+1 The (i+1)th assembly state of the assembly corresponds to the (i+1)th assembly stage; adjacent sequence. It is the set of digital models of two adjacent assembly stages in the assembly process. An assembly has (n-1) sets of adjacent order sequences, where n is the total number of parts in the assembly.

[0109] Step 4: For each adjacent sequence, obtain the set of model differences between the two assembly state digital models involved in it under different perspectives, and take the perspective corresponding to the maximum value of model difference as the preferred perspective of the assembly state involved in it, so as to obtain the preferred perspective of each assembly state.

[0110] For each adjacent sequence, the preferred perspective of the two assembly states involved is determined according to the following steps:

[0111] Step 4.1: For the digital model point cloud Q under a certain viewpoint j... i Local point cloud Q i (j) Perform random sampling and calculate the Euclidean distance between the sampling points:

[0112] Obtaining neighboring sequences Digital Model Point Cloud Q i Read the local point cloud Q from viewpoint j. i (j) In the local point cloud Q i (j) x 2 The random sampling (the specific number of samplings is selected based on the model complexity; the more complex the model, the more samplings are needed, typically 10242) selects a local point cloud Q in each sampling. i (j) For any two sampling points, calculate the Euclidean distance between the two sampling points in each sampling process to obtain the set L.

[0113] l1 is the Euclidean distance between the two sampling points obtained in the first sampling, ..., For the xth 2 The Euclidean distance between two sampling points obtained from the second sampling;

[0114] Step 4.2: Generate a distance statistics histogram:

[0115] The distribution of set L is represented by an x-dimensional isometric histogram, with the class interval d as shown in the following formula:

[0116]

[0117] In the formula:

[0118] l max This represents the maximum value of the Euclidean distance between the sampling points;

[0119] l min This represents the minimum Euclidean distance between the sampling points;

[0120] The height of each interval in the histogram represents the frequency of sampling distances within that interval. The interval height h is shown in the following formula:

[0121]

[0122] In the formula:

[0123] h i It represents the height of the i-th interval in the histogram, and also represents the frequency of being within that interval;

[0124] n i This represents the statistical frequency of the sampling distance falling within the i-th interval;

[0125] Step 4.3: Construct the digital model point cloud Q from viewpoint j. i Local point cloud Q i (j) Shape distribution vector SV i :

[0126] Construct the x-dimensional shape distribution vector SV from the iso-interval histogram in section 4.2. i The value of each dimension is the interval height h of the corresponding histogram. i As shown in the following formula:

[0127]

[0128] In the formula:

[0129] sv i Represents the shape distribution vector SV i The i-th component;

[0130] Step 4.4: Analyze the point cloud of the digital model for the (i+1)th assembly state from the same viewpoint j. Repeat steps 4.1, 4.2, and 4.3 to construct an x-dimensional shape distribution vector, which is the point cloud of the digital model under viewpoint j. Shape distribution vector SV i+1 ;

[0131] Step 4.5: Obtain the neighboring sequence under viewpoint j. Shape distribution vector SV of digital model i and SV i+1 The included angle is Based on this included angle Calculate neighboring sequences The difference between the two digital models involved, Dif(SV) i ,SV i+1 ):

[0132] Neighboring sequences under viewpoint j Local point cloud of digital model and Let its shape distribution vector SV i and SV i+1 The included angle is Similarity between two point clouds (Sim(SV)) i ,SV i+1 It can be derived from the cosine values ​​of two shape distribution vectors. Calculate according to the following formula:

[0133]

[0134] Since the ability to identify perspectives is to solve for the differences between models, and Sim(SV) i ,SV i+1 This reflects the shape distribution vector SV. i and SV i+1 The similarity between them is used to define the model difference denoted by Dif(SV). i ,SV i+1 This is directly used to measure the ability to recognize viewpoints, and is expressed as follows:

[0135] Dif(SV i ,SV i+1 =arccos[Sim(SV) i ,SV i+1 )]

[0136] Dif(SV i ,SV i+1The larger the value of ), the greater the difference between the neighboring sequence models under the perspective j, and the better the recognition ability of the perspective. Conversely, the smaller the value of ), the smaller the difference between the neighboring sequence models under the perspective j, and the poor the recognition ability of the perspective.

[0137] Step 4.6: From other different perspectives, analyze the adjacent-order sequence digital model. Repeat steps 4.1-4.5 to obtain the neighboring sequence. The model difference Dif values ​​of the digital model under different perspectives were analyzed and sorted by size to obtain the neighbor order sequence. The set of digital model discrepancies (or recognition capabilities) DIF(i,i+1):

[0138] DIF(i,i+1)={Dif min (SV i ,SV i+1 ),…,Dif max (SV i ,SV i+1 )}

[0139] In the formula:

[0140] Dif min (SV i ,SV i+1 ),Dif max (SV i ,SV i+1 ) represent adjacent sequences respectively The minimum and maximum values ​​of the downward-view recognition capability. Due to the maximum difference value Dif... max (SV i ,SV i+1 If the recognition ability is best at the corresponding perspective, then that perspective is a neighboring sequence. The preferred perspective.

[0141] Step 5: Build an assembly identification template for in-process products (IT:)

[0142] For each set of adjacent sequences, the assembly recognition template (IT) of the assembled product is jointly constructed based on the local point cloud of the digital model under the preferred perspective and the preferred perspective.

[0143]

[0144] In the formula:

[0145] IT i i+1 Representing adjacent order sequences Recognition template;

[0146] p vThis indicates the coordinates of the preferred viewpoint of the adjacent sequence in the digital model coordinate system;

[0147] The point cloud Q of the digital model for the i-th assembly state from viewpoint j. i Local point cloud;

[0148] The point cloud Q of the digital model for the (i+1)th assembly state under viewpoint j i+1 Local point cloud.

[0149] Step 6: Obtain the current assembly state of the assembled product point cloud from the initial viewpoint. i :

[0150] A depth camera is used to capture images of the assembled products in a real-world assembly scenario from an initial viewpoint (a free viewpoint, i.e., a randomly selected viewpoint). The coordinates p0 of the initial sampling viewpoint are recorded. The acquired point cloud is then denoised, and then transformed from the camera coordinate system to the world coordinate system through translation and rotation transformations to obtain the point cloud R of the assembled products in the current assembly state. i .

[0151] Step 7: Construct the point cloud of the product being assembled in the current assembly state. i Adaptive layered projection contour (LPC) descriptor, and digital model point cloud Q i Layered projection contour descriptor:

[0152] Since both the digital model coordinate system and the world coordinate system use the assembly horizontal plane as the XOY plane, the point cloud R of the assembled product in the current assembly state... i Q-model point cloud in the current assembly state i Alignment was achieved along the Z-axis, but not along the other two coordinate axes. Therefore, an LPC descriptor needs to be constructed to achieve contour registration in the product point cloud R. i With digital model point cloud Q i Alignment in the X and Y axes. The specific method for constructing an LPC descriptor is as follows:

[0153] Step 7.1: Using the positive Z-axis direction as the layering direction, analyze the point cloud R of the assembled product. i Constructing a layered projection contour descriptor (layered projection point set): First, measure the visual height h of each part. k Count all visualized heights and select the minimum value. When the cross-sectional height is less than It can acquire arbitrary incremental information, therefore the cut-off height is set to... The i-th layer of point cloud representation is as follows:

[0154]

[0155]

[0156] In the formula:

[0157] Pc k This represents the k-th segment of the point cloud after adaptive layering.

[0158] h * Represents point cloud R i The maximum height of a point in the middle, i.e., h * =z max ;

[0159] h * ×(k-1) / λ represents the starting ordinate of the k-th segment of the layered point cloud, i.e., the minimum ordinate.

[0160] h * ×k / λ represents the terminating ordinate of the k-th segment of the layered point cloud, i.e., the maximum ordinate.

[0161] This represents the floor function;

[0162] λ represents the total number of layers.

[0163] The k-th segment of the layered point cloud Pc k By projecting along the negative Z-axis onto the XOY plane, the layered projection point set pc can be obtained. k ={(x,y)|(x,y,z)∈Pc k}

[0164] The two-dimensional point set pc obtained in steps 7.2 and 7.1 k The data contains a large number of points. Considering the influence of factors such as noise and point cloud density, the convex hull algorithm is used to project the resulting two-dimensional point set pc into layers. k Expressed as outline As shown in the following formula:

[0165]

[0166] In the formula:

[0167] The outline enclosed by a convex bound in a two-dimensional projection;

[0168] p1, p2, ..., p n Represents outline The vertices are arranged counterclockwise;

[0169] Convex() represents the algorithm for solving the convex hull.

[0170] Step 7.3 The final result is a layered projected profile. The sequence (k takes the values ​​1, 2, ..., λ) forms the point cloud R of the packaged product. i Layered Projected Contour Descriptor (LPC Descriptor):

[0171]

[0172] Step 7.4 Based on the historical assembly state identification, infer the current theoretical assembly state of the product being assembled, and establish a digital model Q for this theoretical assembly state. i Perform steps 7.1-7.3 to obtain the layered projection contour descriptor of the digital model for this assembly state.

[0173] Step 8: Achieve local registration of the in-assembled product contour with the digital model contour using the Hausdorff distance method based on points to edges, and obtain the optimal registration matrix for each contour layer:

[0174] Step 8.1: Using the point cloud of the installed product R i Layered projection profile of any one of the layers For the profile to be registered, use the digital model Q i Layered projection contour For the target contour, arbitrarily select two vertices. and As the registration reference points, calculate the translation matrix between vertices u and v; then extract the counterclockwise neighboring points u0 and v0 of vertices u and v, and calculate the registration edges. and The rotation matrix is ​​obtained; finally, the registration matrix T with vertices u and v as references is obtained based on the translation and rotation matrices. u·v Through the registration matrix T u·v The profile to be registered can be Register to target contour

[0175] Step 8.2: Calculate the registered contour. With target contour Hausdorff distance from point to edge As shown below:

[0176]

[0177]

[0178] In the formula:

[0179] e represents the target outline The edges in the convex hull profile are composed of adjacent vertices.

[0180] v i v represents a point in the outline. i+1 This represents a point adjacent to the given point, and the two points form an edge, e.

[0181] u' represents the projection of vertex u onto the line containing edge e;

[0182] |u,e| represents the distance from vertex u to edge e, and can take the following three values:

[0183] (1) When the projection u' of vertex u onto edge e is within the range of the edge, then |u,e| is the perpendicular distance from vertex u to edge e, i.e.

[0184] (2) When the projection u' of vertex u onto edge e is not within the range of the edge, and vertex u and v i Closer distance, i.e., |u,v i |<|u,v i+1 Then |u,e| equals |u,v i |;

[0185] (3) When the projection u' of vertex u onto edge e is not within the range of the edge, and u and v i+1 Closer distance, i.e., |u,v i+1 |<|u,v i Then |u,e| equals |u,v i+1 |;

[0186] Step 8.3: Traverse the contours to be registered and target outline For all vertices, select one point from each pair as a registration reference point, repeating steps 8.1 and 8.2. Obtain the Hausdorff distance (VHD) from each registered point to the edge. Select the smallest VHD to represent the optimal matching of the two contours. The registration matrix at this point is the optimal registration matrix for that contour layer.

[0187] Step 8.4: Traverse the point cloud R of the product being assembled in the current assembly state. i Layered projection contour descriptor and digital model point cloud Q i Layered contour descriptor For each contour layer, repeat steps 8.1-8.3 to obtain the optimal registration matrix for each contour layer.

[0188] Step 9: Achieve overall registration based on contour overlap, and select the overall optimal registration matrix from the optimal registration matrix obtained in Step 8 based on the overall matching value:

[0189] Step 9.1: For the point cloud R of the product currently being assembled in the assembly state... i Layered projection contour descriptor and digital model point cloud Q i Layered contour descriptor A certain layer of outline and Calculate the optimal registration matrix corresponding to the contour of this layer. The degree of overlap between the two contours after transformation

[0190]

[0191] In the formula:

[0192] This indicates the degree of overlap between the areas of the two contours after registration; the larger the value, the better the matching degree between the two contours.

[0193] Representing the outline and The area;

[0194] S cover Represents outline and The area of ​​the overlapping region.

[0195] Step 9.2: Transfer the point cloud of the product currently being assembled to the current assembly state. i Layered projection contour descriptor Each contour in the model is obtained through the optimal registration matrix of the k-th layer. After transformation, the registered contour is All registration profiles With digital model descriptors Outline The average overlap between the two layers is the optimal registration matrix at the k-th layer. The overall matching value of the two points of cloud below

[0196]

[0197] Step 9.3: Traverse all registration matrices The method in step 9.2 is used to calculate the overall matching value of the two point clouds under each registration matrix, and the maximum overall matching value is obtained by sorting.

[0198] When the overall matching value reaches its maximum, the registration matrix corresponding to this maximum value is the overall optimal registration matrix T. match .

[0199] In other words, each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered descriptor are matched using the optimal registration matrix T1 of the first contour. * The transformation is performed, and the average overlap of the two contours after the transformation is calculated. Then, each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered descriptor are aligned using the optimal registration matrix T2 of the second contour. * Perform the transformation, calculate the average overlap of the two contours after the transformation, ..., and then pass the optimal registration matrix of the λ-th layer for each contour in the product point cloud layered contour descriptor and each contour in the digital model point cloud layered descriptor. Perform the transformation, calculate the average overlap of the two contours after the transformation, and select the optimal registration matrix of the layer corresponding to the maximum average overlap as the overall optimal registration matrix.

[0200] Step 10: Based on the registration relationship, transform the sampling viewpoint to the preferred viewpoint and resample:

[0201] Step 10.1: The current assembly state and the previous assembly state form an adjacent sequence. Assembly identification template Obtain the coordinates p of the preferred viewpoint corresponding to the adjacent order sequence. v The overall optimal registration matrix T obtained through step 9 match The coordinates of the preferred viewpoint p v The coordinates in the world coordinate system after transforming from the digital model coordinate system to the world coordinate system are p. r , as the resampling perspective point.

[0202] Step 10.2: Based on the initial sampling viewpoint p0 and the resampling viewpoint p r Establish perspective transformation relationship The depth camera is guided from the initial sampling position to the preferred sampling position and resampled to obtain the point cloud of the installed product from the preferred viewpoint.

[0203] Step 11: Analyze the overlap difference of the cross-sectional contours between the point cloud of the assembled product and the point cloud of the digital model from the preferred perspective to determine the key cross-sections, and finally use the overlap of the key cross-sections to complete the identification of the assembly status:

[0204] Step 11.1: Under the preferred perspective, the point clouds of adjacent-order sequence digital models are highly similar in the non-newly assembled parts. The point cloud differences are mainly reflected in the newly assembled parts. The adaptive layering method makes the newly assembled parts appear in a few cross-sectional contours. Therefore, the cross-sectional contour overlap threshold range COVER is used. * The cut-off elements of the LPC descriptor are divided into two categories: general cut-off LPC descriptors and general cut-off LPC descriptors. -and key cross-section L * As shown below:

[0205]

[0206]

[0207]

[0208]

[0209]

[0210] In the formula:

[0211] This represents the mean overlap of the contours in the r-th layer;

[0212] n * This indicates the number of digital model point clouds that need to be matched with the products in use;

[0213] This represents the r-th layer contour of the LPC descriptor of the m-th digital model point cloud under the preferred viewpoint.

[0214] This represents the average overlap of all corresponding contours;

[0215] COVER * Indicates the threshold range for cross-sectional profile overlap;

[0216] η represents the threshold range factor, which depends on the actual situation, and is generally taken as (0, 1).

[0217] Step 11.2, at the critical section L * At this point, the product in the installation phase has a high degree of overlap with the target state model, meaning the non-overlapping area is small. The non-overlapping areas of each key section are summed to obtain the total non-overlapping value of each state model. The target state model corresponds to the minimum non-coincident total value:

[0218]

[0219]

[0220] In the formula:

[0221] This represents the total non-overlapping value of the key cross-sections between the m-th digital model point cloud and the product point cloud;

[0222] Let r represent the non-overlapping area between the m-th digital model point cloud and the product point cloud at the key section r, where r∈[1,2,…,r]. *], r * Represents the set {L * The number of key cut-off layers;

[0223] This represents the overlapping contour area of ​​the key cross section at the r-th layer between the digital model point cloud and the product point cloud.

[0224] When the total non-overlap degree is taken When the minimum value is obtained, the assembly state corresponding to the digital model is the assembly state of the product being assembled, which means that the assembly state identification of the product being assembled from the optimal perspective has been completed.

[0225] Example:

[0226] The following is an example Figure 1 Taking Product1 from the four sample groups shown as an example, Product1 has a total of five assembly states. The assembly state identification method based on the preferred viewpoint proposed in this invention will be explained below. The specific steps are as follows:

[0227] Step 1: Create three coaxial circumscribed regular icosahedrons for the digital model of each assembly state of the assembly:

[0228] Step 1.1: For each assembly state of the assembly, create a circumscribed icosahedron for the digital model, ensuring that the outer contour points of the digital model are located at or near the faces of the icosahedron. For example, for assembly Product1, the circumscribed icosahedron of its 5th assembly state P5 is as follows: Figure 2 As shown.

[0229] Step 1.2: Rotate the circumscribed icosahedron of the digital model established in Step 1.1 twice around the Z-axis of the world coordinate system, so that each assembled digital model corresponds to three coaxial circumscribed icosahedrons. An icosahedron has 20 faces and 12 vertices. Since two vertices on the Z-axis coincide, one rotation yields a new set of 20 faces and 10 vertices. The diagrams of the various views after rotation are shown below, using the vertices and the center point of each face of the icosahedron as the perspective. Figure 3 As shown.

[0230] Step 2: Obtain the local point cloud Q of the digital model of each assembly state of the assembly from multiple perspectives. i :

[0231] For each assembly state of the digital model, the point cloud of the digital model is sampled from each vertex and the center point of each face of its corresponding three coaxial circumscribed regular icosahedrons (a total of 92 views), and the local point cloud Q of the digital model of each assembly state under these 92 views is obtained. i Q i =(Qi (1) Q i (2) ,…,Q i (j) ,…,Q i (92) ), i takes values ​​of 1, 2, ..., 5; Q i The digital model point cloud representing the i-th assembly state; Q i (j) This represents the local point cloud of the digital model point cloud at the i-th assembly state, obtained from the j-th viewpoint. The local point cloud model of the digital model of Product1 at the 5th assembly state P5, obtained from a partial viewpoint, is shown below. Figure 4 As shown.

[0232] The point cloud obtained through sampling is located in the sampling coordinate system (i.e., a coordinate system with the sampling point as the origin and the direction from the sampling point to the center of the digital model as the Z-axis). For subsequent registration with the point cloud of the product being assembled, the sampled point cloud is transformed to the digital model coordinate system (a coordinate system with the center of the digital model as the origin and the vertical direction as the Z-axis) through translation and rotation transformations. Schematic diagrams of various coordinate systems in the assembly scenario are shown below. Figure 5 As shown.

[0233] Step 3: Construct the adjacent sequence in the assembly process

[0234] by Figure 1 Taking the assembly Product1 as an example, construct the adjacent sequence. Where P i This refers to the i-th assembly state of assembly Product1, corresponding to the i-th assembly stage; P i+1 This refers to the (i+1)th assembly state of assembly Product1, corresponding to the (i+1)th assembly stage. Adjacent sequence. That is Figure 1 The set of digital models of two adjacent assembly stages during the assembly process of Product1 in the middle assembly, with a total of 4 sets of adjacent-order sequences, such as Figure 6 As shown.

[0235] Step 4: Calculate the neighboring sequences respectively (i takes the values ​​1, 2, 3, and 4 respectively) to determine the set of model differences between two adjacent assembly state digital models under different perspectives, thereby determining the preferred perspective for each assembly state:

[0236] Step 4.1: Select a set of adjacent sequences for assembly Product1. The digital model point clouds involved are Q4 and Q5. The model difference between point clouds Q4 and Q5 under the same viewpoint j is calculated (refer to...). Figure 7 (as shown);

[0237] 4.1.1 Local point cloud of digital model point cloud Q4 under viewpoint j Random sampling, calculate the Euclidean distance between sampling points:

[0238] Obtaining neighboring sequences Digital model point cloud Q4, reading local point cloud from viewpoint j Local point cloud 1024 2 Each random sampling selects a point cloud. For any two sampling points, calculate the Euclidean distance between the two sampling points in each sampling process to obtain the set L. l1 is the Euclidean distance between the two sampling points obtained in the first sampling, ..., For the 1024th 2 The Euclidean distance between two sampling points obtained from the second sampling;

[0239] 4.1.2. Generate a distance statistics histogram:

[0240] The distribution of set L is represented by a 1024-dimensional isometric histogram, and the class interval d is calculated according to the following formula:

[0241]

[0242] In the formula:

[0243] l max This represents the maximum value of the Euclidean distance between the sampling points;

[0244] l min This represents the maximum value of the Euclidean distance between the sampling points.

[0245] The height of each interval in the histogram represents the frequency of sampling distances within that interval. The interval height h is calculated using the following formula:

[0246]

[0247] In the formula:

[0248] h i It represents the height of the i-th interval in the histogram, and also represents the frequency of the sampling distance falling within that interval;

[0249] n i This represents the statistical frequency of the sampling distance falling within the i-th interval;

[0250] The shape distribution histograms for the five assembly states of Product1 are shown below. Figure 8 As shown.

[0251] 4.1.3 Constructing a local point cloud of the digital model point cloud Q4 from viewpoint j Shape distribution vector SV4:

[0252] A 1024-dimensional shape distribution vector SV4 is constructed from the isometric histogram obtained in step 4.1.2. The value of each dimension of the shape distribution vector SV4 corresponds to the interval height h of the histogram. i As shown in the following formula:

[0253]

[0254] In the formula:

[0255] sv i This represents the i-th component of the shape distribution vector SV4.

[0256] 4.1.4 Constructing a local point cloud of the digital model point cloud Q5 from viewpoint j Shape distribution vector SV5:

[0257] Local point cloud of the adjacent-order sequence digital model point cloud Q5 under the same viewpoint j Repeat steps 4.1.1-4.1.3 to construct a 1024-dimensional shape distribution vector SV5;

[0258] 4.1.5 Obtaining the neighboring sequence under viewpoint j The angle between the shape distribution vectors SV4 and SV5 of the digital model is used to calculate the difference Dif(SV4,SV5) between adjacent sequence digital models:

[0259] Neighboring sequences under viewpoint j Local point cloud of digital model and Let the angle between its shape distribution vectors SV4 and SV5 be . The similarity Sim(SV4,SV5) between two point clouds can be expressed as the cosine values ​​of the two shape distribution vectors. Calculate according to the following formula:

[0260]

[0261] Since the ability to recognize viewpoints is to solve for the differences between models, and Sim reflects the similarity between shape distribution vectors, the model difference metric Dif is defined to directly measure the ability to recognize viewpoints, as follows:

[0262] Dif(SV4,SV5)=arccos[Sim(SV4,SV5)]

[0263] The larger the Dif, the larger the neighboring sequence under that perspective j. The greater the difference in the digital model, the better the recognition ability of the perspective j; conversely, the smaller the difference, the worse the recognition ability of the perspective j.

[0264] Step 4.2: From other different perspectives, analyze this group of adjacent sequences. Repeat step 4.1 in the digital model to obtain the neighboring sequence. The model dissimilarity (Dif) of the digital model under each of the other perspectives was calculated and sorted by size to obtain the neighbor sequence. The set of differences in the digital model, DIF(4,5), is shown in the following formula:

[0265] DIF(4,5) = {Dif min (SV4,SV5),…,Dif max (SV4,SV5)}

[0266] In the formula:

[0267] DIF(4,5) represents the neighboring sequence. A collection of recognition capabilities across all perspectives;

[0268] Dif min (SV4,SV5),Dif max (SV4, SV5) represent neighboring sequences, respectively. The minimum and maximum values ​​of the downward-view recognition capability. Due to the maximum difference value Dif... max If the recognition capability is best at the perspective corresponding to (SV4,SV5), then this perspective is the preferred perspective for the current neighboring sequence.

[0269] Step 4.3: Repeat steps 4.1 and 4.2 for all other neighboring sequences of the assembly to obtain the model difference sets of all other neighboring sequences from each viewpoint. Sort these sets by size to obtain the optimal viewpoints for all neighboring sequences. The Dif values ​​of the difference in some viewpoints for all neighboring sequences of assembly Product1 are shown below. Figure 9 As shown.

[0270] Step 5: Construct an assembly identification template for the product being assembled:

[0271] For each set of adjacent sequences, the assembly recognition template (IT) of the assembled product is jointly constructed based on the local point cloud of the digital model under the preferred perspective and the preferred perspective.

[0272]

[0273] In the formula:

[0274] IT i i+1 Representing adjacent order sequences The recognition template, i takes the values ​​1, 2, 3, and 4 respectively;

[0275] p v This indicates the coordinates of the preferred viewpoint in the digital model coordinate system for the adjacent order sequence.

[0276] Step 6: Obtain the current assembly state of the assembled product point cloud from the initial viewpoint. i :

[0277] A depth camera is used to capture images of products being assembled in a real-world assembly scenario from a free-viewpoint perspective. The initial sampling viewpoint coordinates p0 are recorded. The acquired point cloud is then denoised, and translation and rotation transformations are used to convert the point cloud from the camera coordinate system to the world coordinate system, resulting in the current assembly state point cloud R of the products being assembled. i The point cloud of the assembled product in Product1 is as follows: Figure 10 As shown.

[0278] Step 7: Construct the in-assembled product adaptive layered projection contour (LPC) descriptor:

[0279] Since both the digital model coordinate system and the world coordinate system use the assembly horizontal plane as the XOY plane, the product point cloud R... i With digital model point cloud Q i Alignment was achieved in the Z-axis direction, but not in the other two coordinate directions. Therefore, an LPC descriptor was constructed to achieve alignment between the dotted cloud and the digital model point cloud in the X and Y axes by contour registration.

[0280] Step 7.1: Using the positive Z-axis direction as the layering direction, analyze the point cloud R of the assembled product. i To construct a hierarchical projection contour descriptor, the visual height h of each part is first measured. k Calculate the minimum value for all visualized heights. When the cross-sectional height is less than It can acquire arbitrary incremental information and set the cut-off height to... The i-th layer of point cloud representation is as follows:

[0281]

[0282]

[0283] In the formula:

[0284] Pc k This represents the k-th segment of the point cloud after adaptive layering.

[0285] h * Represents point cloud R i The maximum height of a point in the middle, i.e., h * =zmax ;

[0286] h * ×(k-1) / λ represents the starting ordinate of the k-th segment of the layered point cloud, i.e., the minimum ordinate;

[0287] h * ×k / λ represents the terminating ordinate of the k-th segment of the layered point cloud, i.e., the maximum ordinate.

[0288] This represents the function for rounding up.

[0289] The k-th segment of the layered point cloud Pc k By projecting along the negative Z-axis onto the XOY plane, the layered projection pc can be obtained. k ={(x,y)|(x,y,z)∈Pc k}

[0290] The two-dimensional point set pc obtained in steps 7.2 and 7.1 k The data contains a large number of points. Considering the influence of factors such as noise and point cloud density, the convex hull algorithm is used to project the resulting two-dimensional point set pc into layers. k Expressed as outline As shown in the following formula:

[0291]

[0292] In the formula:

[0293] The outline enclosed by a convex bound in a two-dimensional projection;

[0294] p1, p2, ..., p n Represents outline The vertices are arranged counterclockwise;

[0295] Convex() represents the algorithm for solving the convex hull.

[0296] Finally, the point cloud of the installed product was obtained. i Layered projection contour The sequence forms the hierarchical projective contour descriptor (LPC descriptor) of the model:

[0297]

[0298] A schematic diagram of the construction process of the layered projection contour descriptor is shown below. Figure 11 As shown.

[0299] Step 8: Construct a layered projection contour descriptor for the digital model point cloud, and perform local registration between the in-assembled product contour and the digital model contour based on the Hausdorff distance from points to edges, obtaining the optimal registration matrix for each contour layer:

[0300] Step 8.1: Based on the historical assembly status confirmation, deduce the theoretical assembly status of the product being assembled, and establish a digital model Q for this theoretical assembly status. i Perform steps 7.1 and 7.2 to obtain the layered projection contour descriptor of the theoretical assembly state digital model.

[0301] Step 8.2, with The outline to be registered. For the target contour, arbitrarily select two vertices. and As the registration reference point, calculate the translation matrix between u and v, then extract the counterclockwise nearest points u0 and v0 of u and v, and calculate the registration edge. and The rotation matrix is ​​obtained, and finally, the registration matrix T with u and v as references is obtained based on the translation and rotation matrices. u·v Through the registration matrix T u·v The profile to be registered can be Register to target contour

[0302] Step 8.3: Calculate the registered contour. With target contour Hausdorff distance from point to edge As shown below:

[0303] in

[0304]

[0305] In the formula:

[0306] e represents the target outline The edges in the convex hull profile are composed of adjacent vertices.

[0307] v i v represents a point in the outline. i+1 This represents a point adjacent to the given point, and the two points form an edge, e.

[0308] u' represents the projection of vertex u onto the line containing edge e;

[0309] |u,e| represents the distance from vertex u to edge e, and can take the following three values:

[0310] (1) When the projection u' of vertex u onto edge e is within the range of the edge, then |u,e| is the perpendicular distance from u to e, i.e.

[0311] (2) When the projection u' of vertex u onto edge e is not within the range of the edge, and u and v i Closer distance, i.e., |u,v i |<|u,v i+1 Then |u,e| equals |u,v i |;

[0312] (3) When the projection u' of vertex u onto edge e is not within the range of the edge, and u and v i+1 Closer distance, i.e., |u,v i+1 |<|u,v i Then |u,e| equals |u,v i+1 |;

[0313] Step 8.4: Traverse the contours to be registered and target outline For all vertices, select one point from each pair as a registration reference point, repeating steps 8.2 and 8.3. Obtain the Hausdorff distance (VHD) from each registered point to the edge. Select the smallest VHD to represent the optimal matching of the two contours. The registration matrix at this point is the optimal registration matrix for that contour layer. Calculate the profile and outline Optimal registration matrix The process diagram is as follows Figure 12 As shown.

[0314] Step 8.5: Traverse the point cloud descriptors of the assembled product. and digital model point cloud descriptors For each contour layer, repeat steps 8.2-8.4 to obtain the optimal registration matrix for each contour layer.

[0315] Step 9: Achieve overall registration based on contour overlap, as shown in the schematic diagram below. Figure 13 As shown:

[0316] Step 9.1, for and A certain layer of outline and Calculate the optimal registration matrix after this layer The degree of overlap between the two contours after transformation As shown in the following formula:

[0317]

[0318] In the formula:

[0319] This indicates the degree of overlap between the areas of the two contours after registration; the larger the value, the better the matching degree between the two contours.

[0320] Represents outline and The area;

[0321] S cover Represents outline and The area of ​​the overlapping region.

[0322] Step 9.2, in the product descriptor Each contour in the model is obtained through the optimal registration matrix of the k-th layer. Perform the transformation on all registered contours Contours in the digital model descriptor The average overlap between them is the value in the registration matrix. The overall matching value of the two points of cloud below

[0323] Step 9.3: Traverse all registration matrices Calculate the overall matching value of the two point clouds under each registration matrix, and obtain the largest overall matching value by sorting them by size.

[0324]

[0325] When the overall matching value reaches its maximum, the registration matrix corresponding to this maximum value is the overall optimal registration matrix T. match The optimal registration result is as follows: Figure 14 As shown.

[0326] Step 10: Based on the registration relationship, transform the sampling viewpoint to the preferred viewpoint and resample. A schematic diagram of the process is shown below. Figure 15 As shown:

[0327] Step 10.1: The current assembly state and the previous assembly state form an adjacent sequence. In the identification template Obtain the coordinates p of the preferred viewpoint corresponding to the adjacent order sequence. v Through the overall optimal registration matrix T match Transform the coordinates of this point from the digital model coordinate system to the world coordinate system, where the coordinates are p. r , as resampling points.

[0328] Step 10.2: Based on the initial sampling viewpoint p0 and the resampling viewpoint p r Establish perspective transformation relationship The depth camera is guided from the initial sampling position to the preferred sampling position and resampled to obtain the point cloud of the installed product from the preferred viewpoint.

[0329] Step 11: Analyze the overlap difference of the cross-sectional contours between the product point cloud and the digital model point cloud from the preferred perspective to determine the key cross-sections. Finally, the assembly status is identified using the overlap of the key cross-sections.

[0330] Step 11.1: Under the preferred perspective, the point clouds of adjacent sequence digital models are highly similar in the non-newly assembled parts. The point cloud differences are mainly reflected in the newly assembled parts. The adaptive layering method makes the newly assembled parts appear in a few cross-sectional contours. The threshold range factor η = 0.2 is taken, and the cross-sectional contour overlap threshold range COVER is used. * The cut-off elements of the LPC descriptor are divided into two categories: general cut-off LPC descriptors and general cut-off LPC descriptors. - and key cross-section L * As shown below:

[0331]

[0332]

[0333]

[0334]

[0335]

[0336] In the formula:

[0337] This represents the mean overlap of the contours in the r-th layer;

[0338] n * This indicates the number of digital model point clouds that need to be matched with the products in use;

[0339] This represents the r-th layer contour of the LPC descriptor of the m-th digital model point cloud under the preferred viewpoint.

[0340] This represents the average overlap of all corresponding contours;

[0341] COVER * Indicates the threshold range for cross-sectional profile overlap;

[0342] η represents the threshold range factor, which is taken as η = 0.2 here.

[0343] Step 11.2, at the critical section L * At this point, the product in the installation phase has a high degree of overlap with the target state model, meaning the non-overlapping area is small. The non-overlapping areas of each key section are summed to obtain the total non-overlapping value of each state model. The target state model corresponds to the minimum non-coincident total value:

[0344]

[0345]

[0346] In the formula:

[0347] This represents the total non-overlapping value of the key cross-section between the m-th digital model point cloud and the product point cloud;

[0348] Let r represent the non-overlapping area between the m-th digital model point cloud and the product point cloud at the key section r, where r∈[1,2,…,r]. * ], r * Represents the set {L * The number of key cut-off layers;

[0349] This represents the overlapping contour area of ​​the key cross section at the r-th layer between the digital model point cloud and the product point cloud.

[0350] When the total non-overlap degree is taken When the minimum value is obtained, the assembly state corresponding to the digital model is the assembly state of the product being assembled, which means that the assembly state identification of the product being assembled from the optimal perspective has been completed.

[0351] Simulation verification:

[0352] To demonstrate the significance of the perspective optimization recognition process in this invention, we first analyze the recognition process without perspective optimization, such as... Figure 16 As shown in the left figure, the point cloud of the assembled product has significantly low similarity to the point clouds of the digital models in assembly stages 1, 2, and 3, but high similarity to assembly stages 4 and 5. However, the similarity between assembly stages 4 and 5 is very close, indicating that this perspective cannot effectively distinguish between assembly stages 4 and 5. To address the limitations of the initial perspective's distinguishing ability, a recognition process analysis based on perspective optimization is conducted, such as... Figure 16 As shown in the right figure, compared to the initial viewpoint, the similarity of the point clouds of the digital models in assembly stages 4 and 5 differs significantly under the preferred viewpoint. This indicates that the preferred viewpoint has a better ability to distinguish neighboring models than the initial viewpoint, meaning that the preferred viewpoint plays a positive role in the recognition process. The assembly state recognition results of the four groups of samples under the initial viewpoint and the preferred viewpoint are shown below. Figure 17 As shown, the results indicate that the assembly status recognition method based on the preferred perspective proposed in this invention can effectively improve the accuracy of status recognition of assembled products.

Claims

1. A preferred-view-based assembly state recognition method, characterized by, Including the following steps: Step 1: For each assembly state of the assembly, a digital model of a coaxial circumscribed regular polyhedron is created, ; Step 2: For each assembled digital model, the local point cloud of the circumscribed regular polyhedron is obtained from the vertices and the center point of each face as the viewpoints. Step 3: Construct the adjacent sequence in the assembly process , , The first assembly Assembly status. The first assembly Assembly status. The total number of parts in the assembly; Step 4: For each adjacent sequence, obtain the set of model differences between the two assembly state digital models involved in it under different perspectives, and take the perspective corresponding to the maximum value of model difference as the preferred perspective of the assembly state involved in it, so as to obtain the preferred perspective of each assembly state. For each neighbor sequence , 1, 2, …, , step 4 is specifically: Step 4.1) for a certain view angle The digital model point cloud The local point cloud Random sampling , calculate the Euclidean distance between the sampling points in each sampling, and get the set , , is the Euclidean distance between the two sampling points obtained by the first sampling, …, is the Euclidean distance between the two sampling points obtained by the first sampling; and is the Euclidean distance between the two sampling points obtained by the first sampling. Step 4.2) Representing the distribution of the set of with an equidistant histogram of dimension x. Step 4.3) constructing a view from said equidistant histogram a digital model point cloud a local point cloud a shape distribution vector ; Step 4.4) repeating steps 4.1), 4.2) and 4.3) for the same view angle under the digital model point cloud of the local point cloud Step 4.4) repeating steps 4.1), 4.2) and 4.3) for the same view angle ; Step 4.5) Acquiring a viewing angle Lower neighbor sequence Shape distribution vector of the digital model And The angle between Based on the angle Calculate the neighbor sequence The difference between the two digital models involved ; Step 4.6) Repeat steps 4.1)-4.5) under the rest of the view angles to get the neighbor sequence The model difference degrees between the digital models involved under each view angle, and the view angle corresponding to the maximum model difference degree is taken as the neighbor sequence The preferred view angle of the two assembly states involved Step 5: For each assembly state, an assembly recognition template for the product in assembly is jointly constructed based on the local point cloud of the digital model under the preferred view and the preferred view. Step 6: Using any free viewpoint as the initial viewpoint, obtain the point cloud of the assembled product in the current assembly state under this initial viewpoint, and transform it to the world coordinate system; Step 7: Constructing the point cloud adaptive layered projection contour descriptor at the assembly product point cloud in the current assembly state , and the digital model point cloud layered projection contour descriptor in the current assembly state ; Step 8: Perform local registration of each layer of the product outline and the digital model outline to obtain the optimal registration matrix for each layer of outline; Step 9: Achieve overall registration based on contour overlap, and select the optimal overall registration matrix from the optimal registration matrix obtained in Step 8 based on the maximum value of the overall matching value: Step 10: Obtain the coordinates of the preferred viewpoint corresponding to the current neighboring sequence from the assembly recognition template obtained in Step 5. Using the overall optimal registration matrix obtained in Step 9, transform the coordinates of the preferred viewpoint from the digital model coordinate system to the world coordinate system. Use the coordinates after transformation to the world coordinate system as the resampling viewpoint to resample the point cloud of the assembled product to obtain the point cloud of the assembled product under the preferred viewpoint. Step 11: Analyze the overlap of the cross-sectional contours of the product point cloud obtained in Step 10 and the corresponding digital model point cloud searched from the assembly recognition template established in Step 5 under the preferred perspective, so as to determine the key cross-sections, and finally use the overlap of the key cross-sections to complete the recognition of the assembly status.

2. The preferred view-based assembly status recognition method according to claim 1, characterized in that: Step 1 is as follows: Step 1.1) Create a circumscribed regular polyhedron for each assembly state of the digital model; Step 1.2) Rotate the circumscribed regular polyhedra of each assembly state digital model established in Step 1.1) around the same rotation axis by an angle α m-1 times, so that each assembly state digital model has m coaxial circumscribed regular polyhedra.

3. The preferred view-based assembly status recognition method according to claim 2, characterized in that: The angle α ≤ 360° / m.

4. The assembly state recognition method based on a preferred viewpoint according to any one of claims 1-3, characterized in that: Step 7 specifically includes: Step 7.1) Projecting the product point cloud to the layering direction Axis positive direction as layering direction to the product point cloud at the packing point Constructing the layering projection profile projection point set ； Step 7.2) The 2D point set obtained after the layered projection is processed using a convex hull algorithm Expressed as a contour : In the formula: a contour representing a convex hull of the two-dimensional projection; representing the contour of the vertex, arranged in counterclockwise order; convex hull solving algorithm; Step 7.3) In-product point cloud adaptive layering projection contour descriptor is: ； Method for constructing a digital model point cloud layered projection contour descriptor The method for constructing is the same.

5. The assembly state recognition method based on preferred viewpoint according to claim 4, characterized in that: Step 8 specifically involves: using the Hausdorff distance-based contour registration method to achieve local registration between the contour of the assembled product and the contour of the digital model, thereby obtaining the optimal registration matrix for each contour layer.

6. The assembly state recognition method based on preferred viewpoint according to claim 5, characterized in that: Step 9 specifically includes: Each layer profile in the product-in-place point cloud layered profile descriptor and each layer profile in the digital model point cloud layered profile descriptor are transformed by the optimal registration matrix of the first layer profile transforming, calculating the average of the coincidence of the two profiles after transformation, and selecting the optimal registration matrix of the layer corresponding to the maximum average of the coincidence as the optimal registration matrix of the whole transforming, calculating the average of the coincidence of the two profiles after transformation, and selecting the optimal registration matrix of the layer corresponding to the maximum average of the coincidence as the optimal registration matrix of the whole transforming, calculating the average of the coincidence of the two profiles after transformation, and selecting the optimal registration matrix of the layer corresponding to the maximum average of the coincidence as the optimal registration matrix of the whole transforming, calculating the average of the coincidence of the two profiles after transformation, and selecting the optimal registration matrix of the layer corresponding to the maximum average of the coincidence as the optimal registration matrix of the whole 7. A storage medium on which a computer program is stored; characterized in that: The computer program is executed by the processor to perform the method described in any one of claims 1-6.

8. An electronic device, comprising a processor and a storage medium; wherein a computer program is stored on the storage medium; characterized in that: The computer program is executed by the processor to perform the method according to any one of claims 1-6.

Images