A toilet adhesive effect detection system using three-dimensional point cloud recognition method

Abstract

The invention discloses a toilet adhesion effect detection system using a three-dimensional point cloud recognition method, comprising: a structured light scanning system constructed by the invention, completing point cloud scanning of a toilet adhesion model, detecting whether the upper surface parallelism of the adhesion model and the adhesion overlap between the seat ring and the embryo surface meet the product quality requirements through the scanned point cloud image, using a KD-tree algorithm to establish a topological relationship between point cloud data, performing denoising and streamlining of point cloud data by direct filtering combined with grid sampling, and then completing the preprocessing operation of the point cloud data. Subsequently, the SHOT feature descriptor is used to locally describe the extracted feature points in the point cloud, and then the corresponding relationship is searched, and a three-dimensional Hough voting mechanism is introduced to screen and cluster the initial corresponding relationship, thereby completing the target recognition in a complex scene. After completing the recognition of the point cloud data, the adhesion effect of the model can be detected. Finally, after completing the detection of the adhesion effect. According to the invention, it is used to identify qualified, repairable and unqualified products, and to try to save raw materials, reduce costs and improve efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of machinery, and in particular to a toilet pan bonding effect detection system using a three-dimensional point cloud recognition method. Background Art

[0002] Offshore wind power has developed rapidly in recent years due to its advantages such as abundant resources, stable wind speed, less negative impact on the environment, and no occupation of land resources. At present, the pile legs of large-power offshore wind power facilities are large in size, volume, and weight. When they are installed, they need to use lifting equipment mounted on the installation platform, which puts higher requirements on the carrying capacity of the installation platform, resulting in an increase in the quality of the installation platform, and the need to build a stronger support structure.

[0003] At present, offshore wind power installation and piling are basically completed by two types of ships: self-elevating crane platforms and floating crane vessels. Among them, the self-elevating platform is equipped with a crane and 4 to 8 pile legs. Its working process is as follows: after arriving at the site, the pile legs are inserted into the seabed to support and fix the barge. The barge can be completely or partially lifted out of the water through the hydraulic lifting device to form a stable platform that is not affected by waves. In this way, the crane ship can maintain a stable posture when working. The traditional pile column consists of a pile leg and a pile shoe. The huge pile shoe is deeply inserted into the seabed and can provide sufficient support reaction force to support its own weight and various loads generated by the hanging pile column, playing a role in stabilizing the entire installation platform. Because the pile shoe has a huge cross-section (about 100 square meters), its disadvantage is obvious, that is, a lot of time is consumed in the process of inserting and removing the pile, which greatly reduces the work efficiency. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a toilet adhesive effect detection system using a three-dimensional point cloud recognition method, which is used to identify qualified, repairable and unqualified products, and to save raw materials, reduce costs and improve efficiency as much as possible. In order to achieve the above-mentioned purpose and other advantages according to the present invention, a toilet adhesive effect detection system using a three-dimensional point cloud recognition method is provided, comprising:

[0005] A structured light system scanning module, wherein the structured light system scanning module is used to scan the bonded model after the mechanical arm clamps the seat ring and completes the bonding of the seat ring and the toilet embryo, obtain a point cloud distribution map of the bonding model, and pre-process the point cloud data;

[0006] A point cloud data recognition module, which is used to recognize point cloud data and detect the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the embryo body surface;

[0007] A product quality assessment module is used to identify qualified, repairable and unqualified products.

[0008] Preferably, the toilet is formed by bonding the cup body and the seat ring in blocks, and after bonding is completed, the bonding model is scanned by a structured light camera to obtain point cloud data.

[0009] Preferably, it also includes a preprocessing module, which is used to preprocess the collected point cloud data before point cloud target recognition. The preprocessing module includes establishing a topological relationship between point cloud data, filtering and noise reduction, and streamlining processing.

[0010] A toilet bowl bonding effect detection method using a novel three-dimensional point cloud recognition method comprises the following steps:

[0011] S1. Build a light scanning system, scan the upper surface and side of the bonded model through a structured light camera and point cloud data acquisition software, and display the point cloud data acquisition results in the software;

[0012] S2, preprocessing the point cloud data in step S1;

[0013] S3, search for neighboring points of key data points in multidimensional space through KD-Tree algorithm and establish the topological relationship of point cloud set;

[0014] S4. Before recognition, the pre-acquired point cloud data must be filtered and denoised;

[0015] S5. Streamline the point cloud data through the grid sampling algorithm, while reducing the amount of data and retaining the outline and features of the point cloud as much as possible.

[0016] Preferably, after completing the preprocessing of the point cloud data, the point cloud features of the bonded model are identified, and the point cloud feature identification includes feature point extraction, local description of the point cloud for the feature points extracted from the scene point cloud and the template point cloud by the SHOT feature descriptor, and then searching for the corresponding relationship, and finally introducing the three-dimensional Hough voting mechanism to screen and cluster the initial corresponding relationship, thereby completing the target recognition in the complex scene.

[0017] Preferably, after the point cloud data is identified, it is necessary to detect whether the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the blank surface meet the product quality requirements based on the point cloud image of the bonding model.

[0018] Preferably, after completing the point cloud recognition and the detection of the bonding model, an evaluation model is established according to the parallelism ε of the seat ring surface after bonding and the overlap δ between the seat ring and the surface of the blank, and different weights k are assigned to the two parameters according to the actual processing requirements. 1 and k 2 , output result evaluation index k 1 ε+k 2 δ.

[0019] Compared with the prior art, the present invention has the following beneficial effects: a structured light system scanning unit is established, and after the mechanical arm clamps the seat ring to complete the bonding of the seat ring and the toilet embryo, the bonded model is scanned by the structured light system to obtain the point cloud distribution map of the upper surface and side of the bonding model, and pre-processing is performed. Then point cloud data recognition is performed, which mainly includes the recognition of the point cloud data obtained in the previous step, and by detecting the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the embryo surface. Finally, a product quality evaluation unit is established, which is used to identify qualified, repairable and unqualified products after completing the detection of the bonding effect, so as to save raw materials, reduce costs and improve efficiency as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of a structured light scanning system of a toilet adhesive effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0021] Figure 2 A schematic diagram of a three-dimensional point cloud KD-tree of a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0022] Figure 3 A through-filter flow chart of a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0023] Figure 4 A grid sampling principle diagram of a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0024] Figure 5 A flow chart for identifying point cloud features of a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0025] Figure 6 It is a schematic diagram of the SHOT descriptor principle of the toilet adhesion effect detection system using the three-dimensional point cloud recognition method according to the present invention;

[0026] Figure 7 A schematic diagram of SHOT feature space division of a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0027] Figure 8 A schematic diagram of a three-dimensional target recognition algorithm based on Hough voting in a toilet adhesion effect detection system using a three-dimensional point cloud recognition method according to the present invention;

[0028] Fig. 9Schematic diagrams of two common unqualified bonding models of a toilet bonding effect detection system using a three-dimensional point cloud recognition method according to the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Reference Figure 1-9 , a toilet pan bonding effect detection system using a three-dimensional point cloud recognition method, comprising:

[0031] A structured light system scanning module, wherein the structured light system scanning module is used to scan the bonded model after the mechanical arm clamps the seat ring and completes the bonding of the seat ring and the toilet embryo, obtain a point cloud distribution map of the bonding model, and pre-process the point cloud data;

[0032] A point cloud data recognition module, which is used to recognize point cloud data and detect the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the embryo body surface;

[0033] A product quality assessment module is used to identify qualified, repairable and unqualified products.

[0034] Furthermore, the toilet is formed by bonding the cup body and the seat ring in blocks, and after bonding is completed, the bonding model is scanned by a structured light camera to obtain point cloud data.

[0035] Furthermore, it also includes a preprocessing module, which is used to preprocess the collected point cloud data before point cloud target recognition. The preprocessing module includes establishing a topological relationship between point cloud data, filtering and denoising, and streamlining processing.

[0036] A toilet bowl bonding effect detection method using a novel three-dimensional point cloud recognition method comprises the following steps:

[0037] S1. Build a light scanning system, scan the upper surface and side of the bonded model through a structured light camera and point cloud data acquisition software, and display the point cloud data acquisition results in the software;

[0038] S2, preprocessing the point cloud data in step S1;

[0039] S3. Use the KD-Tree algorithm to search for neighboring points of key data points in multidimensional space and establish the topological relationship of the point cloud set. When applying the algorithm, select the dimension with the largest corresponding variance in the point cloud data set, and then divide the point cloud data on this dimension using the median as the basis for splitting;

[0040] S4. Before recognition, the pre-acquired point cloud data must be filtered and denoised by defining a threshold range on a specified coordinate axis and finding point cloud data within and outside the threshold on the axis, and then choosing whether to filter;

[0041] S5. Streamline the point cloud data through the grid sampling algorithm, while reducing the amount of data and retaining the outline and features of the point cloud as much as possible.

[0042] Furthermore, after completing the preprocessing of the point cloud data, the point cloud features of the bonded model are identified. The point cloud feature identification includes feature point extraction, local description of the feature points extracted from the scene point cloud and the template point cloud through the SHOT feature descriptor, and then searching for corresponding relationships. Finally, a three-dimensional Hough voting mechanism is introduced to screen and cluster the initial corresponding relationships, thereby completing target recognition in complex scenes.

[0043] Furthermore, after the extraction of feature points is completed, it is necessary to establish a feature descriptor, which is a key step in point cloud recognition. Currently, there are many types of point cloud descriptors, mainly two types of description-based methods (Signature) and histogram statistics-based methods (Histogram). The SHOT feature descriptor used in the present invention is a combination of the above two methods.

[0044] Furthermore, the SHOT descriptor is established in the local coordinate system within the spherical support domain of each feature point. First, a local coordinate reference system needs to be defined. The process of establishing the local coordinate reference system is mainly divided into three steps: constructing the covariance matrix, solving the eigenvalues ​​and eigenvectors, and eliminating the symbol ambiguity.

[0045] Furthermore, space division is performed, and a local coordinate system is established in the R radius sphere neighborhood at the feature point F, and the sphere neighborhood is divided into spatial grids along the three directions of diameter, azimuth, and elevation. It can be divided into 2 radial areas, 8 azimuth areas, and 2 elevation areas, a total of 32 areas.

[0046] Furthermore, each region is represented by a local histogram, each histogram is divided into 11 small units, and the cosine value of the angle between the normal vector of each point falling into the region and the feature point vector is calculated for each subspace region.

[0047] Furthermore, in each subspace, the cosine value of the angle is divided into 11 equal parts as the horizontal axis, and the vertical axis is the number of corresponding points. The feature histogram of each subspace is then combined and normalized to obtain a 32×11=352-dimensional histogram feature.

[0048] Furthermore, feature point matching and corresponding relationship establishment are performed, and the present invention determines the matching relationship by calculating the Euclidean distance between two feature vectors in the model and the scene.

[0049] Furthermore, considering that the correspondence relationship established in (2.8) only relies on feature vectors for feature matching, this may lead to incorrect matching relationships and reduce recognition accuracy. Therefore, a three-dimensional object recognition method based on Hough voting is designed, which is mainly divided into two stages: offline training and online scene recognition.

[0050] The offline training phase mainly involves establishing a model database, extracting model key points, and establishing model feature descriptions.

[0051] In the online scene recognition stage, the scene data is first simplified and key points are extracted. Then, the feature vector is obtained for each key point and matched one by one with the feature vectors trained in the offline stage to obtain an initial correspondence set. The Hough voting mechanism is then used to determine the instances of the objects to be identified in the scene, and finally the target objects are identified.

[0052] After the target object is identified, the bonding effect between the toilet body and the seat ring can be checked through the identified model point cloud. The main inspection factors are the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the body surface.

[0053] Furthermore, after the point cloud data is identified, it is necessary to detect the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the blank surface based on the point cloud image of the bonding model to see whether they meet the product quality requirements.

[0054] Furthermore, after completing the point cloud recognition and the detection of the bonding model, an evaluation model is established based on the parallelism ε of the seat ring surface after bonding and the overlap δ between the seat ring and the blank surface, and different weights k are assigned to the two parameters according to the actual processing requirements. 1 and k 2 , output result evaluation index k 1 ε+k 2 δ.

[0055] Example 1

[0056] See also Figure 1First, perform point cloud scanning on the bonding model and display the point cloud data acquisition results in the software. Then perform preprocessing on the acquired point cloud data. The specific steps are as follows.

[0057] Step 1: Establish the topological relationship between point cloud data. The point cloud data obtained by the structured light camera are mostly discrete points, and the distribution is extremely uneven. There is no correlation between the points, which makes the subsequent point cloud recognition extremely difficult. Therefore, before point cloud recognition, the topological relationship of the obtained point cloud must be established. The present invention uses the KD-tree algorithm to establish the topological relationship between point cloud data.

[0058] Step 1.1: Refer to Figure 2 ,First, the dimension k with the largest variance in the three-dimensional point cloud dataset is selected, and then the median is selected as the splitting point on this dimension to divide the point cloud data to obtain two subsets, and the tree nodes are created and used for storage.

[0059] Step 1.2: Repeat the operation in step 1.1 for each subset obtained until the subset can no longer be divided, thus completing the construction of the KD-tree.

[0060] Step 2: Perform point cloud filtering and denoising. Please refer to the process of direct filtering. Figure 3 First, filter the noise points on the Z axis and set the filtering range to (μ a ,μ b ), if the target point cloud is within this range, it will be retained, otherwise the target point cloud will be deleted, and then the point cloud will be grid sampled. The specific process is as follows.

[0061] Step 2.1: Please refer to Figure 4 , determine the side length of the bounding box. Find the maximum value of the point cloud coordinates (x max ,y max ,z max ) and the minimum value (x min ,y min ,z min ) as the two points of the diagonal of the bounding box, the lengths of the sides of the bounding box can be expressed as:

[0062]

[0063] Step 2.2: Please refer to Figure 4 , and (x min ,y min ,z min ) as the origin to construct a new coordinate system, the direction of the coordinate system is the same as the direction of the original point cloud data. Set the side length of the voxel to d i, ensure the consistency of discretization of scene point cloud and target point cloud, and complete the construction of voxel space.

[0064] Step 2.3: Please refer to Figure 4 , create a small cube sampling grid in voxel space based on integer coordinates, assuming that the total number of point clouds after straight-through filtering is N, and the maximum bounding box volume V = L x L y L z , the number of small cube sampling grids S = V / d i 3 , then the number of point clouds contained in each cubic sampling grid is n = N / S = Nd i 3 / V.

[0065] Step 2.4: Please refer to Figure 4 , calculate the centroid of each small cube, then use the centroid point to represent each cube, and merge the centroid points of all small cubes to form a constituent point cloud set, and finally obtain the voxel point cloud of the original point cloud set. The formula for solving the centroid is as follows:

[0066]

[0067] Step 3: Simplify the point cloud data. The process of raster sampling the point cloud data in step 2 is also a simplification process. In step 2, the centroid of the cube is used to replace the original point cloud, but this method is generally suitable for denser point clouds. For sparser point clouds, this method will cause point drift. In this case, the point closest to the centroid can be selected to replace the original point cloud. At this time, in any cube, the coordinates of the replacement point (p ix ,p iy ,p iz ) need to meet: At this point, the preprocessing of point cloud data is completed.

[0068] Point cloud data recognition module

[0069] After completing the preprocessing of the point cloud data, it is necessary to identify the point cloud features of the bonding model. Figure 5 According to the process in , point cloud features are recognized. The specific steps are as follows.

[0070] Step 4: First, extract the feature points. Through the grid sampling process in step 2, select the centroid of each cube as the feature point.

[0071] Step 5: Create SHOT feature descriptors. Creating feature descriptors is a key step in the point cloud recognition process. A good descriptor should be stable, resistant to interference, and be able to describe the geometric features of key points well. The SHOT feature descriptor is a combination of the description method and the histogram statistics method. Please refer to the schematic diagram of its basic principle. Figure 6 . First, find a certain point in the area as the key point, and establish a local coordinate system at this point; then establish a feature space at this point and divide this space along different directions to obtain several subspace areas; for the divided subspace areas, calculate the cosine value of the angle between the normal vector of the point falling into the area and the normal vector of the key point, and perform a histogram statistics on the number of points in each subspace area, thus completing the construction of the SHOT descriptor. The specific algorithm flow is as follows:

[0072] Step 5.1: Establish a local coordinate system. Establish a covariance matrix based on the selected key points and their neighborhoods, obtain eigenvalues ​​and mutually perpendicular eigenvectors through eigenvalue decomposition, sort the eigenvalues ​​from large to small, and the eigenvectors corresponding to the three sorted eigenvalues ​​represent the x, y, and z directions of the coordinate system respectively.

[0073] Step 5.1.1: According to step 4, select the extracted feature point (i.e., the cube centroid point) as the key point. Construct the covariance matrix based on the feature point F and its surrounding domain points. Considering that the point cloud distribution in different cube regions is not the same and has no regularity, the distance weight is added when constructing the covariance matrix, that is, the points close to the feature point contribute more to the covariance matrix construction, and the points far from the feature point contribute less to the covariance matrix construction. The final constructed covariance matrix M is as follows:

[0074]

[0075] Among them, R is the neighborhood radius, p i is the coordinate vector of a point in the neighborhood, d i is the Euclidean distance from a point in the neighborhood to the centroid.

[0076] Step 5.1.2: Solve the eigenvalues ​​and corresponding eigenvectors of the matrix M. The solution formula is as follows:

[0077] M.v i =λ i ·v i , i∈{1,2,3}

[0078] Among them, λ i is the eigenvalue of the matrix M, v i is the eigenvector of matrix M. Assume λ 1 >λ 2 >λ 3 , then λ3 The corresponding eigenvector v 3 It is considered to be the normal vector n of the feature point F F .

[0079] Step 5.1.3: Eliminate symbol ambiguity. Since the eigenvalue decomposition method does not clearly define the sign of the eigenvector, in order to achieve the uniqueness of the local coordinate system, the directions of the three coordinate axes need to be redefined. Use the disambiguated singular value decomposition (Disambiguated SVD) method to eliminate symbol ambiguity. Taking the x-axis as an example, the specific method is as follows:

[0080]

[0081] Among them, x + represents the eigenvector obtained in step 5.1.2 along the direction of x, x - With x + In opposite direction. It means that "the distance from a point in the neighborhood to the centroid is less than the neighborhood radius, and the vector The direction of x + The number of points that meet the condition of "same direction". Similarly, It means that "the distance from a point in the neighborhood to the centroid is less than the neighborhood radius, and the vector The number of points that satisfy the condition that the direction of the When there are many points in the condition, the direction of the x-axis is x + direction, otherwise x - direction.

[0082] Similarly, the direction of the z-axis is determined according to the above method. The direction of the y-axis is obtained by y=z×x.

[0083] Step 5.2: Divide the space. Please refer to Figure 7 , a spherical neighborhood space is established with the feature point F as the center and R as the radius, and the neighborhood space is divided into spatial grids along the diameter, azimuth and elevation directions. Finally, it can be divided into 2 radial areas, 8 azimuth areas, and 2 elevation areas, a total of 32 areas.

[0084] Step 5.3: Calculate the histogram. After dividing into subspace regions, calculate the normal vector n of the points falling into each subspace region. i and the feature point normal vector n F The cosine of the angle, i.e. cosθ = n F ·n i Next, divide the cosθ values ​​into 11 parts in each subspace and count the number of points that fall into the corresponding cosθ range. Figure 7, that is, the horizontal axis of the histogram is cosθ, and the vertical axis is the number. The feature histograms of each subspace are combined and normalized, and finally a 32*11=352-dimensional histogram feature is obtained.

[0085] One issue that needs to be noted for step 5.3 is that in order to overcome the boundary effect caused by combining individual histograms together or under the perturbation of the local coordinate system, when each point is accumulated to a specific interval of the histogram, it is necessary to perform quadrilinear interpolation on adjacent cells in the local histogram and the same cell in the adjacent histogram.

[0086] Step 5.4: Match feature points and establish corresponding relationships. The matching of feature points is mainly based on the similarity of feature descriptors to determine the corresponding relationship between the model and the scene. In the present invention, the matching relationship is determined by calculating the Euclidean distance between two feature vectors in the model and the scene. The specific algorithm is as follows:

[0087] Step 5.4.1: Assume that a feature point h in the scene i The feature descriptor is f i h ={a 1 ,a 2 ,a 3 ,…,a 352}, a feature point g in the model i The feature descriptor is f i g = {b 1 ,b 2 ,b 3 ,…,b 352}.

[0088] Step 5.4.2: Using the obtained SHOT feature descriptor, calculate h i With g i The Euclidean distance between them is as follows:

[0089]

[0090] Step 5.4.3: Set the threshold σ, if D(h i ,g i )<σ, then it is considered that there is a corresponding relationship between the scene feature points and the model feature points. On the contrary, if D(h i ,g i )>σ, then discard the model feature point. Traverse all feature points in the scene, repeat steps 5.4.1 to 5.4.3, and establish a corresponding point set for all feature points that meet the corresponding relationship.

[0091] Step 5.5: This is an improved method designed for step 5.4. By designing a three-dimensional Hough space voting mechanism, the target can be identified more accurately. The basic principle of this method is to use the correspondence between the model point cloud and the current scene point cloud, and associate each scene feature point with the relative position of the model centroid through the local coordinate system, so that each corresponding scene feature can vote in the three-dimensional Hough space, and finally enough votes are used to confirm the actual location of the workpiece in the three-dimensional space. The specific algorithm flow of three-dimensional target recognition based on Hough voting is as follows:

[0092] Step 5.5.1: Please refer to Figure 8 , assuming that all point cloud coordinates of the model are based on the model global coordinate system {B M}, the centroids of the model point cloud and the scene point cloud are C M ,C S , the feature point in the model is F i M , the feature point in the scene is F j S , establish a local reference coordinate system for the model feature points as {L M}, the vector composed of each feature point and the centroid in the model point cloud is recorded as the voting vector

[0093] Step 5.5.2: Please refer to Figure 8 , so that the voting vector In any reference coordinate system, it has the property of rotation and translation invariance. From the model global coordinate system {B S}Transform to the model local coordinate system {L M}, the conversion process is expressed as follows:

[0094] in From the model global coordinate system {B M}Transform to the model local coordinate system {L M}, each row of which consists of feature points F i M The local reference frame unit vector is composed of Therefore, each model feature point F i M The voting vector Connected together.

[0095] Step 5.5.3: Please refer to Figure 8 , and then for each scene feature point F j S , establish its relationship with the model feature point F iM Due to the rotation and translation invariance of the local reference system, it can be considered that in the scene local coordinate system {L S The voting vector under The voting vector corresponding to the one in the model are equal.

[0096] Step 5.5.4: Please refer to Figure 8 , and finally Transform to the scene global coordinate system {B S}, the conversion process is expressed as follows:

[0097] in From the scene local coordinate system {L S}Transform to the scene global coordinate system {B S}, each column of which consists of feature points F j S The local reference frame unit vector is composed of

[0098] Voting is performed based on these transformation relationships. Each vote determines a centroid position. The actual position of the target centroid can be determined based on the voting peak. The voting counting process is repeated until all feature points have voted. Finally, the category and position of the scene target can be determined based on the voting peak and the centroid position. In addition, after Hough voting, incorrect matching correspondences are eliminated, and the model can be aligned with the scene target through translation and rotation based on the correct correspondence subset, thereby completing target recognition.

[0099] After the above steps, the recognition of the bonding model point cloud data is completed. For common incorrect bonding effects, please refer to Fig. 9 Through the identified point cloud image, the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the blank surface are detected to see if they meet the processing requirements of the product.

[0100] Product quality assessment module

[0101] After completing the point cloud recognition and the detection of the bonding model, an evaluation model is established based on the parallelism ε (ε∈[0,1]) of the seat ring surface after bonding and the overlap δ (δ∈[0,1]) between the seat ring and the blank surface, and different weights k are assigned to the two parameters according to the actual processing requirements. 1 (k 1 =3) and k 2 (k 2 =7), output result evaluation index k 1 ε+k 2δ. The evaluated products are mainly divided into the following levels:

[0102]

[0103] According to the obtained product quality grade, match the preset evaluation statement and output the corresponding evaluation information.

[0104] The number of devices and processing scales described here are used to simplify the description of the present invention, and the application, modification and variation of the present invention are obvious to those skilled in the art. Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation mode, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, other modifications can be easily realized, so without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the legends shown and described here.

Claims

1. A toilet pan bonding effect detection system using a three-dimensional point cloud recognition method, characterized in that: include: A structured light system scanning module, wherein the structured light system scanning module is used to scan the bonded model after the mechanical arm clamps the seat ring and completes the bonding of the seat ring and the toilet embryo, obtain a point cloud distribution map of the bonding model, and pre-process the point cloud data; Preprocessing module, the preprocessing module is used to preprocess the collected point cloud data before point cloud target recognition, the preprocessing module includes establishing topological relationship between point cloud data, filtering and denoising and streamlining processing; preprocessing the point cloud data specifically searches for neighboring points of key data points in multidimensional space through KD-Tree algorithm, establishes the topological relationship of the point cloud set, and when applying the algorithm, selects the dimension with the largest corresponding variance in the point cloud data set, and then divides the point cloud data on this dimension based on the median; before recognition, the pre-acquired point cloud data must be filtered and denoised, by defining a threshold range on a specified coordinate axis, and finding point cloud data within and outside the threshold on the axis, and then choosing whether to filter; the point cloud data is streamlined through the grid sampling algorithm, while reducing the amount of data, retaining the outline and features of the point cloud as much as possible; The toilet bowl bonding effect detection method is as follows: S1. Build a light scanning system, scan the upper surface and side of the bonded model through a structured light camera and point cloud data acquisition software, and display the point cloud data acquisition results in the software; S2, preprocessing the point cloud data in step S1; S3, search for neighboring points of key data points in multidimensional space through KD-Tree algorithm and establish the topological relationship of point cloud set; S4. Before recognition, the pre-acquired point cloud data must be filtered and denoised; S5. Streamlining point cloud data through grid sampling algorithm, while reducing the amount of data, retaining the outline and features of the point cloud as much as possible; Point cloud data recognition module, the point cloud data recognition module is used to recognize point cloud data and detect the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the embryo body surface; the specific process is as follows: After completing the preprocessing of the point cloud data, the point cloud features of the bonded model are identified. The point cloud feature identification includes feature point extraction, local description of the feature points extracted from the scene point cloud and the template point cloud through the SHOT feature descriptor, and then the corresponding relationship is searched, and finally the three-dimensional Hough voting mechanism is introduced to screen and cluster the initial corresponding relationship, thereby completing the target recognition in the complex scene; after completing the extraction of the feature points, it is necessary to establish a feature descriptor. The use of the SHOT feature descriptor is a combination of the description-based method and the histogram statistics-based method; the SHOT descriptor is established in the local coordinate system within the spherical support domain of each feature point; first, a local coordinate reference system needs to be defined. The process of establishing the local coordinate reference system is mainly divided into three steps: constructing the covariance matrix, solving the eigenvalues ​​and eigenvectors, and eliminating the symbol ambiguity; performing space division, at the feature points Where A local coordinate system is established in the radius sphere neighborhood, and the sphere neighborhood is divided into spatial grids along three directions: diameter, azimuth, and elevation. It is divided into 2 radial areas, 8 azimuth areas, and 2 elevation areas, a total of 32 areas. Each area is represented by a local histogram, and each histogram is divided into 11 small units. For each subspace area, the cosine value of the angle between the normal vector of each point in the area and the feature point vector is calculated. In each subspace, the cosine value of the angle is divided into 11 parts as the horizontal axis, and the vertical axis is the number of corresponding points. The feature histograms of each subspace are combined and normalized to obtain a 32×11=352-dimensional histogram feature. The feature points are matched and the corresponding relationship is established. The matching relationship is determined by calculating the Euclidean distance between the two feature vectors in the model and the scene. A three-dimensional target recognition method based on Hough voting is designed, which is divided into two stages: offline training and online scene recognition. The offline training stage is mainly to establish a model database, extract model key points, and establish a model feature description. In the online scene recognition stage, the scene data is first simplified and key points are extracted. Then, the feature vector is obtained for each key point and matched one by one with the feature vectors trained in the offline stage to obtain the initial correspondence set. The Hough voting mechanism is then used to determine the instance of the object to be identified in the scene, and finally the target object is identified. After the target object is identified, the bonding effect between the toilet embryo and the seat ring can be checked through the identified model point cloud. The detection factors are the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the embryo surface. After the point cloud data is identified, it is necessary to check the parallelism of the upper surface of the bonding model and the bonding overlap between the seat ring and the blank surface based on the point cloud image of the bonding model to see if they meet the product quality requirements. After completing the point cloud recognition and bonding model detection, according to the parallelism of the seat ring surface after bonding The overlap between the seat ring and the embryo surface Establish an evaluation model and assign different weights to the two parameters according to actual processing requirements and , output result evaluation index ; The process of point cloud data recognition mold body is as follows: After completing the preprocessing of the point cloud data, it is necessary to identify the point cloud features of the bonding model. First, the feature points are extracted. Through the grid sampling process, the centroid point of each cube is selected as the feature point. Establish SHOT feature descriptor; Establish Feature descriptor is a key step in the point cloud recognition process. The SHOT feature descriptor is a combination of the description method and the histogram statistics method. First, a certain point is found in the area as the key point, and a local coordinate system is established at the point. Then, a feature space is established at the point and the space is divided in different directions to obtain several subspace areas. For the divided subspace area, the cosine value of the angle between the normal vector of the point falling into the area and the normal vector of the key point is calculated, and the number of points in each subspace area is counted by histogram, thus completing the construction of the SHOT descriptor. The specific algorithm flow is as follows: Step 5.1: Establish a local coordinate system; establish a covariance matrix based on the selected key points and their neighborhoods, obtain eigenvalues ​​and eigenvectors perpendicular to each other through eigenvalue decomposition, sort the eigenvalues ​​from large to small, and the eigenvectors corresponding to the three eigenvalues ​​after sorting represent the coordinate system direction; Step 5.1.1: According to step S4, select the extracted feature point, i.e., the centroid of the cube, as the key point; according to the feature point The covariance matrix is ​​constructed with the surrounding domain points. Considering that the distribution of point clouds in different cubic areas is different and irregular, the distance weight is added when constructing the covariance matrix, that is, the points close to the feature points contribute more to the construction of the covariance matrix, and the points far from the feature points contribute less to the construction of the covariance matrix; the covariance matrix finally constructed is as follows: ； in, is the neighborhood radius, is the coordinate vector of a point in the neighborhood, is the Euclidean distance from a point in the neighborhood to the centroid; Step 5.1.2: Solve the matrix The eigenvalues ​​and corresponding eigenvectors of ; the solution formula is as follows: ； in, For the matrix The characteristic value of For the matrix The eigenvector of ,but The corresponding eigenvector It is considered as a feature point The normal vector ; Step 5.1.3: Eliminate symbol ambiguity; Since the eigenvalue decomposition method does not clearly define the sign of the eigenvector, in order to achieve the uniqueness of the local coordinate system, the directions of the three coordinate axes need to be redefined; use the disambiguated singular value decomposition (Disambiguated SVD) method to eliminate symbol ambiguity. The specific method for the x-axis is as follows: ； in, Indicates that the eigenvector obtained in step 5.1.2 is along Direction, and in the opposite direction; It means that the distance from a point in the neighborhood to the centroid is less than the neighborhood radius, and the vector Direction and The number of points that meet the condition of "same direction"; similarly, It means that the distance from a point in the neighborhood to the centroid is less than the neighborhood radius, and the vector Direction and The number of points that meet the condition of "same direction"; when When there are many conditional points, The direction of the axis is direction, otherwise direction; Similarly, determine The direction of the axis; The direction of the axis is through to obtain; Step 5.2: Perform spatial division: based on feature points is the center of the ball, A spherical neighborhood space is established for the radius, and the neighborhood space is divided into spatial grids along the diameter, azimuth and elevation directions; finally, it is divided into 2 radial areas, 8 azimuth areas, and 2 elevation areas, a total of 32 areas; Step 5.3: Statistical calculation of histogram; after dividing into subspace regions, calculate the normal vector of the points falling into each subspace region and the feature point normal vector The cosine of the angle, ; Next, in each subspace, The equal value is divided into 11 parts, and the corresponding The number of points within the range; that is, the horizontal axis of the histogram is , the vertical axis is the quantity, and then the feature histograms of each subspace are combined and normalized, and finally a 32*11=352-dimensional histogram feature is obtained; For step 5.3, when each point is accumulated to a specific interval of the histogram, it is necessary to perform quadrilinear interpolation on the adjacent cells in the local histogram and the same cell in the adjacent histogram; Step 5.4: Match feature points and establish corresponding relationships; the matching of feature points is to determine the corresponding relationship between the model and the scene based on the similarity of feature descriptors, and the matching relationship is determined by calculating the Euclidean distance between the two feature vectors in the model and the scene: the specific algorithm is as follows: Step 5.4.1: Assume a feature point in the scene The feature descriptor is , a feature point in the model The feature descriptor is ; Step 5.4.2: Using the obtained SHOT feature descriptor, calculate and The Euclidean distance between is as follows: ； Step 5.4.3: Set the threshold ,like , then it is considered that there is a corresponding relationship between the scene feature points and the model feature points; otherwise, if , then discard the model feature point; traverse all feature points in the scene, repeat steps 5.4.1 to 5.4.3, and establish a corresponding relationship point set for all feature points that satisfy the corresponding relationship; Or match feature points and establish corresponding relationships by designing a 3D Hough space voting mechanism to identify the target more accurately; the basic principle of this method is to use the corresponding relationship between the model point cloud and the current scene point cloud, and associate the relative position of each scene feature point with the model centroid through the local coordinate system, so that each corresponding scene feature votes in the 3D Hough space, and finally enough votes are used to confirm the actual location of the workpiece in the 3D space; the specific algorithm flow of 3D target recognition based on Hough voting is as follows: Step 5.5.1: Assume that all point cloud coordinates of the model are based on the model global coordinate system The centroids of the model point cloud and scene point cloud are , the feature points in the model are , the feature points in the scene are , establish a local reference coordinate system for the model feature points , the vector composed of each feature point and the centroid in the model point cloud is recorded as the voting vector ; Step 5.5.2: To make the voting vector In any reference coordinate system, it has the property of rotation and translation invariance. From the model global coordinate system Transform to model local coordinate system , the conversion process is expressed by the formula: ; in From the model global coordinate system Transform to model local coordinate system The rotation matrix, each row of which consists of feature points The local reference frame unit vector is composed of ; Thus, each model feature point The voting vector corresponding to them to be connected; Step 5.5.3: Then for each scene feature point , establish its relationship with the model feature points Due to the rotation and translation invariance of the local reference system, it is considered that in the scene local coordinate system The voting vector The voting vector corresponding to the one in the model are equal; Step 5.5.4: Finally Convert to the scene global coordinate system The conversion process is expressed as follows: ; in From the scene local coordinate system Convert to the scene global coordinate system The rotation matrix, each column of which consists of feature points The local reference frame unit vector is composed of ; Voting is performed based on these transformation relationships. Each vote determines a centroid position. The actual position of the target centroid is determined based on the voting peak. The voting counting process is repeated until all feature points have voted. Finally, the category and position of the scene target can be determined based on the voting peak and the position of the centroid. In addition, after Hough voting, incorrect matching correspondences are eliminated, and the model is aligned with the scene target through translation and rotation based on the correct correspondence subset, thereby completing target recognition. The above steps complete the recognition of the bonding model point cloud data. For common erroneous bonding effects, the recognized point cloud image is used to detect whether the upper surface parallelism of the bonding model and the bonding overlap between the seat ring and the blank surface meet the product processing requirements. Product quality assessment module: After completing the point cloud recognition and bonding model detection, according to the parallelism of the seat ring surface after bonding The overlap between the seat ring and the embryo surface Establish an evaluation model and assign different weights to the two parameters according to actual processing requirements and , output result evaluation index ; The evaluated products are divided into the following levels: ； According to the obtained product quality level, match the preset evaluation sentence and output the corresponding evaluation letter; The product quality assessment module is used to identify qualified, repairable and unqualified products.

2. A toilet adhesive effect detection system using a three-dimensional point cloud recognition method as claimed in claim 1, characterized in that: The toilet is formed by bonding a cup body and a seat ring in blocks. After bonding, a structured light camera is used to scan the bonding model to obtain point cloud data.

Images