A fine-grained three-dimensional recognition method for aircraft engine clamps

Abstract

A fine-grained three-dimensional recognition method for aircraft engine clamps includes: collecting standard aircraft engine three-dimensional point cloud data, using the clamp point cloud data as the clamp shape and position standard, and defining a model standard for each clamp; firstly performing semantic segmentation on the three-dimensional point cloud data of the aircraft engine to be tested, learning the local geometric structure of each point through a semantic segmentation network, and generating point-by-point semantic features; performing clamp instantiation and combining it with a clustering algorithm to obtain a complete clamp instance; performing fine-grained recognition of the clamp to obtain the model category of each clamp; and finally obtaining the point cloud data and model of each clamp. Compared with aircraft engine detection technologies based on two-dimensional image data, the present invention can obtain complete detection data without image stitching, thereby ensuring detection consistency and efficiency; and is not affected by factors such as uneven lighting, shadow occlusion, clamp posture changes, and scale changes.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional vision technology, and in particular to a fine-grained three-dimensional recognition method for an aircraft engine clamp. Background Art

[0002] Aircraft engines are the engines that power aircraft and are the heart of aircraft, as well as a crucial reflection of a nation's scientific, industrial, and national defense strength. As crucial components on the surface of aircraft engines, clamps are subject to higher safety and reliability requirements than other components due to their high-temperature, high-pressure, and high-intensity operating environments. Given the high cost and high standards of aerospace components, they undergo multiple tests before they are officially put into use, including pre-assembly testing of individual components and post-assembly testing of the entire component. Before the advent of automated inspection technology, manual visual inspection was the primary method for defect detection. However, due to the intensity of the inspection work and the limited experience and efficiency of inspectors, there were potential risks of missed or incorrect inspections. Automated inspection methods utilize various test instruments and devices to obtain useful information about the target, thereby shortening the inspection cycle and significantly improving inspection efficiency.

[0003] The goal of fine-grained 3D clamp recognition is to automatically identify each clamp within the complex background of an aircraft engine and perform fine-grained differentiation to determine the shape, location, and model of each clamp in the point cloud data. The core of fine-grained 3D recognition is to distinguish the local structural differences between clamp points and background points, as well as between clamp points of different models, based on the spatial geometric characteristics of the point cloud.

[0004] Most existing clamp detection methods use 2D images as raw data, primarily extracting features from the image to achieve detection. These features describe the clamp's shape details. For example, a combination of corner and edge features can be used to detect the clamp's shape, and combined with a pattern classification model, it can classify clamps and non-clamps. End-to-end deep neural network feature representation can extract local and global clamp features and detect the position and model of all clamps in the image. In general, 2D clamp detection is often affected by sensitive factors in 2D image information, including viewing angle, uneven image lighting distribution, shadows, and variations in clamp pose and scale. Clamp detection based on 3D point cloud data is a promising technology, yet it remains a less explored branch. 3D point cloud data offers excellent continuity and is not restricted by data acquisition range, ensuring consistent and efficient clamp detection. The challenges of fine-grained 3D clamp recognition lie in identifying clamp points within a complex, large-scale background and distinguishing target features with only local structural differences. Summary of the Invention

[0005] The technology of the present invention adopts the detection process of semantic segmentation of clamp scenes, clamp instantiation, and fine-grained recognition of clamps to perform fine-grained three-dimensional recognition of clamps in aircraft engines. First, a standard is established based on the clamp shape and model in the standard engine data of a certain model; when performing online detection, for the point cloud data of the aircraft engine to be tested, semantic segmentation of the clamp scene is first performed, and point-by-point semantic features are extracted using a semantic segmentation network to distinguish clamp points from background points; then, clamp instantiation is performed, and a clustering algorithm is used to obtain complete and independent clamp instances; finally, fine-grained recognition of clamps is performed, and the model of each clamp is obtained through a fine-grained classification network. The present invention provides a fine-grained three-dimensional automatic recognition technology for clamps, which can identify complete clamp instances and obtain their models, and has the characteristics of good environmental adaptability and high detection efficiency.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A fine-grained three-dimensional recognition method for an aircraft engine clamp comprises the following steps:

[0008] Collect standard aircraft engine 3D point cloud data, use the clamp point cloud data as the clamp shape and position standard, and define the model standard for each clamp;

[0009] For the 3D point cloud data of the aircraft engine to be tested, semantic segmentation of the clamp scene is first performed. The semantic segmentation network learns the local geometric structure of each point, generates point-by-point semantic features, and divides the point cloud into two categories: clamp points and background points.

[0010] Perform clamp instantiation, predict the offset of each clamp point relative to the clamp center through the semantic features of the clamp point, and combine with the clustering algorithm to obtain a complete clamp instance;

[0011] Fine-grained recognition of clamps is performed. Based on the fine-grained classification network, the model category of each clamp is obtained according to the fine-grained feature differences between clamps of different models; finally, the point cloud data and model of each clamp are obtained.

[0012] Furthermore, the step of "collecting standard aircraft engine three-dimensional point cloud data, using the clamp point cloud data therein as the clamp shape and position standard, and defining a model standard for each clamp" includes:

[0013] Select a standard aerospace engine body S without any clamps being installed incorrectly or missing, and use a 3D sensor to scan S to collect the 3D point cloud data M of the standard aerospace engine body. S , and annotate the clamp point cloud therein: use the clamp point cloud to define the shape and position of the clamp, and define a model standard for each clamp.

[0014] Furthermore, the step of "first performing semantic segmentation of the clamp scene on the three-dimensional point cloud data of the aircraft engine to be tested, learning the local geometric structure of each point through a semantic segmentation network, generating point-by-point semantic features, and dividing the point cloud into two categories: clamp points and background points" includes:

[0015] During online testing, a 3D sensor is used to scan the entire aircraft engine T to obtain the 3D point cloud data M of the aircraft engine. T ;

[0016] Semantic segmentation of clamp scenes: Through the semantic segmentation network, point-by-point semantic features are generated based on the neighborhood geometry of each point, such as the neighborhood spatial position relationship, normal vector information, relative angle information, etc. Where D is the feature vector dimension, n is M T Then, according to the feature differences between different models of standard parts, the point cloud data is divided into two categories: clamp points and background points, so as to obtain the three-dimensional coordinates of each clamp point in n c represents the total number of clamp points.

[0017] Furthermore, the step of "instantiating the clamp, predicting the offset of each clamp point relative to the clamp center based on the semantic features of the clamp point, and combining the clustering algorithm to obtain a complete clamp instance" includes:

[0018] According to the point-by-point semantic features f of the clamp point j , predict each clamp point relative to the clamp center Offset in is the j-th clamp instance The three-dimensional coordinates of the center point, m j Indicates the number of points in the instance point cloud; then the three-dimensional coordinates x of each clamp point i The corresponding offset o i Add together to get the three-dimensional coordinates of the clamp point after collapsing toward the center of the clamp Here, these points are defined as shift clamp points; then all shift clamp points are clustered to obtain multiple complete instances. Where m is the number of instances, m j is the number of points in the point cloud of the jth instance; then the three-dimensional coordinates of each shifted clamp instance are subtracted from the corresponding offset to obtain a complete, non-offset clamp instance

[0019] Furthermore, the step of "performing fine-grained recognition of the clamp, obtaining the model category of each clamp based on the fine-grained classification network and the fine-grained feature differences between clamps of different models; and finally obtaining the point cloud data and model of each clamp" includes:

[0020] Through the fine-grained classification network, for each clamp instance C obtained in step d j Extracting global features Where D is the dimension of the global feature; then, according to the feature differences between different models in the standard parts, all clamp instances are classified to obtain the model category to which they belong; finally, the three-dimensional coordinates C of the clamp in the aircraft engine are output. j and its model category cls j {cls j ∈{1…N}}(j=1…m), where N is the number of clamp models.

[0021] Furthermore, the three-dimensional point cloud data M of the aircraft engine S S First, the clamps are annotated: the clamp point cloud is used to define the clamp shape and position, which is used to assist the semantic segmentation network to identify the clamp points from the engine point cloud data; then, a model standard is defined for each clamp to assist the fine-grained recognition network to distinguish the geometric structure differences of different models of clamps; further, the specific implementation process of the clamp scene semantic segmentation is as follows: traverse M T Each point in the point cloud Where n represents M T The number of points in and search its neighborhood point set The geometric structure of the neighborhood is modeled through the semantic segmentation network, and the three-dimensional spatial relationship between the neighborhood points is used to represent the neighborhood structure, construct a local geometric paradigm, and then generate the semantic features of each point. Where D is the dimension of the feature vector; then, based on the feature differences between different models of standard parts, the geometric feature differences between the clamp and other parts are distinguished, and finally the semantic category of each point is output. To indicate whether the point is on the clamp, the clamp point is used as the target point to obtain the three-dimensional coordinates of each clamp point where n c Indicates the number of points in the clamp point cloud. Here, "clamp points" refer to points located on the clamp instance.

[0022] Furthermore, the specific implementation process of the clamp instantiation is as follows: point-by-point semantic features f i Predicting point-by-point 3D vectors Used to indicate the distance between a point on the clamp and the center of the clamp The offset of The calculation method of is shown in formula (1):

[0023]

[0024] Among them C j is the point cloud 3D coordinate of the j-th clamp instance, m j Represents the number of points in the instance point cloud, mean is the average value along the dimension; then the three-dimensional coordinate x of each clamp point i The corresponding three-dimensional offset o i Add together to get the three-dimensional coordinates of the shift clamp point At this time, the displaced clamp point cloud will collapse toward the instance center, thereby further increasing the gap with other instance point clouds. Then, the displaced clamp points are clustered: based on the distance between the clamp points, points with closer distances are aggregated, points with farther distances are separated, and points with farther distances are divided into different groups. Finally, multiple independent collapsed clamp instances are obtained. Where m is the number of instances, m j is the number of points in the jth instance point cloud; then the three-dimensional coordinate x′ of each instance point i Subtract the corresponding offset o i , get the clamp instance without offset

[0025] Furthermore, the specific implementation process of the clamp fine-grained recognition is as follows: According to each clamp instance C j (j=1…m), through the fine-grained classification network, traverse each point in the clamp By modeling the neighborhood structure, constructing its local features, and finally aggregating the global features of each clamp instance Where D is the dimension of the global feature; then according to the feature differences between different models of standard parts, the model cls of each clamp is output j {cls j ∈{1…N}}(j=1…m), where N is the number of clamp models.

[0026] Compared with the prior art, the technical solution of the present invention can achieve the following beneficial technical effects:

[0027] ① Distinguish the clamp points from the background points by point-by-point semantic features and only process the clamp points, making the features of the target point cloud more prominent, reducing the interference of the complex background, enhancing the robustness, and improving the accuracy of the clamp fine-grained recognition;

[0028] ② Add the prediction of the clamp point offset so that the clamp point can be closer to the center of the instance, thereby increasing the distance from other clamp instances and improving the accuracy of instance recognition;

[0029] ③ Perform model detection on each clamp, and make fine-grained distinctions based on the three-dimensional structural differences between different models, and finally obtain the model category of each clamp, making the detection results more intuitive and stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of the overall implementation of the present invention;

[0031] Figure 2 Schematic diagram of the encoding and de-encoding process of the semantic segmentation network;

[0032] Figure 3 This is a schematic diagram of some point cloud data of a certain model of engine body T to be tested obtained by the data acquisition system;

[0033] Figure 4 Schematic diagram of the clamp points and background points predicted by the semantic segmentation network;

[0034] Figure 5 This is a schematic diagram of a clamp instance without offset obtained by instantiating the clamp;

[0035] Figure 6 Schematic diagram of model categories obtained by fine-grained classification of clamps;

[0036] Figure 7 Schematic diagram of the true value of a certain model of standard engine body S obtained by the data acquisition system. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] According to a specific embodiment of the present invention, Figure 1 The above is a flowchart of the overall implementation of the fine-grained three-dimensional recognition method for aircraft engine clamps of the present invention, which specifically includes the following steps:

[0039] The surface of an aircraft engine is assembled with a large number of complex parts, while the size of the clamp is relatively small, with only slight structural differences between different models.

[0040] Step 11: For a certain type of aircraft engine, select a standard aircraft engine body S without any clamps being installed incorrectly or missing, and use a 3D sensor to scan the standard aircraft engine body S to collect 3D point cloud data M S , and the three-dimensional point cloud data M SThe clamp point cloud in is annotated for defining the shape, position and model standard of the clamp. Here, it is assumed that the number of clamp models is N. Figure 7 The local clamp point cloud annotation results are shown.

[0041] Specifically, for the three-dimensional point cloud data M of the aircraft engine S S First, the clamps are annotated: the clamp point cloud is used to define the clamp shape and position, which is used to assist the semantic segmentation network to identify the clamp points from the engine point cloud data; then, a model standard is defined for each clamp to assist the fine-grained recognition network to distinguish the geometric structure differences of different models of clamps;

[0042] Step 12: In the online detection process, first use a 3D sensor to scan the entire aircraft engine T to obtain 3D point cloud data M T .

[0043] Step 13: Semantic segmentation of the clamp scene: The 3D point cloud data M of the aircraft engine T to be tested is processed through the semantic segmentation network. T Generate point-by-point semantic features, and the overall network structure is as follows Figure 2 As shown in the figure, the network is divided into two parts: encoding and de-encoding. During encoding, the point cloud is sampled and dimensioned four times. After completing these two operations, a new layer of point cloud is obtained, whose dimension is n. i ×D i , n i Indicates the number of point clouds in this layer, D i Represents the features of the point cloud layer; during the inverse encoding, four upsampling operations are also performed to finally obtain the point-by-point semantic features consistent with the original number of points.

[0044] During the encoding process, the sampling operation uses a random sampling method, corresponding to Figure 2 (c) in the i-th layer is randomly selected without duplication. sampling points, n i Indicates the number of points in the i-th layer of point cloud; feature dimension increase operation (corresponding to Figure 2 (b)) by searching for each point p i Neighborhood, and each neighborhood point Generate feature vectors Then aggregate the eigenvectors to get the eigenvector r of the point i . Eigenvector It mainly consists of three parts: semantic features learned by the previous layer of network Spatial geometric characteristics Normal vector feature Semantic features The calculation method of is shown in formula (1):

[0045]

[0046] in, express The corresponding normal vector, i represents the number of layers;

[0047] Spatial geometric characteristics The calculation method of is shown in formula (2):

[0048]

[0049] in, represents the concatenation operation of vectors, and ||·|| represents the vector modulus.

[0050] In calculating the normal vector feature When , first construct a point pair for the center point Two auxiliary variables are calculated and See formula (3):

[0051]

[0052] Among them, n i and are the center point and the kth neighboring point respectively The normal vector of .

[0053] Next use Represents the cosine of the angle between the normal vector of the center point and the neighboring points, Represents the cosine of the angle between the normal vector and the distance vector between two points, and uses express respectively and n i The angle formed by the length of the projection is calculated by formula (4):

[0054]

[0055] Finally, the angle is used to fully express the relationship between the normal vectors of the center point and the neighboring points, and the relationship between the normal vector and the distance vector, and the normal vector feature is obtained. See formula (5):

[0056]

[0057] Therefore, the feature vector composed of the center point and the kth neighboring point See formula (6):

[0058]

[0059] When aggregating, weighted summation is used to calculate the weights of the network. Each element in generates an attention score Refer to formula (7), where MLP refers to a multi-layer perceptron (MLP), which contains learnable parameters. Then, the attention score is multiplied by the feature vector as a weight to achieve weighted summation and obtain the center point p i The eigenvector r i , the calculation method is shown in formula (8):

[0060]

[0061] In the de-encoding process, the features of each layer come from two parts: the point features of the i+1th layer after nearest neighbor upsampling and the encoding features of the ith layer, so the calculation formula of the features of the ith layer can be found in formula (9):

[0062]

[0063] Among them, NNU stands for Nearest Neighbor Upsampling. This method can transform the point cloud dimension from (n i+1 ×D) expanded to (n i ×D), note that the number of points in the i+1th layer is less than that in the ith layer, so n i+1 <n i , D is the feature dimension of the point cloud. i+1 -n i The feature of points) is assigned to the point feature closest to the i-th layer point cloud, so The number of points and r i The number of points is the same. Then the point cloud features after de-encoding are obtained through MLP Finally, the semantic features of each point are output Among them, “—” indicates reverse encoding.

[0064] Convert semantic features into categories for each point through MLP This category indicates that the point is a clamp point or background point, such as Figure 4 As shown in the black rectangular box, the points marked in the box are the clamp points, and the rest are background points;

[0065] Step 14: Clamp instantiation: Based on the point-by-point semantic features f generated in step 13 i , the semantic features of the clamp points are converted into three-dimensional vectors through MLP Set it as the clamp point towards the center of the clamp instance The offset, The calculation method of is shown in formula (10):

[0066]

[0067] In the formula, mean means taking the average. Then the three-dimensional coordinates x of each clamp point are i Add the corresponding offset o i , get the three-dimensional coordinates of the offset clamp point Then perform the clustering algorithm. The clustering algorithm uses a breadth-first search strategy, which makes a judgment based on the number of neighborhood points of each point with a radius of r and the threshold th1: if the number of neighborhood points is less than th1, the point is classified as a noise point; if it is greater than th1, the point is classified as a point on the instance, and then the current operation is repeated for the neighborhood of the point until the number of neighborhood points is less than th1. At this time, the point has reached the instance boundary and is classified as an instance boundary point to obtain a complete instance; after obtaining the complete instance, the offset coordinate x′ of each instance point is calculated. i Subtract the corresponding offset o i , get the clamp instance without offset m is the number of clamps without offset. Figure 5 shown.

[0068] Step 15: Clamp fine-grained identification: Each clamp instance C obtained in step 14 j (j=1…m), through the fine-grained classification network, use the encoding method mentioned in step 13 to obtain the global features of each clamp Based on the local structural differences of each clamp, the model category cls of each clamp is generated through MLP j {cls j =MLP(F j )|cls j ∈{1…N}}(j=1…m)6, where N is the total number of clamp models. Figure 6 As shown, the model number of each clamp is marked around the frame.

[0069] So far, each clamp point cloud C has been identified from the engine point cloud data. j , and its model cls j .

[0070] As mentioned above, compared with the clamp detection technology for two-dimensional image data, the present invention is not affected by factors such as uneven lighting, shadow occlusion, clamp posture change and scale change; it has good environmental adaptability, with a clamp point classification accuracy of 87.8%, a clamp recognition accuracy of 86.1%, and a model classification accuracy of 91.9%.

[0071] The present invention provides a fine-grained three-dimensional recognition method for aircraft engine clamps, comprising: defining standard engine clamps and models, collecting point cloud data of the aircraft engine to be tested, semantic segmentation of the clamp scene, clamp instantiation, and fine-grained recognition of the clamps. The method of the present invention was tested on the three-dimensional point cloud data of engines exhibited in the Museum of Beijing University of Aeronautics and Astronautics. The clamp point classification accuracy was 87.8%, the clamp recognition accuracy was 86.1%, and the model classification accuracy was 91.9%. Therefore, the present invention has theoretical feasibility and practical effectiveness. The present invention adopts the starting point of three-dimensional data to avoid the susceptibility of two-dimensional image data to factors such as lighting, shadows, and scale changes. It can continuously obtain complete engine point cloud data, making it more environmentally adaptable.

[0072] In summary, the present invention discloses a fine-grained three-dimensional recognition method for aircraft engine clamps. The surface of an aircraft engine is assembled with a large number of complex parts, while the size of the clamps is relatively small, and there are only slight structural differences between different models. The clamp recognition method disclosed in the present invention uses a three-dimensional visual sensor to collect standard aircraft engine three-dimensional point cloud data, uses the clamp point cloud data as the clamp shape and position standard, and defines a model standard for each clamp; for the three-dimensional point cloud data of the aircraft engine to be tested, firstly, semantic segmentation of the clamp scene is performed, and the local geometric structure of each point is learned through the semantic segmentation network, point-by-point semantic features are generated, and the point cloud is divided into two categories: clamp points and background points; then, the clamp is instantiated, and the offset of each clamp point relative to the clamp center is predicted through the semantic features of the clamp point, and a complete clamp instance is obtained by combining a clustering algorithm; finally, fine-grained recognition of the clamp is performed, and the model category of each clamp is obtained based on the fine-grained classification network and the fine-grained feature differences between different models of clamps; finally, the point cloud data and model of each clamp are obtained. Compared with aviation engine detection technology based on two-dimensional image data, the present invention can obtain complete detection data without image stitching, thereby ensuring the consistency and efficiency of detection; it is not affected by factors such as uneven lighting, shadow occlusion, clamp posture changes and scale changes.

[0073] It should be understood that the above-described specific embodiments of the present invention are merely illustrative or illustrative of the principles of the present invention and do not constitute limitations of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included within the scope of protection of the present invention. In addition, the appended claims are intended to cover all variations and modifications that fall within the scope and metes and bounds of the appended claims, or equivalents thereof.

Claims

1. A fine-grained three-dimensional recognition method for aircraft engine clamps, characterized in that: The method comprises the following steps: collecting standard aircraft engine three-dimensional point cloud data, using the clamp point cloud data as the clamp shape and position standard, and defining a model standard for each clamp; For the 3D point cloud data of the aircraft engine to be tested, semantic segmentation of the clamp scene is first performed. The semantic segmentation network learns the local geometric structure of each point, generates point-by-point semantic features, and divides the point cloud into two categories: clamp points and background points. Perform clamp instantiation, predict the offset of each clamp point relative to the clamp center through the semantic features of the clamp point, and combine with the clustering algorithm to obtain a complete clamp instance; Perform fine-grained recognition of clamps. Based on the fine-grained classification network, the model category of each clamp is obtained according to the fine-grained feature differences between clamps of different models. Finally, the point cloud data and model of each clamp are obtained. The step of "collecting standard aircraft engine three-dimensional point cloud data, using the clamp point cloud data therein as the clamp shape and position standard, and defining a model standard for each clamp" includes: Select a standard aerospace engine body S without any clamps being installed incorrectly or missing, and use a 3D sensor to scan S to collect the 3D point cloud data M of the standard aerospace engine body. S , and annotating the clamp point cloud therein: using the clamp point cloud to define the clamp shape and position, and defining a model standard for each clamp; the step of "first performing semantic segmentation on the clamp scene for the three-dimensional point cloud data of the aircraft engine to be tested, learning the local geometric structure of each point through a semantic segmentation network, generating point-by-point semantic features, and dividing the point cloud into two categories: clamp points and background points" includes: During online testing, a 3D sensor is used to scan the entire aircraft engine T to obtain the 3D point cloud data M of the aircraft engine. T ; Semantic segmentation of clamp scenes: Through the semantic segmentation network, based on the encoding and de-encoding process, point-by-point semantic features are generated according to the neighborhood geometric structure of each point, including the neighborhood spatial position relationship, normal vector information, and relative angle information. Where: D is the feature vector dimension, n is M T The encoding process includes random sampling, feature dimensionality increase and attention weighted aggregation. The de-encoding process restores the point cloud dimension by nearest neighbor upsampling. Then, the semantic features are converted into point-by-point category labels through the multi-layer perceptron MLP, and the point cloud is divided into clamp points and background points, so as to obtain the three-dimensional coordinates of each clamp point. where n c Indicates the total number of clamp points; During the encoding process, the sampling operation uses a random sampling method, randomly selecting without duplication at the i-th layer. sampling points, n i Represents the number of points in the i-th layer of point cloud; the feature dimension increase operation is performed by searching each point p i Neighborhood, and each neighborhood point Generate feature vectors Then aggregate the eigenvectors to get the eigenvector r of the point i ; Eigenvector It mainly consists of three parts: semantic features learned by the previous layer of network Spatial geometric characteristics Normal vector feature Semantic features The calculation method of is shown in formula (1): （1） in, express The corresponding normal vector, i represents the number of layers; Spatial geometric characteristics The calculation method of is shown in formula (2): （2） in, Represents the concatenation operation of vectors, ||·|| represents the vector modulus; In calculating the normal vector feature When , first construct a point pair for the center point Two auxiliary variables are calculated and See formula (3): （3） Among them, n i and are the center point and the kth neighboring point respectively The normal vector of Next use Represents the cosine of the angle between the normal vector of the center point and the neighboring points, Represents the cosine of the angle between the normal vector and the distance vector between two points, and uses express respectively and n i The angle formed by the length of the projection is calculated by formula (4): （4） Finally, the angle is used to fully express the relationship between the normal vectors of the center point and the neighboring points, and the relationship between the normal vector and the distance vector, and the normal vector feature is obtained. See formula (5): （5） Therefore, the feature vector composed of the center point and the kth neighboring point See formula (6): （6） When aggregating, weighted summation is used to calculate the weights of the network. Each element in generates an attention score Refer to formula (7), where MLP refers to a multi-layer perceptron. Then, the attention score is multiplied by the feature vector as a weight to achieve weighted summation and obtain the center point p i The eigenvector r i , the calculation method is shown in formula (8): （7） （8） In the de-encoding process, the features of each layer come from two parts: the point features of the i+1th layer after nearest neighbor upsampling and the encoding features of the ith layer, so the calculation formula of the features of the ith layer can be found in formula (9): （9） Among them, NNU represents the nearest neighbor upsampling. This method can reduce the point cloud dimension from n to i+1 ×D expands to n i ×D, note that the number of points in the i+1th layer is less than that in the ith layer, so n i+1 <n i ; Newly added points, a total of n i+1 -n i The feature of each point is assigned to the point feature closest to the i-th layer point cloud, so The number of points and r i The number of points is consistent; then the point cloud features after de-encoding are obtained through MLP Finally, the semantic features of each point are output Among them, "—" indicates reverse encoding; Convert semantic features into the category l of each point through MLP i =MLP(f i ), This category indicates that the point is a clamp point or background point; Clamp instantiation: point-by-point semantic features f generated based on the semantic segmentation of the clamp scene i , the semantic features of the clamp points are converted into three-dimensional vectors through MLP Set it as the clamp point towards the center of the clamp instance The offset, The calculation method of is shown in formula (10): ； Where, mean represents the average; then the three-dimensional coordinates x of each clamp point are i Add the corresponding offset o i , get the three-dimensional coordinates of the offset clamp point , and then perform the clustering algorithm; the clustering algorithm uses a breadth-first search strategy, which makes a judgment based on the number of neighborhood points of each point with a radius of r and the threshold th1: if the number of neighborhood points is less than th1, the point is classified as a noise point; if it is greater than th1, the point is classified as a point on the instance, and then the current operation is repeated for the neighborhood of the point until the number of neighborhood points is less than th1. At this time, the point has reached the instance boundary and is classified as an instance boundary point to obtain a complete instance; after obtaining the complete instance, the offset coordinate x′ of each instance point is calculated. i Subtract the corresponding offset o i , get the clamp instance without offset , m is the number of clamps without offset; The step of "instantiating the clamp, predicting the offset of each clamp point relative to the clamp center based on the semantic features of the clamp point, and combining the clustering algorithm to obtain a complete clamp instance" includes: The point-by-point semantic features f generated by semantic segmentation of the clamp scene i , the semantic features of the clamp points are converted into three-dimensional vectors through MLP , using it as the clamp point toward the center of the clamp instance The offset of is the j-th clamp instance The three-dimensional coordinates of the center point, m j Indicates the number of points in the instance point cloud; then the three-dimensional coordinates x of each clamp point i The corresponding offset o i Add together to get the three-dimensional coordinates of the clamp point after collapsing toward the center of the clamp , here, these points are defined as shift clamp points; then all shift clamp points are clustered, and finally multiple complete instances are obtained , where m is the number of instances, m j is the number of points in the point cloud of the jth instance; then the three-dimensional coordinates of each shifted clamp instance are subtracted from the corresponding offset to obtain a complete, non-offset clamp instance .

2. The fine-grained three-dimensional recognition method for an aircraft engine clamp according to claim 1, characterized in that: The steps of "performing fine-grained recognition of clamps, obtaining the model category of each clamp based on the fine-grained classification network and the fine-grained feature differences between clamps of different models; and finally obtaining point cloud data and the model of each clamp" include: Through the fine-grained classification network, each clamp instance C j Extracting global features Then, based on the feature differences between different models in the standard parts, the model category cls of each clamp is generated through MLP j {cls j =MLP(F j )|cls j ∈{1...N}}(j=1...m)6, where N is the total number of clamp models.

3. The fine-grained three-dimensional recognition method for an aircraft engine clamp according to claim 2, characterized in that: The 3D point cloud data M of the aircraft engine S S First, the clamps are annotated: the clamp point cloud is used to define the clamp shape and position, which is used to assist the semantic segmentation network to identify the clamp points from the engine point cloud data; then a model standard is defined for each clamp to assist the fine-grained recognition network to distinguish the geometric structure differences of different models of clamps.

Images