Aero-engine point cloud data few-sample instance segmentation method and system
By extracting and aggregating the point cloud data of aero engines, determining the anchor point characteristics, and using the Transformer generator for instance segmentation, the problem of existing detection methods depend on specific models is solved, and flexible and accurate detection of different models of engines is achieved.
Patent Information
- Application Number
- CN202411656462.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-04-04
AI Technical Summary
The existing aircraft engine assembly defect detection methods are usually for specific models. When directly applied to other models of engines, the detection effect is not ideal due to differences in the types, shapes and sizes of the parts.
By obtaining point cloud data of the aircraft engine, global feature extraction is performed on the query scenario and support scenario, multi-scale features are obtained, and anchor points are determined through the feature similarity metric calculation module of the multi-layer perceptron network, and the small sample instance segmentation is performed using Transformer's convolution kernel generator.
The assembly defect detection of different models of aircraft engines is realized, which avoids the problem of unsatisfactory detection results due to the differences in parts between the test models and the training models, and improves the flexibility and accuracy of the detection.
Smart Images
Figure CN119762769A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional vision technology, and in particular to a method and system for segmenting a few-sample instances of aircraft engine point cloud data. Background Art
[0002] With the continuous advancement of aviation technology, aircraft engines, as the key power unit of aircraft, must meet extremely strict standards in design, processing and assembly of small parts. After assembly, comprehensive assembly defect detection is essential to ensure the safety and performance of the engine.
[0003] At present, the assembly defect detection of aircraft engines mainly relies on manual methods. However, due to the complex structure of the engine and the wide variety of small parts, manual detection is not only inefficient but also difficult to ensure accuracy, which makes enterprises face great costs and risks.
[0004] With the development of computer vision and deep learning technologies, automated detection methods based on point cloud data have gradually been applied to assembly defect detection of aero-engines.
[0005] However, existing inspection methods usually perform assembly defect detection on specific aircraft models. If these inspection methods are directly applied to other types of aircraft engines, the inspection results will often be unsatisfactory due to the differences in types, shapes and sizes of parts between the test and training models. Summary of the invention
[0006] In order to solve the technical problem that the existing detection methods usually perform assembly defect detection on specific aircraft models, if these detection methods are directly applied to other models of aircraft engines, the differences in types, shapes and sizes of parts between the test model and the training model often lead to unsatisfactory detection results, the present invention provides a method and system for segmenting a few-sample instances of aircraft engine point cloud data.
[0007] The technical solution provided by the embodiment of the present invention is as follows:
[0008] First aspect:
[0009] An embodiment of the present invention provides a method for segmenting a few-sample instances of aircraft engine point cloud data, comprising:
[0010] S1: Acquire point cloud data of an aircraft engine, wherein the point cloud data includes point cloud data of a query scene and point cloud data of a support scene;
[0011] S2: labeling the point cloud data of the supporting scene;
[0012] S3: performing global feature extraction on the point cloud data of the query scene and the support scene respectively to obtain multi-scale features of the query scene and the support scene;
[0013] S4: performing feature aggregation on the multi-scale features of the query scene and the support scene, and determining a plurality of anchor point features between the query scene and the support scene through a feature similarity measurement calculation module based on a multi-layer perceptron network;
[0014] S5: According to the features of each anchor point, a Transformer-based convolution kernel generator is used to perform few-sample instance segmentation on the point cloud data of the query scene.
[0015] Second aspect:
[0016] An embodiment of the present invention provides a system for segmenting a few-sample instances of aircraft engine point cloud data, comprising: a memory and one or more processors;
[0017] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the above-mentioned aircraft engine point cloud data few-sample instance segmentation method.
[0018] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0019] In the present invention, global feature extraction is performed on the point cloud data of the query scene and the support scene respectively, multi-scale features of the query scene and the support scene are obtained, feature aggregation is performed on the multi-scale features of the query scene and the support scene, a plurality of anchor point features between the query scene and the support scene are determined, and according to each anchor point feature, a few-sample instance segmentation is performed on the point cloud data of the query scene. Assembly defect detection is no longer performed only on a specific model, and it can be directly applied to other models of aircraft engines, thereby avoiding the phenomenon of unsatisfactory detection effect due to differences in type, shape and size of parts between the test model and the training model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 A schematic flow chart of a method for segmenting a few-sample instances of aircraft engine point cloud data provided by an embodiment of the present invention;
[0022] Figure 2 A schematic diagram of the structure of a system for segmenting a few-sample instances of aircraft engine point cloud data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0024] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0025] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0026] Reference Manual Attached Figure 1 , showing a flow chart of a method for segmenting a few-sample instances of aircraft engine point cloud data provided by an embodiment of the present invention.
[0027] The embodiment of the present invention provides a method for segmenting a few-sample instances of aircraft engine point cloud data. The method can be implemented by a device for segmenting a few-sample instances of aircraft engine point cloud data. The device for segmenting a few-sample instances of aircraft engine point cloud data can be a terminal or a server. The processing flow of the method for segmenting a few-sample instances of aircraft engine point cloud data can include the following steps:
[0028] S1: Acquire point cloud data of an aircraft engine, where the point cloud data includes point cloud data of a query scene and point cloud data of a supporting scene.
[0029] Optionally, the 3D point cloud data of the aircraft engine is acquired by a high-precision 3D scanning system GOM ATOS ScanBox 8160, where the point cloud data includes the 3D point cloud data of the query scene and the 3D point cloud data of the support scene.
[0030] It should be noted that the GOM ATOS ScanBox 8160 is a high-precision 3D scanning system specifically designed to obtain 3D point cloud data of objects. The device combines advanced optical measurement technology to quickly and accurately capture complex shapes and details, and is widely used in industrial inspection, reverse engineering, quality control and other fields. It provides a scanning accuracy of up to 0.02 mm through multiple high-resolution cameras and efficient software algorithms, and can accurately scan the surface of objects and generate detailed 3D models. These models provide a reliable foundation for subsequent data analysis and processing, significantly improving measurement efficiency and accuracy.
[0031] It should be noted that the query scene refers to the specific scene to be analyzed or processed in point cloud data processing, which usually contains the target object to be identified or segmented. It provides the actual data to which the algorithm is applied and is used to obtain features, perform segmentation or detection tasks.
[0032] It should be noted that support scenes refer to scenes used to provide reference or annotation information in point cloud data processing. These scenes usually contain objects of known categories and are annotated to help algorithms learn and judge, enabling them to more accurately identify and segment targets in query scenes. Support scenes provide the necessary context and benchmarks for algorithms.
[0033] In the present invention, the high-resolution camera and advanced optical measurement technology of GOM ATOS ScanBox 8160 are used to capture complex shapes and details with an accuracy of 0.02 mm. This ensures that the generated 3D point cloud data has high accuracy and provides a reliable basis for subsequent analysis. By combining the data of the query scene with the supporting scene, the algorithm can still maintain good segmentation performance when facing engines of different models or shapes. Through efficient scanning and accurate data processing, the detection time can be significantly shortened and the overall detection efficiency can be improved.
[0034] S2: Label the point cloud data supporting the scene.
[0035] Optionally, Geomagic Wrap 2015 is used to manually annotate point cloud data of specific categories that support the scene.
[0036] It should be noted that Geomagic Wrap 2015 is a professional 3D point cloud processing software designed to convert point cloud data into 3D models that can be used for design and analysis. The software provides a variety of tools and functions for point cloud cleaning, registration, classification, and mesh generation. Users can use Geomagic Wrap 2015 to manually annotate specific categories of point cloud data to ensure the accuracy and completeness of the data, which is convenient for subsequent modeling and analysis.
[0037] In the present invention, Geomagic Wrap 2015 provides a variety of tools and functions to support point cloud cleaning, registration and classification, allowing users to efficiently process complex data. This flexibility allows users to accurately annotate according to specific needs. Manual annotation through professional software ensures the accuracy and completeness of the annotated data. High-quality annotated data is crucial for the training of deep learning models and can significantly improve the performance and generalization ability of the model.
[0038] In a possible implementation manner, S2 is specifically:
[0039] Annotate the point cloud data of the supporting scene and ensure that the query scene contains point cloud data of the same category as the annotated point cloud data in the supporting scene.
[0040] In the present invention, by ensuring that the query scene contains the same category of point cloud data as that annotated in the support scene when acquiring data, consistent feature information can be provided, making it easier for the model to learn effective feature representations during training, thereby improving the accuracy of classification and segmentation. Similar data in the query scene provides the algorithm with the necessary context, helping the algorithm to better understand the environment of the target object and improve the ability to segment the target. Ensuring that there is annotated category data in the query scene can reduce the confusion that may occur in the algorithm during processing, thereby speeding up data processing and analysis and improving overall efficiency.
[0041] S3: Perform global feature extraction on the point cloud data of the query scene and the support scene respectively to obtain multi-scale features of the query scene and the support scene.
[0042] Specifically, in the three-dimensional point cloud data of the query scene and the supporting scene, global feature extraction is performed through the point cloud feature extraction module to obtain multi-scale features of the query scene and the supporting scene.
[0043] In the present invention, global feature extraction can capture the overall structure and geometric features of the scene, allowing the model to more fully understand the spatial relationship and layout of each object in the point cloud data. Acquiring multi-scale features allows the model to analyze data from different perspectives and scales, helping to identify and segment targets of different sizes and shapes. This diversity can improve the ability to adapt to complex shapes. By acquiring multi-scale features, the model can better distinguish between objects of different categories, especially in complex backgrounds, improving the accuracy of few-sample instance segmentation and detection.
[0044] S4: The multi-scale features of the query scene and the support scene are aggregated, and multiple anchor features between the query scene and the support scene are determined through a feature similarity measurement calculation module based on a multi-layer perceptron network.
[0045] Optionally, the multilayer perceptron network consists of five linear layers, and one-dimensional convolutional layers are used in these linear layers.
[0046] It should be noted that the multi-layer perceptron network (MLP) is a feed-forward neural network consisting of multiple layers, including an input layer, multiple hidden layers, and an output layer. Each layer consists of multiple neurons, and the neurons are connected by weights. MLP introduces nonlinear characteristics through nonlinear activation functions (such as ReLU or Sigmoid), enabling it to learn complex mapping relationships. The network usually contains a series of linear layers and activation functions in each hidden layer, and the feature extraction capability can be enhanced by using a one-dimensional convolutional layer. MLP is widely used in tasks such as classification, regression, and feature similarity measurement.
[0047] In the present invention, MLP can learn complex mapping relationships through multiple hidden layers and nonlinear activation functions, making feature aggregation more flexible and efficient. This helps capture subtle differences and complex features in query scenarios and support scenarios. By using MLP to measure feature similarity, fast and accurate similarity calculations can be achieved, providing a solid foundation for subsequent anchor feature determination and ensuring greater reliability in feature matching. The structure of MLP is relatively simple and can process data efficiently. In particular, in the calculation of feature similarity measures, it can reduce computing costs and time and improve operating efficiency compared to complex models.
[0048] Furthermore, S4 specifically includes sub-steps S401 to S404:
[0049] S401: Aggregate the multi-scale features of the query scene and the multi-scale features of the support scene to obtain context features:
[0050] F context =MLP(W agg *[F query ⊙f support ; F query -f support ; F query ])
[0051] in, Represents context features, represents a real number set, MLP(·) represents a multi-layer perceptron network, represents the linear layer weight matrix, D represents the feature dimension, * represents the convolution operation, [·;·] represents the aggregation operation, and F query represents the multi-scale features of the query scene, ⊙ represents the element-by-element multiplication operation, and f support Representation supports multi-scale features of the scene.
[0052] Optionally, the feature dimension D is set to 16.
[0053] S402: Perform farthest point sampling on the context features to obtain candidate point features of the query scene:
[0054]
[0055] in, represents the candidate point feature obtained by sampling the farthest point of the i-th context feature, FPS(·) represents the farthest point sampling operation, represents the i-th context feature, and N represents the number of farthest point sampling points.
[0056] Optionally, the number of farthest point sampling points is set to 1024.
[0057] It should be noted that Furthest Point Sampling (FPS) is a technique for selecting representative points from a point cloud. This method ensures that the selected points are evenly distributed in space by gradually selecting the points farthest from the selected point set. At each step, the algorithm first calculates the distance between the current selected point set and the unselected points, and then selects the farthest point to be added to the selected set. This process continues until the preset number of sample points is reached. Furthest point sampling can effectively reduce the number of point clouds while retaining their geometric features, and is often used in 3D shape analysis and deep learning tasks.
[0058] S403: Inputting the candidate point features into a feature similarity measurement calculation module based on a multi-layer perceptron network to calculate a similarity score between the candidate point features and the multi-scale features of the supporting scene.
[0059] The similarity score is:
[0060]
[0061] Among them, Sim_Sore represents the similarity score.
[0062] S404: Filter out candidate point features whose similarity scores are greater than a similarity score threshold as anchor point features.
[0063] Optionally, the similarity score threshold is set to 0.5.
[0064] In the present invention, by aggregating the multi-scale features of the query scene and the supporting scene, the obtained context features can more comprehensively reflect the geometric information and spatial relationships in the scene, thereby providing richer information for subsequent analysis. The farthest point sampling (FPS) technology can effectively reduce the amount of point cloud data while retaining its geometric features, ensuring that the complexity and time of subsequent calculations are reduced while maintaining representativeness. FPS helps to improve the expressiveness of the model by selecting the farthest point and ensuring that the selected points are evenly distributed in space, so that each anchor point feature can represent part of the overall structure and improve segmentation accuracy.
[0065] S5: According to the features of each anchor point, the point cloud data of the query scene is segmented into few-shot instances through the Transformer-based convolution kernel generator.
[0066] It should be noted that Transformer is a deep learning model architecture based on the self-attention mechanism, which was originally used for natural language processing tasks. Its main feature is the ability to process input data in parallel and capture long-distance dependencies during processing. Transformer consists of two parts: an encoder and a decoder. The encoder is responsible for extracting input features, while the decoder is used to generate outputs. In each layer, Transformer calculates the relationship between each element in the input sequence through the self-attention mechanism, and combines the feedforward neural network to perform feature transformation. Due to its high efficiency and flexibility, Transformer has been widely used in a variety of tasks such as text, images, and speech, especially in processing sequence data.
[0067] In the present invention, the Transformer architecture can process input data in parallel, significantly improving processing efficiency, especially when processing large-scale point cloud data, which can speed up the segmentation process. The self-attention mechanism enables the Transformer to effectively capture the long-distance dependencies between various features in the point cloud data, which helps to understand the contextual information of the target object in complex scenes. The feedforward neural network in the Transformer can flexibly transform the input features, so that the model can adapt to point cloud data of different types and shapes, and improve the accuracy of segmentation. The convolution kernel generated by the Transformer can dynamically adjust the convolution operation according to different anchor point features. This flexibility improves the model's adaptability to different targets and makes the segmentation more refined.
[0068] Furthermore, S5 specifically includes S501 to S506:
[0069] S501: Calculate the multi-head self-attention mechanism operation results between each anchor point feature through the Transformer-based convolution kernel generator:
[0070]
[0071] Among them, F' represents the result of the multi-head self-attention mechanism operation between anchor features, MultiHeadAttention represents the multi-head attention mechanism operation, Q represents the query matrix, represents the i-th anchor feature, K represents the key matrix, and V represents the value matrix.
[0072] Among them, the multi-head attention mechanism operation is specifically as follows:
[0073] MultiHeadAttention=Concat(head1,head2,head3,...,head h )W O
[0074] head j =Attention(Q,K,V)
[0075]
[0076] Among them, MultiHeadAttention represents the multi-head attention mechanism operation, Concat(·) represents the concatenation operation, and head j represents the weighted result calculated by the jth attention head, j = 1, 2, 3, ..., h, h represents the number of attention heads, W O represents the weight matrix of the output linear layer, Attention(·) represents the single-head attention mechanism operation, Q represents the query matrix, K represents the key matrix, V represents the value matrix, softmax(·) represents the normalized exponential function, d k represents the dimension parameter of the key matrix, and T represents the transpose operator.
[0077] Optionally, h is set to 4.
[0078] Optionally, d q =d k =d v =64, where d q Represents the dimension parameter of the query matrix, d v Represents the dimension parameter of the value matrix.
[0079] S502: Calculate the multi-head cross-attention mechanism operation result between the anchor point feature and the context feature according to the multi-head self-attention mechanism operation result between each anchor point feature:
[0080]
[0081] Among them, F” represents the result of the multi-head cross attention mechanism operation between the anchor feature and the context feature. represents the i-th context feature.
[0082] S503: Generate multiple convolution kernels through a feedforward neural network according to the operation results of the multi-head cross attention mechanism between the anchor point features and the context features.
[0083] In one possible implementation, multiple convolution kernels are generated through a feedforward neural network according to the following formula:
[0084] W i =FC(F″)
[0085] Among them, W i represents the convolution kernel generated for the i-th anchor feature, and FC(·) represents the feedforward neural network.
[0086] Optionally, the feedforward neural network consists of two linear transformation layers and a non-linear activation function ReLU.
[0087] It should be noted that the Feedforward Neural Network (FNN) is a basic artificial neural network architecture, in which information is passed from the input layer to the output layer through multiple hidden layers, and data flows unidirectionally between layers without feedback connections. Each neuron processes the input through a weighted and nonlinear activation function (such as ReLU or Sigmoid) and passes the result to the next layer. Feedforward neural networks are usually composed of multiple linear transformation layers and nonlinear activation functions to capture the characteristics and patterns of input data, and are widely used in tasks such as classification, regression, and feature extraction.
[0088] S504: Obtain mask features based on the multi-scale features of the query scene:
[0089] F mask =MLP(F query )
[0090] Among them, F mask represents mask features, MLP(·) represents multi-layer perceptron network, and F query Representing multi-scale features of the query scene.
[0091] Specifically, the multi-scale features of the query scene are input into a mask feature generation module to obtain mask features.
[0092] S505: Perform convolution operation according to the mask features and each convolution kernel to obtain a predicted instance mask.
[0093] Among them, the prediction instance mask is specifically:
[0094]
[0095] in, represents the predicted instance mask corresponding to the i-th anchor feature, Conv(·) represents the dynamic convolution operation, and F mask represents the mask feature, W i Represents the convolution kernel generated for the i-th anchor feature.
[0096] S506: Performing few-sample instance segmentation on the point cloud data of the query scene according to the predicted instance mask.
[0097] In the present invention, by calculating the operation results of the multi-head self-attention mechanism between each anchor point feature, the anchor point feature relationship is captured to help the model understand the target structure and improve the segmentation accuracy. The operation results of the multi-head cross-attention mechanism between the anchor point feature and the context feature are calculated to enhance the context understanding and improve the accuracy and reliability of the few-sample instance segmentation. The convolution kernel generated according to the mechanism can be dynamically adjusted to adapt to different target features and improve the few-sample instance segmentation effect. The multi-scale features of the query scene are input into the mask feature generation module to effectively extract the relevant mask features and provide subsequent few-sample instance segmentation guidance. The mask features and convolution kernels are used to perform dynamic convolution operations to ensure that the predicted instance mask accurately reflects the target contour and improve the segmentation level. Finally, the few-sample instance segmentation of the point cloud data of the query scene is performed by predicting the instance mask, simplifying the processing flow, enabling the model to complete the task efficiently and quickly, and improving the overall performance.
[0098] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0099] In the present invention, global feature extraction is performed on the point cloud data of the query scene and the support scene respectively, multi-scale features of the query scene and the support scene are obtained, feature aggregation is performed on the multi-scale features of the query scene and the support scene, a plurality of anchor point features between the query scene and the support scene are determined, and according to each anchor point feature, a few-sample instance segmentation is performed on the point cloud data of the query scene. Assembly defect detection is no longer performed only on a specific model, and it can be directly applied to other models of aircraft engines, thereby avoiding the phenomenon of unsatisfactory detection effect due to differences in type, shape and size of parts between the test model and the training model.
[0100] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a system for segmenting a few-sample instances of aircraft engine point cloud data provided by the present invention.
[0101] The present invention further provides a system 30 for segmenting a few-sample instances of aircraft engine point cloud data, comprising: a memory 303 and one or more processors 301 .
[0102] One or more application programs are stored in the memory 303 , and the one or more application programs are suitable for being executed by the one or more processors 301 to implement the method for few-sample instance segmentation of aircraft engine point cloud data described in the method embodiment.
[0103] The aircraft engine point cloud data few-sample instance segmentation system 30 comprises: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302.
[0104] The structure of the aircraft engine point cloud data few-sample instance segmentation system 30 does not constitute a limitation on the embodiment of the present invention.
[0105] Processor 301 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0106] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI bus or an EISA bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0107] The memory 303 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.
[0108] It should be noted that the aircraft engine point cloud data few-sample instance segmentation system 30 can implement the above-mentioned aircraft engine point cloud data few-sample instance segmentation method and can achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.
[0109] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0110] In the present invention, global feature extraction is performed on the point cloud data of the query scene and the support scene respectively, multi-scale features of the query scene and the support scene are obtained, feature aggregation is performed on the multi-scale features of the query scene and the support scene, a plurality of anchor point features between the query scene and the support scene are determined, and according to each anchor point feature, a few-sample instance segmentation is performed on the point cloud data of the query scene. Assembly defect detection is no longer performed only on a specific model, and it can be directly applied to other models of aircraft engines, thereby avoiding the phenomenon of unsatisfactory detection effect due to differences in type, shape and size of parts between the test model and the training model.
[0111] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be loaded and executed by a processor according to the method for segmenting a few-sample instances of aircraft engine point cloud data as described in the first aspect.
[0112] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0113] There are a few points to note:
[0114] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.
[0115] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.
[0116] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.
[0117] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for segmenting a few-sample instances of aircraft engine point cloud data, characterized in that: include: S1: Acquire point cloud data of an aircraft engine, wherein the point cloud data includes point cloud data of a query scene and point cloud data of a support scene; S2: labeling the point cloud data of the supporting scene; S3: performing global feature extraction on the point cloud data of the query scene and the support scene respectively to obtain multi-scale features of the query scene and the support scene; S4: performing feature aggregation on the multi-scale features of the query scene and the support scene, and determining a plurality of anchor point features between the query scene and the support scene through a feature similarity measurement calculation module based on a multi-layer perceptron network; S5: According to the features of each anchor point, a Transformer-based convolution kernel generator is used to perform few-sample instance segmentation on the point cloud data of the query scene.
2. The method for segmenting aerospace engine point cloud data with a few samples according to claim 1, characterized in that: The S2 is specifically: The point cloud data of the supporting scene is labeled, and it is ensured that the query scene contains point cloud data of the same category as the labeled point cloud data in the supporting scene.
3. The method for segmenting aeroengine point cloud data with a few samples according to claim 1, characterized in that: The S4 specifically includes: S401: performing feature aggregation on the multi-scale features of the query scene and the multi-scale features of the support scene to obtain context features: F context =MLP(W agg *[F query ⊙f support ; F query -f support ; F query ])in, Represents context features, represents a real number set, MLP(·) represents a multi-layer perceptron network, represents the linear layer weight matrix, D represents the feature dimension, * represents the convolution operation, [·;·] represents the aggregation operation, and F query represents the multi-scale features of the query scene, ⊙ represents the element-by-element multiplication operation, and f support Representation supports multi-scale features of the scene; S402: Perform farthest point sampling on the context feature to obtain candidate point features of the query scene: in, represents the candidate point feature obtained by sampling the farthest point of the i-th context feature, FPS(·) represents the farthest point sampling operation, represents the i-th context feature, and N represents the number of farthest point sampling points; S403: Inputting the candidate point features into a feature similarity measurement calculation module based on a multi-layer perceptron network to calculate a similarity score between the candidate point features and the multi-scale features of the support scene; S404: Filter out candidate point features whose similarity scores are greater than a similarity score threshold as the anchor point features.
4. The method for segmenting a few-sample instances of aircraft engine point cloud data according to claim 3, characterized in that: The similarity score is specifically: Among them, Sim_Sore represents the similarity score.
5. The method for segmenting aeroengine point cloud data with a few samples according to claim 3, characterized in that: The similarity score threshold is set to 0.
5.
6. The method for segmenting aeroengine point cloud data with a few samples according to claim 1, characterized in that: The S5 specifically includes: S501: Calculate the multi-head self-attention mechanism operation results between the anchor point features through the Transformer-based convolution kernel generator: Among them, F' represents the result of the multi-head self-attention mechanism operation between anchor features, MultiHeadAttention represents the multi-head attention mechanism operation, Q represents the query matrix, represents the i-th anchor feature, K represents the key matrix, and V represents the value matrix; S502: Calculate the multi-head cross-attention mechanism operation result between the anchor point feature and the context feature according to the multi-head self-attention mechanism operation result between each of the anchor point features: Among them, F” represents the result of the multi-head cross attention mechanism operation between the anchor feature and the context feature. represents the i-th context feature; S503: Generate multiple convolution kernels through a feedforward neural network according to the multi-head cross attention mechanism operation result between the anchor point feature and the context feature; S504: Obtain mask features according to the multi-scale features of the query scene: F mask =MLP(F query ) Among them, F mask represents mask features, MLP(·) represents multi-layer perceptron network, and F query Representing multi-scale features of the query scene; S505: performing a convolution operation according to the mask feature and each of the convolution kernels to obtain a predicted instance mask; S506: Performing few-sample instance segmentation on the point cloud data of the query scene according to the predicted instance mask.
7. The method for segmenting aeroengine point cloud data with a few samples according to claim 6, characterized in that: The multi-head attention mechanism operation is specifically as follows: MultiHeadAttention=Concat(head1,head2,head3,...,head h )W O head j =Attention(Q,K,V) Among them, MultiHeadAttention represents the multi-head attention mechanism operation, Concat(·) represents the concatenation operation, and head j represents the weighted result calculated by the jth attention head, j = 1, 2, 3, ..., h, h represents the number of attention heads, W O represents the weight matrix of the output linear layer, Attention(·) represents the single-head attention mechanism operation, Q represents the query matrix, K represents the key matrix, V represents the value matrix, softmax(·) represents the normalized exponential function, d k represents the dimension parameter of the key matrix, and T represents the transpose operator.
8. The method for segmenting aeroengine point cloud data with a few samples according to claim 6, characterized in that: The S503 is specifically as follows: According to the following formula, multiple convolution kernels are generated through a feedforward neural network: W i =FC(F″) Among them, W i represents the convolution kernel generated for the i-th anchor feature, and FC(·) represents the feedforward neural network.
9. The method for segmenting aeroengine point cloud data with a few samples according to claim 6, characterized in that: The prediction instance mask is specifically: in, represents the predicted instance mask corresponding to the i-th anchor feature, Conv(·) represents the dynamic convolution operation, and F mask represents the mask feature, W i Represents the convolution kernel generated for the i-th anchor feature.
10. A system for segmenting aircraft engine point cloud data with few samples, characterized in that: include: memory and one or more processors; One or more applications are stored in the memory, and the one or more applications are suitable for being executed by the one or more processors to implement the method for few-sample instance segmentation of aircraft engine point cloud data as described in any one of claims 1 to 9.