An aircraft shape measurement view point generation method based on deep learning and rule paradigm

Abstract

The present invention discloses a method for generating aircraft shape measurement viewpoints based on deep learning and rule paradigms, including: discretizing the aircraft CAD digital model to obtain the aircraft's entire body point cloud; performing semantic segmentation on the aircraft's entire body point cloud based on the deep learning network AP-SSN to obtain the aircraft point clouds of different components; constructing rule paradigms for generating viewpoints for different parts of the aircraft to generate an initial viewpoint set for different parts of the aircraft; merging the viewpoint sets of different parts to form the aircraft's entire body measurement viewpoint set, and comprehensively considering multiple factors such as the number of viewpoints, view quality, and viewpoint redundancy to globally optimize the aircraft's entire body measurement viewpoint set; the present invention realizes the rapid generation of the viewpoint set by introducing a deep learning network combined with targeted rule paradigms; and globally optimizes from multiple angles to improve the rationality and execution efficiency of viewpoint generation. The present invention integrates deep learning, parametric modeling, and rule paradigms, effectively improving the automation degree and intelligent level of measurement viewpoint generation.

Description

Technical Field

[0001] The present invention belongs to the field of aviation digital inspection, and particularly relates to a method for generating measurement viewpoints for aircraft shapes based on deep learning and rule paradigms. Background Art

[0002] As the core product of modern aviation industry, the aircraft has a complex shape structure, numerous components with different forms. Especially during the whole aircraft assembly and manufacturing process, extremely high requirements are put forward for the external dimensions and geometric precision. In order to ensure the consistency of the whole aircraft shape, assembly precision and flight performance, high-precision measurement of the aircraft surface becomes a key link. Traditional measurement methods mainly rely on manual experience and regularly arranged viewpoints, which have problems such as low measurement efficiency, redundant viewpoints, poor repeatability, etc., and are difficult to meet the high-efficiency measurement requirements of modern large-size complex curved surface structures. How to quickly and intelligently generate scientific and reasonable measurement viewpoints according to the geometric structure of the aircraft has become a technical bottleneck to be solved urgently.

[0003] In recent years, with the rapid development of three-dimensional point cloud processing technology and deep learning, new means have been provided for the intelligent analysis and task planning of complex structures. As an important form for describing the geometric shape of an object, the three-dimensional point cloud can completely retain the spatial characteristics of the aircraft surface. By combining a deep learning model to perform semantic segmentation on the whole aircraft point cloud, not only can the accurate identification of different parts of the aircraft be realized, but also an adaptive viewpoint generation strategy can be further constructed based on the structural characteristics, thereby improving the rationality and automation degree of measurement viewpoint generation. Therefore, proposing a method for generating measurement viewpoints for aircraft shapes based on deep learning and rule paradigms can not only improve the viewpoint generation efficiency, but also significantly reduce manual intervention, which has important significance for promoting the intelligent and automated development of aircraft shape measurement for different models. Summary of the Invention

[0004] Aiming at the problems existing in the above-mentioned prior art, the present invention provides a method for generating measurement viewpoints for aircraft shapes based on deep learning and rule paradigms. By performing semantic segmentation on the whole aircraft point cloud data, constructing rule paradigms for different parts of the aircraft to quickly generate a set of measurement viewpoints, and merging and globally optimizing the set of measurement viewpoints for the whole aircraft, the rapid generation of measurement viewpoints for the whole aircraft is realized, the problems of low efficiency, heavy tasks, and cumbersome processes of traditional methods are solved, the automation and intelligence of measurement viewpoint generation for different models are realized, and the reliability of measurement results is ensured.

[0005] To achieve the above technical objectives, the present invention provides the following technical solutions:

[0006] A method for generating measurement viewpoints for aircraft shapes based on deep learning and rule paradigms specifically includes the following steps:

[0007] S1. Convert the aircraft CAD digital model into a triangular mesh, and then perform Poisson sampling on the triangular mesh to obtain the point cloud of the entire aircraft.

[0008] S2. Design a semantic segmentation deep learning network AP-SSN, and use a variety of annotated point clouds of the entire aircraft for training to obtain a trained AP-SSN. Then input the point cloud of the entire aircraft to be processed into the trained AP-SSN for semantic segmentation to obtain the semantic label of each point. Then group and spatially cluster according to the labels to obtain the point clouds of different parts of the aircraft.

[0009] S3. Construct a rule paradigm for generating viewpoints of different parts of the aircraft, and apply the rule paradigm to the point clouds of different parts of the aircraft to generate an initial viewpoint set for different parts.

[0010] S4. Merge the initial viewpoint sets of different parts to form the initial viewpoint set of the entire aircraft, and perform global optimization on the initial viewpoint set of the entire aircraft to obtain the final measurement viewpoint set of the entire aircraft.

[0011] Furthermore, in step S2:

[0012] The internal structure of AP-SSN includes: a sampling module, a neighborhood construction module, a feature extraction module, a feature upsampling module, and a prediction module connected in sequence.

[0013] The specific process of semantic segmentation based on AP-SSN is as follows:

[0014] 1) The input point cloud of the entire aircraft is the original point set First, in the sampling module, a representative point set is selected by the farthest point sampling method ; where respectively represent the three-dimensional spatial coordinates of the th original point and the th representative point, , are the normal vectors of the th original point and the th representative point; N, M are the total number of original points in the point cloud and the total number of representative points obtained by sampling respectively;

[0015] 2) Then, in the neighborhood construction module, for each representative point, a neighborhood point set is obtained through sphere query within a radius of , and a local coordinate feature is constructed; where, is the three-dimensional spatial coordinate and normal vector of the th neighborhood point;

[0016] 3) Then, in the feature extraction module, for all local features within each neighborhood Perform shared MLP, then perform max pooling within the neighborhood, and aggregate the features of each representative point , which is expressed by the formula:

[0017] ;

[0018] After multi-layer feature extraction, i.e., MLP and pooling operations, a pyramidal sparse feature map is formed, and finally a sparse feature set is obtained ;

[0019] 4) After obtaining the sparse point cloud feature set, in the feature upsampling module, the sparse point cloud features are propagated back to the original point cloud through interpolation, and the final fused features of the original points are obtained; specifically:

[0020] For each original point, select 3 nearest representative points based on KNN for feature interpolation; first define the interpolation weights based on distance weighting, then normalize the weights, and calculate the interpolation features; then fuse the interpolation features and the early shallow features of the original points to get; the formula is expressed as:

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] Among them, is the interpolation weight between the s-th nearest representative point and the i-th original point, is the corresponding normalized interpolation weight; , are the interpolation feature and the final fused feature of the i-th original point respectively; is the early shallow feature of the i-th original point, which is a set of preliminary encoded features obtained by the original point passing through the shared MLP mechanism; is a very small positive number used to prevent the denominator from being 0;

[0026] 5) Finally, in the prediction module, predict the semantic category of each point, specifically:

[0027] For the final fused feature of each point , use shared MLP and Dropout to obtain a high-dimensional representation, and the formula is expressed as:

[0028] ;

[0029] Then, through the Softmax classifier, output the probability distribution of each category ;

[0030] wherein represents the probability that the -th point belongs to the -th class; finally, the class corresponding to the maximum probability is taken as the predicted class ;

[0031] The final overall output result is , that is, the aircraft part category corresponding to each point is predicted.

[0032] Furthermore, step S3 specifically includes:

[0033] S31. Abstract different aircraft parts into different geometric shapes;

[0034] S32. Construct corresponding viewpoint generation rule paradigms for aircraft parts of different geometric shapes;

[0035] S33. Generate an initial viewpoint set for the point clouds of different parts of the aircraft according to the viewpoint generation rule paradigm.

[0036] Furthermore, step S31 is specifically:

[0037] Abstract the aircraft nose and tail into cones, and abstract the aircraft fuselage, left engine, and right engine into cylinders; abstract the left wing, right wing, vertical tail, left horizontal tail, and right horizontal tail of the aircraft into planes.

[0038] Furthermore, the viewpoint generation rule paradigm in step S32 includes a cone viewpoint generation rule paradigm, a cylinder viewpoint generation rule paradigm, and a plane viewpoint generation rule paradigm, specifically:

[0039] The cone viewpoint generation rule paradigm is:

[0040] Let the center of the bottom surface of the cone be , the main axis direction be , the height be , and the bottom radius be , then the parametric cone model is expressed as:

[0041] ;

[0042] wherein, the circumferential direction coordinate value , the main axis direction coordinate value , is the main axis length; and are two orthogonal unit vectors perpendicular to the main axis direction ;

[0043] Let the number of viewpoints in the circumferential direction of the cone be , the number of axial view points , then the discrete step size of coordinate values is:

[0044] ;

[0045] Denote any sampling point as , and the corresponding coordinate values of the surface points of the cone are respectively , , and the corresponding surface point of the cone is expressed as , and the surface normal is: , Denote normalization;

[0046] Suppose the expected viewing point is at a distance of from the surface of the cone, then the corresponding viewing point position is , and the viewing direction is ;

[0047] The generation rule paradigm of the cylinder viewing point is:

[0048] Suppose the main axis direction of the cylinder is , the length is , and the radius is , then the parametric cylinder model is:

[0049] ;

[0050] Among them, is the center point of the bottom surface of the cylinder, the coordinate value in the circumferential direction is , the coordinate value in the main axis direction is , and are the circumferential basis vectors orthogonal to the main axis; Given that the number of viewing points in the circumferential direction of the cylinder is , and the number of viewing points in the axial direction is , then the discrete step size of coordinate values is:

[0051] ;

[0052] Denote any sampling point as , and the corresponding coordinate values of the surface points of the cylinder are , and the corresponding surface point of the cylinder is: , and the surface normal vector of the surface point is: ;

[0053] Suppose the expected viewing point is at a distance of from the surface of the cylinder, then the corresponding viewing point position is: ; The viewing direction is ;

[0054] The generation rule paradigm of the plane viewing point:

[0055] Let the plane parametric model be:

[0056] ;

[0057] where, is the plane starting point, and are the unit vectors in two directions of the plane, , are the coordinate value ranges in two directions; Let the number of viewpoints in two directions of the plane be and respectively, then the discrete step length of the coordinate value is:

[0058] ;

[0059] Denote any sampling point as , and its corresponding plane point coordinate value is , , and the corresponding plane point is expressed as , and the plane point normal vector is ;

[0060] Let the expected distance of the viewpoint from the plane be , then the corresponding viewpoint position is: , and the corresponding viewpoint orientation is .

[0061] Furthermore, step S4 specifically includes:

[0062] S41. Combine the viewpoint sets of different parts of the aircraft to form the aircraft's overall measurement viewpoint set;

[0063] S42. Comprehensively consider the number of viewpoints, view quality, and viewpoint redundancy, construct an objective function to globally optimize the aircraft's overall measurement viewpoint set, and combine an optimization algorithm to solve for the optimal viewpoint set to obtain the final aircraft's overall measurement viewpoint set.

[0064] Furthermore, the specific process of constructing the objective function in step S42 is:

[0065] Comprehensively consider minimizing the number of viewpoints, maximizing the coverage rate, minimizing redundancy, minimizing the invisible area, minimizing the viewpoint collision risk, and maximizing the scanning quality to construct the objective function , and select the optimal subset from the aircraft's overall measurement viewpoint set with minimizing the objective function as the evaluation criterion; The formula of the objective function is expressed as:

[0066] ;

[0067] where, respectively represent the view point quantity evaluation function, the coverage rate evaluation function, the redundancy evaluation function, the invisible area evaluation function, the collision risk evaluation function, and the scanning quality evaluation function; are the weight parameters corresponding to each evaluation function, which are adjusted according to actual needs, where is positive, is negative;

[0068] The specific formulas of each evaluation function are as follows:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] Among them, represents the number of view points in the subset; represents the number of view points in the aircraft's overall measurement view point set; is the number of points in the aircraft point cloud, represents being the number of points scanned by at least one view point in; is the number of times each point is seen by multiple view points; represents the number of invisible points; is the closest distance between each view point and the model surface, is the safety distance; is an indicator function indicating whether the safety distance is violated; is the angle between the view point direction and the point cloud normal, is the view point and the target point distance, is the distance attenuation factor.

[0076] Furthermore, in step S42, the specific process of solving the optimal view point set by combining optimization algorithms is as follows:

[0077] Use the particle swarm optimization algorithm to solve the best view point set , ; The specific process is as follows:

[0078] Ⅰ. Initialize the particle swarm, and each particle represents a selection method of a candidate view point combination; use a 0 / 1 encoded particle position vector , assume that the th particle vector is , where represents the selection of the th viewpoint, represents the non-selection of the th viewpoint. The length of the particle vector is equal to the number of candidate viewpoints, being the total number of viewpoints; for the th particle vector at the th viewpoint, the particle velocity is the probability tendency for the th viewpoint to be selected;

[0079] Ⅱ. Calculate the fitness ;

[0080] where is the viewpoint combination vector represented by the th particle at the th iteration, is the set of selected viewpoints corresponding to the viewpoint combination vector , is the fitness calculation function, is the objective function for viewpoint evaluation;

[0081] Ⅲ. Update the individual historical optimal position. Each particle saves the historical best position and the best value. If , then , that is, if the current particle finds a better viewpoint combination than the historical one, update its individual optimal viewpoint combination record; where is the historical optimal position of the th particle up to the th iteration;

[0082] Ⅳ. Update the global optimum. The viewpoint combination with the best objective function among all particles is selected as the current global optimum combination , then the global optimum combination at the th iteration is:

[0083] ;

[0084] where is the total number of particles;

[0085] Ⅴ. Update the particle velocity to control the current viewpoint combination to explore the local optimum and search for the global optimum simultaneously;

[0086] For the th particle vector at the th viewpoint , its formula is expressed as:

[0087] ;

[0088] where is the inertia weight, , are the individual and swarm learning factors, , are random numbers, , represents the -th iteration, is the -th iteration, the -th velocity of the -th view point of the -th particle, is the -th iteration, the -th position of the -th particle, is the -th iteration, the -th historical best position of the -th particle, is the -th global best position of the

[0089] VI. Update the particle position, that is, update the view point combination ; First, map the particle velocity to probability , then, sample according to the random number. If , then , otherwise ;

[0090] VII. Perform boundary control and feasibility verification;

[0091] VIII. Repeat multiple iterations until the maximum iteration number is reached, and output the final global best combination , so as to obtain the optimal view point set .

[0092] Based on the above technical solution, the present invention has at least the following beneficial effects:

[0093] This method adopts a strategy that combines deep learning with a rule paradigm. First, it uses a semantic segmentation network to achieve intelligent segmentation of the point cloud of the entire aircraft; subsequently, based on a preset geometric rule paradigm, it generates an initial view point set that covers comprehensively and is reasonably arranged for different parts of the aircraft; finally, it screens, fine-tunes, and merges the view points through a global optimization algorithm to achieve the generation of efficient and low-redundancy measurement view points for the entire aircraft. The method proposed by the present invention realizes the full-process automatic view point generation from intelligent recognition to efficient planning, effectively improving the accuracy and efficiency of the external shape measurement of aircraft of different models. Brief Description of the Drawings

[0094] Figure 1 It is a flowchart of a method for generating viewpoints for aircraft shape measurement based on deep learning and rule paradigms proposed by the present invention;

[0095] Figure 2 It is a flowchart of point cloud segmentation of different parts of an aircraft based on the semantic segmentation AP - SNN network in the method proposed by the present invention;

[0096] Figure 3 It is a schematic diagram of creating a rule paradigm for generating viewpoints for different parts of an aircraft in the method proposed by the present invention. Detailed Embodiment

[0097] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the following further elaborates on the present invention in combination with the attached Figures 1 - 3 drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0098] Although the steps in the present invention are numbered, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It can be understood that the term "and / or" used herein relates to and encompasses any and all possible combinations of one or more of the associated listed items.

[0099] As Figure 1 and 2 shown, it shows a flowchart of a method for generating viewpoints for aircraft shape measurement based on deep learning and rule paradigms proposed by the present invention, which specifically includes the following steps:

[0100] S1. Convert the aircraft CAD digital model into a triangular mesh, and then perform Poisson sampling on the triangular mesh to obtain the point cloud of the entire aircraft;

[0101] In this embodiment, assume that the CAD model is represented as a set of surfaces , , where each is a parametric surface, and the parameter ;

[0102] Discretize each into a set of triangular patches: , That is , where is the triangulation function;

[0103] Perform Poisson sampling on the triangular mesh to generate a point set, that is, obtain the point cloud of the whole aircraft wherein, represents Poisson sampling; is the minimum allowable point spacing, which controls the sampling density.

[0104] S2. Design a semantic segmentation deep learning network AP-SSN, and use a variety of labeled whole-aircraft point clouds for training to obtain a trained AP-SSN; then input the whole-aircraft point cloud to be processed into the trained AP-SSN for semantic segmentation to obtain the semantic label of each point; then group and spatially cluster according to the labels to obtain the point clouds of different parts of the aircraft; in this application, a variety of aircraft point cloud datasets are prepared in advance, and different semantic categories are defined for the points belonging to different parts of the aircraft, such as the nose, fuselage, wings, etc., so as to train the semantic segmentation model AP-SSN. During the training process, by continuously optimizing parameters and debugging the model, the network is enabled to automatically extract geometric and spatial features from the original point cloud and map each point to its corresponding aircraft part category with non-linear semantic understanding ability;

[0105] As a preferred embodiment, in step S2:

[0106] As shown in Figure 2 , the internal structure of AP-SSN includes: a sampling module, a neighborhood construction module, a feature extraction module, a feature upsampling module, and a prediction module connected in sequence; this application designs AP-SSN using the ideas of sparse coding, multi-scale feature extraction, and sparse feature backpropagation. Using sparse feature coding can greatly reduce memory and computational complexity; while neighborhood features and pyramid structures can take into account local details and large-scale structures, which are suitable for complex point clouds such as aircraft; finally, through interpolation and feature fusion to return to the original dense point cloud, fine-grained prediction can be achieved to ensure clear boundary details;

[0107] The specific process of semantic segmentation based on AP-SSN is as follows:

[0108] 1) The input whole-aircraft point cloud is the original point set First, use the farthest point sampling method in the sampling module to screen out the representative point set wherein respectively represent the three-dimensional spatial coordinates of the th original point and the th representative point, , are the normal vectors of the th original point and the th representative point; N, M are the total number of original points in the original point cloud and the total number of representative points obtained by sampling respectively;​

[0109] 2) Then, in the neighborhood construction module, for each representative point, within a radius , obtain the neighborhood point set through sphere query , and construct the local coordinate feature ; where is the three-dimensional spatial coordinate and normal vector of the th neighborhood point;

[0110] 3) After that, in the feature extraction module, perform shared MLP on all local features within each neighborhood, and then perform max pooling within the neighborhood to aggregate the features of each representative point , which is expressed by the formula:

[0111] ;

[0112] Through multi-layer feature extraction, namely MLP and pooling operations, a pyramidal sparse feature map is formed, and finally a sparse feature set is obtained;

[0113] Here, it should be noted that in this application, choosing to transform local features into sparse features is to use sparse and few representative points to compress and enhance the local geometric information of the original point cloud, gradually construct higher-level global structure features, and at the same time greatly reduce the computational complexity, which is convenient for subsequent interpolation back to the original point cloud for segmentation;

[0114] 4) After obtaining the sparse point cloud feature set, in the feature upsampling module, propagate the sparse point cloud features back to the original point cloud through interpolation to obtain the final fused features of the original points; specifically:

[0115] For each original point, select 3 nearest representative points based on KNN for feature interpolation; first, define the interpolation weights based on distance weighting, then normalize the weights, and calculate the interpolation features; then fuse the interpolation features and the early shallow features of the original points to obtain; which is expressed by the formula:

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] Among them, is the interpolation weight between the th nearest representative point and the nd They are the interpolation feature and the final fusion feature of the i-th original point respectively; is the early shallow feature of the i-th original point, which is a set of preliminary encoded features obtained by the original point passing through the shared MLP mechanism. It is close to the original geometry and rich in local details. In this embodiment, it is used to make up for the detail loss of the high-level features during feature upsampling, making the network output more accurate; is a very small positive number used to prevent the denominator from being zero;

[0121] 5) Finally, in the prediction module, predict the semantic category of each point, specifically:

[0122] For the final fusion feature of each point , use the shared MLP and Dropout to obtain a high-dimensional representation, and the formula is expressed as:

[0123] ;

[0124] Then, through the Softmax classifier, output the probability distribution of each category ;

[0125] Among them , represents the probability that the -th point belongs to the -th category; finally, take the category corresponding to the maximum probability as the predicted category ;

[0126] The final overall output result is , that is, each point predicts its corresponding aircraft part category.

[0127] In addition, in this embodiment, during the training of AP-SSN, the cross-entropy loss function is used to measure the prediction error of each point; the formula of the loss function is expressed as:

[0128] ;

[0129] Among them, is the predicted probability distribution, is the true label;

[0130] Use the Adam optimizer to update the network parameters: , where is the network weight, is the learning rate, is the gradient of the loss function with respect to the parameter.

[0131] After training the deep learning network, input the point cloud of the whole aircraft to be processed, and through two stages of label grouping and stewardess clustering, obtain the point cloud of different parts of the aircraft; as follows:

[0132] Label grouping stage: Extract points of the same type of labels to form component-level point clouds; for each type of label, extract all points of this category to form a set, and obtain sub-point clouds of 6 categories: , corresponding to the nose, fuselage, wings, tail, engine, and empennage respectively;

[0133] Spatial clustering stage: Divide the structural areas of point clouds of the same type, and distinguish the structures with spatial distribution differences in each type of point cloud; in this embodiment, first assign category labels to each point through the semantic segmentation deep learning network AP-SSN to distinguish six sub-categories of the aircraft; then further subdivide through spatial clustering, such as separating the left and right wings in the wings; after the above operations, finally obtain point clouds of different parts of the aircraft.

[0134] S3. Construct a rule paradigm for generating viewpoints of different parts of the aircraft, and apply the rule paradigm to the point clouds of different parts of the aircraft to generate an initial viewpoint set for different parts;

[0135] As a preferred implementation method, step S3 specifically includes:

[0136] S31. Abstract different parts of the aircraft into different geometric shapes; in this embodiment, abstract the nose and tail of the aircraft into cones, abstract the fuselage, left engine, and right engine of the aircraft into cylinders; abstract the left wing, right wing, vertical tail, left horizontal tail, and right horizontal tail of the aircraft into planes.

[0137] S32. Construct corresponding viewpoint generation rule paradigms for different parts of the aircraft with different geometric shapes;

[0138] More specifically, as Figure 3 shown, the viewpoint generation rule paradigm includes a cone viewpoint generation rule paradigm, a cylinder viewpoint generation rule paradigm, and a plane viewpoint generation rule paradigm, specifically:

[0139] The cone viewpoint generation rule paradigm is:

[0140] Let the center of the bottom surface of the cone be , the main axis direction be , the height be , and the bottom radius be , then the parametric cone model is expressed as:

[0141] ;

[0142] Among them, the circumferential direction coordinate value , the main axis direction coordinate value , is the main axis length; and are perpendicular to the main axis direction Two orthogonal unit vectors;

[0143] Let the number of viewing points in the circumferential direction of the cone be , and the number of viewing points in the axial direction be . Then the discrete step size of the coordinate value is:

[0144] ;

[0145] Denote any sampling point as , and the corresponding coordinate values of the cone surface point are respectively , . The corresponding cone surface point is expressed as , and the surface normal is: , Indicates normalization;

[0146] Let the expected viewing point be at a distance of from the cone surface. Then the corresponding viewing point position is , and the viewing direction is ;

[0147] The viewing point generation rule paradigm for the cylinder is:

[0148] Let the main axis direction of the cylinder be , the length be , and the radius be . Then the parameterized cylinder model is:

[0149] ;

[0150] Among them, is the center point of the bottom surface of the cylinder, the circumferential direction coordinate value is , the main axis direction coordinate value is , and are the circumferential basis vectors orthogonal to the main axis; Given the number of viewing points in the circumferential direction of the cylinder is , and the number of viewing points in the axial direction is . Then the discrete step size of the coordinate value is:

[0151] ;

[0152] Denote any sampling point as , and the corresponding coordinate values of the cylinder surface point are . The corresponding cylinder surface point is: , and the surface point normal vector is: ;

[0153] Let the expected viewing point be at a distance of from the cylinder surface. Then the corresponding viewing point position is: ; The viewpoint orientation is ;

[0154] Plane viewpoint generation rule paradigm:

[0155] Let the plane parametric model be:

[0156] ;

[0157] Among them, is the plane starting point, , are the unit vectors of the plane in two directions, , are the coordinate value ranges of the two directions; Let the number of viewpoints in the two directions of the plane be and respectively, then the discrete step length of the coordinate value is:

[0158] ;

[0159] Denote any sampling point as , and the corresponding plane point coordinate value is , , and the corresponding plane point is expressed as , and the plane point normal vector is ;

[0160] Let the expected distance of the viewpoint from the plane be , then the corresponding viewpoint position is: , and the corresponding viewpoint orientation is .

[0161] S33. According to the viewpoint generation rule paradigm, generate an initial viewpoint set for the point clouds of different parts of the aircraft; In this embodiment, according to the geometric shapes abstracted from different parts in step S31, generate an initial viewpoint set for the nose and tail of the aircraft according to the cone viewpoint generation rule paradigm, and denote them as respectively; Generate an initial viewpoint set for the fuselage, left engine, and right engine of the aircraft according to the cylinder viewpoint generation rule paradigm, and denote them as respectively; Generate an initial viewpoint set for the left wing, right wing, vertical tail, left horizontal tail, and right horizontal tail of the aircraft according to the plane viewpoint generation rule paradigm, and denote them as respectively. So far, the present application designs different viewpoint generation rule paradigms for the geometric shapes of different parts of the aircraft, and obtains an initial viewpoint set that covers all parts of the aircraft and has a reasonable layout as shown in Figure 3 (the direction of the viewpoint is from the apex of the visible cone to the center of the bottom circle of the visible cone).

[0162] S4. Combine the initial view point sets of different parts to form the initial view point set of the whole aircraft, and perform global optimization on the initial view point set of the whole aircraft to obtain the final measurement view point set of the whole aircraft;

[0163] As a preferred embodiment, step S4 specifically includes:

[0164] S41. Combine the view point sets of different parts of the aircraft to form the measurement view point set of the whole aircraft; in this embodiment, all the initial view point sets generated for different parts of the aircraft are combined to form the measurement view point set of the whole aircraft, denoted as , ;

[0165] S42. S42. Considering the number of view points, view quality and view point redundancy comprehensively, construct an objective function to perform global optimization on the measurement view point set of the whole aircraft, and combine the optimization algorithm to solve the optimal view point set to obtain the final measurement view point set of the whole aircraft;

[0166] In this embodiment, the specific process of constructing the objective function is as follows:

[0167] Considering the minimization of the number of view points, maximization of coverage, minimization of redundancy, minimization of invisible areas, minimization of view point collision risk, and maximization of scanning quality comprehensively, construct the objective function , and select the optimal subset from the measurement view point set of the whole aircraft with the minimization of the objective function as the evaluation criterion; the formula of the objective function is expressed as:

[0168] ;

[0169] Among them, respectively represent the view point number evaluation function, coverage evaluation function, redundancy evaluation function, invisible area evaluation function, collision risk evaluation function, and scanning quality evaluation function; for the objective function , the smaller the better; the smaller it is, the better the coverage, and the ideal value is 0; the smaller the better, to avoid repeated scanning of the same area by multiple view points; the smaller the better; the smaller the better; the higher the better; are the weight parameters corresponding to each evaluation function, which are adjusted according to actual needs, where is positive, is negative;

[0170] The specific formulas of each evaluation function are as follows:

[0171] ;

[0172] ;

[0173] ;

[0174] ;

[0175] ;

[0176] ;

[0177] Among them, represents the number of viewpoints in the subset; represents the number of viewpoints in the aircraft's overall measurement viewpoint set; is the number of points in the aircraft point cloud, represents the number of points scanned by at least one viewpoint in ; is the number of times each point is seen by multiple viewpoints; represents the number of invisible points; is the closest distance from each viewpoint to the model surface, is the safety distance; is an indicator function indicating whether the safety distance is violated. If , it means there is a potential collision risk for this viewpoint, and the indicator function will output 1; otherwise, it outputs 0; is the angle between the viewpoint direction and the point cloud normal, is the viewpoint and the target point distance, is the distance attenuation factor.

[0178] After constructing the objective function as above, combined with the optimization algorithm to solve the optimal viewpoint set, the specific process is as follows:

[0179] Use the particle swarm optimization algorithm to solve the best viewpoint set , ; The specific process is as follows:

[0180] Ⅰ. Initialize the particle swarm, and each particle represents a selection method for a candidate viewpoint combination; Use a 0 / 1 encoded particle position vector , assuming the th particle vector is , among which, represents the selection of the th viewpoint, represents not selecting the th viewpoint. The length of the particle vector is equal to the number of candidate viewpoints, is the total number of viewpoints; For the th particle vector at the viewpoints, particle speed For the The probability tendency of a viewpoint being selected;

[0181] II. Calculating Fitness ;

[0182] in, For the The first iteration The combined viewpoint vector represented by the particles, is the viewpoint combination vector In corresponds to the selected viewpoint set, is the fitness calculation function, is the objective function for viewpoint evaluation;

[0183] III. Update the individual best historical position. Each particle saves the best historical position and best value. If ,but , that is, if the current particle finds a better viewpoint combination than the historical one, then update its individual optimal viewpoint combination record; where, It is to Until the iteration The historical optimal position of a particle;

[0184] IV. Update the global optimum. The viewpoint combination with the best objective function among all particles is selected as the current global optimal combination. , then The global optimal combination at the iteration is:

[0185] ;

[0186] in, is the total number of particles;

[0187] V. Update particle speed and control the current viewpoint combination to explore both local optimum and global optimum;

[0188] For The particle vector Viewpoints , its formula is expressed as:

[0189] ;

[0190] in is the inertia weight, , is the individual and group learning factor, , is a random number, , Indicates Iterations, is the velocity of the th iteration of the th particle at the th viewing point, is the position of the th iteration of the th particle at the th viewing point, is the historical optimal position of the th iteration of the th particle at the th viewing point, is the global optimal position of the th iteration of the th viewing point;

[0191] Ⅵ. Update the particle position, that is, update the viewing point combination ; First, map the particle velocity to probability , then, according to random number sampling, if , then , otherwise ;

[0192] Ⅶ. Perform boundary control and feasibility verification;

[0193] Ⅷ. Repeat multiple iterations until the maximum number of iterations is reached, and output the final global optimal combination , so as to obtain the best viewing point set .

[0194] So far, the optimal viewing point set for aircraft shape measurement has been obtained, which simultaneously satisfies the requirements of the fewest viewing points, the most complete coverage, the lowest redundancy, the fewest invisible areas, the lowest collision risk, and the highest scanning quality.

[0195] In summary, the method proposed by the present invention combines multiple technical means such as deep learning, parametric modeling, and rule paradigms, effectively improving the automation degree and intelligent level of measurement viewing point generation.

[0196] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0197] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An aircraft shape measurement view point generation method based on deep learning and rule paradigm, characterized in that Specifically, it includes the following steps: S1. Convert the aircraft CAD digital model into a triangular mesh, and then perform Poisson sampling on the triangular mesh to obtain the aircraft's whole-body point cloud; S2. Design a semantic segmentation deep learning network AP-SSN, and use a variety of annotated aircraft whole-body point clouds for training to obtain the trained AP-SSN; Then input the aircraft whole-body point cloud to be processed into the trained AP-SSN for semantic segmentation to obtain the semantic label of each point; then group and spatially cluster according to the labels to obtain the aircraft point clouds of different parts; S3. Construct a regular paradigm for generating viewpoints of different parts of the aircraft, and apply the regular paradigm to the aircraft point clouds of different parts to generate the initial viewpoint sets of different parts; S4. Merge the initial viewpoint sets of different parts to form the initial viewpoint set of the whole aircraft, and globally optimize the initial viewpoint set of the whole aircraft to obtain the final measurement viewpoint set of the whole aircraft, specifically including: S41. Merge the viewpoint sets of different parts of the aircraft to form the measurement viewpoint set of the whole aircraft; S42. Considering the number of viewpoints, view quality, and viewpoint redundancy comprehensively, construct an objective function to globally optimize the measurement viewpoint set of the whole aircraft. The specific process of constructing the objective function is as follows: Construct an objective function by comprehensively considering minimizing the number of viewpoints, maximizing the coverage rate, minimizing redundancy, minimizing the invisible area, minimizing the risk of viewpoint collision, and maximizing the scanning quality , and select the optimal subset from the aircraft's overall measurement viewpoint set with minimizing the objective function as the evaluation criterion; the formula of the objective function is expressed as: ； Among them, respectively represent the viewpoint quantity evaluation function, the coverage rate evaluation function, the redundancy evaluation function, the invisible area evaluation function, the collision risk evaluation function, and the scanning quality evaluation function; are the weight parameters corresponding to each evaluation function, which are adjusted according to actual needs, where is positive, is negative; The specific formulas of each evaluation function are as follows: ； ； ； ； ； ； Among them, represents the number of viewpoints in the subset; represents the number of viewpoints in the aircraft's overall measurement viewpoint set; is the number of points in the aircraft point cloud, represents the number of points scanned by at least one viewpoint in ; is the number of times each point is seen by multiple viewpoints; represents the number of invisible points; is the closest distance from each viewpoint to the model surface, is the safety distance; is an indicator function indicating whether the safety distance is violated; is the angle between the viewpoint direction and the point cloud normal, is the viewpoint and the target point distance, is the distance attenuation factor; And combine the optimization algorithm to solve the optimal viewpoint set to obtain the final measurement viewpoint set of the whole aircraft.

2. The method for generating aircraft shape measurement viewpoints based on deep learning and rule paradigm according to claim 1, wherein In step S2, The internal structure of AP-SSN includes: a sampling module, a neighborhood construction module, a feature extraction module, a feature upsampling module, and a prediction module connected in sequence; The specific process of semantic segmentation based on AP-SSN is as follows: 1) The input point cloud of the whole aircraft, i.e., the original point set First, use the farthest point sampling method in the sampling module to screen out the representative point set ; where respectively represent the 3D spatial coordinates of the -th original point and the -th representative point, , are the normal vectors of the -th original point and the -th representative point; N, M are the total number of original points in the point cloud and the total number of representative points obtained by sampling respectively; 2) Then, for each representative point in the neighborhood construction module, obtain the neighborhood point set through sphere query within the radius and construct the local coordinate feature ; where is the three-dimensional spatial coordinate and normal vector of the th neighborhood point; ​ 3) Then, in the feature extraction module, for all local features in each neighborhood a shared MLP is performed, and then max pooling within the neighborhood is carried out to aggregate the features of each representative point , which is expressed by the formula: ； After multiple layers of feature extraction, namely MLP and pooling operations, a pyramidal sparse feature map is formed, and finally a sparse feature set is obtained ; 4) After obtaining the sparse point cloud feature set, in the feature upsampling module, propagate the sparse point cloud features back to the original point cloud through interpolation to obtain the final fused features of the original points; specifically: For each original point, select 3 nearest representative points based on KNN for feature interpolation; first define the interpolation weights based on distance weighting, then normalize the weights, and calculate the interpolation features; then fuse the interpolation features and the early shallow features of the original points; the formula is expressed as: ； ； ； ； wherein, is the interpolation weight between the s-th nearest representative point and the i-th original point, , is the corresponding normalized interpolation weight; , are the interpolated feature and the final fused feature of the i-th original point respectively; is the early shallow feature of the i-th original point, which is a set of preliminary encoded features obtained by the original point through the shared MLP mechanism; is a very small positive number used to prevent the denominator from being zero; 5) Finally, in the prediction module, predict the semantic category of each point, specifically: Final fused features for each point , a high-dimensional representation is obtained using a shared MLP and Dropout, which is expressed by the formula: ； Then, it passes through the Softmax classifier to output the probability distribution of each category ; where , indicates the probability that the -th point belongs to the -th class; finally, the class corresponding to the maximum probability is taken as the predicted class ; The final overall output result is , that is, each point predicts its corresponding aircraft part category.

3. A method for generating viewpoints for aircraft shape measurement based on deep learning and rule paradigms according to claim 1, characterized in that, Step S3 specifically includes: S31. Abstract different parts of the aircraft into different geometric shapes; S32. Construct corresponding viewpoint generation regular paradigms for different geometric-shaped aircraft parts; S33. According to the viewpoint generation regular paradigm, generate the initial viewpoint sets for the aircraft point clouds of different parts.

4. A method for generating an aircraft shape measurement view point based on deep learning and rule paradigm according to claim 3, characterized in that, Step S31 is specifically: Abstract the aircraft nose and tail into cones, and abstract the aircraft fuselage, left engine, and right engine into cylinders; abstract the left wing, right wing, vertical tail, left horizontal tail, and right horizontal tail of the aircraft into planes.

5. A method for generating viewpoints for aircraft shape measurement based on deep learning and rule paradigm according to claim 3, characterized in that, In step S32, the viewpoint generation regular paradigm includes the cone viewpoint generation regular paradigm, the cylinder viewpoint generation regular paradigm, and the plane viewpoint generation regular paradigm, specifically: The cone viewpoint generation regular paradigm is: Let the center of the base of the cone be , the main axis direction be , the height be , and the base radius be . Then the parametric cone model is expressed as: ； Among them, the circumferential direction coordinate value , the main axis direction coordinate value , is the main axis length; and are two orthogonal unit vectors perpendicular to the main axis direction . Number of viewing points in the circumferential direction of the cone , number of viewing points in the axial direction , then the discrete step size of the coordinate value is as follows: ； Denote any sampling point as , and the coordinate values of the corresponding points on the conical surface are respectively , . The corresponding points on the conical surface are denoted as , and the surface normal is: , denotes normalization; Let the distance between the desired viewing point and the conical surface be , then the corresponding viewing point position is , and the viewing direction is ; The cylinder viewpoint generation regular paradigm is: Let the main axis direction of the cylinder be , the length be , and the radius be . Then the parametric cylinder model is as follows: ； Among them, is the center point of the bottom surface of the cylinder, and the coordinate value in the circumferential direction is , and the coordinate value in the main axis direction is , and are the circumferential basis vectors orthogonal to the main axis; the number of viewpoints in the circumferential direction of the given cylinder is , and the number of viewpoints in the axial direction is , then the discrete step length of the coordinate value is: ； Denote any sampling point as , and the coordinate value of the corresponding point on the cylinder surface is , and the corresponding point on the cylinder surface is: , and the normal vector of the surface point is: ; Let the distance between the desired viewing point and the surface of the cylinder be , then the corresponding viewing point position is: ; the viewing direction is ; The plane viewpoint generation regular paradigm: Let the plane parametric model be: ； Among them, is the planar starting point, , are the unit vectors of the plane in two directions, , is the coordinate value range in two directions; let the number of viewing points in the two directions of the plane be and respectively, then the discrete step size of the coordinate value is: ； Denote any sampling point as , and the corresponding plane point coordinate values are , , and the corresponding plane point is represented as , and the plane point normal vector is ; Let the distance of the desired viewing point from the plane be , then the corresponding viewing point position is: , and the corresponding viewing point orientation is .

6. A method for generating aircraft shape measurement viewpoints based on deep learning and rule paradigms according to claim 1, characterized in that In step S42, combining the optimization algorithm to solve the optimal viewpoint set is specifically: Using the particle swarm optimization algorithm to solve the optimal viewpoint set , ; The specific process is as follows: Ⅰ. Initialize the particle swarm, where each particle represents a selection method for a candidate viewpoint combination; use 0 / 1 encoding for the position vector of the particle , assume the -th particle vector is , where represents the selection of the -th viewpoint, represents not selecting the -th viewpoint, the length of the particle vector is equal to the number of candidate viewpoints, is the total number of viewpoints; for the -th particle vector and the -th viewpoint, the particle velocity is the probability tendency for the -th viewpoint to be selected; Ⅱ. Calculate fitness ; Among them, is the th iteration, and is the view point combination vector represented by the th particle, is the selected view point set corresponding to the view point combination vector, is the fitness calculation function, is the objective function for view point evaluation. Ⅲ. Update the individual historical optimal position. Each particle saves the historical best position and the best value. If , then , that is, if the current particle finds a better view point combination than the historical one, update its individual optimal view point combination record; where is the historical optimal position of the th particle up to the th iteration; Ⅳ. Update the global optimum. The viewpoint combination with the optimal objective function among all particles is selected as the current global optimum combination , then the global optimum combination at the -th iteration is as follows: ； Among them, is the total number of particles; Ⅴ. Update the particle velocity to control the current viewpoint combination to explore both local and global optima; For the th particle vector, the th viewpoint , its formula representation is: ； wherein is the inertia weight, , are the individual and swarm learning factors, , are random numbers, , denotes the -th iteration, is the velocity of the -th iteration at the -th view point of the -th particle, is the position of the -th iteration at the -th view point of the -th particle, is the historical best position of the -th iteration at the -th view point of the -th particle, is the global best position of the -th iteration at the -th view point; Ⅵ. Update the particle position, i.e., update the viewpoint combination ; First, map the particle velocity to probability , then, sample according to the random number. If , then , otherwise ; Ⅶ. Perform boundary control and feasibility verification; Ⅷ. Repeat the iteration multiple times until the maximum number of iterations is reached , and output the final globally optimal combination , thereby obtaining the best viewpoint set .

Images