An identification method for feature surface pieces of an intelligent manufacturing structural part of an aircraft

Abstract

The application discloses a kind of identification methods of intelligent manufacturing structural parts feature surface sheet of aircraft, the image pre-processing of the aircraft structural parts image intercepted is obtained Training sample and prediction sample of rough identification;It is carried out rough identification model training and sample prediction, and the rough identification result is output.For rough identification data, the training sample and prediction sample of fine identification are obtained by processing;It is carried out fine identification model training and sample prediction, and the accurate identification result is output.For accurate identification result, it is carried out accurate result information original map mapping and feature vector construction.Based on feature vector design measure vector, it is judged to accessory sheet;Identification accessory sheet, and find the label information of corresponding sheet based on the TAG value saved in step S100, the result attribute information of identification is added to three-dimensional model and is displayed.The application can realize the identification judgment of class attribute feature of aircraft structural parts automatic programming sheet under any visual angle, with good practicability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of feature recognition of aviation manufacturing parts, and particularly relates to a method for recognizing feature surfaces of an aircraft intelligent manufacturing structural part. BACKGROUND

[0002] In the entire manufacturing process of an aircraft structural part from a blank to a finished product in the aviation field, machining needs to be completed based on a numerical control (NC) machine tool, and NC program information is the basis and core for realizing accurate machining. How to obtain the required NC program of the aircraft structural part to be machined becomes the focus of the entire work. Currently, in the industry, the generation methods of the NC program mainly include two types of technical means, namely, a manual point selection design method and an automatic recognition generation method. The manual design method mainly selects the required feature surfaces in a three-dimensional modeling software based on a manual method, sets corresponding machining parameters, and then obtains the required NC program. The main implementation process of the automatic recognition generation method is to automatically recognize features through a designed criterion or algorithm, and based on the recognition result of the feature surfaces, the three-dimensional modeling software is combined and corresponding machining parameters are set to obtain the NC program. The main feature surfaces of the common frame and beam structural parts in the aviation field include a web surface, a sidewall surface, a cylindrical surface, a bottom corner surface, and a top surface. How to accurately recognize the work is particularly important. For skilled NC machining program designers, the defects of the manually selected machining feature surfaces are obvious, that is, for a part with multiple feature elements, the efficiency of manually selecting all the feature surfaces is low, and the automation and high efficiency requirements cannot be met.

[0003] To realize automatic acquisition of the NC program, how to accurately judge the surface attribute is an important basis, and the accurate surface attribute information is greatly dependent. Therefore, to realize automatic acquisition of the NC program and reduce human intervention in part programming, accurate feature judgment and recognition of the surface attribute are the key to realizing automatic programming (automatic acquisition of the NC program) and are also a problem to be solved in the industry. Under this background, the application designs an aircraft structural part intelligent manufacturing surface class visual recognition and analysis method, which combines characteristic vector measurement judgment, multi-step judgment and other means to judge the surface attribute and type while favorably ensuring the integrity and accuracy of the surface recognition result. SUMMARY

[0004] The application aims to provide a method for recognizing feature surfaces of an aircraft intelligent manufacturing structural part to solve the problem of low efficiency of manual point selection. The application realizes accurate judgment of the surface type, the recognition result can provide feature data for automatic generation of the NC program, is beneficial to the development of automatic programming, solves the problem of low efficiency of manual point selection, and provides a guiding reference method for recognizing features of an aircraft structural part based on a three-dimensional model. The application is mainly realized through the following technical scheme:

[0005] A method for identifying feature surfaces of an aircraft intelligent manufacturing structural part, comprising the following steps:

[0006] Step S100: design a display mode of an aircraft structural part image, and intercept aircraft structural part images from different perspectives, and save a TAG value corresponding to each image;

[0007] Step S200: image preprocessing is performed on the intercepted images to obtain training samples and prediction samples of rough identification; Step S300: rough identification model training and sample prediction are performed, and a rough identification result is output;

[0008] Step S400: training samples and prediction samples of fine identification are obtained by processing the rough identification data;

[0009] Step S500: fine identification model training and sample prediction are performed, and a fine identification result is output;

[0010] Step S600: for the fine identification result, a feature vector is constructed by mapping the original image of the fine result information; Step S700: based on the feature vector, a measurement vector is designed to judge the attached surface;

[0011] Step S800: the attached surface is identified, and the label information of the corresponding surface is found based on the TAG value saved in step S100, the attribute information of the identified result is added to the three-dimensional model and displayed. Preferably, the attached surface includes a web surface, a large bottom corner surface and a top surface.

[0012] In order to better realize the present application, further, in step S100, the display mode of the aircraft structural part image is designed based on software, and the image is intercepted by an intercepting tool, the first perspective range of the intercepting tool is 0-360 degrees, the second perspective range is 0-360 degrees, the magnification is 3 times, and the transformation interval angle of the perspective is 10 degrees.

[0013] In order to better realize the present application, further, step S200 comprises the following steps:

[0014] Step S210: a blank image image_dst of the same size and monochrome channel as the obtained structural part image is newly created, the gray value of each position pixel point in the structural part image is obtained, and the gray value of the pixel point at the same position of image_dst is transformed;

[0015] Step S220: all pixel points in the image are processed, and then image_dst is operated:

[0016]

[0017] In the formula,

[0018] colums and rows represent the number of columns and rows of the image respectively;

[0019] x and y represent the number of columns and rows of the image respectively;

[0020] h R represents the value corresponding to the R color space at the (x, y) position in the image_dst image;

[0021] max r represents the maximum gray value number set;

[0022] Step S230: curve transformation processing:

[0023] image_dst(x, y) - kernel eight = {min→result(x+x', y+y')(x', y'):kernel eight (x', y')≠0}

[0024] wherein, image_dst(x, y) represents the transformed image;

[0025] kernel eight represents the 8-neighborhood transformation factor;

[0026] result(x, y) represents the pixel value after processing at the (x, y) position in the image;

[0027] x', y' represent the position offset of the number of columns and rows of the image respectively;

[0028] Step S240: standard processing is performed on the feature curve in the image_dst image, so that the thickness of the normal vector direction of the curve along any point is the same.

[0029] In order to better realize the present application, further, in the step S210, when the gray value of the 3 channels is 0 during the gray value transformation, the gray value of the pixel point at the same position in the image_dst is transformed to 0; if the gray values of the 3 channels are not all 0, the gray value of the pixel point at the same position in the image_dst remains unchanged.

[0030] In order to better realize the present application, further, in the step S300, the coarse recognition model obtains feature maps of different resolutions based on multi-scale differences, and obtains feature maps with high resolution and high semantic information by using the upsampling and fusion manner, and then obtains potential regions of interest based on the region candidate network, and performs classification and regression processing on the regions of interest.

[0031] For better implementation of the present application, further, in the step S400, the rough recognition result contains the total number of recognition features corresponding to the image result_all and the coordinate data point(x, y) corresponding to each feature; the coordinate region mapping and transformation processing of the rough recognition result is performed to obtain the sub-face feature image, so as to obtain the training sample and the prediction sample of the fine recognition.

[0032] For better implementation of the present application, further, the step S400 includes the following steps:

[0033] Step S410: Transformation preprocessing of the rough recognition data, reading each rough recognition result region R result , the upper left corner position point and the lower right corner position of the rectangular frame region R result are Point l-result and Point r-result respectively, and the new coordinates of the sample after the transformation processing are as follows:

[0034] Point l-result-new (x, y)→(Point l-result (x)-pro x-round )≥0, (Point l-result (y)-pro y-round )≥0 (1)

[0035] Point r-result-new (x, y)→(Point r-result (x)+pro x-round )<cols, (Point r-result (y)+pro y-round )<

[0036] rows (2)

[0037] Wherein, Point l-result-new (x, y), Point r-result-new (x, y) respectively represent the new coordinate positions of the upper left corner position point and the lower right corner position point of the result frame after the processing, and the corresponding values are positive integers;

[0038] pro x-round and pro y-round respectively represent the scale of the x and y transformation processing;

[0039] cols and rows respectively represent the number of columns and rows of the image_dst image;

[0040] Step S420: Based on the new coordinate data obtained by the transformation, a rectangular sub-image is captured in the image_dst image at the position based on the coordinate mapping. The size of the image is consistent with the size of the processed rectangular area. All rectangular areas in the image are captured in a batch process, and the sub-images {image1, image2, ...imageN} are saved.

[0041] Step S430: Transform the intercepted sub-images, starting from the upper left corner and counting counterclockwise, to obtain the corner coordinates P1(x,y), P2(x,y), P3(x,y), P4(x,y) of each sub-image; use the corner coordinates P1(x,y), P2(x,y), P3(x,y), P4(x,y) of each sub-image as anchor points, iteratively search for points with the same grayscale value as each anchor point at a distance of 1 pixel, and reassign their grayscale values ​​to h. r (x,y)=numb r1 、h g (x,y)=numb g1 、h b (x,y)=numb b1 ;

[0042] Among them, h r (x,y),h g (x,y),h b (x, y) represent the feature values ​​corresponding to the r, g, and b channels of the pixel at position (x, y) in the imageN image;

[0043] numb r1 、numb g1 、numb b1 They are the values ​​set in the r, g, and b channels respectively;

[0044] Step S440: adding random grayscale label information to independent areas outside the non-edge area of ​​the sub-image imageN in different ways based on grayscale differences;

[0045] Step S450: After adding random grayscale label information to the independent area outside the non-edge area of ​​the sub-image, read the sub-image with the grayscale label information except (max r 、max g 、max b )、(numb r1 、numb g1 、numb b1 )Number of different grayscale pixel types gray , and the corresponding grayscale values ​​(gray1 R 、gray1 G 、gray1B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B );

[0046] max r , max g , max b respectively represent the corresponding values of the arbitrary contour line of the sub-image imageN in the r, g, b color space;

[0047] numb r , numb g , numb b respectively represent the set values for judging in the r, g, b channels;

[0048] gray R , gray G , gray B respectively represent the corresponding values of the different gray pixel regions in the r, g, b color space;

[0049] Step S460: Read the gray value of each pixel point in the sub-image respectively, and create a blank image image_new with the same size and corresponding label. The initial color of the blank image image_new is white. Based on whether the gray value of the read pixel point (x, y) belongs to the grouped gray value, new gray information is added to the pixel at the (x, y) position in the image_new, and the grouped gray value corresponds to the obtained images image_new1-1, image_new1-2, … image_new1-n;

[0050] Step S470: After processing each sub-image, a plurality of images containing only one closed region are obtained:

[0051]

[0052] wherein N represents the number of sub-images,

[0053] n represents the number of closed regions corresponding to (max r , max g , max b ), (numb r1 , numb g1 , numb b1 ) in the sub-image.

[0054] For better implementation of the present application, further, in the step S430, the search mode is: taking P1(x, y), P2(x, y), P3(x, y), P4(x, y) as anchor point 1, taking the anchor point 1 as the starting point, judging the gray values of the pixel points which are one pixel distance above, below, left and right of the anchor point 1, and the orders are respectively recorded as step1, step2, step3, step4, when the gray value of the last position point of the order point is the same as the gray value of the anchor point, the point is recorded as anchor point 2, and the coordinates of the point are recorded, however, taking the anchor point 2 as the starting point to search above, below, left and right adopts the same mode to obtain the anchor point 3, until all the points are searched iteratively.

[0055] For better implementation of the present application, further, in the step S440, the random gray label information is added:

[0056]

[0057] Wherein, numb r1 , numb g1 , numb b1 are respectively the values set in the r, g, b channel judgment;

[0058] max r , max g , max b respectively represent the corresponding values of the r, g, b color space of the arbitrary contour line of the subgraph imageN;

[0059] h R (x, y), h G (x, y), h B (x, y) respectively represent the gray values of the corresponding pixel points R, G, B channels at the (x, y) point in the intercepted image;

[0060] continue represents skipping the current processing to the next processing;

[0061] rand() represents a random number generated based on time, that is, corresponding to 3 random number seeds number1, number2, number3;

[0062] S xy represents the same piece area with the same gray value in the image.

[0063] For better implementation of the present application, further, in the step S400, the class 3 of the sample is respectively a cylindrical surface, a bottom corner surface and other surfaces, the gray values of the areas corresponding to the surfaces are all 0 after the gray uniform processing of all the surfaces contained in the sample, and the gray values of the areas corresponding to the surfaces are uniformly max r .

[0064] To better implement the present application, further, in the step S500, the accurate recognition model first extracts features using a small-scale convolution kernel, then performs feature transformation using pooling, a nonlinear function, and batch normalization to obtain a feature map, and uses a fully connected layer to reduce dimensions to obtain highly abstract features, and finally uses a classifier to implement attribute classification.

[0065] To better implement the present application, further, the step S600 includes the following steps:

[0066] Step S610: Express the accurate recognition result as File_numb_Lx_Ly_Rx_Ry.png,

[0067] wherein File is a judgment header name;

[0068] numb represents the number of screenshots;

[0069] Lx and Ly respectively represent the x value and y value of the top-left corner of the predicted image taken from the rough recognition of the original image, i.e., the row value and column value of the top-left corner;

[0070] Rx and Ry respectively represent the x value and y value of the bottom-right corner of the predicted image taken from the rough recognition of the original image, i.e., the row value and column value of the top-left corner;

[0071] In all the result images File_numb_Lx_Ly_Rx_Ry.png of the recognition, the recognized cylindrical surface adopts the labeling method of numb r2 , numb g2 , and numb b2 , and the corresponding values correspond to r, g, and b; while the recognized bottom corner surface adopts the labeling method of numb r3 , numb g3 , and numb b3 ; the labeled results are based on numb and the images corresponding to image1, image, …imageN, and the corresponding numerical values Lx, Ly, Rx, and Ry of the images are used to color the images;

[0072] Step S620: In the colored images, the cluster centers of the feature surface pieces with the gray values of numb r2 , numb g2 , numb b2 , and numb r3 , numb g3 , and numb b3 are calculated, respectively;

[0073] Step S630: The cluster centers of the feature surface pieces with the gray values of numb r2 , numb g2 , and numbb2 Patch clustering center center n_z (x, y) and its nearest neighbor gray value is numb r3 , numb g3 , numb b3 Patch clustering center center n_d (x, y) line, center n_z (x, y) is the coordinate of the point at the 1 / 3 length position from the starting point center n_1 / 3 (x, y);

[0074] Step S640: For the nearest neighbor pair of points in the image, the coordinates of the center n_d (x, y) and center n_1 / 3 (x, y) are constructed into a directed vector, the starting point of the vector is center n_1 / 3 (x, y), and the end point is center n_d (x, y); the normal direction of the vector is calculated, respectively denoted as and In the X-axis direction, the unit vector can be represented as:

[0075] i_shiliang=(center n_1 / 3 (x)-center n_d (x)) / sqrt((center n_1 / 3 (x)-center n_d (x))*(center n_1 / 3 (x)-

[0076] center n_d (x))+(center n_1 / 3 (y)-center n_d (y))*(center n_1 / 3 (y)-center n_d (y))

[0077] (7) In the Y-axis direction, the unit vector can be represented as:

[0078] j_shiliang=(center n_1 / 3 (y)-center n_d (y)) / sqrt((center n_1 / 3 (x)-center n_d (x))*(center n_1 / 3 (x)-

[0079] center n_d (x))+(center n_1 / 3(y)-center n_d (y))*(center n_1 / 3 (y)-center n_d (y))

[0080] (8) Normal vector It can be expressed as:

[0081]

[0082] In order to better implement the present invention, further, in step S620, the cluster center is calculated as follows: in the row direction, x=1 / num_point(∑x i ), the column direction has y=1 / num_point(∑y i ).

[0083] In order to better implement the present invention, further, in the step S700, the starting point center of the directed vector n_1 / 3 (x,y) is the initial point, along and The first search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not 0; the second search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not max. r ;

[0084] Search along the normal direction of the vector, and the following characteristics exist and the gray value of the moving seed point changes from 0 to max r , and then by max r changes to 0, from max r The area corresponding to 0 is the side of the groove area, and the cylindrical surface is the interface. and There is a side to each.

[0085] In order to better implement the present invention, further, in the step S800, the web surface search method is: the directed vector center n_d (x,y) and center n_1 / 3 (x,y) constructs a directed vector with the endpoint as center n_d (x, y), with the endpoint as the initial point, search along the directed vector direction; the first search, each movement distance is 1 unit pixel, until the gray value corresponding to the seed point is not 0; the second search, each movement distance is 1 unit pixel, until the gray value corresponding to the seed point is not max r ; The second step is to search, where max rThe area corresponding to 0 corresponds to the web surface corresponding to the groove area, and there is only one web surface for one groove area.

[0086] To better realize the present application, further, in the step S800, the bottom corner surface searching method is as follows: taking center n_d The angle between the starting point (x, y) and the main direction is angle The angle between the starting point (x, y) and the main direction is angle The first search is performed, and the distance of each movement is one unit pixel until the gray value corresponding to the seed point is not 0; the second search is performed, and the distance of each movement is one unit pixel until the gray value corresponding to the seed point is not max r The search is performed along the angle direction of the vector, and the following characteristics exist: the gray value of the seed point of the search movement changes from 0 to max r , and the gray value of the seed point of the search movement changes from 0 to max r , and the gray value of the seed point of the search movement changes from 0 to max r , and the gray value of the seed point of the search movement changes from 0 to max

[0087] To better realize the present application, further, in the step S800, the top surface searching method is as follows: the cluster center center n_c (x, y) of the adjacent bottom corner surface of the side surface is calculated respectively, and the cluster center center n_j (x, y) of the adjacent bottom corner surface of the side surface is calculated respectively, and the cluster center center n_j (x, y) of the adjacent bottom corner surface of the side surface is calculated respectively, and the cluster center center n_c (x, y) of the adjacent bottom corner surface of the side surface is calculated respectively, and the cluster center center

[0088] To better realize the present application, further, the values of numb r2 , numb g2 , numb b2 , numb r3 , numb g3 , numb b3 are located between 0-255, and there is no equal value. r1 , numb g1 , numb b1 , numb r2 , numb g2 , numb b2 , numb r3 , numb g3 , numb b3 are located between 0-255, and there is no equal value.

[0089] The present application has the following beneficial effects:

[0090] This method can identify and judge the categorical attribute features of aircraft structural component patches from any viewing angle for automated programming. By employing coarse and fine design recognition, as well as directed feature vector metric recognition, the method increases its adaptability to various scenarios and enhances its recognition integrity. The recognition results provide useful feature information for automated programming, laying the foundation for the early implementation of automated programming based on recognition features. It also offers a guiding reference method for similar research projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 is a flow chart of the present invention;

[0092] Figure 2 This is a schematic diagram of an aircraft structure designed based on 3D modeling software;

[0093] Figure 3 Images of aircraft structural parts from different perspectives obtained using the cropping tool;

[0094] Figure 4 is the coarse recognition sample image obtained after preprocessing;

[0095] Figure 5 The schematic diagram of the rough recognition result and its mapping transformation processing;

[0096] Figure 6 Schematic diagram of calculating cluster centers for feature patches with grayscale values ​​of numbr2, numbg2, numbb2 and numbr3, numbg3, numbb3;

[0097] Figure 7 The schematic diagram is for searching points on a directed vector along the direction of the starting point.

[0098] Figure 8 Schematic diagram of the final recognition results displayed on the 3D model. DETAILED DESCRIPTION

[0099] Example 1:

[0100] A method for identifying characteristic facets of aircraft intelligent manufacturing structural parts, such as Figure 1 As shown, the following steps are included:

[0101] ① Design the display mode of aircraft structural parts and perform image acquisition;

[0102] ② Design the structural part feature display mode, i.e., feature representation mode, and make a rough recognition sample;

[0103] ③ Design and construction of coarse recognition network, coarse recognition model training and sample prediction, and output of coarse recognition result coordinate values;

[0104] ④Coordinate region mapping and transformation processing is performed and a sub-face feature image is obtained;

[0105] ⑤A fine recognition network is designed and constructed, a fine recognition model is trained and sample prediction is performed, and fine recognition result data information is output;

[0106] ⑥A fine result information original map mapping and feature vector design and construction are performed;

[0107] ⑦Based on the feature vector, a measurement vector is designed to judge the attached face;

[0108] ⑧The attached face is further judged and the result is valued by TAG, and the final recognition structure is obtained in the three-dimensional model and displayed.

[0109] Preferably, an image display mode for aircraft structure piece face classification recognition is designed based on three-dimensional modeling software CITIA V5.21, and the display parameters, display mode, light, and other parameters of the structure and background in the CITAI modeling software are all set to the set parameters. The display mode gray scale of the background is 131, 193, and 232, the structure piece display mode gray scale is 169, 169, and 16, the boundary line display mode gray scale is 0, 0, and 0, and the light source lighting mode is a single light source. The numerical parameters of scattering, diffusion, and reflection are 0.34, 0.49, and 0.78.

[0110] Preferably, a multi-view structure piece image cutting tool based on CATIA is developed, and the cutting tool is designed with a first view range of 0-360 degrees, a second view range of 0-360 degrees, a magnification of 3 times, and a cutting view angle transformation interval of 10 degrees. The image is saved while the corresponding TAG value of each image is saved in a txt file.

[0111] Preferably, the obtained structure piece image is designed for feature display mode and feature representation mode, and the design method is as follows: (1) a blank image image_dst of the same size and single color channel as the obtained structure piece image is newly created, and the initial gray scale value is max r , the gray scale value of each position pixel point in the structure piece image is obtained, and if the gray scale values of the three channels are all 0, the gray scale value of the pixel point at the same position in image_dst is transformed to 0;

[0112] (2) if the gray scale values of the three channels are not all 0, the gray scale value of the pixel point at the same position in image_dst remains unchanged; (3) all pixel points in the image are processed, and then image_dst is operated

[0113] wherein x and y represent the number of columns and rows of the image, respectively; hR represents the value corresponding to the R color space at the (x, y) position in the image_dst image; maxr represents the set maximum gray value number.

[0114] (4) Curve transformation image_dst(x, y) - kernel eight = {min→result(x+x', y+y')(x', y'):kernel eight (x', y')≠0} where image_dst(x, y) represents the transformed image; kernel eight represents the 8-neighborhood transformation factor; result(x, y) represents the pixel value after processing at the (x, y) position in the image;

[0115] where x and y respectively represent the column number and the row number of the image; image_dst(x, y) represents the transformed image; kernel eight represents the 8-neighborhood transformation factor; result(x, y) represents the pixel value after processing at the (x, y) position in the image; x', y' respectively represent the position offset.

[0116] (5) Standard processing is performed on the feature curve in the image_dst image, so that the thickness of the normal vector direction of the curve along any point is the same, and is 5 pixels.

[0117] Preferably, the image_dst image obtained by processing is used as a training and prediction original sample, and the training sample is labeled, and the features labeled are the corresponding cylindrical surface and the small bottom angle surface feature along the downward direction of the cylindrical surface in the image, which are used as the training and prediction sample.

[0118] Preferably, a coarse recognition network is designed, the coarse recognition network acquires feature maps of different resolutions based on multi-scale differences, and adopts an upsampling and fusion manner to acquire feature maps with high resolution and high semantic information, acquires potential regions of interest based on a region candidate network, and then performs classification and regression processing on the regions of interest, num_stages=4, batch normalization is adopted to accelerate the network learning rate, the input channels of the multi-scale feature extraction module are [256, 512, 1024, 2048], the channels of the output feature layer are 256, the number of output feature layers num_outs is 5, the screening thresholds of positive and negative samples during training are 0.7 and 0.3 respectively, the maximum number of initial judgment boxes generated is 2000, and the set non-maximum suppression threshold is 0.7. During prediction, the maximum number of initial judgment boxes generated is 1000, the non-maximum suppression threshold is 0.7, and num_classes=1.

[0119] The hyperparameters such as batch_size, epoch, img_size, learn_rate are set to start training, and the numerical changes corresponding to different parameters are recorded. The network model weight and the corresponding hyperparameters with the optimal loss are screened to initialize the prediction network. The image_dst prediction image obtained by the same preprocessing step is predicted to obtain the potential feature region coordinate data in the image to be identified. The result of the identification process is a coarse identification result. The coarse identification result contains the total number of identification features result_all corresponding to the image and the coordinate data point(x, y) corresponding to each feature.

[0120] Preferably, the coarse identification network includes three core modules: a multi-scale feature extraction module, a target preliminary judgment module, and a deviation correction module. The multi-scale feature extraction module solves the problem of insufficient feature expression ability of a single scale feature extractor, thereby improving the network's ability to express different scale features. The target preliminary judgment module performs preliminary judgment analysis on the image to be identified to determine whether there is a potential feature to be identified, thereby providing input data for the deviation correction module. The deviation correction module reduces the overfitting problem and mismatch risk caused by directly using a large threshold in the model, and improves the accuracy of the detection result through layer-by-layer refinement.

[0121] Preferably, there are a large number of position deviation coordinates in the coarse identification result, and based on the existing coarse identification result, it is not possible to accurately determine which face in the three-dimensional model corresponds to the class information and the accurate position information. Further identification and judgment are needed, and the present application also designs a fine data strategy. The main reason is that part of the frame in the result area truncates the region feature, that is, the result frame does not completely contain the combined feature. In order to ensure the identification accuracy of subsequent processing, a transformation preprocessing step is designed.

[0122] Preferably, the coarse identification data is transformed and preprocessed, and each coarse identification result region R result is read. result If the coordinates of the upper left corner point and the lower right corner point of the rectangular frame region R l-result are Point r-result and Point l-result-new , the new coordinates of the sample after transformation processing can be represented as follows:

[0123] Point l-result (x)-pro x-round )≥0, (Point l-result (y)-pro y-round )≥0 (1)

[0124] Point r-result-new(x, y)→ (Poin r-result (x) + pro x-round ) < cols, (Point r-result (y) + pro y-round ) < rows

[0125] rows(2)

[0126] wherein, Point l-result-new (x, y), Point r-result-new (x, y) represent the new coordinate positions of the left upper corner position point and the right lower corner position point of the processed result box respectively, and the corresponding values are positive integers; pro x-round and pro y-round represent the scales of the transformation processing, and the values set in the application are 15; cols and rows represent the column number and the row number of the image_dst image respectively.

[0127] According to the obtained new coordinate data, the rectangular sub-image is intercepted in the image_dst image based on the coordinate mapping position, and the size of the image is consistent with the size of the processed rectangular region. The batch processing intercepts all the rectangular regions in the image, and saves the sub-images (image1, image2, … imageN). The intercepted sub-image imageN is subjected to transformation processing, and the corner point coordinates Pn(x, y) of each sub-image are obtained, wherein n respectively equals 1, 2, 3, 4 starting from the left upper corner in a counterclockwise direction. Taking the corner point coordinates P1(x, y), P2(x, y), P3(x, y), P4(x, y) of each sub-image as anchor points, an iterative method is used to search for the points with the same pixel gray value within a 1-pixel unit distance from each anchor point, and the gray values of the points are re-assigned as h r (x, y) = numb r1 , h g (x, y) = numb g1 , h b (x, y) = numb b1 .

[0128] wherein, h r (x, y), h g (x, y), h b (x, y) represent the feature values of the pixels r, g, b channels at the position (x, y) in the imageN image; numb r1 , numb g1 , numb b1 are the values set for the judgment in the r, g, b channels respectively.

[0129] Preferably, the search mode is: respectively taking P1(x, y), P2(x, y), P3(x, y), P4(x, y) as the anchor point 1 as the starting point to judge the pixel points above, below, left and right and one pixel distance, the order is respectively recorded as step1, step2, step3, step4, the last position point gray value of the order point is the same as the anchor point gray value, the point is recorded as anchor point 2, and the coordinates of the point are recorded. However, the same way is adopted to search above, below, left and right from the anchor point 2 as the starting point to obtain the anchor point 3, and the iteration of all points is searched.

[0130] Preferably, the independent region outside the non-edge region of imageN is added with random gray label information based on different ways of gray difference. The point point with 4-neighbor adjacency relationship is defined by Rect, the random number measured by time change is obtained by rand(), and the independent region S outside the non-edge region of imageN is added with random gray label information based on the formula. xy Add random gray label information:

[0131]

[0132] Wherein, h R (x, y), h G (x, y), h B (x, y) respectively represent the gray values of the corresponding pixel points R, G and B channels at the point (x, y) in the intercepted image; continue represents skipping the current processing to the next processing; rand() represents a random number generated based on time, that is, corresponding to three random number seeds number1, number2 and number3; S xy represents the same region with the same gray value in the image.

[0133] cols and rows respectively represent the number of columns and rows of imageN image; x and y respectively represent the column number and row number of the image; ∑ represents the cumulative symbol; if represents the judgment condition; max r , max g , max b respectively represent the corresponding values of the r, g and b color spaces of the arbitrary contour line of the sub-image imageN; numb r1 , numb g1 , numb b1 are respectively set values for judgment in r, g and b channels.

[0134] After adding random gray label information to the independent region outside the non-edge region of the sub-image (image1, image2, … imageN), the independent region outside the non-edge region of the sub-image (image1, image2, … imageN) is read, which has max r , max g , maxb ), (numb r1 , numb g1 , numb b1 ) the number of types of different gray pixels gray , and the corresponding group gray values (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ). The different gray pixel regions correspond to the specific values in the r, g, b color space.

[0135] where max r , max g , max b respectively represent the corresponding values of the r, g, b color space of any contour line of the sub-image imageN; numb r1 , numb g1 , numb b1 are the set values for judgment in the r, g, b channel respectively. Read the gray value of each pixel point in the sub-image, and create a new blank image corresponding to the label, such as image1 corresponding to image_new1, the size of the image is consistent with image, and the initial color of the image is white. Based on whether the gray value of the read pixel point (x, y) belongs to the group gray value (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ) in the image_new1 image, add new gray information to the pixel at the (x, y) position, and at the same time, the group gray value (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B) as many images as groups are newly created and are represented by image_new1-1, image_new1-2, … image_new1-n. The corresponding processing formula in image_new1-1, image_new1-2, … image_new1-n can be represented as:

[0136]

[0137] wherein gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ) represent the specific values corresponding to the different gray pixel areas in the r, g, b color space; h R , h G , h B represent the values corresponding to the R, G, B color space in the (x, y) position in the image_new1-1 … image_new1-n images.

[0138] After processing each sub-image, a number of images containing only one closed area can be obtained, which can be represented by the following relationship:

[0139]

[0140] wherein N represents the number of sub-images, and n represents the number of closed areas corresponding to (max r , max g , max b ), (numb r1 , numb g1 , numb b1 ) in the sub-image.

[0141] Preferably, training samples are made from the obtained images, and the classes 3 of the samples are respectively cylindrical surface, base angle surface and other surface. Meanwhile, the surfaces contained in all the training samples are uniformly processed in gray, i.e. the gray values corresponding to the areas other than the surface are all 0, and the gray values corresponding to the areas where the surface is located are uniformly max r . Each sample image and the prediction image are processed in this way, and the gray values of each channel are consistent.

[0142] Preferably, the fine recognition network is designed and constructed, the fine recognition network is implemented by using a residual network, the number of corresponding four BasicBlocks in the structure of the network is 4, 5, 7, and 4 respectively, the value of num_classes is consistent with the number of categories, and the value of num_classes is 3. Other hyperparameters are set and trained, and the weight values contained in the corresponding epoch are preferably selected based on the principles of minimum loss and strong generalization ability. Based on the weight values, the network is initialized, the predicted samples are input into the network after being processed in the same way as the training samples, and the category of the face sheet can be judged. Since the corresponding category during training is only three categories, the result is only a cylindrical surface, a bottom corner surface, and other surfaces. Preferably, the fine recognition network uses a small-scale convolution kernel to extract features, combines pooling, a nonlinear function, and batch normalization to perform feature transformation to obtain a feature map, uses full connection to reduce dimension to obtain highly abstract features, and finally uses a classifier to realize attribute classification.

[0143] Preferably, the predicted samples adopt the following naming format: File_numb_Lx_Ly_Rx_Ry.png. Wherein File is a unified judgment header naming method; numb represents the number of screenshots, corresponding to image1, image, …imageN and their subgraphs; Lx and Ly respectively represent the x value and y value of the top left corner of the predicted image obtained by coarse recognition from the original image, that is, the row value and column value of the top left corner; Rx and Ry respectively represent the x value and y value of the bottom right corner of the predicted image obtained by coarse recognition from the original image, that is, the row value and column value of the top left corner; and the corresponding image format is all saved in png lossless format, avoiding the problem of position deviation caused by value loss during image analysis.

[0144] Preferably, in all the result images File_numb_Lx_Ly_Rx_Ry.png obtained by recognition, the cylindrical surface obtained by recognition is labeled by using numb r2 , numb g2 , and numb b2 , and the corresponding values correspond to r, g, and b; and the bottom corner surface obtained by recognition is labeled by using numb r3 , numb g3 , and numb b3 . The labeled results are colored based on the images corresponding to numb and image1, image, …imageN, and the values Lx, Ly, Rx, and Ry in the images. In the colored images, the gray values of numb r2 , numb g2 , and numb b2 , and the gray values of numb r3 , numb g3 , and numb b3The feature patch calculates its cluster center center n_z (x,y) and center n_d (x,y); center n_z (x,y) satisfies the pixel value numb r2 、numb g2 、numb b2 The centroid of the corresponding area. The cluster center is calculated as follows: x = 1 / num_point (∑xi) in the row direction and y = 1 / num_point (∑yi) in the column direction. The grayscale value is calculated as numb r2 、numb g2 、numb b2 patch cluster center n_z (x,y) and its nearest neighbor grayscale value numb r3 、numb g3 、numb b3 Patch Center n_d (x,y) connecting the line with center n_z (x,y) is the coordinate center of the point at the starting 1 / 3 length position n_1 / 3 (x,y). is the coordinate center of the nearest neighbor pair in the image n_d (x,y) and center n_1 / 3 (x,y) constructs a directed vector, the starting point of the vector is center n_1 / 3 (x,y) endpoint is center n_d (x,y).

[0145] Preferably, the normal direction of the calculated vector is recorded as and The unit vector in the X-axis direction can be expressed as:

[0146] i_shiliang=(center n_1 / 3 (x)-center n_d (x)) / sqrt((center n_1 / 3 (x)-center n_d (x))*(center n_1 / 3 (x)-

[0147] center n_d (x))+(center n_1 / 3 (y)-center n_d (y))*(center n_1 / 3 (y)-center n_d (y))(7)

[0148] The unit vector in the Y-axis direction can be represented as:

[0149] j_shiliang = (center n_1 / 3 (y) - center n_d (y)) / sqrt((center n_1 / 3 (x) - center n_d (x))*(center n_1 / 3 (x)-

[0150] center n_d (x)) + (center n_1 / 3 (y) - center n_d (y))*(center n_1 / 3 (y) - center n_d (y))(8)

[0151] The normal vector can be represented as:

[0152]

[0153] Based on the starting point of the directional vector center n_1 / 3 (x, y), the search is performed along the and directions, respectively, (1) the first search, the distance moved each time is 1 unit pixel, until the seed point corresponding to the gray value is not 0; (2) the second search, the distance moved each time is 1 unit pixel, until the seed point corresponding to the gray value is not max r . The search is performed along the direction of the normal vector of the vector, and there are the following characteristics and the gray value of the seed point of the search movement changes from 0 to max r , and then from max r to 0. Wherein from max r to 0, the region corresponding to 0 is the side surface corresponding to the groove region, and there is one side surface with the cylindrical surface as the interface in and .

[0154] Preferably, the values of numb r1 , numb g1 , numb b1 , numb r2 , numb g2 , numb b2 , numb r3 , numb g3 , numb b3 are between 0-255, and there is no equal value.

[0155] Preferably, the web surface, the ground corner surface, and the top surface are identified by using the designed search method, and the identified results are added to the three-dimensional model based on the saved tag values and displayed, thereby completing the entire identification process. Based on the directed vector center n_d (x,y) and the directed vector center n_1 / 3 (x,y), the end point of the directed vector is center n_d (x,y), and the search is performed from the end point in the direction of the directed vector. (1) The first search, in which the distance of each movement is 1 unit pixel, until the gray value of the seed point corresponding to the search is not 0; (2) the second search, in which the distance of each movement is 1 unit pixel, until the gray value of the seed point corresponding to the search is not max r . The second search, in which max r varies to 0, and the region corresponding to 0 is the web surface corresponding to the groove region.

[0156] Based on the angle between the starting point in center n_d (x,y) and the main direction being angle degrees and , (1) the first search, in which the distance of each movement is 1 unit pixel, until the gray value of the seed point corresponding to the search is not 0; (2) the second search, in which the distance of each movement is 1 unit pixel, until the gray value of the seed point corresponding to the search is not max r . The search is performed in the angle direction of the vector, and the following characteristics and the gray value of the seed point of the search movement vary from 0 to max r . r , in which max r varies to 0, and the region corresponding to 0 is the ground corner surface corresponding to the groove region.

[0157] Preferably, after all the nearest neighbors in the image that satisfy the conditions are processed, the gray values numb r2 , numb g2 , and numb b2 are calculated, and the face patch clustering centers and the gray values numb r3 , numb g3 , and numb b3 are obtained. The face patch attributes of all the groove regions in the image can be obtained after the face patch clustering centers are calculated. The clustering center center n_c (x,y) of the side surface and the clustering center center n_j (x,y) of the adjacent ground corner surface are calculated, and the starting point center n_j (x,y) is taken as the starting point.n_c Construct a directed vector with (x, y) as the endpoint. The face that the vector points to, which intersects the side face, is the top face. Based on the TAG value txt file saved in the previous step, find the corresponding facet label information and add the attribute information of the recognition result to the 3D model to display the recognition results intuitively. After processing all the side faces and their adjacent large bottom angle faces in this step, the attribute information of all the facets in the image can be obtained.

[0158] The present invention can realize the recognition and judgment of category attribute features of aircraft structural parts automatic programming patches at any viewing angle, increase the adaptability of the method to scenarios and the integrity of recognition, lay the foundation for the early realization of automatic programming based on recognition features, and provide a guiding reference method for similar research objects.

[0159] Example 2:

[0160] A method for identifying characteristic facets of aircraft intelligent manufacturing structural parts comprises the following steps:

[0161] The program of the present invention is based on Windows10 system 64, and processor is Intel (R) Xeon (R) W-2223, and running memory size is 32GB, and frequency is 3200Mhz, and the graphics card used for deep learning training and detection is Nvidia-P2200 (video memory capacity 5GB), and deep learning platform is based on Pytorch framework, and Python version is 3.7.1, and Pytorch version is 1.5.0, and Torchvision version is 0.6.0, and sample annotation tool is labelimg, and matrix operation tool library Numpy version is 1.21.4, SDK version is 10.0.22000.0, and graphics driver version is 441.66, and CUDA version is 10.2, and hard disk capacity is 256GB. Software development platform is based on VS2019, and programming language is C++, and corresponding three-dimensional modeling software is CATIA V5.21, and corresponding image processing library is OpenCV4.5.2Release X64 platform. The main steps of the visual recognition and analysis method of facet categories in intelligent manufacturing of aircraft structural parts are as follows: Figure 1 shown.

[0162] 1. The image display mode of the aircraft structure part surface sheet category identification researched by the present application is designed based on the three-dimensional modeling software CATIA V5.21, and the display parameters, display mode, and light of the structure part and background in the CATIA modeling software are all set by using the set parameters. The display mode of the background is designed with the gray scale of 131, 193, and 232, the display mode of the structure part surface sheet is designed with the gray scale of 169, 169, and 16, the display mode of the boundary line is designed with the gray scale of 0, 0, and 0, the light source illumination mode is a single light source, and the numerical parameters of scattering, diffusion, and reflection are 0.34, 0.49, and 0.78. The aircraft structure part obtained based on the designed display mode is shown in FIG. 1. Figure 2

[0163] 2. A multi-view structure part image cutting tool is designed based on the secondary development function of CATIA. The first view range of the designed cutting tool is 0-360 degrees, the second view range is 0-360 degrees, the magnification is 3 times, and the transformation interval angle of the cutting view angle is 10 degrees. The different view angle aircraft structure part images obtained based on the cutting tool are shown in FIG. 2. The TAG value corresponding to each image is saved at the same time when the image is cut, and is saved in a txt file. Figure 3

[0164] 3. The feature display mode and feature representation mode of the structure part images obtained under different view angles are designed in the following manner: (1) a blank image of the same size and single color channel as the obtained structure part image image_dst is newly created, and the initial gray scale value is max r , max g , and max b . The gray scale value of each position pixel point in the structure part image is obtained, and if the gray scale values of the three channels are all 0, the gray scale value of the pixel point at the same position of image_dst is transformed to 0; (2) if the gray scale values of the three channels are not all 0, the gray scale value of the pixel point at the same position of image_dst remains unchanged; (3) all the pixel points in the image are processed, and then image_dst is operated (4) curve transformation processing image_dst(x,y)-kernel eight ={min→result(x+x′,y+y′)(x′,y′):kernel eight (x′,y′)≠0} wherein, image_dst(x,y) represents the transformation processing image; kernel eight represents the 8-neighborhood transformation factor; and result(x,y) represents the pixel value processed at the (x,y) position in the image; (5) the standard processing is performed on the feature curve in image_dst image, so that the thickness of the normal vector direction of the curve along any point is the same, and is 5 pixels.

[0165] ​​4. As shown in Figure 4

[0166] 5. The coarse recognition network is designed, the network obtains feature maps of different resolutions based on multi-scale differences, and high-resolution and high-semantic information feature maps are obtained by using up-sampling and fusion, potential regions of interest are obtained based on the region candidate network, and then the regions of interest are classified and regressed, num_stages=4, batch normalization is used to speed up the network learning rate, the input channels of the multi-scale feature extraction module are [256, 512, 1024, 2048], the output feature layer channels are 256, the number of output feature layers is num_outs=5, the screening thresholds of positive and negative samples during training are 0.7 and 0.3 respectively, the maximum number of initial judgment boxes generated is 2000, and the set non-maximum suppression threshold is 0.7. The maximum number of initial judgment boxes generated during prediction is 1000, the non-maximum suppression threshold is 0.7, and num_classes=1.

[0167] 6. The hyperparameters such as batch_size, epoch, img_size, learn_rate are set to start training, and the numerical changes corresponding to different parameters are recorded, the network model weight and the corresponding hyperparameters are screened by using the loss optimal strategy, and are used to initialize the prediction network.

[0168] 7. The image_dst prediction image obtained by the same preprocessing step is predicted, and the potential feature region coordinate data in the image is obtained, and the corresponding result of the recognition process is a coarse recognition result, as shown in Figure 5

[0169] 8. Because there are a large number of position deviation coordinates in the coarse recognition result, and based on the existing coarse recognition result, it is not possible to accurately determine which face in the three-dimensional model corresponds to the category information and the accurate position information. Further recognition and judgment are needed, and the fine data strategy is designed in the present application. The main reason is that part of the frame in the result area truncates the region feature, that is, the result frame does not completely contain the combined feature. In order to ensure the recognition accuracy of subsequent processing, the transformation preprocessing step is designed.

[0170] 9. The coarse recognition data needs to be transformed and preprocessed before fine recognition, and each coarse recognition result region R result ​​, if the upper left corner position point and the lower right corner position of the rectangular frame region R result are Point l-result and Point r-result respectively, the new coordinates of the sample after the transformation processing can be expressed as follows:

[0171] Point l-result-new (x,y)→(Point l-result (x)-pro x-round )≥0, (Point l-result (y)-pro y-round )≥0 (1)

[0172] Point r-result-new (x,y)→(Point r-result (x)+pro x-round )<cols, (Point r-result (y)+pro y-round )<rows (2)

[0173] wherein Point l-result-new (x,y) and Point r-result-new (x,y) represent the new coordinate positions of the upper left corner position point and the lower right corner position point of the processed result frame respectively, and the corresponding values are positive integers; pro x-round and pro y-round represent the scale of the transformation processing, and the value set in the invention is 15; cols and rows represent the column number and the row number of the image_dst image respectively. The transformation processing effect is shown in Fig. Figure 5 (b).

[0174] 10. Based on the obtained new coordinate data, the rectangular sub-image is intercepted in the image_dst image based on the coordinate mapping position, and the size of the image is consistent with the size of the processed rectangular region. The batch processing intercepts all the rectangular regions in the image, and saves the sub-images (image1, image2, … imageN).

[0175] 11. The intercepted sub-image imageN is processed, and the corner point coordinates Pn(x,y) of each sub-image are obtained, wherein n respectively equals 1, 2, 3, and 4 from the left upper corner as the starting point counterclockwise.

[0176] 12. The corner point coordinates P1(x,y), P2(x,y), P3(x,y), and P4(x,y) of each sub-image are taken as anchor points, and an iterative method is used to search for the points with the same pixel point gray value within 1 pixel unit distance from each anchor point, and the gray value h r (x,y) of the points is re-assigned as numbr1 , h g (x, y) = numb g1 , h b (x, y) = numb b1 . Preferably, the search method is as follows: take P1(x, y), P2(x, y), P3(x, y), P4(x, y) as anchor point 1 as the starting point to judge the pixel points above, below, left and right and one pixel distance, and the order is step1, step2, step3, step4 respectively. When the gray value of the last position point of the order point is the same as the gray value of the anchor point, the point is recorded as anchor point 2, and the coordinates of the point are recorded. However, the same method is used to search above, below, left and right from anchor point 2 as the starting point to get anchor point 3, until all points are searched iteratively. In this paper, h r (x, y) = 25, h g (x, y) = 125, h b (x, y) = 225.

[0177] 13. Random gray label information is added to the independent region outside the non-edge region of imageN in different ways of gray difference. Rect is used to define point with 4-neighbor adjacency relationship, rand() is used to get random number based on time change measure, and formula is used to add random gray label information to the independent region S xy of imageN outside the non-edge region:

[0178]

[0179] wherein, h R (x, y), h G (x, y), h B (x, y) respectively represent the gray value of the pixel point R, G, B channel corresponding to the (x, y) point in the intercepted image; continue represents skipping this processing to the next processing; rand() represents a random number based on time, that is, corresponding to 3 random number seeds number1, number2, number3; S xy represents the same region with the same gray value in the image.

[0180] 14. After adding random gray label information to the independent region outside the non-edge region of the sub-image (image1, image2, … imageN), read the type number num of different gray pixels (max r , max g , max b ), (numb r1 , numb g1 , numb b1 )gray and corresponding group gray level (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ).

[0181] 15. Read the gray level of each pixel in the sub-image respectively, and create a new blank image with the corresponding label, such as image1 corresponding to image_new1, the size of the image is consistent with image, and the initial color of the image is white. Based on whether the read pixel (x, y) gray level belongs to the group gray level (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ) in image_new1 image, add new gray level information to the pixel at (x, y) position, and at the same time, create as many images as there are groups of gray levels (gray1 R , gray1 G , gray1 B ), (gray2 R , gray2 G , gray2 B ), … (grayN R , grayN G , grayN B ) and use image_new1-1, image_new1-2, … image_new1-n to represent them. The corresponding processing formula in image_new1-1, image_new1-2, … image_new1-n can be represented as:

[0182] 16. After processing each sub-image, a number of images containing only one closed region can be obtained, which can be represented by the following relationship:

[0183]

[0184] Among them, N represents the number of sub-images, n represents the number of sub-images with grayscale values ​​(max r 、max g 、max b )、(numb r1 、numb g1 、numb b1 ) corresponds to the number of closed areas.

[0185] 17. Create training samples from the acquired images. The three categories of samples are cylindrical surfaces, bottom angle surfaces, and other surfaces. At the same time, all the surfaces contained in the training samples are grayscale unified, that is, the grayscale values ​​corresponding to the areas other than the surfaces are all 0, and the grayscale values ​​corresponding to the areas where the surfaces are located are unified to max. r , each sample image and predicted image adopts this processing method, and the grayscale value of each channel is consistent.

[0186] 18. Design and Construction of the Precision Recognition Network. The precision recognition network is implemented using a residual network. The number of BasicBlocks in the network structure corresponds to 4, 5, 7, and 4, respectively. The value of num_classes corresponds to the number of classes, which is 3. Other hyperparameters are set and training is performed. The weight values ​​contained in the corresponding epoch are optimized based on the principles of minimizing loss and improving generalization ability.

[0187] 19. The network is initialized based on the weight values. The predicted samples are processed in the same way as the training samples and then input into the network to determine the category of the face. Since there are only three corresponding categories during training, the results are only cylindrical faces, bottom corner faces, and other faces.

[0188] 20. For each sample to be predicted, use the following naming format: File_numb_Lx_Ly_Rx_Ry.png. File is a unified naming format for the judgment header; numb represents the number of screenshots, corresponding to image1, image, ...imageN, and their sub-images; Lx and Ly represent the x and y values ​​of the upper left corner of the predicted image, i.e., the row and column values ​​of the upper left corner, respectively; Rx and Ry represent the x and y values ​​of the lower right corner of the predicted image, i.e., the row and column values ​​of the upper left corner, respectively. All corresponding screenshots are saved in a lossless PNG format to avoid value loss and position deviation problems caused by image parsing.

[0189] 21. In all the recognition result images File_numb_Lx_Ly_Rx_Ry.png, use numb for the recognized cylindrical surface r3 、numb g3 、numbb3 The corresponding values are r, g, b, respectively; and the identified bottom corner surface adopts the labeling method of numb r2 , numb g2 , numb b2 The labeled results are based on numb and the corresponding images of image1, image, …imageN, and the corresponding values Lx, Ly, Rx, Ry in the images to color the images.

[0190] 22. In the colored image, respectively, the clustering centers of the feature surface patches with gray values of numb r3 , numb g3 , numb b3 , and numb r2 , numb g2 , numb b2 are calculated. The clustering center is calculated as follows: x = 1 / num_point(∑xi) in the row direction and y = 1 / num_point(∑yi) in the column direction, as shown in Figure 6

[0191] 23. The clustering centers center r3 (x,y) of the patches with gray values of numb g3 , numb b3 and the nearest neighbor gray values of numb r2 , numb g2 , numb b2 are calculated, respectively, and the line connecting the center n_d (x,y) of the center n_z (x,y) is the starting point of 1 / 3 length position of the point center n_1 / 3 (x,y).

[0192] 24. The directed vector is constructed for the nearest neighbor pair of points center n_d (x,y) and center n_1 / 3 (x,y) in the image, the starting point of the vector is center n_1 / 3 (x,y) and the end point is center n_d (x,y). The normal direction of the vector is calculated and recorded as and The unit vector in the X-axis direction can be represented as:

[0193] i_shiliang = (center n_1 / 3 (x) - center n_d (x)) / sqrt((center n_1 / 3 (x) - center n_d (x))2)(x) - center n_d (x) ) * ( center n_1 / 3 (x) -

[0194] center n_d (x) ) + ( center n_1 / 3 (y) - center n_d (y) ) * ( center n_1 / 3 (y) - center n_d (y) ) (7) The unit vector in the Y-axis direction can be represented as:

[0195] j_shiliang = ( center n_1 / 3 (y) - center n_d (y) ) / sqrt ( ( center n_1 / 3 (x) - center n_d (x) ) * ( center n_1 / 3 (x) -

[0196] center n_d (x) ) + ( center n_1 / 3 (y) - center n_d (y) ) * ( center n_1 / 3 (y) - center n_d (y) ) (8) The normal vector can be represented as:

[0197]

[0198] 25. Taking the starting point center n_1 / 3 (x, y) of the directed vector as the initial point, search along the and directions respectively, as shown in Figure 7 (1) The first search, the distance moved each time is 1 unit pixel, until the gray value corresponding to the seed point of the search is not 0; (2) The second search, the distance moved each time is 1 unit pixel, until the gray value corresponding to the seed point of the search is not max r .

[0199] 26. Search along the direction of the normal vector of the vector, there are the following characteristics and the gray value of the seed point of the search moves from 0 to max r , and then from max r to 0. Wherein from max r to 0, the region corresponding to 0 is the side surface corresponding to the groove region, and there is one side surface in and respectively with the cylindrical surface as the interface.

[0200] 27. The directed vector center n_d (x,y) and center n_1 / 3 (x,y) to construct a directed vector with the end point as center n_d (x,y), and search along the direction of the directed vector with the end point as the initial point. (1) The first search, the distance of each movement is 1 unit pixel, until the gray value corresponding to the seed point is not 0; (2) The second search, the distance of each movement is 1 unit pixel, until the gray value corresponding to the seed point is not max r . The second search, where the gray value changes from max r to 0, the region corresponding to 0 is the web face corresponding to the groove region. There is only one web face for each groove region.

[0201] 28. Search along the direction of angle n_d (x,y) and the main direction with the initial point of center (x,y) and the main direction, respectively, (1) the first search, the distance of each movement is 1 unit pixel, until the gray value corresponding to the seed point is not 0; (2) the second search, the distance of each movement is 1 unit pixel, until the gray value corresponding to the seed point is not max r . Search along the direction of angle r (x,y) and the main direction with the initial point of center r (x,y) as the end point to construct a directed vector, there are the following characteristics and the gray value of the seed point changes from 0 to max r . The region corresponding to 0 is the large base corner face corresponding to the groove region.

[0202] 29. After processing all the nearest neighbors in the image, the gray values are calculated respectively as numb r3 , numb g3 , and numb b3 . After clustering the face patches with the gray values numb r2 , numb g2 , and numb b2 , the face patch attributes corresponding to all the groove regions in the image can be obtained.

[0203] 30. Calculate the clustering center center n_c (x,y) of the side face and the clustering center center n_j (x,y) of the large base corner face adjacent to it, respectively, with center n_j (x,y) as the end point to construct a directed vector with the initial point center n_c (x,y), the face intersecting with the side face pointed by the vector is the top face.

[0204] 31. Based on the TAG value txt file saved in the previous step, find the label information corresponding to the facet, add the attribute information of the recognition result to the three-dimensional model, and display the intuitive recognition result.

[0205] 32. After processing all the side surfaces and their adjacent large bottom corner surfaces according to the principle steps, the attribute information of all the facets in the image can be obtained. Based on this information, the attribute information of the feature facets is provided for the generation of the automatic numerical control processing program, and the foundation is laid for automatic programming. The display of the recognition result based on the design on the three-dimensional model is shown in FIG. 8. Figure 8

[0206] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Any simple modification or equivalent change made according to the technical essence of the present application to the above embodiment falls within the protection scope of the present application.​

Claims

1. A method for identifying characteristic facets of aircraft intelligent manufacturing structural parts, characterized in that: The following steps are involved: Step S100: Designing a display mode for aircraft structural component images, capturing images of aircraft structural components from different viewing angles, and saving the TAG value corresponding to each image; Step S200: performing image preprocessing on the captured image to obtain a coarsely recognized training sample and a prediction sample; said step S200 includes the following steps: Step S210: Create a new blank image image_dst with the same size and monochrome as the acquired structural component image, obtain the grayscale value of each pixel in the structural component image, and transform the grayscale value of the pixel at the same position in image_dst; Step S220: Process all pixels in the image, and then operate on image_dst: in, colums and rows represent the number of columns and rows of the image respectively; x and y represent the number of columns and rows of the image respectively; h R Indicates the value corresponding to the (x, y) position in the R color space in the image_dst image; max r Indicates the maximum grayscale value set; Step S230: Curve transformation processing: image_dst(x,y)-kernel eight ={min→result(x+x′,y+y′)(x′,y′):kernel eight (x′,y′)≠0} Among them, image_dst(x,y) represents the transformed image; kernel eight represents the 8-neighborhood transformation factor; result(x,y) represents the pixel value at position (x,y) in the image after processing; x′, y′ represent the position offset of the image columns and rows respectively; Step S240: performing standard processing on the characteristic curve in the image_dst image so that the thickness of the curve along the normal vector direction of any point is the same; Step S300: Perform coarse recognition model training and sample prediction, and output coarse recognition results; Step S400: Processing the coarse recognition data to obtain training samples and prediction samples for fine recognition; the coarse recognition result includes the total number of recognition features corresponding to the image (result_all) and the coordinate data point (x, y) corresponding to each feature; performing coordinate region mapping and transformation processing on the coarse recognition result and obtaining a sub-facet feature image to obtain training samples and prediction samples for fine recognition; The step S400 includes the following steps: Step S410: Perform transformation preprocessing on the coarse recognition data and read each coarse recognition result area R result , rectangular area R result The coordinates of the upper left corner and the lower right corner are Point l-result and Point r-result , the new coordinates of the sample after transformation are as follows: Point l-result-new (x,y)→(Point l-result (x)-pro x-round )≥0,(Point l-result (y)-pro y-round )≥0(1) Point r-result-new (x,y)→(Poin r-result (x)+pro x-round )<cols,(Point r-result (y)+pro y-round )< rows(2) Among them, Point l-result-new (x,y),Point r-result-new (x, y) represent the new coordinate positions of the upper left corner and lower right corner of the result box after processing, and the corresponding values ​​are positive integers; pro x-round With pro y-round Respectively represent the scale of x and y transformation processing; cols and rows represent the number of columns and rows of the image_dst image respectively; Step S420: Based on the new coordinate data obtained by the transformation, a rectangular sub-image is captured in the image_dst image at the position based on the coordinate mapping. The size of the image is consistent with the size of the processed rectangular area. All rectangular areas in the image are captured in a batch process, and the sub-images {image1, image2, ...imageN} are saved. Step S430: Transform the intercepted sub-images, starting from the upper left corner and counting counterclockwise, to obtain the corner coordinates P1(x,y), P2(x,y), P3(x,y), P4(x,y) of each sub-image; use the corner coordinates P1(x,y), P2(x,y), P3(x,y), P4(x,y) of each sub-image as anchor points, iteratively search for points with the same grayscale value as each anchor point at a distance of 1 pixel, and reassign their grayscale values ​​to h. r (x,y)=numb r1 、h g (x,y)=numb g1 、h b (x,y)=numb b1 ; Among them, h r (x,y),h g (x,y),h b (x, y) represent the feature values ​​corresponding to the r, g, and b channels of the pixel at position (x, y) in the imageN image; numb r1 、numb g1 、numb b1 They are the values ​​set in the r, g, and b channels respectively; Step S440: adding random grayscale label information to independent areas outside the non-edge area of ​​the sub-image imageN in different ways based on grayscale differences; Step S450: After adding random grayscale label information to the independent area outside the non-edge area of ​​the sub-image, read the sub-image with the grayscale label information except (max r 、max g 、max b )、(numb r1 、numb g1 、numb b1 )Number of different grayscale pixel types gray , and the corresponding grayscale values ​​(gray1 R 、gray1 G 、gray1 B )、(gray2 R 、gray2 G 、gray2 B ),…(grayN R 、grayN G 、grayN B ); Among them: max r 、max g 、max b Respectively represent the values ​​corresponding to any contour line of sub-image imageN in r, g, and b color spaces; numb r 、numb g 、numb b They are the values ​​set in the r, g, and b channels respectively; gray R 、gray G 、gray B Respectively represent the specific values ​​corresponding to different grayscale pixel areas in the r, g, and b color spaces; Step S460: Read the grayscale value of each pixel in the sub-image respectively, and create a blank image image_new of the same size and corresponding label. The initial color of the blank image image_new is white. Based on whether the read grayscale value of the pixel point (x, y) belongs to the grouped grayscale value, add new grayscale information to the pixel at the (x, y) position in the image_new image, and group the grayscale value at the same time, to obtain the corresponding images image_new1-1, image_new1-2, ..., image_new1-n; Step S470: After processing each sub-image, a number of images containing only one closed area are obtained: Where N represents the number of subgraphs. n means that in the sub-image, the gray value is (max r 、max g 、max b )、(numb r1 、numb g1 、numb b1 ) corresponds to the number of closed areas; Step S500: Perform precise recognition model training and sample prediction, and output accurate recognition results; Step S600: For the accurate recognition result, accurate result information original image mapping and feature vector construction are performed; the step S600 includes the following steps: Step S610: The accurate recognition result is represented as File_numb_Lx_Ly_Rx_Ry.png. Among them, File is the name of the judgment header; numb indicates the number of screenshots; Lx and Ly represent the x value and y value of the upper left corner of the predicted image obtained by rough recognition from the original image, that is, the row value and column value of the upper left corner; Rx and Ry represent the x value and y value of the lower right corner of the predicted image obtained by rough recognition from the original image, that is, the row value and column value of the upper left corner; In all the recognition result images File_numb_Lx_Ly_Rx_Ry.png, the recognized cylindrical surface is represented by numb r2 、numb g2 、numb b2 The corresponding values ​​correspond to r, g, and b; the bottom angle face is identified using numb r3 、numb g3 、numb b3 The annotation method is used to color the image based on the image corresponding to numb and image1, image, ...imageN, and the corresponding values ​​Lx, Ly, Rx, Ry in the image; Step S620: In the colored image, the gray value is numb r2 、numb g2 、numb b2 with numb r3 、numb g3 、numb b3 Calculate the cluster center of the feature patch; Step S630: Set the grayscale value to numb r2 、numb g2 、numb b2 patch cluster center n_z (x,y) and its nearest neighbor grayscale value numb r3 、numb g3 、numb b3 patch cluster center n_d (x,y) connection, with center n_z (x,y) is the coordinate of the point at the starting 1 / 3 length position and is the center n_1 / 3 (x,y); Step S640: center the coordinates of the nearest neighbor paired points in the image n_d (x,y) and center n_1 / 3 (x,y) constructs a directed vector, the starting point of the vector is center n_1 / 3 (x,y), the endpoint is the center n_d (x,y); calculate the normal direction of the vector, respectively recorded as and- The unit vector in the X-axis direction can be expressed as: i_shiliang=(center n_1 / 3 (x)-center n_d (x)) / sqrt((center n_1 / 3 (x)-center n_d (x))*(center n_1 / 3 (x)-center n_d (x))+(center n_1 / 3 (y)-center n_d (y))*(center n_1 / 3 (y)-center n_d (y)) (7) The unit vector in the Y-axis direction can be expressed as: j_shiliang=(center n_1 / 3 (y)-center n_d (y)) / sqrt((center n_1 / 3 (x)-center n_d (x))*(center n_1 / 3 (x)-center n_d (x))+(center n_1 / 3 (y)-center n_d (y))*(center n_1 / 3 (y)-center n_d (y)) (8) Normal vector It can be expressed as: Step S700: Designing a metric vector based on the feature vector to determine the attached facets; Step S800: Identify the attached facets and find the label information of the corresponding facets based on the TAG value saved in step S100, add the attribute information of the identified result to the three-dimensional model and display it.

2. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S100, a display mode of an aircraft structural component image is designed based on software, and an image is captured using a capture tool. The capture tool has a first viewing angle range of 0-360 degrees, a second viewing angle range of 0-360 degrees, a magnification of 3 times, and a capture viewing angle change interval of 10 degrees.

3. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S210, when the grayscale value is converted, if the grayscale values ​​of the three channels are all 0, the grayscale value of the pixel at the same position of image_dst is converted to 0; if the grayscale values ​​of the three channels are not all 0, the grayscale value of the pixel at the same position of image_dst remains unchanged.

4. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S300, the coarse recognition model obtains feature maps of different resolutions based on multi-scale differences, and obtains feature maps with high resolution and high semantic information by up-sampling and fusion. Then, the potential region of interest is obtained based on the region candidate network, and the region of interest is classified and regressed.

5. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S430, the search method is: take P1(x,y), P2(x,y), P3(x,y), and P4(x,y) as anchor point 1 respectively, and take anchor point 1 as the starting point to determine the grayscale values ​​of the pixel points above, below, left, and right that are one pixel away. The order is recorded as step1, step2, step3, and step4 respectively. When the grayscale value of the last position point in the sequence is the same as the grayscale value of the anchor point, the point is recorded as anchor point 2, and the coordinates of the point are recorded. However, starting from anchor point 2, the same method is used to search up, down, left, and right to obtain anchor point 3 until all points are searched and iterated.

6. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S440, random grayscale label information is added: Among them, numb r1 、numb g1 、numb b1 They are the values ​​set in the r, g, and b channels respectively; max r 、max g 、max b Respectively represent the values ​​corresponding to any contour line of sub-image imageN in r, g, and b color spaces; h R (x,y),h G (x,y),h B (x, y) represents the grayscale value of the R, G, and B channels of the corresponding pixel at point (x, y) in the intercepted image; Continue means skipping this process and proceeding to the next process; rand() represents a random number generated based on time, which corresponds to three random number seeds number1, number2, and number3; S xy Represents the same area with the same grayscale value in the image.

7. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S400, the three categories of samples are cylindrical surfaces, bottom angle surfaces and other surfaces. All surfaces included in the samples are grayscale unified. The grayscale values ​​corresponding to the areas other than the surfaces are all 0, and the grayscale values ​​corresponding to the areas where the surfaces are located are unified to max. r .

8. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S500, the precise recognition model first uses a small-scale convolution kernel to extract features, then combines pooling, nonlinear functions, and batch normalization to perform feature transformation to obtain a feature map, and uses a fully connected layer to perform dimensionality reduction to obtain highly abstract features, and finally uses a classifier to implement attribute classification.

9. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S620, the cluster center is calculated as follows: in the row direction, x=1 / num_point(∑x i ), the column direction has y=1 / num_point(∑y i ).

10. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: In step S700, the starting point of the directed vector is center n_1 / 3 (x,y) is the initial point, along and The first search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not 0; the second search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not max. r ; Search along the normal direction of the vector, and the following characteristics exist and the gray value of the moving seed point changes from 0 to max r , and then by max r changes to 0, from max r The area corresponding to 0 is the side of the groove area, and the cylindrical surface is the interface. and There is a side to each.

11. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 10, characterized in that: In step S700, the web surface search method is: directed vector center n_d (x,y) and center n_1 / 3 (x,y) constructs a directed vector with the endpoint as center n_d (x, y), with the endpoint as the initial point, search along the directed vector direction; the first search, each movement distance is 1 unit pixel, until the gray value corresponding to the seed point is not 0; the second search, each movement distance is 1 unit pixel, until the gray value corresponding to the seed point is not max r ; The second step is to search, where max r The area corresponding to 0 is the web surface corresponding to the slot area. There is only one web surface in a slot area.

12. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 10, characterized in that: In step S700, the large bottom angle face search method is: center n_d The angle between the starting point and the main direction in (x,y) is angle degrees and The first search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not 0; the second search is performed with a distance of one unit pixel each time until the gray value corresponding to the seed point is not max. r ; Search along the angle direction of the vector, with the following characteristics and the gray value of the moving seed point changes from 0 to max r , in the max r changes to 0, from max r The area corresponding to the change to 0,0 is the large bottom angle surface corresponding to the groove area.

13. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 10, characterized in that: In step S700, the top surface search method is to calculate the cluster center of the side surfaces respectively. n_c (x,y) and the cluster center of its adjacent large-angle faces n_j (x,y), with center n_j (x,y) to the starting point center n_c (x, y) is the endpoint to construct a directed vector, and the surface that the vector points to that intersects with the side surface is the top surface.

14. The method for identifying characteristic facets of aircraft intelligent manufacturing structural parts according to claim 1, characterized in that: numb r2 、numb g2 、numb b2 、numb r3 、numb g3 、numb b3 The value is between 0 and 255, and there are no equal values.

Images