Precision casting defect intelligent identification method based on deep learning algorithm

Abstract

The invention discloses a precision casting defect intelligent identification method based on a deep learning algorithm, and particularly relates to the technical field of industrial detection.The method comprises the steps that multi-source heterogeneous data of historical precision castings are collected and preprocessed, image feature vectors, process feature vectors and structural feature vectors are extracted respectively, and then dynamic weight feature fusion is carried out to obtain a precise casting defect intelligent identification result; inputting the fusion feature vector into an intelligent identification model for training; and finally, inputting a multi-source heterogeneous data set, which is acquired in real time, of a new sample of the precision casting into the trained intelligent defect identification model, and outputting a defect identification result. By introducing the attention mechanism and the improved deep learning model of the multi-scale feature fusion module, the key features in the precision casting defect data can be extracted more accurately, the detection capability of tiny defects under a complex background is effectively improved, and the detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial detection, in particular to a precision casting defect intelligent identification method based on a deep learning algorithm. BACKGROUND

[0002] Precision castings are widely used in many key fields such as aerospace, automobile manufacturing, medical devices, etc., and their quality is directly related to the performance and safety of the products. However, during the casting process, due to the influence of various factors such as material properties, casting process parameter fluctuations, and mold state changes, various defects such as pores, cracks, and shrinkage are prone to occur inside and on the surface of the castings.

[0003] Traditional precision casting defect detection methods mainly include manual visual inspection and non-destructive testing technology based on physical principles. Manual detection relies on the experience and visual observation of the detection personnel, which is inefficient and subjective, and it is difficult to find small defects and internal defects. Non-destructive testing technologies such as X-ray detection, ultrasonic detection, and magnetic powder detection can detect casting defects to some extent, but they have problems such as expensive detection equipment, high requirements for the professional skills of operators, and limited detection accuracy for complex shape and structure castings.

[0004] With the development of deep learning technology, it has shown great potential in the field of industrial detection. However, there are still many challenges in applying deep learning to precision casting defect detection. The data sources of precision casting defects are diverse, including image data, process parameter data, and structure data, and how to effectively fuse multi-modal data features is a major problem. In the data feature extraction process, the potential features of the defect data cannot be fully mined, resulting in inaccurate determination of the defect type and severity. The generalization ability of the model is insufficient, making it difficult to adapt to the casting defect detection needs under different production batches and process conditions. Therefore, it is of great practical significance to develop a method that can effectively process precision casting defect data features and accurately identify defect types. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a precision casting defect intelligent identification method based on a deep learning algorithm to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a precision casting defect intelligent identification method based on a deep learning algorithm, comprising: S1: collecting and storing image data, process parameter data, and three-dimensional structure point cloud data of precision castings without defects and with defect types, and simultaneously receiving data generated during the precision casting defect intelligent identification process, and constructing a precision casting defect intelligent identification database; S2: Based on the intelligent identification database of precision casting defects, the image data set, the process parameter time series data set and the three-dimensional structure point cloud data set of historical precision castings are obtained, and the process parameters are screened to obtain a first multi-source heterogeneous data set for intelligent identification of defects; S3: Through preprocessing technology, the first multi-source heterogeneous data set is respectively subjected to image data preprocessing, process parameter preprocessing and three-dimensional structure point cloud data preprocessing to obtain a second multi-source heterogeneous data set; S4, through feature extraction technology, the second multi-source heterogeneous data set is subjected to defect feature extraction to obtain image feature vectors, process feature vectors and structure feature vectors, and through dynamic weight feature fusion, a fusion feature vector for intelligent identification of defects is obtained; S5: for inputting the fusion feature vector for intelligent identification of defects and the corresponding sample defect category label into the intelligent identification model for training to obtain a trained defect intelligent identification model; S6: for inputting the first multi-source heterogeneous data set of the real-time collected new sample of precision castings into the trained defect intelligent identification model, obtaining a defect intelligent identification result according to the model output result, and feeding back to the management terminal for human-computer interaction.

[0007] Technical effects and advantages of the present application: 1、The present application collects image data, process parameter data and three-dimensional structure point cloud data through the intelligent identification database of precision casting defects, and effectively fuses multi-modal data features through dynamic feature fusion technology, so that the deep learning algorithm can more comprehensively understand the state of the casting, thereby more accurately identifying defects; 2、The improved deep learning model of the present application introduces an attention mechanism and a multi-scale feature fusion module, which can more accurately extract key features in the precision casting defect data, effectively improve the detection ability of micro-defects in a complex background, and improve the detection accuracy compared with the traditional deep learning-based detection method; 3、The present application adopts FocalLoss function, Adam optimizer and adaptive learning rate adjustment strategy, which effectively improves the generalization ability of the model, ensures that the model can be optimized in time with the change of production data and the adjustment of process, always maintains the best detection performance, and reduces the false detection and missed detection rate of the model in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 The present application is a whole process schematic diagram.

[0009] Figure 2 The present application is a method flowchart. DETAILED DESCRIPTION

[0010] With reference to the accompanying drawings on the basis of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0011] Please refer to Figure 1 As shown in the drawings, the present application provides a precision casting defect intelligent identification system based on a deep learning algorithm, which comprises a precision casting defect intelligent identification database, a multi-source heterogeneous defect data acquisition module, a multi-source heterogeneous defect data preprocessing module, a defect multi-modal feature extraction module, an intelligent identification model training module, and a defect intelligent identification and feedback module.

[0012] The precision casting defect intelligent identification database is connected with the rest of the modules, the multi-source heterogeneous defect data acquisition module is connected with the multi-source heterogeneous defect data preprocessing module, the defect multi-modal feature extraction module is connected with the multi-source heterogeneous defect data preprocessing module and the intelligent identification model training module respectively, and the defect intelligent identification and feedback module is connected with the intelligent identification model training module and the defect intelligent identification and feedback module respectively.

[0013] The precision casting defect intelligent identification database is used for collecting and storing image data, process parameter data and three-dimensional structure point cloud data of historical precision castings without defects and with defects, and simultaneously receiving data generated in the precision casting defect intelligent identification process in real time, and constructing a precision casting defect intelligent identification database; The multi-source heterogeneous defect data acquisition module is based on the precision casting defect intelligent identification database, acquires image data sets, process parameter time series data sets and three-dimensional structure point cloud data sets of historical precision castings, and screens the process parameters to obtain a first multi-source heterogeneous data set for defect intelligent identification, and transmits the first multi-source heterogeneous data set to the multi-source heterogeneous defect data preprocessing module; The multi-source heterogeneous defect data preprocessing module performs image data preprocessing, process parameter preprocessing and three-dimensional structure point cloud data preprocessing on the first multi-source heterogeneous data set respectively through preprocessing technology, obtains a second multi-source heterogeneous data set, and transmits the second multi-source heterogeneous data set to the defect multi-modal feature extraction module; The defect multi-modal feature extraction module extracts defect features from the second multi-source heterogeneous data set through feature extraction technology, obtains image feature vectors, process feature vectors and structure feature vectors, and obtains a fusion feature vector for defect intelligent identification through dynamic weight feature fusion, and transmits the fusion feature vector to the intelligent identification model training module; The intelligent recognition model training module is configured to input the fusion feature vector of the defect intelligent recognition and the corresponding sample defect category label into an intelligent recognition model, train the intelligent recognition model, obtain a trained defect intelligent recognition model, and transmit the trained defect intelligent recognition model to the defect intelligent recognition and feedback module; The defect intelligent recognition and feedback module is configured to input a first multi-source heterogeneous data set of a new sample of the precision casting collected in real time into the trained defect intelligent recognition model, obtain a defect intelligent recognition result according to a model output result, and feed back the defect intelligent recognition result to the management terminal for human-computer interaction.

[0014] Referring to Figure 2 As shown in FIG. 1, a precision casting defect intelligent recognition method based on a deep learning algorithm includes the following steps: S1, acquiring and storing image data, process parameter data and three-dimensional structure point cloud data of precision castings of no defect and defect types, simultaneously receiving data generated in a precision casting defect intelligent recognition process in real time, and constructing a precision casting defect intelligent recognition database; S2, based on the precision casting defect intelligent recognition database, obtaining an image data set, a process parameter time series data set and a three-dimensional structure point cloud data set of historical precision castings, and filtering process parameters to obtain a first multi-source heterogeneous data set for defect intelligent recognition; S3, through a preprocessing technology, image data preprocessing, process parameter preprocessing and three-dimensional structure point cloud data preprocessing are performed on the first multi-source heterogeneous data set respectively to obtain a second multi-source heterogeneous data set; S4, through a feature extraction technology, defect feature extraction is performed on the second multi-source heterogeneous data set to obtain an image feature vector, a process feature vector and a structure feature vector, and a fusion feature vector for defect intelligent recognition is obtained through dynamic weight feature fusion; S5, the fusion feature vector for defect intelligent recognition and a corresponding sample defect category label are input into an intelligent recognition model, and the intelligent recognition model is trained to obtain a trained defect intelligent recognition model; S6, a first multi-source heterogeneous data set of a new sample of the precision casting collected in real time is input into the trained defect intelligent recognition model, a defect intelligent recognition result is obtained according to a model output result, and the defect intelligent recognition result is fed back to a management terminal for human-computer interaction.

[0015] S1, acquiring and storing image data, process parameter data and three-dimensional structure point cloud data of precision castings of no defect and defect types, simultaneously receiving data generated in a precision casting defect intelligent recognition process in real time, and constructing a precision casting defect intelligent recognition database; The embodiment needs to be specifically explained that the image data of the defect-free and defective types refers to that according to the industry field experts, using professional image labeling tools, the defects in the images of different batches of precision castings meeting various defect types obtained by image acquisition equipment (for example, an X-ray machine) are finely labeled, the labeling content includes the type (porosity, crack, shrinkage, etc.), position coordinates and other information of the defects, and a labeling file conforming to the PASCAL VOC format is generated; the process parameter data is obtained through sensor technology, and the three-dimensional structure point cloud data is obtained through a three-dimensional scanning (for example, CT scanning) device.

[0016] S2: Based on the precision casting defect intelligent recognition database, image data sets, process parameter time series data sets and three-dimensional structure point cloud data sets of historical precision castings are obtained, and the process parameters are filtered to obtain a first multi-source heterogeneous data set for defect intelligent recognition, including the following steps: S2.1: Based on the precision casting defect intelligent recognition database, the image data set I, the process parameter time series data set P and the three-dimensional point cloud data set S are obtained to jointly constitute a multi-source heterogeneous data set DY, Y is a corresponding sample defect category label data set, the sample defect category label k includes a defect-free sample numerical label 0 and a defect sample numerical label, that is, k ∈ (0, 1, 2,..., K), K+1 is the type of defect category, which respectively corresponds to a specific defect type (for example, 0 = no defect, 1 = porosity, 2 = crack, 3 = shrinkage, etc.); I i is the i-th sample image data collected, and N is the number of collections, wherein P i is the process parameter time series data set of the i-th sample (for example, P1 is temperature, P2 is pressure, P3 is pouring speed, etc.), p m (t) is the m-th process parameter time series data set of the i-th sample, (for example, the time series value of temperature, ), p m (tn) is the n-th time node value of the m-th process parameter, and m is the number of process parameters, wherein S i is the i-th point cloud data set, s m1 is the m1-th three-dimensional coordinate point of the N-th group, and m1 is the number of three-dimensional coordinate points; Y i,k is the defect category label of the i-th sample, which is k; In this embodiment, it is necessary to specifically explain that the image data N, process parameter data N, point cloud data N, and defect numerical label N have the same value and are all in a one-to-one correspondence. The process parameters are based on the process parameters required in the precision casting production process of the industry. The defect types of the N samples meet the requirement that each category has a sufficient number of samples (e.g., at least 50 samples for each defect type).

[0017] S2.2: Obtain the process parameter set X and the corresponding defect numerical label set y based on the process parameter dataset of N samples. (For example, X1 is temperature, X2 is pressure, and X3 is pouring speed), where (e.g., temperature) ), X m For the m-th process parameter dataset (e.g., 50 temperature data values), p m,N The process parameters p for the i-th sample m The numerical values, and the defect value labels include a defective value of 1 and a non-defective value of 0. y i This represents the defect value of the i-th sample (e.g., the defect value of the 50th sample is 1). S2.3: First, the process parameter dataset of N samples is filtered using the Pearson correlation coefficient to obtain the m-th process parameter p. m With the defect numerical dataset y m Pearson correlation coefficient r m , p m,i For the i-th and m-th process parameters p m y i Let p be the defect value of the i-th sample, i∈N; then calculate the m-th process parameter p. m The average value μ(p) of N data points m ) and standard deviation σ(p m ), thus obtaining the process parameter p m coefficient of variation η(p) m ), η(p m )=σ(p m ) / μ(p m Secondly, iterate through all process parameters and retain the absolute value of the Pearson correlation coefficient |r m |≥ the corresponding threshold r m th (e.g., 0.2) and the coefficient of variation η(p) m )≥ the corresponding threshold η(p) m ) th Process parameter p (e.g., 0.02) m Otherwise, they are discarded to obtain the time series dataset P of process parameters. N , , p m2 (t) is the m2th process parameter time series data set, m2 is the number of screened process parameters, m2≤m; and a first multi-source heterogeneous data set DY is finally obtained, , P s is the screened process parameter time series data set; It needs to be specifically explained in this embodiment that different batches of precision castings refer to the same batch of precision castings having the same process parameters and the same three-dimensional structure, and different batches of process parameters and three-dimensional structures are different; the fluctuation degree of the parameters in the batch is measured by the coefficient of variation, and the parameters with "too small fluctuation, which have no actual effect even if associated" are excluded.

[0018] S3: Through a preprocessing technique, the first multi-source heterogeneous data set is respectively subjected to image data preprocessing, process parameter preprocessing and three-dimensional structure point cloud data preprocessing, to obtain a second multi-source heterogeneous data set, including the following steps: S3.1: Image data preprocessing: first, the denoising of the i th sample image data I i is performed through a Gaussian filtering technique, to obtain denoised image data I i de , I i de =G 1.5 (I i )*I i , G 1.5 is a Gaussian kernel with a standard deviation σ = 1.5, and * is a convolution operation; then the pixel value is normalized to [0, 1] through a Min-Max function, to obtain I i no , , min(I i de ) and max(I i de ) are the maximum and minimum values of the pixels, respectively; finally, the N sample image data are traversed, to obtain a preprocessed image data set I no , ; It needs to be specifically explained in this embodiment that the preprocessed image is a single-channel grayscale image, and the size is set to the commonly used size of industrial detection implied by the document (such as 224x224 pixels).

[0019] S3.2: Process parameter preprocessing: first, the z-score standardization function is used to standardize the screened m2th process parameter p m2 s (t) at time t, to obtain the standardized process parameter p m2 s,no (t), , μ[p m2 s (t)] and σ[p m2 s (t)] are the mean and standard deviation of the m2th process parameter p m2 s (t) at the n time points, respectively; then traverse the m2th process parameter p m2 s (t) at the n time points to obtain the standardized time series data set p m2 no (t) of the m2th process parameter, then traverse the screened m2th process parameters to obtain the standardized process parameter time series data set p i no (t) of the ith sample after preprocessing, p m2 no (t) is the time series data set of the m2th process parameter; finally traverse the process parameter time series data set p i no (t) of the N sample image data to obtain the preprocessed process parameter time series data set P no , P i no (t) is the standardized process parameter time series set of the ith sample; S3.3: Preprocessing of three-dimensional structure point cloud data: first, downsample the ith sample three-dimensional point cloud data set S i using Voxel Filter to obtain the downsampled three-dimensional point cloud data set S i down , i down = Voselfilter(S i , voxel_size = 0.5mm), where voxel_size is the voxel size; then calculate the centroid C i dowm of the point cloud data set S i , s j is the jth three-dimensional coordinate point, s j includes x-direction coordinate, y-direction coordinate and z-direction coordinate, m3 is the number of downsampled three-dimensional coordinate points, C i = [μ(x i ), μ(y i ), μ(z i )], μ(x i ), μ(y i ) and μ(zi are the average values of m3 three-dimensional coordinate points in x, y and z directions respectively, and the coordinates of each point are subtracted from the centroid to obtain s j ce , which realizes decentralization, s j ce = [x j - μ(x i ), y j - μ(x i ), z j - μ(x i )], j is the jth three-dimensional coordinate point, j ∈ m3; secondly, through the maximum distance normalization method, the maximum distance d max,i , of each decentralized point to the centroid is calculated max,i to obtain the jth scale-normalized three-dimensional coordinate point s j sc , , the coordinates of the m3 decentralized points are traversed to obtain the ith sample scale-normalized three-dimensional point cloud data set S i sc , , the scale-normalized three-dimensional point cloud data sets S sc , of N samples are traversed N sc to obtain the Nth group of scale-normalized three-dimensional point cloud data sets; finally, a second multi-source heterogeneous data set is obtained, including the preprocessed image data set I no , the preprocessed process parameter time series data set P no and the scale-normalized three-dimensional point cloud data set S sc ; It is particularly pointed out in this embodiment that the voxel filter (Voxel Filter) is used for downsampling to retain key structure points (for example, points of key positions such as edges, corners and holes of the casting), which is a mature technology in the field of three-dimensional point cloud processing, and there are ready-to-use functions and modules in many open-source three-dimensional point cloud processing libraries (such as PCL - Point Cloud Library); downsampling refers to the process of reducing the amount of data, and in casting detection, the voxel filter is used for downsampling to retain key structure points, thereby improving the accuracy and efficiency of detection.

[0020] S4, defect feature extraction is performed on the second multi-source heterogeneous data set through feature extraction technology to obtain image feature vectors, process feature vectors and structure feature vectors, and through dynamic weight feature fusion, a fusion feature vector for intelligent defect recognition is obtained, including the following steps: S4.1: image feature vector: S4.1.1: The pre-processed image data I of the i-th sample is first input into an improved ResNeSt-50 network, the improved ResNeSt-50 network comprising five stages Stage, Stage ∈ (0, 1, 2, 3, 4) being a network stage number (for example, network stage 0, network stage 3, network stage 4, etc.); then a defect attention enhancement module is added after each residual block of Stage 3 to improve the ResNeSt-50 network, including channel attention weight M i no (F c ) and spatial attention weight M Stage3 (F s ) of the defect attention enhancement module, F t Stage3 ) being a feature map output by one of the residual blocks of Stage 3, M Stage3 (F c ) = σ(MLP(AvgPool(F Stage3 ))+MLP(MaxPool((F Stage3 ))), indicating that the spatial dimension is compressed by average pooling AvgPool and maximum pooling MaxPool, and then the channel attention weight M Stage3 (F c ) is obtained through a multi-layer perceptron MLP. Stage3 (F s ) = σ(Conv t (Concat(AvgPool(F Stage3 1×1 ), MaxPool(F t Stage3 ))), F t Stage3 ) being a feature map after channel weighting, and the feature map is subjected to average pooling and maximum along the channel dimension, and then the spatial weight M t (F Stage3 ) is obtained through 1x1 convolution after splicing. s (F t Stage3 ) = M t (F Stage3 ) ⊙ F c , obtaining a spatially weighted feature map F Stage3 Stage3 k Stage3 = M s (F t Stage3 ) ⊙ F t Stage3; each output 1 feature map of all residual blocks in Stage3 is weighted by the defect attention enhancement module to obtain the final output feature map F of Stage3 la Stage3 ; S4.1.2: Add a sample feature calibration layer F after the last residual block of Stage4 i,stage4 ca (removing the influence of imaging fluctuations between samples on features), , F i,stage4 is the feature map output by Stage4, μ(F la Stage3 is the mean value of the N image data I i no through the final output feature map F la Stage3 of Stage3; finally, the image feature vector F img,i of the i-th sample based on the improved ResNeSt-50 network is obtained i no ); It needs to be specifically explained in this embodiment that ResNeSt-50 is a variant network based on ResNet (Residual Network), which contains 5 stages, Stage0: pre-processes the input image, including convolution, Batch Normalization and ReLU activation function, and a maximum pooling layer; Stage1 to Stage4: features are extracted layer by layer through residual blocks (Bottleneck), and each stage contains different number of residual blocks, for example, Stage1 contains 3 residual blocks, Stage2 contains 4, Stage3 contains 6, and Stage4 contains 3; the improved ResNeSt-50 includes a defect attention enhancement module and a sample feature calibration layer.

[0021] S4.2: Process feature vector: first, the standardized process parameter time series data set p i no (t) of the i-th sample is taken as the input of the bidirectional LSTM network, and the input shape of the network is (n, m2), n is the number of time points, and m2 is the number of process parameters, then the forward hidden state sequence h t q from t1 to tn is calculated through the forward LSTM t q =(h t1 q ,h t2 q ,...,h tn q), the backward hidden state sequence h from time point tn to t1 is calculated by the backward LSTM t h , h t h = (h tn h , h t(n-1) h ,..., h t1 h ); finally, the process feature vector F of the i-th sample is obtained by a fully connected layer FC proc,i = FC([h tn q ; h t1 h ]); It is particularly pointed out in this embodiment that the bidirectional LSTM (BiLSTM) is composed of a forward LSTM and a backward LSTM. The forward LSTM processes data from front to back in time sequence, capturing information dependence from the past to the present. The backward LSTM processes data from back to front in time sequence, capturing information dependence from the present to the past. The outputs of the two are spliced, which can simultaneously utilize the front and back context information of the time sequence. For example, the length of the pressure maintaining time may be related to the subsequent pouring effect, and may also be affected by the previous pressure change.

[0022] S4.3: Structure feature vector: the i-th sample scale normalized three-dimensional point cloud data set S i sc As the input of the PointNet++ network, the PointNet++ network adopts a 3-layer feature extraction structure, each layer containing local features f j L (encoded by MLP) with dimensions of 128, 256, 256, respectively) and MaxPool global aggregation F glob L glob L is the L-th layer feature extraction structure, F glob L = MaxPool(f1 L ,f2 L ,...f j L ,...,f m4 L ), m4 is the number of local features of the L-th layer, and the number of local features of each layer is different; finally, the 3-layer global features are obtained by splicing operation Concat to obtain cross-layer global features F glob,all glob,all = (F glob 1 ,F glob 2 ​​F glob 3 ), the structure feature vector F str,i of the i-th sample is obtained i sc =PointNet++(S glob,all glob ), m3 is the number of three-dimensional coordinate points after down-sampling of the i-th sample; It needs to be particularly pointed out that the first layer feature extraction structure F 1 1 =MaxPool(f1 1 ,f2 j ,...f 1 m3 ,...,f 1 j ), for each three-dimensional coordinate point s sc j After MLP encoding, the local feature f 1 j 1 =MLP 1 (s j sc ) is obtained; first, M center points are sampled from m3 points by FPS, M << m3 (for example, m3=1024, M=128), the J-th point ∈M, then for each center point c J 2 The corresponding local field N(c J 2 ) generates the J-th local feature f 2 J 2 , f J 2 =MLP 2 (Concat(f j 1 ,F glob 1 )|s j sc ∈N(c J 2 )) by MLP, and Concat is a concatenation operation, and the second layer feature extraction structure F glob 2 =MaxPool(f1 2 ,f2 2 ,...f J 2 ,...,f M 2 ​​); the third layer repeated FPS sampling center point M1, M1 << M, the Jjth point ∈ M1, through MLP coding to obtain local feature f Jj 3 , through maximum pooling global aggregation to obtain F glob 3 glob 3 = MaxPool(f1 3 ,f2 3 ,...f Jj 3 ,...,f M1 3 ), f Jj 3 = MLP 3 (Concat(f J 2 ,F glob 2 )|s j sc ∈ N(c Jj 3 )), N(c Jj 3 ) is the local domain of the Jjth center point; MLP L is the Lth layer MLP; the local domain is centered on each center point, and a fixed radius R (for example, R = 0.1) is set to screen out the points s j sc in the domain, for example, N(c J 2 ) = [s j sc |d(s j sc ,c J 2 ) ≤ R], d(s j sc ,c J 2 ) represents the Euclidean distance between two points; for example, in the second layer feature extraction structure, the domain N(c 10 10 2 ) of the center point c 25 sc contains points s 37 sc , s 88 sc , and s 10 2 , when calculating f 25 1 , f glob 1 will be spliced with F 37 ​​1 With F glob 1 splicing and f 88 1 With F glob 1 Concatenate the vectors, and then input the concatenated vectors into the MLP. 2 In the middle, we get f 10 2 .

[0023] This embodiment requires a specific explanation: PointNet++ is an improvement upon PointNet, addressing the issue of PointNet's inability to capture local features. Essentially, it involves "multi-layer local feature extraction + global aggregation at each layer (F...)". glob + Cross-layer fusion; FPS (Farthest Point Sampling) is an existing technology that is widely used in the field of 3D point cloud processing, especially as a preprocessing step in models such as PointNet++.

[0024] S4.4: Feature Vector Fusion: First, the image data I after preprocessing the i-th sample... i no Using Gaussian kernels G with sizes of 3×3 and 5×5 3×3 and G 5×5 to I i no Perform a Gaussian filtering operation, where std() is the standard deviation function, to obtain the image noise level σ(img,i), where σ(img,i) = std(I i no -G 3×3 (I i no )-G 5×5 (I i no Then calculate the time series dataset p of the i-th sample. N no The degree of fluctuation of the process parameters of (t) is σ(proc,i). w m2 σ(m2,i) and σ(m2,i) are the weight and standard deviation of the m2th process parameter, respectively. The weight of each process parameter is determined by its Pearson correlation coefficient r. m The ratio of the Pearson correlation coefficients with all process parameters is obtained; then, the scale-normalized 3D point cloud dataset S of the i-th sample is calculated. i sc With the corresponding standard model M 3d (Ideal 3D CAD model of sample i casting) Calculate distance error d(s) j sc M 3d(Calculated using Euclidean distance), the three-dimensional structural error σ(str,i) is obtained, σ(str,i)=std(d(s) j sc M 3d )), s j sc ∈S i sc d(s j sc M 3d () is point s j sc To the corresponding standard model M 3d The shortest distance between all 3D point clouds on the surface is obtained; then the fused feature vector F of the i-th sample is obtained. fu,i F fu,i =w img,i ×F img,i +w proc,i ×F proc,i +w str,i ×F str,i w img,i w proc,i and w proc,i These represent the corresponding weights, w img,i =exp(-σ(img,i)) / Z,w proc,i =exp(-σ(proc,i)) / Z,w img,i =exp(-σ(str,i)) / Z, Z=exp(-σ(img,i))+exp(-σ(proc,i))+exp(-σ(str,i)); finally, the fused feature vector F of N samples is obtained. fu , ; S5: This step involves inputting the fused feature vector and corresponding sample defect category label into the intelligent recognition model, and simultaneously training the model to obtain a trained intelligent defect recognition model. The steps include: S5.1: First, the fused feature vector F of the i-th sample... fu,i Connecting fully connected layers and performing a flattening operation yields the original fractional vector z(F). fu,i ), z(F fu,i )=W×Flatten(F fu,i W+b, where W is the weight matrix of the fully connected layer, b is the bias term, and each element in the vector corresponds to a defect category label Y. i,K The number of neurons in the fully connected layer is equal to the number of defect categories in the sample; then, the probability PP of sample i belonging to defect category K is output through the Softmax activation function. i,K K represents the type of defect. , z(F fu,i ) K is the score of defect category K, traverse all defect categories to get the sample i defect category probability distribution vector PP i =(PP i,0 ,PP i,1 ,PP i,2 ,...,PP i,K ), traverse the fusion feature vector F fu of N samples to get the defect category probability distribution vector PP of all samples, PP=(PP1,PP2,...,PP N ); S5.2: First, based on the true defect category label Y i,k of sample i, use the FocalLoss function to calculate the loss FL i of sample i, FL i =-a K ×(1-PP i,K ) γ ×log2(PP i,K ), a K is the weight of defect category K (for example a K =0.75), γ is the adjustment parameter (for example γ=2), traverse the real defect category label data set Y of N samples to get the batch total loss L tot (θ), ; then use the Adam optimizer to take the partial derivative of the total loss L tot (θ) with respect to all trainable parameters θ of the model (such as the weights W and bias b of the fully connected layer, and the parameters of the pre-feature extraction / fusion module, etc.), to get the gradient ∇L tot (θ), through the first-order momentum m r-1 of the last r-1 step, calculate the first-order momentum m r of the current r step, the second-order momentum v r-1 of the last r-1 step, calculate the second-order momentum v r of the current r step; secondly, based on the first-order momentum m r and the second-order momentum v r , update all trainable parameters θ of the model, the parameters θ r-1 of the last r-1 step get the training parameters θ r of the current r step, , η is the initial learning rate (for example 0.001), e is a constant (for example 0.001) to prevent the denominator from being 0, m r =ß1×m r-1 +(1-ß1)∇L tot (θ), v r =ß2×v r-1 +(1-ß2)(∇Ltot (θ)) 2 , where β1 is the decay rate of the first moment (typically taken as 0.9), β2 is the decay rate of the second moment (typically taken as 0.999), and the learning rate η is dynamically adjusted by the cosine annealing strategy new , , epoch and epoch max are the current iteration number and the maximum iteration number, respectively; and the steps S5.1 and S5.2 are repeated until the batch total loss L tot (θ) meets ≤ the corresponding threshold value (for example, 0.05) for n1 consecutive times (for example, 3 consecutive times) or reaches the maximum iteration number, and a trained defect intelligent recognition model M θ is output. It is particularly pointed out in this embodiment that the Adam optimizer updates the model parameters (such as the weights W and the bias b of the full connection layer of the classification head, and the parameters of the previous feature extraction / fusion module) by calculating the gradient, combining the momentum and the adaptive learning rate.

[0025] S6: For inputting the first multi-source heterogeneous data set of the real-time collected new sample of the precision casting into the trained defect intelligent recognition model, obtaining the defect intelligent recognition result according to the model output result, and feeding back to the management terminal for human-computer interaction, including the following steps: S6.1: Input the first multi-source heterogeneous data set of the real-time collected new sample of the precision casting into the trained defect intelligent recognition model M θ , repeat steps S3 and S4 to obtain the fusion feature vector F fu,new of the new sample of the precision casting, output the defect class probability distribution PP new of the sample, PP new =M θ (F fu,new )=(PP new,0 , PP new,1 , PP new,2 ,..., PP new,K ), PP new,0 is the probability of no defect, and PP new,K is the probability of the Kth defect. S6.2: Set the no defect threshold value as PP0 th (for example, 0.9) and the defect threshold value as PP1 th (for example, 0.8), if PP new,0 ≥ PP0 th , output no defect; if there is a Kth defect probability PP new,K ≥ PP1 thIf K is the defect category, then output the defect category K; otherwise trigger manual review; then feed the defect intelligent recognition result to the management terminal for human-computer interaction, the defect intelligent recognition result including the defect category probability and the corresponding defect category, the sample process parameter and the sample three-dimensional structure point cloud data.

[0026] It is particularly pointed out in the embodiment that F fu,new Input the model M θ , and obtain the sample defect category probability distribution PP new , PP new =(PP new,0 ,PP new,1 ,PP new,2 ), PP new,0 =0.2 (no defect probability), PP new,1 =0.2 (porous defect probability) and PP new,2 =0.05 (crack defect probability), PP new,0 =0.2 th and there is no PP new,K ≥PP1 th , so manual review is triggered, and it is further needed to confirm whether the precision casting exists defects and the defect type.

[0027] Secondly, in the drawings of the disclosed embodiments, only the structures involved in the disclosed embodiments are involved, other structures can be referred to the general design, and in the case of no conflict, the same embodiments and different embodiments of the present application can be combined with each other; Finally, the above only describes the preferred embodiments of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A deep learning algorithm-based intelligent identification method for defects in precision castings, characterized in that: Comprise: S1: for collecting and storing the image data of the defect-free and defective types of precision castings, process parameter data and three-dimensional structure point cloud data, while receiving the data generated in the intelligent identification process of precision casting defects in real time, and constructing an intelligent identification database of precision casting defects; S2: based on the intelligent identification database of precision casting defects, obtain the image data set, process parameter time series data set and three-dimensional structure point cloud data set of historical precision castings, and filter the process parameters to obtain a first multi-source heterogeneous data set for intelligent identification of defects; S3: through preprocessing technology, the first multi-source heterogeneous data set is respectively preprocessed for image data, process parameters and three-dimensional structure point cloud data to obtain a second multi-source heterogeneous data set; S4, through feature extraction technology, the second multi-source heterogeneous data set is extracted for defect features to obtain image feature vectors, process feature vectors and structure feature vectors, and through dynamic weight feature fusion, a fusion feature vector for intelligent identification of defects is obtained; S5: for inputting the fusion feature vector for intelligent identification of defects and the corresponding sample defect category label into the intelligent identification model, and training to obtain a trained defect intelligent identification model; S6: for inputting the first multi-source heterogeneous data set of the real-time collected new sample of precision castings into the trained defect intelligent identification model, obtaining the intelligent identification result of defects according to the model output result, and feeding back to the management terminal for human-computer interaction.

2. The intelligent precision casting defect identification method based on a deep learning algorithm according to claim 1, characterized in that: The S2 implementation comprises: S2.1: Based on the intelligent identification database of precision casting defects, obtain the image data set I, the process parameter time series data set P and the three-dimensional point cloud data set S to jointly constitute a multi-source heterogeneous data set DY, Y is a corresponding sample defect category label data set, the sample defect category label k includes a defect-free sample numerical label 0 and a defect sample numerical label, that is, k∈(0, 1, 2,..., K), K+1 is the type of defect category, which corresponds to a specific defect type respectively; I i is the i-th sample image data collected, and N is the number of collection, wherein P i is the process parameter time series data set of the i-th sample, p m (t) is the m-th process parameter time series data set of the i-th sample, p m (tn) is the m-th process parameter n-th time node value, m is the number of process parameters, wherein S i is the i-th point cloud data set, s m1 is the m1-th three-dimensional coordinate point of the N-th group, and m1 is the number of three-dimensional coordinate points; Y i,k is the defect category label of the i-th sample, which is k. 3.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The S2 implementation further comprises: S2.2: Obtain a set of process parameters X and a corresponding set of defect value labels y based on the N-sample process parameter dataset, wherein X m is the mth process parameter dataset, p m,N is the value of the process parameter p m of the ith sample, the defect value label includes a defect value of 1 and a non-defect value of 0, y i is the defect value of the ith sample; S2.3: First, the Pearson correlation coefficient of the N sample process parameter dataset is screened to obtain the mth process parameter p m The Pearson correlation coefficient r m of the defect numerical data set y m ; Then calculate the average value μ(p m ) and the standard deviation σ(p m ) of the N data of the mth process parameter p m , get the coefficient of variation η(p m ) of the process parameter p m ; Second, traverse all the process parameters, retain the process parameters p m |≥ the corresponding threshold r m |≥ the corresponding threshold r th and the coefficient of variation η(p m )≥ the corresponding threshold η(p m ) th , otherwise, the process parameters p m are eliminated, and the process parameter time series data set P N of the sample is obtained, , p m2 (t) is the m2th process parameter time series data set, m2 is the number of screened process parameters, m2≤m; Finally, the first multi-source heterogeneous data set DY, , P s is the screened process parameter time series data set.

4. The intelligent precision casting defect identification method based on a deep learning algorithm according to claim 1, characterized in that: The image feature vector in S4 is obtained: S4.1.1: first, the pre-processed image data I of the i-th sample is input into the improved ResNeSt-50 network, the improved ResNeSt-50 network includes 5 stages Stage, Stage ∈ (0, 1, 2, 3, 4) is the network stage number; then the defect attention enhancement module is added after each residual block of Stage3 to improve the ResNeSt-50 network, including channel attention weight M i no Input into the improved ResNeSt-50 network, the improved ResNeSt-50 network includes 5 stages Stage, Stage ∈ (0, 1, 2, 3, 4) is the network stage number; then the defect attention enhancement module is added after each residual block of Stage3 to improve the ResNeSt-50 network, including channel attention weight M c (F Stage3 ) and spatial attention weight M s (F t Stage3 ), F Stage3 is the feature map output by one of the residual blocks of Stage3, F t Stage3 represents the feature map after channel weighting, F t Stage3 =M c (F Stage3 )⊙F Stage3 , to get the spatially weighted feature map F k Stage3 =M s (F t Stage3 )⊙F t Stage3 ; each output 1 feature map of all residual blocks of Stage3 is weighted by the defect attention enhancement module, to get the last output feature map F la Stage3 of Stage3; S4.1.2: Add a sample feature calibration layer F after the last 1 residual block of Stage 4 i,stage4 ca The ResNeSt-50 network is improved; and finally the image feature vector F of the i-th sample based on the improved ResNeSt-50 network is obtained img,i = ResNeSt-50(I i no ). 5.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The process feature vector and the structure feature vector obtained in S4: S4 .2: Process feature vector: First, the standardized process parameter time series dataset p of the ith sample i no (t) as input of the bidirectional LSTM network, the input shape of the network is (n, m2), n is the number of time points, m2 is the number of process parameters, then the forward hidden state sequence h from time point t1 to tn is calculated by forward LSTM t q , h t q =(h t1 q ,h t2 q ,...,h tn q ) is calculated by backward LSTM from time point tn to t1 t h , h t h =(h tn h ,h t(n-1) h ,...,h t1 h ); finally, the process feature vector F of the ith sample is obtained by a fully connected layer FC proc,i =FC([h tn q ;h t1 h ]) S4.3: Structure feature vector: the i-th sample scale normalized three-dimensional point cloud data set S i sc As the input of the PointNet++ network, the PointNet++ network adopts a 3-layer feature extraction structure, each layer containing local features f j L and MaxPool global aggregation F glob L , F glob L is the i-th sample scale normalized three-dimensional point cloud data set S glob L = MaxPool (f1 L , f2 L ,...f j L ,..., f m4 L ), m4 is the number of local features of the L-th layer, and the number of local features of each layer is different; finally, the three global features are obtained by concatenating operation Concat to obtain cross-layer global feature F glob,all glob,all = (F glob 1 , F glob 2 , F glob 3 ), the structure feature vector F str,i of the i-th sample is obtained i sc = PointNet++ (S glob,all ), FC is a fully connected layer, and m3 is the number of three-dimensional coordinate points after downsampling of the i-th sample.​ 6.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The fusion feature vector in S4 is obtained: firstly, the pre-processed image data I i no Gaussian kernels G of size 3x3 and 5x5 are adopted 3×3 and G 5×5 The Gaussian filtering operation is performed on I i no , std() is the standard deviation function, and the image noise level σ(img, i) is obtained; then, the weight w m2 of the m2th process parameter and the standard deviation σ(m2, i) are combined to calculate the process parameter fluctuation degree σ(proc, i) of the ith sample time series data set p N no (t); secondly, the scale-normalized three-dimensional point cloud data set S i sc of the ith sample is calculated 3d The distance error d(s j sc , M 3d ) is calculated, and the three-dimensional structure error σ(str, i) is obtained; then, the fusion feature vector F fu,i of the ith sample is obtained, F fu,i = w img,i x F img,i + w proc,i x F proc,i + w str,i x F str,i , w img,i , w proc,i , and w proc,i represent the corresponding weights; finally, the fusion feature vectors F fu of the N samples are obtained . 7.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The S5 implementation includes: S5.1: firstly, the fusion feature vector F fu,i of the i-th sample is obtained fu,i , and each element in the vector corresponds to a defect category label Y i,K ; then the probability PP i,K that the sample i belongs to the defect category K is output through the Softmax activation function, K is the category of the defect category, and the defect category probability distribution vector PP i of the sample i is obtained by traversing all defect categories, PP i,0 =(PP i,1 ,PP i,2 ,...,PP i,K ), the fusion feature vector F fu of N samples is traversed, and the defect category probability distribution vector PP of all samples is obtained, PP=(PP1,PP2,...,PP i ,...,PP N ). 8.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The S5 implementation further comprises: S5.2: first, based on the true defect category label Y of sample i i,K , using the FocalLoss function, calculate the loss FL of sample i i , traverse the real defect category label data set Y of N samples, get the batch total loss L tot (θ);Then use the Adam optimizer to derive the gradient ∇L tot (θ) with respect to all trainable parameters θ of the model, get the gradient ∇L tot (θ) of the current r step, through the first order momentum m r-1 of the last r-1 step, calculate the first order momentum m r of the current r step, the second order momentum v r-1 of the last r-1 step, calculate the second order momentum v r of the current r step;Second, based on the first order momentum m r and the second order momentum v r , update all trainable parameters θ of the model, the parameters θ of the last r-1 step r-1 get the training parameters θ of the current r step r , , η is the initial learning rate, e is a constant to prevent the denominator from being 0, and the learning rate η is dynamically adjusted by the cosine annealing strategy new , , epoch and epoch max are the current iteration number and the maximum iteration number respectively;Finally, repeat the steps S5.1 and S5.2 until the batch total loss L tot (θ) is less than or equal to the corresponding threshold for n1 consecutive times or the maximum iteration number is reached, output the trained defect intelligent recognition model M θ . 9.The intelligent precision casting defect identification method based on deep learning algorithm according to claim 1, characterized in that: The S6 implementation comprises: S6.1: input the first multi-source heterogeneous data set of the real-time collected new sample of precision casting into the trained intelligent defect recognition model M θ , repeat steps S3 and S4 to obtain the fusion feature vector F of the new sample of precision casting fu,new , output the sample defect category probability distribution PP new , PP new =M θ (F fu,new )=(PP new,0 ,PP new,1 ,PP new,2 ,...,PP new,K ), PP new,0 is the probability of no defect, and PP new,K is the probability of the Kth defect; S6.2: Set the threshold of no defect as PP0 th and the threshold of defect as PP1 th , if PP new,0 ≥ PP0 th , output no defect; if the probability of the Kth defect exists PP new,K ≥ PP1 th , output the Kth defect; otherwise, trigger manual review; then feed the intelligent defect recognition result to the management terminal for human-computer interaction, which includes the probability of defect category, the corresponding defect category, the sample process parameters and the sample three-dimensional structure point cloud data.

Images