Casting defect identification model training method, defect identification method, device and system

Abstract

The application provides a cast defect recognition model training method, a defect recognition method, a device and a system. A data set of a cast X-ray image is determined, and the data set is divided into two subsets according to the categories of defects. The defect types of the images in the first subset include linear defects and hole-shaped defects, and the defect types of the images in the second subset include shrinkage defects and high-density inclusion defects. The first subset is augmented based on a first network model to obtain a first data set. The second subset is augmented based on a second network model to obtain a second data set. The first data set, the second data set and a cast real X-ray image data set are mixed, and the mixed data set is augmented again to obtain a training set. The neural network is trained based on the training set, and a cast defect recognition model is obtained after the training. The application increases the diversity of defect appearances in the training set in a simulation manner, realizes a breakthrough from manual detection to machine detection, and improves the detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of casting defect recognition, and more particularly relates to a casting defect recognition model training method, a defect recognition method, a device and a system. BACKGROUND

[0002] With the increasing requirements of core equipment in the fields of aerospace, weapon equipment, etc. on parts, metal parts develop in the direction of structural complexity, such as large wall thickness difference, multi-intersection structure, multi-ring, curved surface, blind hole, complex internal cavity, etc. Hole porosity, inclusion, and incomplete fusion of welding are main internal defects of castings, and the main detection means is "X-ray flaw detection two-dimensional DR imaging + manual evaluation of film", that is, a special person visually detects flaw images to detect defects and manually marks and records defect information.

[0003] The existing manual film evaluation mode cannot realize rapid online identification of defects, and it is difficult to meet the current intelligent and efficient production needs of major national defense equipment manufacturing. X-ray image and deep learning driven defect detection are the frontiers and development trends of automatic evaluation of casting hole porosity, inclusion, and incomplete fusion of welding. However, the automatic evaluation technology based on deep learning relies on a large number of image data sets, and the defect images generated by the existing data augmentation method have single morphology, and the identification of part defect categories is difficult to unify. There are problems such as insufficient samples, imbalance, and lack of scientificity in the data set production, which leads to the quality of the data set used for training not meeting the engineering requirements, hindering the transformation from "manual offline film evaluation mode" to "machine online automatic film evaluation mode". SUMMARY

[0004] In view of the defects of the prior art, the purpose of the present application is to provide a casting defect recognition model training method, a defect recognition method, a device and a system, which aims to solve the problems that the data augmentation method of the existing casting X-ray image data set is single and the identification of part defect categories is difficult to unify, leading to the inability to transform from manual film selection mode to machine online automatic mode.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present application provides a casting defect recognition model training method, comprising the following steps:

[0006] determining a data set of casting X-ray images, and dividing the data set into two subsets according to the categories of defects; wherein the defect types of images in the first subset are linear defects and hole defects, and the defect types of images in the second subset are shrinkage defects and high-density inclusion defects; wherein there is only one type of casting defect on each image in the data set;

[0007] augment the first subset based on a first network model to obtain an augmented first dataset; the first network model is obtained by combining a generative adversarial network with a deep convolutional neural network; the generative adversarial network focuses on the whole of an image more than the local part of the image, and can generate a simulated image that is greatly different in defect morphology from images in the first subset and contains the same type of defects, effectively improving the morphology diversity of linear defects and hole defects, so that the morphology of linear defects and hole defects in the first dataset is rich and diverse;

[0008] augment the second subset based on a second network model to obtain an augmented second dataset; the second network model is obtained by combining a variational autoencoder with a deep convolutional neural network; the variational autoencoder focuses on the local part of an image more than the whole of the image, and can generate a simulated image that is greatly different in defect morphology from images in the second subset and contains the same type of defects, effectively improving the morphology diversity of loose defects and high-density inclusion defects, so that the morphology of loose defects and high-density inclusion defects in the second dataset is rich and diverse;

[0009] mix the first dataset, the second dataset, and a cast real X-ray image dataset, and perform regular secondary augmentation on the mixed dataset to obtain a training set;

[0010] train a neural network based on the training set, and obtain a cast defect recognition model after training; the neural network is used to recognize cast defects and defect types based on received images.

[0011] In one possible example, the first network model includes a generator and a discriminator; wherein:

[0012] The first layer of the generator uses a full connection layer, a ReLu activation function, and a Reshape function; the second to fourth layers all use a deconvolution layer, a BN layer, and a ReLu activation function; the fifth layer uses a deconvolution layer and a Hard Tanh activation function; the Reshape function is used to change the output dimension, and the BN layer is used to speed up the training;

[0013] The first layer of the discriminator uses a convolution layer and a Leaky ReLu activation function; the second to fourth layers all use a convolution layer, a BN layer, and a Leaky ReLu activation function, and the fifth layer uses a Reshape function, a full connection layer, and a Sigmoid activation function.

[0014] In one possible example, the second network model includes an encoder and a decoder; wherein:

[0015] The first to fourth layers of the encoder use convolutional layers, BN layers and LeakyReLu activation functions, the fifth layer uses a reshape function, a fully connected layer and a ReLu activation function, and the sixth layer uses a fully connected layer;

[0016] The first layer of the decoder uses a fully connected layer, a ReLu activation function and a reshape function, the second to fourth layers all use deconvolutional layers, BN layers and ReLu activation functions, and the fifth layer uses a deconvolutional layer and a Sigmoid activation function.

[0017] In one possible example, the LeakyReLu activation function is used to prevent the case where the operation result is 0 in the convolution operation and the deconvolution operation, resulting in image distortion.

[0018] In one possible example, the first subset is augmented based on the first network model to obtain an augmented first data set, and the first subset is augmented based on the second network model to obtain an augmented second data set, specifically as follows:

[0019] When the first network model augments the first subset or the second network model augments the second subset, different convolution kernel sizes and image resolutions are set to generate multiple groups of augmented simulation data sets;

[0020] From the generated simulation data set under each defect, multiple groups of data sets with relatively real defect topographies are selected as screening simulation data sets for each defect according to the topography of the defect in the simulation image;

[0021] The screening simulation data sets of the four defects and the real X-ray image data set of the castings are mixed according to a preset proportion to obtain a mixed data set;

[0022] When the mis-screening rate of the simulation images selected from the mixed data set is not close to the preset proportion, it is indicated that the final simulation data set of the four defects is substandard, then the simulation images that are not selected as real images are combined into a new training subset, which is mixed with the real X-ray images of the castings as a new data set of X-ray images, and new first and second subsets are divided from the new data set according to the defect types, and then the new subsets are augmented based on the first and second network models, respectively, and the above steps are iteratively executed until the mis-screening rate is close to the preset proportion; the mis-screening rate refers to the proportion of real images in the selected simulation images;

[0023] When the mis-screening rate of the simulation images selected from the mixed data set is close to the preset proportion, it is indicated that the screening simulation data sets of the four defects are up to standard, indicating that the simulated defect images are close to the real defect images, and the screening simulation data sets of the four defects are respectively matched as the first and second data sets according to the defect types.

[0024] In a second aspect, the present application provides a cast defect identification method, comprising the following steps:

[0025] determining a cast X-ray image;

[0026] inputting the cast X-ray image into a cast defect identification model to identify defects of a corresponding region of the cast based on the cast X-ray image; the cast defect identification model is trained by the training method of any one of claims 1 to 5.

[0027] In one possible example, the neural network used by the cast defect identification model is a neural network for automatic segmentation, positioning and classification of defects in a cast flaw detection image.

[0028] In a third aspect, the present application provides a cast defect identification model training device, comprising:

[0029] a data set determination module for determining a data set of a cast X-ray image, and dividing the data set into two subsets according to the categories of defects; wherein the defect types of the images in the first subset are linear defects and hole-shaped defects, and the defect types of the images in the second subset are shrinkage defects and high-density inclusion defects; wherein there is only one type of cast defect on each image in the data set;

[0030] a defect diversity simulation module for augmenting the first subset based on a first network model to obtain an augmented first data set; the first network model is obtained by combining a generative adversarial network with a deep convolutional neural network; the generative adversarial network focuses on the overall image rather than the local image, and can generate simulated images with large differences in defect morphology and containing the same type of defects in the first subset, effectively improving the morphology diversity of linear defects and hole-shaped defects, so that the morphology of linear defects and hole-shaped defects in the first data set is rich and diverse; and augmenting the second subset based on a second network model to obtain an augmented second data set; the second network model is obtained by combining a variational autoencoder with a deep convolutional neural network; the variational autoencoder focuses on the local image rather than the overall image, and can generate simulated images with large differences in defect morphology and containing the same type of defects in the second subset, effectively improving the morphology diversity of shrinkage defects and high-density inclusion defects, so that the morphology of shrinkage defects and high-density inclusion defects in the second data set is rich and diverse;

[0031] a training set determination module for mixing the first data set, the second data set and a cast real X-ray image data set, and performing regular secondary augmentation on the mixed data set to obtain a training set;

[0032] The recognition model training module is configured to train a neural network based on the training set, and a casting defect recognition model is obtained after the training.

[0033] In a fourth aspect, the application provides a casting defect recognition model training system, comprising a memory and a processor.

[0034] The memory is configured to store a computer program.

[0035] The processor is configured to implement the training method of the first aspect when executing the computer program.

[0036] In a fifth aspect, the application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the training method of the first aspect.

[0037] Compared with the prior art, the above technical solutions of the application have the following advantages:

[0038] The application provides a casting defect recognition model training method, a defect recognition method, a device and a system, two models of DCGAN and DCVAE are constructed and optimized, high-quality generation of various defects can be achieved, the poor simulation generation quality of traditional models for casting defect images is changed, the problem of insufficient data set balance is solved, the defect recognition model with high detection reliability can be trained by achieving the defect morphology diversity and balance of the training data set, the breakthrough from manual detection to machine detection of casting defects is achieved, the defect detection efficiency and accuracy are improved by using machine detection, and the time-consuming and difference of manual detection are avoided.

[0039] The application provides a casting defect recognition model training method, a defect recognition method, a device and a system, the dependence on real samples of the previous data set augmentation method is reduced, so that the defect sample deficiency can be simulated and generated for augmentation, and the problem of few data set samples is solved.

[0040] The application provides a casting defect recognition model training method, a defect recognition method, a device and a system, when the "mutation-evolution" training is performed, the simulation image with large difference from the real defect image morphology can be generated, and the defect morphology diversity of the data set can be improved.

[0041] The application provides a casting defect recognition model training method, a defect recognition method, a device and a system, the expert cross-evaluation method is proposed, the defect calibration accuracy can be improved, the quality of the simulation generated image can be detected, the problem of insufficient scientific nature of the data set is solved, and the data set made according to the application has high reliability. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a cast defect recognition model training method flowchart provided by the embodiment of the present application;

[0043] Figure 2 is a cast X-ray image data set augmentation and production method flowchart for a neural network provided by the embodiment of the present application;

[0044] Figure 3 is a linear defect and hole defect image example diagram generated by the DCGAN provided by the embodiment of the present application;

[0045] Figure 4 is a shrinkage defect and high-density inclusion defect image example diagram generated by the DCVAE provided by the embodiment of the present application;

[0046] Figure 5 is a cast defect recognition model training device architecture diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0049] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0050] Figure 1 is a cast defect recognition model training method flowchart provided by the embodiment of the present application;As shown in Figure 1 , comprising the following steps:

[0051] S101, determine the data set of the casting X-ray image, and divide the data set into two subsets according to the category of defects; wherein the defect types of the images in the first subset are linear defects and hole defects, and the defect types of the images in the second subset are shrinkage defects and high-density inclusion defects; wherein there is only one type of casting defect on each image in the data set;

[0052] S102, augment the first subset based on the first network model to obtain the augmented first data set; the first network model is obtained by combining a generative adversarial network with a deep convolutional neural network; the generative adversarial network focuses on the whole more than the local of the image, can generate simulated images with large morphological differences from the images in the first subset and containing the same type of defects, effectively improving the morphological diversity of linear defects and hole defects, and making the morphologies of linear defects and hole defects in the first data set rich and diverse;

[0053] S103, augment the second subset based on the second network model to obtain the augmented second data set; the second network model is obtained by combining a variational autoencoder with a deep convolutional neural network; the variational autoencoder focuses on the local more than the whole of the image, can generate simulated images with large morphological differences from the images in the second subset and containing the same type of defects, effectively improving the morphological diversity of shrinkage defects and high-density inclusion defects, and making the morphologies of shrinkage defects and high-density inclusion defects in the second data set rich and diverse;

[0054] S104, mix the first data set, the second data set and the real X-ray image data set of the casting, and perform regular secondary augmentation on the mixed data set to obtain a training set;

[0055] S105, train the neural network based on the training set, and obtain a casting defect recognition model after training; the neural network is used for identifying casting defects and defect types in the received image.

[0056] Specifically, in view of the problems of single data augmentation method and difficult to unify the identification of part of defect categories in the data set, since the quality requirement of the data set is very high in engineering, the traditional position transformation, gray level transformation, image distortion and other data set expansion methods and traditional data labeling methods cannot be applied to the data augmentation and production of casting X-ray images with complex conditions. The application first starts from deep learning, constructs two models of DCGAN and DCVAE which can generate high-quality casting defect images, and determines the most suitable model structure and image resolution for defect image generation around the convolution kernel size and image resolution of the generation model. Secondly, the expert cross-evaluation method is proposed, which can accurately label the area where the defect position and category are difficult to unify and improve the quality of the data set.

[0057] To address the issues of insufficient sample size, lack of balance, and inadequate scientific rigor in images generated by data augmentation methods, Figure 2 This is a flowchart of the method for augmenting and fabricating X-ray image datasets of castings for neural networks, provided in an embodiment of the present invention. Figure 2 As shown, it includes the following steps:

[0058] 1. Based on Generative Adversarial Networks (GANs), a deep convolutional neural network is combined with GANs to construct and optimize a simulation generative model for linear and pore defects in castings, abbreviated as DCGAN. This model combines a deep convolutional neural network with a GAN, replacing spatial pooling layers with convolutional layers. During the downsampling process, the network can learn the sampling method itself, changing the original method of discarding fixed pixel positions.

[0059] Wherein, the generator convolution kernel size is w f ×h f (3×3, 5×5, or 7×7 can be selected), with a stride of 2; the first layer uses Fully Connected+ReLU+Reshape; layers 2-4 use Transposed Conv+BN+ReLU, and the fifth layer is a deconvolutional layer using Conv+Hard Tanh. The discriminator uses Conv+Leaky ReLU in the first layer; Conv+BN+LeakyReLU is used in layers 2-4, with the LeakyReLU hyperparameter set to 0.2; the fifth layer uses Reshape+Fully Connected+Sigmoid. The structures of the model generator and discriminator are shown in Tables 1 and 2.

[0060] Table 1. Detailed information on the DCGAN generator structure.

[0061]

[0062]

[0063] Table 2. Detailed information on the DCGAN discriminator structure.

[0064]

[0065] Specifically, in the table above, Fully Connected is a fully connected layer, Reshape is a reshape function used to change the output dimension, Transposed Conv is a deconvolution, BN layer is used to speed up training, Conv is a convolution, and ReLU, Hard Tanh, Leaky ReLU and Sigmoid are different activation functions.

[0066] Specifically, the kernel size in the generator is variable, which is w. f ×h f (e.g., 3×3, 5×5, 7×7), the first layer output size is h / 16×w / 16×512; the second layer output size is h / 8×w / 8×256; the third layer output size is h / 4×w / 4×128; the fourth layer output size is h / 2×w / 2×64; the fifth layer output size is h×w×3.

[0067] The convolution kernel size in the discriminator is w f ×h f The first layer output size is h / 2×w / 2×64; the second layer output size is h / 4×w / 4×128; the third layer output size is h / 8×w / 8×256; the fourth layer output size is h / 16×w / 16×512; and the fifth layer output size is 1.

[0068] In some layers of the generator and discriminator, the LeakyReLu function is used to effectively prevent image distortion caused by the result of 0 in convolution and deconvolution operations.

[0069] Specifically, the DCGAN workflow is as follows: z is sampled from the ld-dimensional latent space (ld-dimensional normal distribution space). i And input it into the generator (generative model) G to generate data G(z) i The generated data is input into the discriminator (discrimination model) D, and the discriminator D determines G(z). i The probability of a given data point being real is represented by the generator (or discriminator). The DCGAN training process involves alternately training the generator and discriminator. When training the generator, the discriminator is kept constant to ensure that the data generated by the generator is correctly identified as real data by the discriminator. Conversely, when training the discriminator, the generator is kept constant to ensure that the data generated by the generator is accurately identified as non-real data.

[0070] 2. Using the model constructed above, simulations of linear and porosity defects in castings were generated. Nine sets of training parameters were generated based on different convolution kernel sizes (3×3, 5×5, 7×7) and image resolutions (64×64, 128×128, 256×256). Simultaneous training was performed, and nine batches of augmented simulation images of linear and porosity defects were generated, such as... Figure 3 As shown, Figure 3 (a) is the generated image of the linear defect; (b) is the generated image of the pore defect.

[0071] 3. Based on the variational autoencoder (VAE), the deep convolutional neural network is combined with it, the spatial pooling layer is replaced with the convolution layer, the casting shrinkage defect and high density inclusion defect simulation generation model is constructed and optimized, which is called DCVAE. Among them, the size of the encoder convolution kernel is 5*5, the super parameter of LeakyReLu is 0.2, layers 1-4 use Conv+BN+LeakyReLu, layer 5 uses Reshape+Fully Connected+ReLu, and layer 6 uses Fully Connected. The first layer of the decoder is Fully Connected+ReLu+Reshape, layers 2-4 are Transposed Conv+BN+ReLu, and the fifth layer is Transposed Conv+Sigmoid. Among them, the structure of the encoder and the decoder of the model is shown in table 3 and table 4.

[0072] Table 3. Detailed information example of DCVAE encoder structure

[0073]

[0074] Table 4. Detailed information example of DCVAE decoder structure

[0075]

[0076] Specifically, in the encoder, the convolution kernel size is 5*5, the output size of the first layer is 64*64*16; the output size of the second layer is 32*32*32; the output size of the third layer is 16*16*64; the output size of the fourth layer is 8*8*128; the output size of the fifth layer is 64; the output size of the sixth layer is 2

[0077] In the decoder, the convolution kernel size is 5*5, the output size of the first layer is 8*8*128; the output size of the second layer is 16*16*64; the output size of the third layer is 32*32*32; the output size of the fourth layer is 64*64*16; the output size of the fifth layer is 128*128*3

[0078] In some layers of the encoder and the decoder, LeakyReLu function is used to effectively prevent the image distortion caused by the operation result of 0 in convolution operation and deconvolution operation.

[0079] Specifically, the workflow of DCVAE is as follows: an image x i After inputting into the deep autoencoder, the mean m and the variance v of the normal distribution to which the image belongs are generated by the encoder, and the constraints that the l2 norms of the mean m and the variance v are both zero are applied. The image x is sampled from the normal distribution As the multiplicative noise of the variance v, the image x is obtained by combining m and vi The mapping z in the latent space. The decoder decodes z to generate The update of the model parameters is achieved by minimizing the reconstruction error and the regularization term imposed on the mean m and the variance v. After the DCVAE training is completed, the decoder is used as a generative model to decode the samples obtained by sampling from the normal distribution to generate images.

[0080] 4. Through the model constructed above, the shrinkage defects and high-density inclusion defects of the castings are simulated and generated. Nine groups of training parameters are generated according to different convolution kernel sizes (3x3, 5x5, 7x7) and image resolutions (64x64, 128x128, 256x256), and the nine batches of augmented shrinkage defect and high-density inclusion defect simulation images are generated, as shown in Figure 4 Figure 4 Fig. (a) is a generated shrinkage defect image; and (b) is a generated high-density inclusion image.

[0081] 5. By extracting the gray scale, edge curvature, background texture, and hue information of real images of linear defects, hole defects, shrinkage defects, and high-density inclusion defects, a multi-dimensional defect reality evaluation standard is constructed.

[0082] 6. According to the above standard, the nine batches of images of linear defects and hole defects, shrinkage defects and high-density inclusion defects are judged respectively, and the best three groups of images are selected for subsequent operation.

[0083] 7. The four types of defect images are mixed, and according to the actual image quantity and the simulation image quantity, the actual images and the simulation images are mixed in a certain proportion (such as 1:1) to obtain a sample set for expert evaluation.

[0084] 8. The experts screen the simulation images in the total sample set, and according to the number of simulation images, the simulation images in the sample set that the experts think are screened out, to ensure that the mis-screening rate of the screened simulation images and the proportion of the actual images in the total sample set are close, such as a mis-screening rate of 48%, that is, 100 expert-considered simulation images are selected, of which 52 are simulation images and 48 are actual images. The mis-screening rate refers to the proportion of images that are actually real images but are screened as simulation images, or are actually simulation images but are screened as real images.

[0085] If the above standard is not met, the optimal parameters are selected, the simulation images that are not screened out by the experts and have large differences with the real defect image morphology are mixed with the original real images in a certain proportion (determined according to the total number of images required) to construct a new batch of training set, and the "mutation-evolution" defect generation is repeated from steps 1-8 until the 48% mis-screening rate is met or the number of simulation images not screened out in several training reaches the expected number. ​

[0086] 9. The last obtained evaluated simulation image is mixed with existing real defect images as an augmented sample, and secondary augmentation is performed by using conventional mathematical methods such as regular folding, rotation, linear transformation, etc., to obtain defect image data for making a data set.

[0087] 10. For the defect image data, expert cross-evaluation is performed, different experts independently label the images in the total sample, and different defects are re-evaluated according to the labeling results, to obtain defect samples after expert cross-evaluation, and to solve the problems of mixed defect structures and difficult-to-distinguish defects in the data set making process.

[0088] 11. According to the results of expert cross-evaluation, a cast X-ray image data set for neural network training is made.

[0089] Specifically, the neural network used by the cast defect recognition model is all neural network models used for automatic segmentation, positioning and classification of cast flaw images. If such models want to obtain higher detection accuracy, they depend on the number of defects in the data set and the accuracy of labeling. In actual application, the number of defect images is very limited, and different people have different standards for segmentation, positioning and classification of different defects, resulting in low detection accuracy of the neural network after training. The simulation image generation method mentioned above will effectively solve the problem of the small number of defect images, and the expert cross-evaluation will effectively solve the problem of labeling accuracy.

[0090] Figure 5 is a cast defect recognition model training device architecture provided by an embodiment of the present application, as shown in Figure 5 , comprising:

[0091] The data set determination module 510 is used to determine the data set of the cast X-ray image, and divide the data set into two subsets according to the category of the defect; wherein the defect types of the images in the first subset are linear defects and hole-shaped defects, and the defect types of the images in the second subset are shrinkage defects and high-density inclusion defects; wherein there is only one kind of cast defect on each image in the data set;

[0092] The defect diversity simulation module 520 is configured to augment the first subset based on a first network model to obtain an augmented first data set; the first network model is obtained by combining a generative adversarial network with a deep convolutional neural network; the generative adversarial network focuses on the whole of an image more than the part of the image, and can generate a simulated image that is significantly different in defect morphology from images in the first subset and contains the same type of defects, effectively improving the morphology diversity of linear defects and hole-shaped defects, so that the morphology of linear defects and hole-shaped defects in the first data set is rich and diverse; and augment the second subset based on a second network model to obtain an augmented second data set; the second network model is obtained by combining a variational autoencoder with a deep convolutional neural network; the variational autoencoder focuses on the part of an image more than the whole of the image, and can generate a simulated image that is significantly different in defect morphology from images in the second subset and contains the same type of defects, effectively improving the morphology diversity of loose defects and high-density inclusion defects, so that the morphology of loose defects and high-density inclusion defects in the second data set is rich and diverse;

[0093] The training set determination module 530 is configured to mix the first data set, the second data set, and a cast real X-ray image data set, and perform regular secondary augmentation on the mixed data set to obtain a training set.

[0094] The recognition model training module 540 is configured to train a neural network based on the training set to obtain a cast defect recognition model; the neural network is used to recognize cast defects and defect types in a received image based on the image.

[0095] It can be understood that the detailed function implementation of each of the above modules can be referred to the description in the foregoing method embodiments, which will not be described herein.

[0096] In addition, the embodiment of the present application provides a cast defect recognition model training system, which comprises a memory and a processor.

[0097] The memory is configured to store a computer program.

[0098] The processor is configured to implement the method in the above embodiments when executing the computer program.

[0099] In addition, the present application also provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method in the above embodiments is implemented.

[0100] Based on the method in the above embodiments, the embodiment of the present application provides a computer program product, which, when executed on a processor, causes the processor to execute the method in the above embodiments.

[0101] Based on the method in the above embodiment, the embodiment of the application further provides a chip comprising one or more processors and interface circuitry. Optionally, the chip can further comprise a bus. Wherein:

[0102] The processor can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The processor described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method and step disclosed in the embodiment of the application can be implemented or executed. The general processor can be a microprocessor or any conventional processor.

[0103] The interface circuitry can be used for the transmission or reception of data, instructions or information. The processor can process the data, instructions or other information received by the interface circuitry, and can send the processed information out through the interface circuitry.

[0104] Optionally, the chip further comprises a memory, which can include a read-only memory and a random access memory, and provide operation instructions and data for the processor. Part of the memory can also include a non-volatile random access memory (NVRAM).

[0105] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which can be stored in an operating system).

[0106] Optionally, the interface circuitry can be used to output the execution result of the processor.

[0107] It should be noted that the functions of the processor and the interface circuitry respectively can be realized by hardware design, software design or a combination of hardware and software, which is not limited here.

[0108] It should be understood that each step of the above method embodiment can be completed by logic circuits in the form of hardware or instructions in the form of software in the processor.

[0109] It can be understood that the size of the serial number of each step in the above embodiment does not mean the execution order. The execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application. In addition, in some possible implementation ways, each step in the above embodiment can be selectively executed, partially executed or fully executed according to the actual situation, which is not limited here.

[0110] It can be appreciated that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0111] The method steps in the embodiments of the present application can be implemented in the form of hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0112] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0113] Those skilled in the art will readily understand that the above description is only preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a casting defect identification model, characterized in that, Includes the following steps: A dataset of X-ray images of castings is determined, and the dataset is divided into two subsets according to the type of defects. The first subset contains images with linear defects and porosity defects, while the second subset contains images with shrinkage defects and high-density inclusion defects. Each image in the dataset contains only one type of casting defect. The first subset is augmented based on the first network model to obtain the augmented first dataset; the first network model is obtained by combining generative adversarial network and deep convolutional neural network; the generative adversarial network focuses on the whole of the image rather than the local, and can generate simulated images with large differences in defect morphology from the images in the first subset but containing the same type of defects, effectively improving the morphological diversity of linear defects and pore defects, making the morphology of linear defects and pore defects in the first dataset rich and diverse. The second subset is augmented based on the second network model to obtain the augmented second dataset; the second network model is obtained by combining variational autoencoder and deep convolutional neural network; the variational autoencoder focuses on the local of the image more than the whole, and can generate simulated images with large differences in defect morphology from the images in the second subset but containing the same type of defects, effectively improving the morphological diversity of shrinkage defects and high-density inclusion defects, making the morphology of shrinkage defects and high-density inclusion defects in the second dataset rich and diverse; The first dataset, the second dataset, and the dataset of real X-ray images of castings are mixed, and the mixed dataset is subjected to conventional secondary augmentation to obtain the training set. The neural network is trained based on the training set to obtain a casting defect recognition model; the neural network is used to identify casting defects and defect types in received images. Specifically, the augmentation of the first subset based on the first network model to obtain the augmented first dataset, and the augmentation of the first subset based on the second network model to obtain the augmented second dataset, are as follows: When the first network model augments the first subset or the second network model augments the second subset, different convolutional kernel sizes and image resolutions are set to generate multiple sets of augmented simulation datasets. From the simulation datasets generated for each type of defect, select multiple datasets with relatively realistic defect morphology based on the morphology of the defect in the simulation image, and use them as the screening simulation datasets for each type of defect. The four types of defect screening simulation datasets are mixed with the real X-ray image datasets of castings according to a preset ratio to obtain a mixed dataset; If the false screening rate of the simulated images selected from the mixed dataset is not close to the preset ratio, it indicates that the final simulated dataset of the four types of defects does not meet the standard. Then, the simulated images that were not selected as real images are combined into a new training subset, which is mixed with real X-ray images of castings to form a new X-ray image dataset. From this dataset, a new first subset and a second subset are divided according to the defect type. Then, the new subsets are augmented based on the first network model and the second network model, respectively. The above steps are iteratively executed until the false screening rate is close to the preset ratio. The false screening rate refers to the proportion of real images in the selected simulated images. When the false screening rate of the simulated images selected from the mixed dataset is close to the preset ratio, it indicates that the screening simulation datasets of the four types of defects meet the standard, indicating that the simulated defect images are close to the real defect images. The screening simulation datasets of the four types of defects are then compared with the first dataset and the second dataset according to the defect type.

2. The method according to claim 1, characterized in that, The first network model includes a generator and a discriminator; wherein: The generator's first layer uses a fully connected layer, a ReLU activation function, and a Reshape function; layers 2 through 4 all use deconvolutional layers, batch normalization (BN) layers, and a ReLU activation function; the fifth layer uses a deconvolutional layer and a Hard Tanh activation function; the Reshape function is used to change the output dimension, and the BN layer is used to speed up training. The first layer of the discriminator uses a convolutional layer and the Leaky ReLU activation function; the second to fourth layers all use convolutional layers, BN layers, and the Leaky ReLU activation function; and the fifth layer uses the Reshape function, a fully connected layer, and the Sigmoid activation function.

3. The method according to claim 1, characterized in that, The second network model includes an encoder and a decoder; wherein: The encoder uses convolutional layers, batch normalization (BN) layers, and Leaky ReLU activation function in layers 1 to 4; it uses reshape function, fully connected layer, and ReLU activation function in layer 5; and it uses fully connected layer in layer 6. The decoder uses a fully connected layer, ReLU activation function, and Reshape function in layer 1; deconvolution layer, BN layer, and ReLU activation function in layers 2 to 4; and deconvolution layer and Sigmoid activation function in layer 5.

4. The method according to claim 2 or 3, characterized in that, The LeakyReLu activation function is used to prevent image distortion caused by the result of 0 in convolution and deconvolution operations.

5. A method for identifying casting defects, characterized in that, Includes the following steps: Determine the X-ray image of the casting; The X-ray image of the casting is input into the casting defect recognition model to identify defects in the corresponding area of ​​the casting based on the X-ray image; the casting defect recognition model is trained using the training method described in any one of claims 1 to 4.

6. The method according to claim 5, characterized in that, The neural network used in the casting defect identification model is a neural network for automatic segmentation, localization, and classification of defects in casting flaw detection images.

7. A training device for a casting defect identification model, characterized in that, include: The dataset determination module is used to determine the dataset of X-ray images of castings and divide the dataset into two subsets according to the type of defects. The first subset contains images with linear defects and porosity defects, while the second subset contains images with shrinkage defects and high-density inclusion defects. Each image in the dataset contains only one type of casting defect. A defect diversity simulation module is used to augment the first subset based on a first network model to obtain an augmented first dataset. The first network model is obtained by combining a generative adversarial network (GAN) with a deep convolutional neural network (CNN). The GAN focuses on the overall image over the local image, and can generate simulation images that have significantly different defect morphologies from those in the images in the first subset but contain the same type of defects, effectively improving the morphological diversity of linear and porosity defects, making the morphologies of linear and porosity defects in the first dataset rich and diverse. The module also augments the second subset based on a second network model to obtain an augmented second dataset. The second network model combines a variational autoencoder (VAE) with a deep convolutional neural network (CNN). The VAE focuses on the local image over the overall image, and can generate simulation images that have significantly different defect morphologies from those in the images in the second subset but contain the same type of defects, effectively improving the morphological diversity of shrinkage defects and high-density inclusion defects, making the morphologies of shrinkage defects and high-density inclusion defects in the second dataset rich and diverse. The training set determination module is used to mix the first dataset, the second dataset, and the real X-ray image dataset of castings, and perform conventional secondary augmentation on the mixed dataset to obtain the training set. The identification model training module is used to train the neural network based on the training set, and obtain a casting defect identification model after training; the neural network is used to identify casting defects and defect types in the received images; Specifically, the augmentation of the first subset based on the first network model to obtain the augmented first dataset, and the augmentation of the first subset based on the second network model to obtain the augmented second dataset, are as follows: When the first network model augments the first subset or the second network model augments the second subset, different convolutional kernel sizes and image resolutions are set to generate multiple sets of augmented simulation datasets. From the simulation datasets generated for each type of defect, select multiple datasets with relatively realistic defect morphology based on the morphology of the defect in the simulation image, and use them as the screening simulation datasets for each type of defect. The four types of defect screening simulation datasets are mixed with the real X-ray image datasets of castings according to a preset ratio to obtain a mixed dataset; If the false screening rate of the simulated images selected from the mixed dataset is not close to the preset ratio, it indicates that the final simulated dataset of the four types of defects does not meet the standard. Then, the simulated images that were not selected as real images are combined into a new training subset, which is mixed with real X-ray images of castings to form a new X-ray image dataset. From this dataset, a new first subset and a second subset are divided according to the defect type. Then, the new subsets are augmented based on the first network model and the second network model, respectively. The above steps are iteratively executed until the false screening rate is close to the preset ratio. The false screening rate refers to the proportion of real images in the selected simulated images. When the false screening rate of the simulated images selected from the mixed dataset is close to the preset ratio, it indicates that the screening simulation datasets of the four types of defects meet the standard, indicating that the simulated defect images are close to the real defect images. The screening simulation datasets of the four types of defects are then compared with the first dataset and the second dataset according to the defect type.

8. A training system for a casting defect identification model, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is configured to implement the training method as described in any one of claims 1-4 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the training method as described in any one of claims 1-4.

Images