An industrial CT image recognition method and device and an industrial CT image recognition system

Abstract

The application provides an industrial CT image recognition method and device and an industrial CT image recognition system, and belongs to the technical field of defect recognition.The industrial CT image recognition method comprises the following steps: color feature vectors, texture feature vectors and shape feature vectors of an industrial CT image are extracted; the color feature vectors, the texture feature vectors and the shape feature vectors are fused to obtain fused feature vectors; the industrial CT image and the fused feature vectors are spliced to serve as the input of a first convolutional neural network and a second convolutional neural network, the first convolutional neural network outputs a first feature vector, and the second convolutional neural network outputs a second feature vector; the first feature vector and the second feature vector are weighted and fused to obtain a third feature vector; and defects in the industrial CT image are recognized according to the third feature vector and a template image feature vector.The application can improve the accuracy, robustness and generalization of defect recognition by fusing the extracted feature vectors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect identification, and in particular to an industrial CT image identification method and device and an industrial CT image identification system. BACKGROUND

[0002] Industrial Computed Tomography (industrial CT) technology has developed rapidly and is considered to be the best non-destructive testing method at present, with the development and progress of science and technology, the performance of industrial CT equipment improves year by year, and the cost gradually decreases.

[0003] However, when identifying defects in industrial CT images, only independent features are simply extracted, resulting in low defect diagnosis and recognition accuracy and poor robustness. SUMMARY

[0004] Therefore, the present application provides an industrial CT image identification method and device and an industrial CT image identification system to solve the problem of low defect diagnosis and recognition accuracy and poor robustness when identifying defects in industrial CT images.

[0005] To solve the above technical problems, in a first aspect, the present application provides an industrial CT image identification method, comprising:

[0006] extracting color feature vectors, texture feature vectors and shape feature vectors in the industrial CT image;

[0007] fusing the color feature vectors, the texture feature vectors and the shape feature vectors to obtain a fused feature vector;

[0008] splicing the industrial CT image and the fused feature vector to obtain a splicing result, taking the splicing result as the input of a first convolutional neural network and a second convolutional neural network respectively, outputting a first feature vector from the first convolutional neural network and a second feature vector from the second convolutional neural network, wherein the first convolutional neural network and the second convolutional neural network have different structures;

[0009] weighting and fusing the first feature vector and the second feature vector to obtain a third feature vector;

[0010] identifying defects in the industrial CT image according to the third feature vector and a feature vector of a template image.

[0011] Optionally, the first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, the convolutional layers comprise a plurality of 3x3 convolutional kernels and a plurality of 2x2 small pooling kernels.

[0012] Optionally, the second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, the size of the convolutional kernel of the first layer of convolutional layers is 11x11x3, the size of the convolutional kernel of the second layer of convolutional layers is 5x5, the size of the convolutional kernel of the third layer of convolutional layers is 3x3, the size of the convolutional kernel of the fourth layer of convolutional layers is 3x3, and the size of the convolutional kernel of the fifth layer of convolutional layers is 3x3.

[0013] Optionally, the weighting and fusing the first feature vector and the second feature vector to obtain a third feature vector comprises:

[0014] calculating a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector;

[0015] determining the weights of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value;

[0016] performing weighted summation on the first feature vector and the second feature vector according to the weights of the first feature vector and the second feature vector to obtain a third feature vector.

[0017] Optionally, the identifying the defects in the industrial CT image according to the third feature vector and the feature vector of the template image comprises:

[0018] calculating the similarity between the third feature vector and the feature vector of the template image;

[0019] determining the defects in the target template image as the defects in the industrial CT image, wherein the target template image has a similarity with the third feature vector greater than a preset similarity threshold, and different template images have different defects.

[0020] In a second aspect, the present application further provides an industrial CT image recognition device, comprising:

[0021] an extraction module configured to extract a color feature vector, a texture feature vector and a shape feature vector in an industrial CT image;

[0022] a first fusion module configured to fuse the color feature vector, the texture feature vector and the shape feature vector to obtain a fused feature vector;

[0023] The processing module is configured to splice the industrial CT image and the fusion feature vector to obtain a splicing result, take the splicing result as an input of a first convolutional neural network and a second convolutional neural network respectively, output a first feature vector by the first convolutional neural network, and output a second feature vector by the second convolutional neural network, wherein the first convolutional neural network and the second convolutional neural network are different in structure.

[0024] The second fusion module is configured to perform weighted fusion on the first feature vector and the second feature vector to obtain a third feature vector.

[0025] The recognition module is configured to recognize a defect in the industrial CT image according to the third feature vector and a feature vector of a template image.

[0026] Optionally, the first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, and the convolutional layers comprise a plurality of 3×3 convolutional kernels and a plurality of 2×2 small pooling kernels.

[0027] Optionally, the second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, a size of a convolutional kernel of a first convolutional layer is 11×11×3, a size of a convolutional kernel of a second convolutional layer is 5×5, a size of a convolutional kernel of a third convolutional layer is 3×3, a size of a convolutional kernel of a fourth convolutional layer is 3×3, and a size of a convolutional kernel of a fifth convolutional layer is 3×3.

[0028] Optionally, the second fusion module comprises:

[0029] The first calculation unit is configured to calculate a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector.

[0030] The first determination unit is configured to determine weights of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value.

[0031] The weighted summation unit is configured to perform weighted summation on the first feature vector and the second feature vector according to the weights of the first feature vector and the second feature vector to obtain the third feature vector.

[0032] Optionally, the recognition module comprises:

[0033] The second calculation unit is configured to calculate a similarity between the third feature vector and a feature vector of a template image.

[0034] The second determining unit is configured to determine the defects in the target template image as the defects in the industrial CT image, wherein the target template image has a similarity to the third feature vector greater than a preset similarity threshold, and different template images have different defects.

[0035] In a third aspect, the present application provides an industrial CT image recognition system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor implements any of the above industrial CT image recognition methods when executing the computer program.

[0036] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps of any of the above industrial CT image recognition methods.

[0037] The above technical solutions of the present application have the following advantages:

[0038] In the embodiments of the present application, the fusion processing of the extracted feature vectors can improve the accuracy, robustness and generalization of defect recognition. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 FIG. 1 is a flowchart of an industrial CT image recognition method according to an embodiment of the present application;

[0040] Figure 2 FIG. 2 is a structural diagram of an industrial CT image recognition device according to an embodiment of the present application;

[0041] Figure 3 FIG. 3 is a structural diagram of an industrial CT image recognition system according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0043] Please refer to Figure 1 , Figure 1 FIG. 1 is a flowchart of an industrial CT image recognition method according to an embodiment of the present application, which comprises the following steps:

[0044] Step 11: Extracting color feature vectors, texture feature vectors and shape feature vectors in the industrial CT image.

[0045] Wherein, the industrial CT image of the workpiece is obtained by adopting the industrial CT technology, if the workpiece has defects, the defects will be reflected in the industrial CT image; different defects in the industrial CT image are usually different corresponding feature vectors, by extracting the color feature vector, the texture feature vector and the shape feature vector, the defect type in the industrial CT image can be identified.

[0046] Step 12: the color feature vector, the texture feature vector and the shape feature vector are fused to obtain a fused feature vector;

[0047] In the embodiment of the application, in order to comprehensively consider each feature vector and improve the recognition accuracy of defects, the extracted multiple feature vectors can be fused to obtain a fused feature vector, and the feature fusion is a mature algorithm, which will not be described here.

[0048] Step 13: the industrial CT image and the fused feature vector are spliced to obtain a splicing result, the splicing result is input into a first convolutional neural network and a second convolutional neural network respectively, a first feature vector is output by the first convolutional neural network, and a second feature vector is output by the second convolutional neural network, wherein the structures of the first convolutional neural network and the second convolutional neural network are different;

[0049] In this step, optionally, the splicing process of the industrial CT image and the fused feature vector can be before the input of the first convolutional neural network and the second convolutional neural network, or after the input of the first convolutional neural network and the second convolutional neural network, that is, the industrial CT image and the fused feature vector can be spliced first, and then the splicing result is input into the first convolutional neural network and the second convolutional neural network, or the industrial CT image and the fused feature vector can be input into the first convolutional neural network at the same time, spliced in the first convolutional neural network, and input into the second convolutional neural network at the same time, spliced in the second convolutional neural network. Wherein, the industrial CT image is 3 channels, and the fused feature vector becomes 4 channels after being spliced with the industrial CT image.

[0050] In the embodiment of the present application, the first convolutional neural network and the second convolutional neural network with different structures are used to process the industrial CT image and the fusion feature vector, so that different processing results, i.e., the first feature vector and the second feature vector, are obtained. The two convolutional neural networks are used to process the industrial CT image, so that the feature vector obtained by using a single convolutional neural network due to the defects of the algorithm principle can be avoided to comprehensively and accurately describe the defects in the industrial CT image. That is, by comprehensively considering the first feature vector and the second feature vector obtained by using different convolutional neural networks, the biased results caused by a single algorithm mechanism can be avoided.

[0051] Step 14: The first feature vector and the second feature vector are fused by weighting to obtain a third feature vector.

[0052] Step 15: According to the third feature vector and the feature vector of the template image, the defects in the industrial CT image are identified.

[0053] In the embodiment of the present application, the so-called template image is an image in which the defects have been accurately identified. The number of template images is usually multiple, and the defects in different template images are usually different. By comparing the third feature vector with the feature vector of the template image, a template image with high similarity can be found from the template images, so that the defects in the corresponding template image are taken as the defects in the industrial CT image.

[0054] Therefore, the industrial CT image recognition method provided by the embodiment of the present application can improve the accuracy, robustness and generalization of defect recognition by fusing the extracted feature vectors.

[0055] The above industrial CT image recognition method is described below.

[0056] In one of the optional specific embodiments, before the color feature vector, the texture feature vector and the shape feature vector in the industrial CT image are extracted, the method further includes:

[0057] The industrial CT image is preprocessed, and the preprocessing changes the target parameters of the industrial CT image.

[0058] That is, the industrial CT image is preprocessed, i.e., the image enhancement processing is performed on the industrial CT image to reduce the noise in the image and change the original image parameters such as brightness, color distribution, contrast, gray scale and saturation.

[0059] For example, the image enhancement processing can use linear transformation to adjust the contrast and brightness of the image. The linear transformation formula is y=a*x+b, wherein the parameter a affects the contrast of the image, and the parameter b affects the brightness of the image. Specifically, the following cases can be divided:

[0060] 1) a>1: increase the contrast of the image, the image looks more clear;

[0061] 2) a<1: reduce the contrast of the image, the image looks more blurred;

[0062] 3) a<0 and b=0: the bright area of the image becomes dark, and the dark area becomes bright

[0063] 4) a=1 and b≠0: the gray value of the image as a whole is shifted up or down, that is, the image as a whole is brightened or darkened, the contrast of the image does not change, b>0 the image is brightened, b<0 the image is darkened

[0064] 5) a=-1 and b=255: the image is flipped.

[0065] In some embodiments of the application, cv::convertScaleAbs() can be used to realize fast operation of image enhancement and related operations, specifically as follows:

[0066] void cv::convertScaleAbs(

[0067] cv::InputArray src, / / input array

[0068] cv::OutputArray dst, / / output array

[0069] double alpha=1.0, / / multiplier factor

[0070] double beta=0.0 / / offset).

[0071] In other embodiments of the application, the extraction of color feature vectors in industrial CT images can be based on HSV color space color feature extraction technology, the extraction of texture feature vectors can be based on gray level co-occurrence matrix texture feature extraction technology, and the extraction of shape feature vectors can be based on semantic segmentation technology, i.e. Hough technology. The representation of shape feature vectors includes contour feature representation and region feature representation, the former only uses the outer boundary of the object, and the latter relates to the entire shape region.

[0072] In the embodiments of the application, optionally, the first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, the convolutional layers comprise a plurality of 3*3 convolutional kernels and a plurality of 2*2 small pooling kernels.

[0073] The specific structure of the first convolutional neural network and the processing process of inputting the industrial CT image and the fusion feature vector into the first convolutional neural network after splicing are introduced below.

[0074] Step 1.1: First, input the spliced image into a small convolution kernel.

[0075] Step 1.1.1: First input the image of 224*224*3, and perform two times of convolution + ReLU (Rectified Linear Unit, also known as modified linear unit) through 64 3*3 convolution kernels. The image size after convolution becomes 224*224*64.

[0076] Step 1.1.2: Perform max pooling operation, and the pooling unit size is 2*2 (the image size is finally halved). The image size after pooling becomes 112*112*64.

[0077] Step 1.2: Continuous convolution layer

[0078] Step 1.2.1: Perform two times of convolution + ReLU through 128 3*3 convolution kernels. The image size becomes 12*112*128.

[0079] Step 1.2.2: Perform 2*2 pooling (max pooling). The image size becomes 56*56*128.

[0080] Step 1.2.3: Perform three times of convolution + ReLU through 256 3*3 convolution kernels. The size becomes 56*56*256.

[0081] Step 1.2.4: Perform 2*2 pooling (max pooling). The image size becomes 28*28*256.

[0082] Step 1.2.5: Perform three times of convolution + ReLU through 512 3*3 convolution kernels. The size becomes 28*28*512.

[0083] Step 1.3: Small pooling kernel

[0084] Step 1.3.1: Perform 2*2 pooling (max pooling). The image size becomes 14*14*512.

[0085] Step 1.3.2: Perform three times of convolution + ReLU through 512 3*3 convolution kernels. The image size becomes 14*14*512.

[0086] Step 1.3.3: Perform pooling (max pooling). The image size becomes 7*7*512.

[0087] Step 1.3.4: pass through two layers 1*1*4096, one layer 1*1*1000 to perform full connection + ReLU (total three layers).

[0088] After the first convolutional neural network with the above structure, the first feature vector can be obtained.

[0089] In some embodiments of the present application, the second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, the size of the convolution kernel of the first layer of convolutional layer is 11*11*3, the size of the convolution kernel of the second layer of convolutional layer is 5*5, the size of the convolution kernel of the third layer of convolutional layer is 3*3, the size of the convolution kernel of the fourth layer of convolutional layer is 3*3, and the size of the convolution kernel of the fifth layer of convolutional layer is 3*3.

[0090] The specific structure of the second convolutional neural network and the processing process of inputting the industrial CT image and the fusion feature vector into the second convolutional neural network after splicing are introduced below.

[0091] Step 2.1:

[0092] The first layer: the convolutional layer 1, the input is an image of 224*224*3, the number of convolution kernels is 96, and the size of the convolution kernel is 11*11*3; stride (step length) = 4;

[0093] Step 2.2:

[0094] The second layer: the convolutional layer 2, the input is the feature map (feature map) of the convolution of the last layer, the number of convolution kernels is 256, the size of the convolution kernel is 5*5*48, pad (padding) = 2, stride = 1; then LRN is done, and finally max pooling is done, pool_size = (3, 3), stride = 2;

[0095] Step 2.3:

[0096] The third layer: the convolutional layer 3, the input is the output of the second layer, the number of convolution kernels is 384, kernel_size = (3*3*256), padding = 1, and the third layer does not have LRN and Pool;

[0097] Step 2.4:

[0098] The fourth layer: the convolutional layer 4, the input is the output of the third layer, the number of convolution kernels is 384, kernel_size = (3*3), padding = 1, and the fourth layer does not have LRN and Pool as the third layer;

[0099] Step 2.5:

[0100] The fifth layer: convolution 5, the input is the output of the fourth layer, the number of convolution kernels is 256, kernel_size=(3*3), padding=1, then directly max_pooling, pool_size=(3,3), stride=2;

[0101] Step 2.6:

[0102] The 6th, 7th and 8th layers are fully connected layers, the number of neurons of each layer is 4096, the final output softmax is 1000, and ReLU and Dropout are used in the fully connected layer.

[0103] After the second convolutional neural network with the above structure, the second feature vector can be obtained.

[0104] In some embodiments of the application, the first feature vector and the second feature vector are weighted and fused to obtain a third feature vector.

[0105] The first Euclidean distance value of the first feature vector and the second Euclidean distance value of the second feature vector are calculated.

[0106] The weights of the first feature vector and the second feature vector are determined according to the first Euclidean distance value and the second Euclidean distance value.

[0107] The first feature vector and the second feature vector are weighted and summed according to the weights of the first feature vector and the second feature vector to obtain a third feature vector.

[0108] That is, after obtaining the first feature vector and the second feature vector, the Euclidean distance values of the first feature vector and the second feature vector are calculated, and then the weights of the first feature vector and the second feature vector are determined, and finally the third feature vector is calculated.

[0109] For example, the first Euclidean distance value of the first feature vector feature1 is L1, and the second Euclidean distance value of the second feature vector feature2 is L2, then the weight A of the first feature vector feature1 is L1 / (L1+L2), the weight B of the second feature vector feature2 is L2 / (L1+L2), and the third feature vector feature3 can be calculated by the following formula:

[0110] feature3=A*feature1+B*feature2.

[0111] Through the above weighted fusion operation, the first feature vector and the second feature vector can be comprehensively considered, and the accuracy and robustness of defect recognition can be improved.

[0112] In some other embodiments of the present application, the identifying the defects in the industrial CT image according to the third feature vector and the feature vectors of the template images comprises:

[0113] calculating the similarity of the third feature vector and the feature vectors of the template images;

[0114] determining the defects in the target template image as the defects in the industrial CT image, wherein the target template image has a similarity greater than a preset similarity threshold with the third feature vector, and different template images have different defects.

[0115] In some other embodiments of the present application, the feature vectors of the template images can be extracted and determined in advance, different template images have different defects, that is, the feature vectors in different template images are different, and the similarity can be a cosine similarity, which refers to calculating the cosine value of the included angle of two feature vectors to evaluate the similarity of the two vectors, the cosine value ranges between [-1, 1], the value tends to 1, indicating that the included angle of the vectors tends to 0 degrees, representing that the two feature vectors are more similar, and the value tends to -1, representing that the two feature vectors are less similar.

[0116] In some embodiments of the present application, if the similarity of the feature vector of a certain template image and the third feature vector is greater than a preset similarity threshold, the template image can be determined as a target template image, that is, the defects in the target template image are determined as the defects in the industrial CT image to be identified and diagnosed, and the defects are identified.

[0117] The following are the Euclidean distances obtained after processing by different feature vectors and different convolutional neural networks:

[0118] 1) When a single color feature vector + VGG convolutional neural network, the Euclidean distance is: 0.8076;

[0119] 2) When a single texture feature vector + VGG convolutional neural network, the Euclidean distance is: 0.7909;

[0120] 3) When a single shape feature vector + VGG convolutional neural network, the Euclidean distance is: 0.6424;

[0121] 4) When feature fusion + VGG convolutional neural network, the Euclidean distance is: 0.5834;

[0122] 5) When feature fusion + AlexNet convolutional neural network, the Euclidean distance is: 0.5387;

[0123] 6) When feature fusion + VGG convolutional neural network + AlexNet convolutional neural network + feature vector weighted fusion, the Euclidean distance is: 0.4823;

[0124] Therefore, the recognition accuracy after the feature fusion of the plurality of feature vectors, i.e., the color feature vector, the texture feature vector and the shape feature vector, is higher than that of a single color feature vector, a single texture feature vector or a single shape feature vector, and the recognition accuracy after the feature fusion of the plurality of feature vectors, the extraction of the feature vectors by the plurality of convolutional neural networks and the weighted fusion is higher than that of a single feature vector detection mode and a single convolutional neural network detection mode.

[0125] In summary, in the embodiment of the present application, the accuracy, robustness and generalization of defect recognition can be improved by the fusion processing of the extracted feature vectors.

[0126] Please refer to Figure 2 , Figure 2 is a structural schematic diagram of an industrial CT image recognition device provided by the second embodiment of the present application. The industrial CT image recognition device 20 comprises:

[0127] An extraction module 21 is configured to extract a color feature vector, a texture feature vector and a shape feature vector in an industrial CT image.

[0128] A first fusion module 22 is configured to fuse the color feature vector, the texture feature vector and the shape feature vector to obtain a fused feature vector.

[0129] A processing module 23 is configured to splice the industrial CT image and the fused feature vector to obtain a spliced result, and take the spliced result as the input of a first convolutional neural network and a second convolutional neural network respectively, output a first feature vector by the first convolutional neural network, and output a second feature vector by the second convolutional neural network, wherein the structures of the first convolutional neural network and the second convolutional neural network are different.

[0130] A second fusion module 24 is configured to perform weighted fusion on the first feature vector and the second feature vector to obtain a third feature vector.

[0131] An identification module 25 is configured to identify defects in the industrial CT image according to the third feature vector and the feature vector of a template image.

[0132] Optionally, the first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, the convolutional layers comprise a plurality of 3x3 convolutional kernels and a plurality of 2x2 small pooling kernels.

[0133] Optionally, the second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, the size of a convolution kernel of a first convolutional layer is 11*11*3, the size of a convolution kernel of a second convolutional layer is 5*5, the size of a convolution kernel of a third convolutional layer is 3*3, the size of a convolution kernel of a fourth convolutional layer is 3*3, and the size of a convolution kernel of a fifth convolutional layer is 3*3.

[0134] Optionally, the second fusion module comprises:

[0135] a first calculation unit configured to calculate a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector;

[0136] a first determination unit configured to determine weights of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value;

[0137] a weighted summation unit configured to perform weighted summation on the first feature vector and the second feature vector according to the weights of the first feature vector and the second feature vector to obtain a third feature vector.

[0138] Optionally, the recognition module comprises:

[0139] a second calculation unit configured to calculate a similarity between the third feature vector and a feature vector of a template image;

[0140] a second determination unit configured to determine a defect in a target template image as a defect in the industrial CT image, wherein the target template image has a similarity with the third feature vector greater than a preset similarity threshold, and different template images have different defects.

[0141] The embodiment of the application is a product corresponding to the above-mentioned method embodiment one, and thus will not be described here again, and please refer to the above-mentioned embodiment one for details.

[0142] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of an industrial CT image recognition system provided by the embodiment three, the industrial CT image recognition system 30 comprises a processor 31, a memory 32, and a computer program stored in the memory 32 and capable of running on the processor 31; the processor 31 implements the following steps when executing the computer program:

[0143] extracting a color feature vector, a texture feature vector and a shape feature vector in an industrial CT image;

[0144] fuse the color feature vector, the texture feature vector and the shape feature vector to obtain a fused feature vector;

[0145] splice the industrial CT image and the fused feature vector to obtain a spliced result, take the spliced result as input of a first convolutional neural network and a second convolutional neural network respectively, output a first feature vector from the first convolutional neural network, and output a second feature vector from the second convolutional neural network, wherein the first convolutional neural network and the second convolutional neural network are different in structure;

[0146] weight fuse the first feature vector and the second feature vector to obtain a third feature vector;

[0147] identify defects in the industrial CT image according to the third feature vector and a feature vector of a template image.

[0148] In the embodiment of the application, the accuracy, robustness and generalization of defect identification can be improved by fusing the extracted feature vectors.

[0149] Optionally, the first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, and the convolutional layers comprise a plurality of 3*3 convolutional kernels and a plurality of 2*2 small pooling kernels.

[0150] Optionally, the second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, the size of the convolutional kernel of the first convolutional layer is 11*11*3, the size of the convolutional kernel of the second convolutional layer is 5*5, the size of the convolutional kernel of the third convolutional layer is 3*3, the size of the convolutional kernel of the fourth convolutional layer is 3*3, and the size of the convolutional kernel of the fifth convolutional layer is 3*3.

[0151] Optionally, the weight fusion of the first feature vector and the second feature vector to obtain the third feature vector comprises:

[0152] calculate a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector;

[0153] determine the weight of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value;

[0154] weight sum the first feature vector and the second feature vector according to the weight of the first feature vector and the second feature vector to obtain the third feature vector.

[0155] Optionally, the identifying the defect in the industrial CT image according to the third feature vector and the feature vector of the template image comprises:

[0156] calculating the similarity between the third feature vector and the feature vector of the template image;

[0157] determining the defect in the target template image as the defect in the industrial CT image, wherein the target template image has a similarity to the third feature vector greater than a preset similarity threshold, and different template images have different defects.

[0158] The specific working process of the embodiment of the present application is consistent with that of the above-mentioned method embodiment one, and thus will not be described here again. For details, please refer to the description of the method steps in the above-mentioned embodiment one.

[0159] The embodiment four of the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps in the industrial CT image identification method of any one of the above-mentioned embodiment one. For details, please refer to the description of the method steps in the corresponding embodiment above.

[0160] The above-mentioned computer readable storage medium includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0161] The above-mentioned is the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. An industrial CT image recognition method, characterized by, The method comprises the following steps: extracting a color feature vector, a texture feature vector and a shape feature vector in an industrial CT image; fusing the color feature vector, the texture feature vector and the shape feature vector to obtain a fused feature vector; splicing the industrial CT image and the fused feature vector to obtain a splicing result, taking the splicing result as the input of a first convolutional neural network and a second convolutional neural network respectively, outputting a first feature vector by the first convolutional neural network, and outputting a second feature vector by the second convolutional neural network, wherein the structures of the first convolutional neural network and the second convolutional neural network are different; performing weighted fusion on the first feature vector and the second feature vector to obtain a third feature vector; recognizing defects in the industrial CT image according to the third feature vector and a feature vector of a template image.

2. The method of claim 1, wherein, The first convolutional neural network is a VGG convolutional neural network, the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, the convolutional layers comprise a plurality of 3×3 convolutional kernels and a plurality of 2×2 small pooling kernels.

3. The method of claim 1, wherein, The second convolutional neural network is an AlexNet convolutional neural network, the second convolutional neural network comprises five convolutional layers and three fully connected layers, the size of the convolutional kernel of the first layer of convolutional layers is 11×11×3, the size of the convolutional kernel of the second layer of convolutional layers is 5×5, the size of the convolutional kernel of the third layer of convolutional layers is 3×3, the size of the convolutional kernel of the fourth layer of convolutional layers is 3×3, and the size of the convolutional kernel of the fifth layer of convolutional layers is 3×3.

4. The method of claim 1, wherein, The weighted fusion of the first feature vector and the second feature vector to obtain the third feature vector comprises the following steps: calculating a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector; determining the weights of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value; performing weighted summation on the first feature vector and the second feature vector according to the weights of the first feature vector and the second feature vector to obtain the third feature vector.

5. The method of claim 4, wherein, The recognition of defects in the industrial CT image according to the third feature vector and the feature vector of the template image comprises the following steps: calculating the similarity between the third feature vector and the feature vector of the template image; determining the defects in a target template image as the defects in the industrial CT image, wherein the similarity between the target template image and the third feature vector is greater than a preset similarity threshold, and different template images have different defects.

6. An industrial CT image recognition apparatus characterized by comprising: The method comprises the following steps: a extraction module for extracting a color feature vector, a texture feature vector and a shape feature vector in an industrial CT image; a first fusion module for fusing the color feature vector, the texture feature vector and the shape feature vector to obtain a fused feature vector; The processing module is configured to splice the industrial CT image and the fusion feature vector to obtain a splicing result, take the splicing result as input of a first convolutional neural network and a second convolutional neural network respectively, output a first feature vector by the first convolutional neural network, and output a second feature vector by the second convolutional neural network, wherein the first convolutional neural network and the second convolutional neural network are different in structure. The second fusion module is configured to perform weighted fusion on the first feature vector and the second feature vector to obtain a third feature vector. The recognition module is configured to recognize a defect in the industrial CT image according to the third feature vector and a feature vector of a template image.

7. The apparatus of claim 6, wherein, The first convolutional neural network is a VGG convolutional neural network, and the first convolutional neural network comprises a plurality of continuous convolutional layers and a plurality of fully connected layers, the convolutional layers comprise a plurality of 3×3 convolutional kernels and a plurality of 2×2 small pooling kernels.

8. The apparatus of claim 6, wherein, The second convolutional neural network is an AlexNet convolutional neural network, and the second convolutional neural network comprises five convolutional layers and three fully connected layers, a size of a convolutional kernel of a first convolutional layer is 11×11×3, a size of a convolutional kernel of a second convolutional layer is 5×5, a size of a convolutional kernel of a third convolutional layer is 3×3, a size of a convolutional kernel of a fourth convolutional layer is 3×3, and a size of a convolutional kernel of a fifth convolutional layer is 3×3.

9. The apparatus of claim 6, wherein, The second fusion module comprises: The first calculation unit is configured to calculate a first Euclidean distance value of the first feature vector and a second Euclidean distance value of the second feature vector. The first determination unit is configured to determine weights of the first feature vector and the second feature vector according to the first Euclidean distance value and the second Euclidean distance value. The weighted summation unit is configured to perform weighted summation on the first feature vector and the second feature vector according to the weights of the first feature vector and the second feature vector to obtain a third feature vector.

10. The apparatus of claim 9, wherein, The recognition module comprises: The second calculation unit is configured to calculate a similarity between the third feature vector and a feature vector of a template image. The second determination unit is configured to determine a defect in a target template image as a defect in the industrial CT image, wherein the target template image has a similarity to the third feature vector greater than a preset similarity threshold, and different template images have different defects.

11. An industrial CT image recognition system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor; characterized in that, The processor executes the computer program to implement the industrial CT image recognition method in any one of claims 1 to 5.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the industrial CT image recognition method in any one of claims 1 to 5.

Images