A deep learning-based method and system for detecting large-scale defects in welds

Abstract

The present application belongs to the field of X-ray weld seam image defect detection, and relates to a weld seam large-scale defect detection method and system based on deep learning, comprising: inputting a preprocessed weld seam image into a backbone network model for training, wherein the backbone network model comprises two or more convolution units, and the convolution kernel of the last convolution unit is a hollow convolution kernel with different hollow rates; inputting the output result of the backbone network model into a feature pyramid network for feature fusion and enhancement; inputting the fused and enhanced features into a regression network and a classification network respectively to obtain the classification confidence and position of the weld seam defects. The present application improves the detection rate and accuracy of large-scale defects, and the detection rate and accuracy of small defects are not affected.

Description

TECHNICAL FIELD

[0001] The present application relates to a deep learning-based large-scale weld defect detection method and system, belonging to the field of X-ray weld image defect detection. BACKGROUND

[0002] With the rapid development of modern industry and manufacturing, the requirement for welding quality is getting higher and higher in production activities, and the welding quality determines the safety and effective life of the workpiece. Defect detection of welded workpieces is the key work to determine the welding quality. In the traditional detection process, X-ray weld images are analyzed by manual, and the type, position and size of defects are determined by experience, and the corresponding rating is given. In manual evaluation, misjudgment and omission are easily caused due to the difference in technical quality and experience of personnel, and the efficiency is low and the consistency is poor, which cannot meet the needs of rapid production development of modern manufacturing industry.

[0003] The computer vision technology based on deep learning is adopted to analyze and detect defects of X-ray defect detection images, which can better solve the above-mentioned problems of manual evaluation, and more scientifically, objectively and normatively evaluate the welding quality. The existing defect detection technology based on deep learning mostly uses convolutional neural network, which is limited by the number of training data and the amount of calculation. In order to avoid overfitting and inefficiency, the convolutional neural network cannot be made very large and deep. The size of the defect that can be detected by the convolutional network has a great relationship with the receptive field of the convolutional network. The receptive field can be simply understood as the field of view that can be seen by the network, and its size directly affects the size of the defect that can be detected by the network. The deeper the network is, the larger the receptive field is, and the theoretical receptive field is not all effective, and its effectiveness decreases to the edge in the form of Gaussian distribution. Therefore, the actual effective receptive field is much smaller than the theoretical receptive field. Thus, the contradiction between the relatively shallow network which cannot identify large-scale defects and the relatively deep network which greatly increases the amount of calculation is caused. Moreover, due to the difficulty in collecting welding defect data, the data set is relatively small, and overfitting is more likely to occur, which reduces the generalization ability of the model. SUMMARY

[0004] In view of the above problems, the present application provides a deep learning-based large-scale weld defect detection method and system, which improves the detection rate and accuracy of large-scale defects, and the detection rate and accuracy of small defects are not affected.

[0005] To achieve the above object, the application provides the following technical scheme: a welding seam large-scale defect detection method based on deep learning, comprising: inputting a preprocessed welding seam image into a backbone network model for training, wherein the backbone network model comprises two or more convolution units, and the convolution kernel of the last convolution unit is a hollow convolution kernel with different hollow rates; inputting the output result of the backbone network model into a feature pyramid network for feature fusion and enhancement; inputting the fused and enhanced features into a regression network and a classification network respectively to obtain the classification confidence and position of the welding seam defect.

[0006] Further, the preprocessing method comprises: cutting the welding seam image; stretching the gray value of the cut welding seam image; performing random contrast enhancement on the welding seam image according to a set proportion; performing random blur enhancement on the welding seam image; labeling the welding seam image subjected to random blur enhancement; and performing statistical analysis on the labeled welding seam image and calculating the size of the defect in the welding seam image.

[0007] Further, the hollow rate and the number of layers of the hollow convolution kernel are determined by the size of the defect in the welding seam image, and the receptive field is obtained by the hollow convolution kernel, and the calculation formula is:

[0008] j out =j in ×s

[0009] r out =r in +((k+(k+1)×(d-1))-1)×j in

[0010] wherein, j out is the overall span of the output feature map; j in is the overall span of the input feature map; s is the step of the convolution layer; r out is the receptive field of the output feature map; r in is the receptive field of the input feature map; k is the size of the convolution kernel; and d is the hollow rate of the hollow convolution kernel.

[0011] Further, the backbone network model comprises a first convolution unit, a second convolution unit and a third convolution unit, each of the convolution units comprises a plurality of convolution stages; the first convolution unit comprises a plurality of 3x3 convolution networks for feature extraction and down-sampling of the input welding seam image; the second convolution unit comprises a plurality of convolution modules, each of the convolution modules comprises convolution networks with sizes of 1x1, 3x3 and 1x1, and the number of the second convolution units is determined according to the number of features of the welding seam image; and the third convolution unit is a hollow convolution layer.

[0012] Further, the processing method of the atrous convolution layer is: reducing the dimension of the features of the weld image through a plurality of 1x1 convolution networks, or reducing the dimension of the features of the weld image through one 1x1 convolution network, and dividing the reduced features into a plurality of groups; extracting features of the weld image with reduced dimension through a plurality of atrous convolution layers with different atrous rates; splicing the features extracted by different atrous convolution layers together for feature fusion or feature fusion and dimension increase; increasing the dimension of the input features of the weld image, and adding the features after feature fusion to obtain the final output features.

[0013] Further, the processing method of the atrous convolution layer is: reducing the dimension of the features of the weld image through a plurality of 1x1 convolution networks; extracting features of the feature map with reduced dimension through a plurality of atrous convolution layers with different atrous rates; increasing the dimension of the features extracted by different atrous convolution layers respectively; adding the features with increased dimension; increasing the dimension of the input features of the weld image, and adding the features obtained in the previous step to obtain the final output features.

[0014] Further, the method for inputting the output result of the backbone network model into the feature pyramid network for feature fusion and enhancement is: performing dimension reduction operation on the output result of the backbone network model; extracting features of the feature map obtained by the last convolution unit of the backbone network model, and down-sampling the feature map to obtain a new feature map; up-sampling all the feature maps after dimension reduction, adding the feature maps to the feature maps obtained in the next convolution stage, adding the feature map obtained by the last convolution unit to the new feature map obtained in the previous step, and taking the added result as the network output feature.

[0015] Further, the processing method of the regression network is: inputting the features after fusion and enhancement into the regression network, and the regression network extracts features; further extracting features of the feature map obtained by the previous step of feature extraction, and performing position regression on the extracted features to obtain a first stage offset; converting the first stage offset into a pseudo box; adding the first stage offset to the feature map obtained after the first feature extraction, and further extracting features of the feature map to which the first stage offset is added; performing position regression on the feature map obtained by the previous step of feature extraction to obtain a second stage offset; adding the second stage offset to the pseudo box to obtain the final defect position.

[0016] Further, the processing method of the classification network is: inputting the fused and enhanced features into a regression network, performing feature extraction on the features by the regression network; adding the first stage offset to the feature map obtained in the last step of feature extraction, and performing further feature extraction on the feature map to which the first stage offset is added; and classifying the feature map obtained in the last step to obtain the final defect type confidence.

[0017] The application further discloses a deep learning-based weld large-scale defect detection system, which comprises a backbone network model module, a neck network model module and a head network model module.

[0018] The application has the following advantages due to the above technical scheme.

[0019] 1. The scheme in the application can effectively expand the receptive field of the convolutional network, improve the detection rate and accuracy of large-scale defects, and does not affect the detection rate and accuracy of small defects, and the detected defect types are as many as 11, i.e. pores, slag inclusions, deep holes, cracks, un-fusion, incomplete penetration, dense, chain, concave, undercut and metal slag, which is more suitable for industrial application.

[0020] 2. The scheme in the application uses a hollow convolution to expand the receptive field without increasing the depth and width of the network, so that the complexity of the model is hardly changed, the calculation amount is not increased, and the overfitting problem caused by the increase of the complexity of the model is avoided, and the pressure of collecting data is relieved. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flowchart of the deep learning-based weld large-scale defect detection method in an embodiment of the application;

[0022] Figure 2 is a structural schematic diagram of the backbone network model in an embodiment of the application;

[0023] Figure 3 is a structural schematic diagram of the first hollow convolution module in an embodiment of the application;

[0024] Figure 4 is a structural schematic diagram of the second hollow convolution module in an embodiment of the application;

[0025] Figure 5 is a structural schematic diagram of a third hollow convolution module in an embodiment of the present application;

[0026] Figure 6 is a structural schematic diagram of a fourth hollow convolution module in an embodiment of the present application;

[0027] Figure 7 is a structural schematic diagram of a neck network model in an embodiment of the present application;

[0028] Figure 8 is a structural schematic diagram of a head network model in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order for those skilled in the art to better understand the technical direction of the present application, the present application is described in detail through specific embodiments. However, it should be understood that the specific embodiments are provided only for better understanding of the present application, and they should not be understood as limiting the present application. In the description of the present application, it should be understood that the terms used are only for the purpose of description, and should not be understood as indicating or implying relative importance.

[0030] The deep learning-based weld large-scale defect detection method and system provided in the present application research the field of X-ray weld image defect detection by using artificial intelligence algorithms. Artificial intelligence is the theory, method, technology and application system for using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0031] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0032] With the development of science and technology, artificial intelligence technology has been widely used in defect detection technology field, which overcomes some misjudgment and missed judgment due to experience and physical limitations of artificial detection, and can more accurately locate the defect type and defect position. And the defect detection method and system introducing artificial intelligence algorithm have higher detection efficiency and good consistency, and generally do not need manual supervision, can automatically generate detection results, have high automation degree, and reduce labor cost. However, for welding defects, not only small defects exist in general material detection, but also large-scale defects may exist, and large-scale defects have greater impact on welding quality than small-scale defects. However, the existing welding defect detection technology using deep learning algorithm often uses a shallow network, and such a shallow network cannot identify large-scale defects. If large-scale defects are to be identified, the network depth needs to be increased, but increasing the network depth will greatly increase the calculation amount. Even if the calculation amount is not considered, a deeper network is more likely to overfit, reducing the model generalization ability.

[0033] In order to solve the problems existing in the prior art, the present application provides a welding large-scale defect detection method and system based on deep learning, which is aimed at high-resolution welding images. In order to make it suitable for existing GPU (Graphics Processing Unit) devices for model training, it is preprocessed by cutting and the like. The cut welding image is subjected to data enhancement. The enhanced data is sent to the backbone network model for feature extraction. The features output by the backbone network are sent to the neck network for feature fusion and enhancement. Finally, the features fused by the neck network are sent to the classification network and the regression network of the head network for defect classification and position recognition, and the final defect classification confidence and position are obtained. It improves the detection rate and accuracy of large-scale defects, while the detection rate and accuracy of small defects are not affected. The present application will be described below in conjunction with the drawings through embodiments.

[0034] Embodiment one

[0035] The present embodiment discloses a welding large-scale defect detection method based on deep learning, as shown in Figure 1 , comprising:

[0036] S1 inputting the preprocessed welding image into the backbone network model for training, wherein the backbone network model comprises two or more convolution units, and the convolution kernel of the last convolution unit is a hollow convolution kernel with different hollow rates;

[0037] S2 inputting the output result of the backbone network model into the feature pyramid network for feature fusion and enhancement;

[0038] S3 inputs the fused and enhanced features into the regression network and the classification network respectively to obtain the classification confidence and the position of the weld defect.

[0039] The preprocessed weld image is input into the detection network with various parameters set, and the weld image is extracted by the backbone network model, and the extracted features are fused by the neck network model, that is, the feature pyramid network is input for feature fusion and enhancement, and the fused features are input into the head network for defect type classification and defect position regression to obtain the final defect type confidence and defect prediction position. The head network is a network model including the regression network and the classification network.

[0040] Among them, the preprocessing of the weld image can make it suitable for the existing GPU device, make up for the problem of insufficient data, and improve the robustness of the model. By introducing a convolution kernel with different hole rates, the receptive field of the convolution network is expanded, so that large-scale defects can be more effectively identified. At the same time, since the receptive field of the first three convolution stages is not changed, the detection of small defects is not affected.

[0041] Optionally, the preprocessing process can include the following steps:

[0042] S1.1 According to the actual situation of the GPU device, the weld image is divided to make it suitable for the existing GPU device;

[0043] S1.2 The divided weld image is stretched in gray value, such as stretching the image in gray value to 16bit, and converting it to an 8bit image;

[0044] S1.3 The weld image after gray value stretching is randomly enhanced in contrast according to the set proportion to simulate images with different contrasts in actual application;

[0045] S1.4 The weld image after random contrast enhancement is randomly blurred according to the set proportion to simulate images with blurred defects in actual application;

[0046] S1.5 Labeling the weld image after random blur enhancement; statistically analyzing the labeled weld image and calculating the size of the defect in the weld image.

[0047] In addition to the above steps, the preprocessing of the weld image in this embodiment also includes color inversion, contrast enhancement, brightness enhancement, sharpness enhancement and other conventional operations, which are not listed one by one here.

[0048] The hole rate and the number of layers of the hole convolution kernel are determined by the size of the defect in the weld image. The receptive field is obtained by the hole convolution kernel, and the calculation formula is:

[0049] j out =j in ×s

[0050] r out =r in +((k+(k+1)×(d-1))-1)×j in

[0051] Where, j out It is the overall span of the output feature map; j in The input feature map span is s, with an initial value of 1; s is the stride of the convolutional layer; r out It is the receptive field of the output feature map; r in is the receptive field of the input feature map; k is the size of the convolution kernel; d is the dilation rate of the dilated convolution kernel. All the physical quantities mentioned above are in units of pixels.

[0052] The backbone network model in this embodiment includes a first convolutional unit, a second convolutional unit, and a third convolutional unit. It should be noted that the first, second, and third convolutional units are used only to differentiate the structure of the convolutional units; they do not mean that a convolutional unit can only be a single layer. For example, a second convolutional unit can be configured in multiple ways. In this text, "multiple" should be understood as two or more. The number of each convolutional unit can be determined according to actual needs. Each convolutional unit includes several convolutional stages.

[0053] The first convolutional unit comprises several 3×3 convolutional networks used for feature extraction and downsampling of the input weld seam image;

[0054] The second convolutional unit includes several sets of convolutional modules. Each convolutional module includes convolutional networks with sizes of 1×1, 3×3 and 1×1, respectively. The number of second convolutional units is determined according to the number of features in the weld image.

[0055] The third convolutional unit is a hollow convolutional layer.

[0056] Specifically, such as Figure 2 As shown, the backbone network model includes a first convolutional stage, a second convolutional stage, a third convolutional stage, a fourth convolutional stage, and a fifth convolutional stage. The first convolutional stage is the first convolutional unit, and the second, third, and fourth convolutional stages are all second convolutional units, as the only difference between them is the number of convolutional modules; their network structures are essentially the same. The fifth convolutional stage is the third convolutional unit.

[0057] The first convolutional stage extracts features through three 3×3 convolutional networks and performs a 2x downsampling, while increasing the number of channels from 3 to 64.

[0058] The second convolution stage is composed of three groups of convolution modules stacked by 1x1, 3x3 and 1x1 convolution layers. When entering the second convolution stage for the first time, the step is 2, and 2 times down-sampling is performed. After the down-sampled feature map passes through the first 1x1 convolution layer, the channel number is 1 / 2 of the channel number of the first convolution stage, and after passing through the 3x3 convolution layer, the channel number is 2 times of the channel number of the first convolution stage. The last convolution layer in this stage outputs a feature F1, and the dimension of F1 is 256.

[0059] As described above, the third convolution stage has the same structure as the second convolution stage, and the only difference is that the second convolution stage includes three groups of convolution modules, and the third convolution stage includes four groups of convolution modules. The last convolution layer in this stage outputs a feature F2, and the dimension of F2 is 512.

[0060] As described above, the fourth convolution stage also has the same structure as the second convolution stage, and the only difference is that the second convolution stage includes three groups of convolution modules, and the fourth convolution stage includes six groups of convolution modules. The last convolution layer in this stage outputs a feature F3, and the dimension of F3 is 1024.

[0061] The fifth convolution stage is an important improvement point in this embodiment. It improves the detection capability of large-scale defects by introducing a hollow convolution layer to expand the receptive field of the detection network. In order to set a suitable hole rate to adapt to the size of the defect, the receptive field of the defect in the data set needs to be counted. Table 1 is a statistical table of the receptive field of the training set in this embodiment. According to the receptive field statistics as shown in Table 1, a suitable hole rate is set, and this embodiment provides the following four different hollow convolution modules to balance the accuracy and speed of the model. The appropriate hollow convolution module can be selected according to the actual application requirements.

[0062] Table 1 is a statistical table of the receptive field of the training set in this embodiment

[0063]

[0064] Note: ① for air hole, ② for slag inclusion, ③ for dense, ④ for chain, ⑤ for incomplete fusion, ⑥ for incomplete penetration, ⑦ for crack; * for the receptive field output by the backbone network model.

[0065] The structure diagrams of the four different hollow convolution modules are shown in Figure 3 、 Figure 4 、 Figure 5 and Figure 6 , and the specific description is as follows:

[0066] The structure diagram of the first convolution module is shown in Figure 3 , which is divided into the following steps:

[0067] The features of the weld image are reduced in dimension by 4 1x1 convolutional networks, and the features of the weld image of 1024 dimensions are reduced in dimension to 512 dimensions;

[0068] The features of the weld image after feature reduction are extracted by a plurality of hollow convolutional layers with different hollow rates, that is, the features of the weld image after feature reduction are extracted by 4 3x3 group convolutional networks with hollow rates of (1, 1), (1, 2), (2, 1) and (2, 2) respectively, wherein the number of groups of each branch is 32;

[0069] The features extracted by different hollow convolutional layers are spliced together, that is, the 4 features of 512 dimensions are spliced together for feature fusion to generate features of 2048 dimensions;

[0070] The features of the input weld image are increased in dimension, and the features after feature fusion are added to obtain the final output features, that is, the features of the original weld image of 1024 dimensions are increased in dimension by using a 1x1 convolutional network to obtain features of 2048 dimensions, the features of 2048 dimensions generated in the previous step are added to the features of 2048 dimensions obtained by increasing the dimension, and the final output features are obtained.

[0071] The structure diagram of the second convolutional module is shown in Figure 4 The structure diagram of the second convolutional module is shown in

[0072] The features of the weld image are reduced in dimension by 4 1x1 convolutional networks, and the features of the weld image of 1024 dimensions are reduced in dimension to 256 dimensions;

[0073] The features of the weld image after feature reduction are extracted by a plurality of hollow convolutional layers with different hollow rates, that is, the features of the weld image after feature reduction are extracted by 4 3x3 group convolutional networks with hollow rates of (1, 1), (1, 2), (2, 1) and (2, 2) respectively, wherein the number of groups of each branch is 8;

[0074] The features extracted by different hollow convolutional layers are spliced together, that is, the 4 features of 256 dimensions are spliced together for feature fusion to generate features of 1024 dimensions, and the generated features of 1024 dimensions are increased in dimension by a 1x1 convolutional network to obtain features of 2048 dimensions;

[0075] The features of the input weld image are increased in dimension, and the features after feature fusion are added to obtain the final output features, that is, the features of the original weld image of 1024 dimensions are increased in dimension by using a 1x1 convolutional network to obtain features of 2048 dimensions, the features of 2048 dimensions generated in the previous step are added to the features of 2048 dimensions obtained by increasing the dimension, and the final output features are obtained.

[0076] The structure diagram of the third convolutional module is shown inFigure 5 As shown in FIG. 4, the fourth convolution module is divided into the following steps:

[0077] The features of the weld image are reduced in dimension by using one 1x1 convolution network, and the 1024-dimensional features of the weld image are reduced to 512-dimensional features.

[0078] The features of the weld image after the dimension reduction are extracted by using several hollow convolution layers with different hollow rates, that is, the features of the weld image after the dimension reduction are extracted by using four 3x3 group convolution networks with hollow rates of (1, 1), (1, 2), (2, 1), and (2, 2) respectively, wherein the number of groups in each branch is 8.

[0079] The features extracted by different hollow convolution layers are spliced together, that is, the four 128-dimensional features are spliced together to generate 512-dimensional features, and the generated 512-dimensional features are fused and upgraded in dimension to obtain 2048-dimensional features by using a 1x1 convolution network.

[0080] The features of the input weld image are upgraded in dimension and added to the features after the feature fusion to obtain the final output features, that is, the 1024-dimensional features of the original weld image are upgraded in dimension by using a 1x1 convolution network to obtain 2048-dimensional features, the 2048-dimensional features generated in the previous step are added to the 2048-dimensional features obtained by the dimension upgrade to obtain the final output features.

[0081] The structure of the fourth convolution module is shown in FIG. 4. Figure 6 As shown in FIG. 4, the fourth convolution module is divided into the following steps:

[0082] The features of the weld image are reduced in dimension by using four 1x1 convolution networks, and the 1024-dimensional features of the weld image are reduced to 512-dimensional features.

[0083] The features of the weld image after the dimension reduction are extracted by using several hollow convolution layers with different hollow rates, that is, the features of the weld image after the dimension reduction are extracted by using four 3x3 group convolution networks with hollow rates of (1, 1), (1, 2), (2, 1), and (2, 2) respectively, wherein the number of groups in each branch is 32.

[0084] The features extracted by different hollow convolution layers are upgraded in dimension respectively, that is, the four 512-dimensional features extracted in the previous step are upgraded in dimension by using a 1x1 convolution to obtain four 2048-dimensional features.

[0085] The several features after the dimension upgrade are added together, that is, the four 2048-dimensional features are added together.

[0086] The features of the input weld image are dimensioned, and the features obtained in the previous step are added to obtain the final output features. That is, the features of the original 1024-dimensional weld image are dimensioned using a 1x1 convolutional network to obtain 2048-dimensional features, and the 2048-dimensional features generated in the previous step are added to the 2048-dimensional features obtained by dimensioning to obtain the final output features.

[0087] The feature output of the last convolutional layer of the fifth convolutional stage is output, and the output feature is F4 with a dimension of 2048.

[0088] The features output by the backbone network model as a whole are F1, F2, F3, and F4.

[0089] As shown in Figure 7 , the method of inputting the output results of the backbone network model into the feature pyramid network for feature fusion and enhancement, i.e., the calculation method of the neck network model, is as follows:

[0090] The output results of the backbone network model, i.e., F1, F2, F3, and F4, are respectively subjected to dimension reduction operation using a 1x1 convolutional network to obtain F1', F2', F3', and F4';

[0091] The feature map F4' obtained by the last convolutional unit of the backbone network model is further subjected to feature extraction using a 3x3 convolutional network, a step of 2 is used, and the feature map F4' is subjected to 2 times down-sampling to obtain a new feature map F5'. After this step, the feature maps F1', F2', F3', F4', and F5' are obtained.

[0092] All the feature maps subjected to dimension reduction, i.e., F1', F2', F3', and F4', are respectively subjected to two times up-sampling, and the feature maps are added to the feature maps obtained in the next convolutional stage. The feature map obtained in the last convolutional stage is added to the new feature map obtained in the previous step, and the result after addition is taken as the network output feature. That is, F1' and F2' are added to obtain F2", F2' and F3' are added to obtain F3", F3' and F4' are added to obtain F4", and F4' and F5' are added to obtain F5". The output features of the neck network model are F1", F2", F3", F4", and F5". Among them, F1" is F1'.

[0093] The features F1", F2", F3", F4", and F5" are input into the head network model for defect detection to give the class confidence of the defect and the position of the defect. The structure diagram of the head network model is shown in Figure 8 . The head network model is divided into two parallel branches, i.e., a classification network and a regression network. The classification network is responsible for defect classification, and the regression network is responsible for defect positioning.

[0094] The processing method of the regression network is as follows:

[0095] The fused and enhanced features F1'', F2'', F3'', F4'' and F5'' are input into the regression network, and the regression network extracts features through three 3x3 convolution networks to obtain a feature map F S1 ;

[0096] The feature map F S1 extracted in the previous step is further extracted through a 3x3 convolution network, and the extracted features are regressed through a 1x1 convolution network to obtain a first-stage offset, i.e., offset 1.

[0097] The first-stage offset is converted into a pseudo box.

[0098] The first-stage offset is added to the feature map F S1 obtained after the first feature extraction, and the feature map to which the first-stage offset is added is further extracted through a 3x3 convolution network to obtain a feature map F S2 .

[0099] The feature map F S2 extracted in the previous step is regressed through a 1x1 convolution network to obtain a second-stage offset, i.e., offset 2.

[0100] The second-stage offset is added to the pseudo box to obtain the final defect position.

[0101] The processing method of the classification network is as follows:

[0102] The fused and enhanced features F1'', F2'', F3'', F4'' and F5'' are input into the regression network, and the regression network extracts features through three 3x3 convolution networks.

[0103] The first-stage offset is added to the feature map obtained in the previous step, and the feature map to which the first-stage offset is added is further extracted through a 3x3 convolution network.

[0104] The feature map obtained in the previous step is classified through a 1x1 convolution network to obtain the final defect type confidence.

[0105] The results obtained in the regression network and the classification network are integrated to obtain the output of the final defect detection, i.e., the type confidence of the defect and the position of the defect.

[0106] Embodiment Two

[0107] Based on the same inventive concept, the embodiment discloses a welding seam large-scale defect detection system based on deep learning, which comprises:

[0108] The backbone network model module is configured to input the preprocessed weld seam image into a backbone network model for training, and the backbone network model comprises two or more convolution units, and a convolution kernel of a last layer of the convolution units is a dilated convolution kernel with different dilated rates.

[0109] The neck network model module is configured to input an output result of the backbone network model into a feature pyramid network for feature fusion and enhancement.

[0110] The head network model module is configured to input the fused and enhanced features into a classification network and a regression network respectively to obtain a classification confidence and a position of a weld seam defect.

[0111] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product comprising instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0114] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0115] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the technical solutions of the present application. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the technical solutions of the present application can still be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement should be covered in the protection scope of the claims of the present application. The above content is merely a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all changes or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.​​

Claims

1. A deep learning-based weld large-scale defect detection method, characterized in that, The application relates to a welding seam defect detection method based on a feature pyramid network and a regression network. The preprocessed welding seam image is input into a backbone network model for training, wherein the backbone network model comprises two or more convolution units, and the convolution kernel of the last convolution unit is a hollow convolution kernel with different hollow rates; The output result of the backbone network model is input into a feature pyramid network for feature fusion and enhancement; The fused and enhanced features are input into a regression network and a classification network respectively to obtain the classification confidence and position of the welding seam defect; The backbone network model comprises a first convolution unit, a second convolution unit and a third convolution unit, and each convolution unit comprises a plurality of convolution stages; The first convolution unit comprises a plurality of 3*3 convolution networks for feature extraction and down-sampling of the input welding seam image; The second convolution unit comprises a plurality of convolution modules, each of which comprises 1*1, 3*3 and 1*1 convolution networks, and the number of the second convolution unit is determined according to the feature quantity of the welding seam image; The third convolution unit is a hollow convolution layer; The processing method of the regression network is as follows: The fused and enhanced features are input into the regression network for feature extraction; The feature map obtained through the feature extraction is further subjected to feature extraction, and the extracted features are subjected to position regression to obtain the first-stage offset; The first-stage offset is converted into a pseudo box; The first-stage offset is added to the feature map obtained after the first feature extraction, and the feature map added with the first-stage offset is further subjected to feature extraction; The feature map obtained through the feature extraction is subjected to position regression to obtain the second-stage offset; The second-stage offset is added to the pseudo box to obtain the final defect position. 2.The deep learning-based weld large-scale defect detection method of claim 1, wherein, The preprocessing method is as follows: The welding seam image is cut; The cut welding seam image is subjected to gray value stretching; The welding seam image subjected to the gray value stretching is subjected to random contrast enhancement according to a set proportion; The welding seam image subjected to the random contrast enhancement is subjected to random blur enhancement; The welding seam image subjected to the random blur enhancement is labeled; The labeled welding seam image is subjected to statistical analysis, and the size of the defect in the welding seam image is calculated. 3.The deep learning-based weld large-scale defect detection method of claim 1, wherein, The hollow rate and the number of layers of the hollow convolution kernel are determined by the size of the defect in the welding seam image, and the receptive field is obtained through the hollow convolution kernel, and the calculation formula is as follows: wherein, is the overall span of the output feature map; is the overall span of the input feature map; is the stride of the convolutional layer; is the receptive field of the output feature map; is the receptive field of the input feature map; is the size of the convolutional kernel; is the dilation rate of the dilated convolutional kernel. 4.The deep learning-based weld large-scale defect detection method of claim 3, wherein, The processing method of the hollow convolution layer is as follows: The features of the welding seam image are reduced in dimension through a plurality of 1*1 convolution networks, or the features of the welding seam image are reduced in dimension through one 1*1 convolution network, and the reduced features are divided into a plurality of groups; The features of the welding seam image subjected to the feature reduction are extracted through a plurality of hollow convolution layers with different hollow rates; The features extracted by different hollow convolution layers are spliced together for feature fusion or feature fusion and dimension increase; The features of the input welding seam image are increased in dimension, and the features subjected to the feature fusion are added to obtain the final output features. 5.The deep learning-based weld large-scale defect detection method of claim 4, wherein, The processing method of the hollow convolution layer is as follows: The features of the weld seam image are reduced in dimension by several 1x1 convolutional networks; The feature maps obtained after feature reduction are extracted by several dilated convolutional layers with different dilated rates; The features extracted by different dilated convolutional layers are respectively increased in dimension; The several features obtained after dimension increase are added together; The features of the input weld seam image are increased in dimension, and the features obtained in the previous step are added to obtain the final output features. 6.The deep learning-based weld large-scale defect detection method of claim 4, wherein, The method for inputting the output result of the backbone network model into the feature pyramid network for feature fusion and enhancement is: The output result of the backbone network model is reduced in dimension; The feature map obtained by the last convolutional layer of the backbone network model is extracted, and the feature map is down-sampled to obtain a new feature map; All the feature maps obtained after dimension reduction are up-sampled, and the feature maps are added to the feature maps obtained in the next convolutional stage. The feature map obtained by the last convolutional unit is added to the new feature map obtained in the previous step, and the added result is taken as the network output feature.

7. The deep learning-based weld large-scale defect detection method of claim 6, wherein, The processing method of the classification network is: The fused and enhanced features are input into the regression network, and the regression network extracts features therefrom; The feature map obtained in the previous step is classified to obtain the final defect type confidence. It comprises:

8. A deep learning-based weld large-scale defect detection system, characterized by, A backbone network model module for inputting the preprocessed weld seam image into a backbone network model for training, wherein the backbone network model comprises two or more convolutional units, and the convolution kernel of the last convolutional unit is a dilated convolution kernel with different dilated rates; A neck network model module for inputting the output result of the backbone network model into a feature pyramid network for feature fusion and enhancement; A head network model module for inputting the fused and enhanced features into a classification network and a regression network respectively to obtain the classification confidence and position of the weld seam defects; The backbone network model comprises a first convolutional unit, a second convolutional unit, and a third convolutional unit, each of which comprises several convolutional stages; The first convolutional unit comprises several 3x3 convolutional networks for feature extraction and down-sampling of the input weld seam image; The second convolutional unit comprises several convolutional modules, each of which comprises 1x1, 3x3, and 1x1 convolutional networks, and the number of the second convolutional units is determined according to the number of features of the weld seam image; The third convolutional unit is a dilated convolutional layer; The processing method of the regression network is: The fused and enhanced features are input into the regression network, and the regression network extracts features therefrom; The feature map obtained in the previous step is further extracted, and the extracted features are position-regressed to obtain a first-stage offset; The first-stage offset is converted into a pseudo box; ​ The first stage offset is added to the feature map obtained after the first time feature extraction, and further feature extraction is performed on the feature map to which the first stage offset is added; Position regression is performed on the feature map obtained by the feature extraction in the last step, and a second stage offset is obtained. The second stage offset is added to the pseudo frame, and the final defect position is obtained.

Images