Target plate damage detection method and system based on twin network

Abstract

The present application relates to a target plate damage detection method and system based on a twin network, the method comprising: acquiring a target plate damage image to be detected; constructing a target plate damage detection model; inputting the target plate damage image to be detected into the target plate damage detection model; and acquiring a segmentation mask image, wherein the target plate damage detection model is constructed based on a U-Net network and is obtained by training a training set, the training set comprising: an image of a target plate damage area and a label identifying the target plate damage area; and quantitatively analyzing the segmentation mask image to acquire geometric information of the target plate damage area. The present application constructs a prediction network with twin branches for predicting a damage area of an input image. The introduction of regional attention increases the attention to the target area and suppresses the expression of useless information. A multi-scale convolution module is added to improve the perception of edge details and improve the prediction accuracy. Finally, effective application is achieved in target plate damage area identification and quantification.

Description

[0001] The present invention relates to the fields of machine vision and artificial intelligence, and in particular to a target plate damage detection method and system based on a twin network. Background Art

[0002] In modern high-tech warfare, it is of great significance to control the damaging effect of explosives and achieve precise strikes on targets. In order to evaluate the damaging effect of explosives, field experiments are often required. In the damaging experiment of explosives containing fragments, field tests are an important basis for evaluating the damaging effect of explosive fragments. In the damaging experiment of explosives containing fragments, the use of witness target plates can realize the collection of fragments and retain the dispersion pattern of fragments. Fragments with different impact speeds and angles will leave damaged areas (holes) of different sizes on the steel plate. Statistical analysis and recording of the distribution, number and size of holes on the witness steel plate are of great significance for evaluating the damaging effect. At present, traditional manual detection methods are mostly relied on. This method has the problems of low detection efficiency, poor accuracy, strong subjectivity and high detection cost.

[0003] Therefore, there is an urgent need for a target plate damage detection method and system based on twin networks to improve detection efficiency and accuracy and reduce subjectivity and cost. Summary of the Invention

[0004] The purpose of the present invention is to provide a target plate damage detection method and system based on a twin network, with the aim of improving the detection efficiency and accuracy of the target plate damage area.

[0005] To achieve the above objectives, the present invention provides a target plate damage detection method based on a twin network and a regional attention mechanism, comprising:

[0006] Acquire damage images of the target plate to be tested;

[0007] Constructing a target plate damage detection model, inputting the target plate damage image to be tested into the target plate damage detection model, and obtaining a segmentation mask image, wherein the target plate damage detection model is constructed based on a U-Net network and obtained by training with a training set, wherein the training set includes: an image of a target plate damage area and a label identifying the target plate damage area;

[0008] The segmentation mask image is quantitatively analyzed to obtain geometric information of the target plate damage area.

[0009] Optionally, before inputting the target plate damage image to be tested into the target plate damage detection model, the method includes: compressing the target plate damage image to be tested by nearest neighbor interpolation and normalization operations to obtain three-dimensional matrix data.

[0010] Optionally, constructing the target plate damage detection model based on the U-Net network includes:

[0011] A multi-scale convolution module and a CBR module are respectively added to the encoder and decoder of the U-Net network, and a twin classification module and a regional attention module are added between the encoder and the decoder to construct the target plate damage detection model.

[0012] Optionally, inputting the target plate damage image to be tested into the target plate damage detection model to obtain a segmentation mask image includes:

[0013] Inputting the three-dimensional matrix data of the target plate damage image to the multi-scale convolution module to perform multi-scale local information extraction and global information depth extraction to obtain high-dimensional feature information;

[0014] Processing the high-dimensional feature information through normalization and activation function to obtain nonlinear high-dimensional feature information;

[0015] Inputting the nonlinear high-dimensional feature information into the twin classification module to obtain the damage area attention weight parameter;

[0016] Inputting the damaged area attention weight parameter after upsampling and the nonlinear high-dimensional feature information of the corresponding scale into the regional attention module to obtain a spatial weighted feature map;

[0017] The spatial weighted feature map is input into the CBR module to obtain the segmentation mask map.

[0018] Optionally, the CBR module includes a convolution module, BN regularization and activation function connected in sequence.

[0019] Optionally, obtaining a spatially weighted feature map includes:

[0020] Perform global average pooling on the nonlinear high-dimensional feature map to obtain channel attention weights;

[0021] Input the channel attention weight into the FC fully connected layer to obtain the final channel attention weight;

[0022] Performing weighted multiplication of the final channel attention weight and the nonlinear high-dimensional feature map to obtain a channel weighted feature map;

[0023] The channel weighted feature map is multiplied by the damage area attention weight parameter to obtain the spatial weighted feature map.

[0024] Optionally, during the process of training the target plate damage detection model using the training set, loss calculation is performed using a loss function with multiple constraints:

[0025] L m =L c +L seg

[0026] wherein, L m is a multi-constraint loss function; L c is a classification loss of the twin classification module; L seg is a segmentation loss; L c s CE s is a similarity loss, L CE is a weight cross-entropy loss function; L seg s BCE Dice BCE is a cross-entropy loss function, L Dice is a set similarity loss.

[0027] Optionally, the segmentation mask graph is subjected to quantitative analysis to obtain geometric information of the target plate damage area, including:

[0028] The boundary pixel points of the adhesion area in the segmentation mask graph are eliminated by a watershed algorithm to obtain a non-adhesion binary image;

[0029] Connected threshold calculation is performed on the non-adhesion binary image to obtain the geometric information of the target plate damage area, and the labeling of the target plate damage area is completed.

[0030] In another aspect, a target plate damage detection system based on a twin network is also provided, including an image acquisition module, a data preprocessing module, a data transmission module, a data analysis module, and a visualization module.

[0031] The image acquisition module is configured to acquire damage area image data.

[0032] The data preprocessing module is configured to preprocess the damage area image data.

[0033] The data transmission module is configured to transmit the preprocessed damage area image to the data analysis module.

[0034] The data analysis module is configured to construct a target plate damage detection model, input the target plate damage image to be detected into the target plate damage detection model, obtain a segmentation mask graph, and subject the segmentation mask graph to quantitative analysis to obtain geometric information of the target plate damage area.

[0035] The visualization module is configured to present the geometric information of the target plate damage area on a display, and simultaneously present damage area color image data.

[0036] ​​​​​​​The beneficial effects of the present invention are as follows: the present invention utilizes the advantages of high precision, high efficiency and contactlessness of machine vision technology to construct a prediction network with twin branches for predicting the damaged area of ​​the input image. The network with twin branches improves the generalization ability of the model by learning implicit mapping relationships with similarity constraints; introduces regional attention to increase the focus on the target area and suppress the expression of useless information; adds a multi-scale convolution module to improve the perception of edge details and improves prediction accuracy. Ultimately, this method has been effectively applied in the identification and quantification of target plate damaged areas, changing the traditional manual detection method for most target plate damaged area identification and quantification, avoiding the problems of low detection efficiency, poor accuracy, strong subjectivity and high detection cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 Flowchart of a target plate damage detection method based on a twin network according to an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the network structure of a target plate damage detection model according to an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of a target plate damage detection system based on a twin network according to an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of a target plate damage detection device based on a twin network according to an embodiment of the present invention;

[0042] Figure 5 Schematic diagram of the structure of the regional attention module of an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] The rapid development of artificial intelligence (AI) and machine vision in recent years has made intelligent detection of damaged areas on targets possible. For example, AI algorithms have achieved breakthroughs in areas such as facial recognition, image detection, natural language processing, emotion recognition, and voice recognition. AI has become a key technology in the new round of scientific and technological revolution and industrial transformation. Explosive breakthroughs in machine learning and big data are key to driving the advancement of AI technology. With the advancement of computer hardware, big data, and deep learning technologies, machine learning models are becoming increasingly sophisticated and are developing at a massive scale, making their application to damage area identification feasible in this field.

[0046] Machine vision technology uses computers to simulate human vision and the human brain, extracting, processing, and analyzing image information through methods such as feature extraction and nonlinear combination. A typical machine vision system consists of an image acquisition device and a computing device. Improving production automation and efficiency are the two main goals of machine vision systems in industrial production. For example, in large-scale and repeatable industrial production activities, the use of machine vision methods can improve production efficiency and automation. Machine vision can also perform automated defect detection through neural networks. The advantages of machine vision technology, such as high precision, high efficiency, and non-contact nature, make this technology promising for application in damage area identification. Therefore, a target plate damage detection method based on a twin network and a regional attention mechanism is provided.

[0047] Example 1

[0048] like Figure 1 As shown, this embodiment provides a target plate damage detection method based on a twin network, including:

[0049] S1. Obtain damage images of the target plate to be tested.

[0050] S2. Construct a target plate damage detection model, input the target plate damage image to be tested into the target plate damage detection model, and obtain a segmentation mask image, wherein the target plate damage detection model is constructed based on a U-Net network and obtained through training with a training set, and the training set includes an image of the target plate damage area and a label identifying the target plate damage area.

[0051] (1) The target plate damage detection model is trained by the target plate damage image training set to obtain the target plate damage detection model including:

[0052] Target plate damage images are obtained, and data enhancement is used to expand the dataset by rotating, mirroring, adding noise, and other operations on the image set. The dataset is used to train a target plate damage detection model based on the twin network structure.

[0053] The Siamese network has a dual-branch parallel network architecture. Both branches have identical structures. During training, two randomly selected samples are run simultaneously on the dataset. The two branches perform parallel computations and cross-learning. Two output masks are generated, which are used to calculate the loss. The loss function is a multi-constraint loss function composed of contrastive loss, classification loss, and Dice loss.

[0054] L m =L c +L seg

[0055] Where, L m is the loss function of multiple constraints; L c is the classification loss of the twin classification module; L seg For segmentation loss.

[0056] L c =L s +L CE

[0057] Where, L s is the similarity loss; L CE is the weighted cross entropy loss function.

[0058]

[0059] Where Y is the similarity label. When the sample pair belongs to the same class, Y = 1; when the sample pair belongs to different classes, Y = 0. w : Euclidean distance of the classification network output vector.

[0060]

[0061] Where X is the predicted output; Y is the true output; N is the number of demarcated regions, and CN is the total number of categories.

[0062] L seg =L s +L BCE +L Dice

[0063] Where, L seg is the total segmentation loss; L s is the pixel-level similarity loss; L BCE is the cross entropy loss function; L Dice is the set similarity loss. L s Mainly focuses on the similarity between two pixels. L BCE The model is constrained to converge at each pixel without paying attention to the information between pixels. The calculation formula is:

[0064]

[0065] Where N is the total number of pixels; Y i ∈{0, 1} is the true label value; X i ∈{X i |0≤X i ≤1} is the probability value predicted by the model.

[0066]

[0067] Where X is the predicted probability matrix and Y is the true label matrix. |X∩Y| is the dot product of matrices X and Y and the element-wise summation.

[0068] (2) Input the target plate damage image to the target plate damage detection model to obtain the segmentation mask map, such as Figure 2 The following figure shows the network structure of the target damage detection model, where the multi-scale feature extraction module is composed of multiple multi-scale convolution modules:

[0069] The first step is image data preprocessing. The 800*800*3 three-dimensional color image is compressed into a three-dimensional matrix data with a value range of [-1, 1] and a size of 512*512*3 through nearest neighbor interpolation and normalization operations to reduce the amount of calculation and improve the calculation efficiency.

[0070] In the second step, multi-scale convolution and pooling operations are performed on the matrix data to extract local and global information in depth four times at multiple scales, with the number of channels each time being 32, 64, 128, and 256, ultimately obtaining high-dimensional feature information with a size reduced by 16 times.

[0071] The third step is normalization and adding activation function. The high-dimensional feature information is normalized again and the activation function is used to obtain nonlinear high-dimensional feature information.

[0072] The fourth step is twin classification processing, which uses the twin classification network to process and compress nonlinear high-dimensional features to obtain the 16*16*1 damage area attention weight parameters.

[0073] In the fifth step, a gradual upsampling process is performed. The regional attention weights and nonlinear high-dimensional feature information are upsampled four times using a regional attention mechanism combined with nearest neighbor interpolation and bilinear interpolation. The upsampled damage region attention weight parameters are then fed into the regional attention module along with the nonlinear high-dimensional feature information at the corresponding scale. The output of the regional attention module is then fed into the CBR module, generating a 512x512x1 binary mask. The CBR module consists of a series of 3x3 convolutions coupled with batch normalization and ReLU activation functions, increasing model depth and improving the accuracy of pixel classification at the target damage edge.

[0074] like Figure 5 As shown in the figure, the regional attention model first performs global average pooling on the nonlinear high-dimensional feature information to obtain preliminary channel attention weights. It then passes multiple FC fully connected layers to obtain the final channel weights, which are weighted and multiplied with the original feature map to obtain a channel-weighted feature map. This is then multiplied with the upsampled classification matrix (i.e., the damage region attention weight parameter) to obtain a spatially weighted feature map. The feature map output by the regional attention module significantly enhances the feature representation of the target damage region and suppresses the representation of background information, thereby improving the model's focus on the target damage region.

[0075] S3. Quantitatively analyze the segmentation mask image to obtain geometric information of the target plate damage area.

[0076] Connectivity threshold calculation is combined with the watershed algorithm to determine the geometric size and number of damaged areas in the binary mask image, while also labeling the damaged areas on the image. Connectivity threshold calculation can find the entire target area, but it can identify two contiguous areas as a single region. The watershed algorithm can separate contiguous areas, but it can easily miss smaller areas.

[0077] The fusion process first uses the watershed algorithm to obtain the position of the boundary pixels in the adhesion area, eliminates the boundary, and obtains a binary image without adhesion. Then, the connectivity threshold can be used to calculate the accurate result.

[0078] (4) A verification experiment was designed, and 31 samples were selected for 10 repeated experiments. At the same time, the results were compared with standard measurement parameters. The average error was 6%, which fully demonstrated the effectiveness of the method of the present invention.

[0079] Example 2

[0080] like Figure 3 As shown, this embodiment provides a target plate damage detection system based on a twin network, including:

[0081] An image data acquisition module, used to acquire image data of the damaged area;

[0082] The image acquisition module comprises a pulley, a liftable bracket and an armrest. A visible light camera and an auxiliary light source are placed on the liftable bracket.

[0083] The image data acquisition module is designed to standardize image acquisition. During acquisition, the movable image data acquisition module is positioned in front of a steel target plate, and the visible light camera and light source bracket are adjusted to a specific height. The light source is a 50x50 LED array with adjustable brightness and color. The image information captured by the visible light camera is a 3D color image of 800x800x3 at 30 frames per second.

[0084] A data transmission module, used for transmitting the processed damage image data to the data analysis module;

[0085] A data preprocessing module is used to preprocess the collected image data;

[0086] a data analysis module, configured to construct a target plate damage detection model, input the target plate damage image to be tested into the target plate damage detection model, obtain a segmentation mask image, perform quantitative analysis on the segmentation mask image, and obtain geometric information of the target plate damage area;

[0087] The visualization module is used to present the statistical geometric size and quantity information of the target plate damage area on the display, and at the same time present the color image data of the damage area.

[0088] Example 3

[0089] The target plate damage detection device based on the twin network and regional attention mechanism in this embodiment includes a central processing unit, a graphics processing unit, a memory, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it controls the device and implements the steps of the method. The device structure is as follows: Figure 4 shown.

[0090] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A target plate damage detection method based on twin networks, characterized in that: include: Acquire damage images of the target plate to be tested; Constructing a target plate damage detection model, inputting the target plate damage image to be tested into the target plate damage detection model, and obtaining a segmentation mask image, wherein the target plate damage detection model is constructed based on a U-Net network and obtained by training with a training set, wherein the training set includes: an image of a target plate damage area and a label identifying the target plate damage area; Constructing the target plate damage detection model based on the U-Net network includes: A multi-scale convolution module and a CBR module are added to the encoder and decoder of the U-Net network, respectively, and a twin classification module and a regional attention module are added between the encoder and the decoder to construct the target plate damage detection model; Inputting the target plate damage image to be tested into the target plate damage detection model to obtain a segmentation mask image includes: Inputting the three-dimensional matrix data of the target plate damage image to the multi-scale convolution module to perform multi-scale local information extraction and global information depth extraction to obtain high-dimensional feature information; Processing the high-dimensional feature information through normalization and activation function to obtain nonlinear high-dimensional feature information; Inputting the nonlinear high-dimensional feature information into the twin classification module to obtain the damage area attention weight parameter; Inputting the damaged area attention weight parameter after upsampling and the nonlinear high-dimensional feature information of the corresponding scale into the regional attention module to obtain a spatial weighted feature map; Inputting the spatial weighted feature map into the CBR module to obtain the segmentation mask map; The CBR module includes a convolution module, a BN regularization and an activation function connected in sequence; The segmentation mask image is quantitatively analyzed to obtain geometric information of the target plate damage area.

2. The target plate damage detection method based on twin network according to claim 1 is characterized in that: Before inputting the target plate damage image to be tested into the target plate damage detection model, the method includes: compressing the target plate damage image to be tested through nearest neighbor interpolation and normalization operations to obtain three-dimensional matrix data.

3. The target plate damage detection method based on twin network according to claim 1 is characterized in that: Obtaining a spatially weighted feature map includes: Perform global average pooling on the nonlinear high-dimensional feature map to obtain channel attention weights; Input the channel attention weight into the FC fully connected layer to obtain the final channel attention weight; Performing weighted multiplication of the final channel attention weight and the nonlinear high-dimensional feature map to obtain a channel weighted feature map; The channel weighted feature map is multiplied by the damage area attention weight parameter to obtain the spatial weighted feature map.

4. The target plate damage detection method based on twin network according to claim 1 is characterized in that: In the process of training the target plate damage detection model through the training set, the loss function with multiple constraints is used to calculate the loss: L m =L c +L seg Among them, L m is the loss function of multiple constraints; L c is the classification loss of the twin classification module; L seg is the segmentation loss; L c =L s +L CE , L s is the similarity loss, L CE is the weighted cross entropy loss function; L seg =L s +L BCE +L Dice , L BCE is the cross entropy loss function, L Dice is the set similarity loss.

5. The target plate damage detection method based on twin network according to claim 1 is characterized in that: Quantitatively analyzing the segmentation mask image to obtain geometric information of the target plate damage area includes: Eliminating the boundary pixels of the adhesion area in the segmentation mask image by using a watershed algorithm to obtain a non-adhesion binary image; A connectivity threshold calculation is performed on the unattached binary image to obtain geometric information of the target plate damaged area, and at the same time, the target plate damaged area is labeled.

6. A system for implementing the target plate damage detection method based on twin networks according to any one of claims 1 to 5, characterized in that: include: Image acquisition module, data preprocessing module, data transmission module, data analysis module and visualization module; The image acquisition module is used to acquire image data of the damaged area; The data preprocessing module is used to preprocess the damaged area image data; The data transmission module is configured to transmit the pre-processed image of the damaged area to the data analysis module; The data analysis module is used to construct a target plate damage detection model, input the target plate damage image to be tested into the target plate damage detection model, obtain a segmentation mask image, perform quantitative analysis on the segmentation mask image, and obtain geometric information of the target plate damage area; The visualization module is used to present the geometric information of the target plate damage area on the display and simultaneously present the color image data of the damage area.

Images