Lightweight robust tunnel disease detection method based on pulse large model
Through a lightweight robust tunnel disease detection method based on pulsed large model, using U-Net network and adversarial network to generate fake images, combined with high-frequency feature generator and pulse cross attention layer, the tunnel in the prior art is solved The high cost of disease detection and poor accuracy have achieved low power consumption and high accuracy detection effects.
Patent Information
- Application Number
- CN202411285041.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-13
AI Technical Summary
The existing tunnel disease detection methods based on machine vision and deep learning have problems such as high power consumption and poor generalization capabilities, resulting in high detection costs and poor accuracy of detection results.
Using a lightweight robust tunnel disease detection method based on pulsed large models, false images are generated through U-Net networks and adversarial networks, combined with high-frequency feature generators, patch embedding layers and Couple Net networks, and using pulse cross attention layers and self-attention mechanisms generate high-quality embeddings to perform tunnel disease bounding boxes and category predictions.
It achieves low power consumption and strong generalization capabilities, reduces detection costs and improves the accuracy of detection results.
Smart Images
Figure CN119130993A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of tunnel disease detection, and in particular to a lightweight robust tunnel disease detection method based on a pulsed large model. Background Art
[0002] Tunnels are important engineering structures for transportation. During operation, cracks, leakages, potholes and other defects are very likely to occur. Tunnel defect detection, as a detection method that can timely detect the type, form and location of defects, plays an important role in ensuring tunnel safety. At present, with the continuous development of computer vision technology, tunnel defect detection methods based on machine vision and deep learning have been widely used. However, in practical applications, the deep learning models used in existing tunnel defect detection methods based on machine vision and deep learning generally have the problems of high power consumption and poor generalization ability, which leads to the problems of high detection cost and poor accuracy of detection results in existing tunnel defect detection methods based on machine vision and deep learning. Based on this, it is necessary to invent a lightweight and robust tunnel defect detection method based on a pulsed large model to solve the problems of high detection cost and poor accuracy of detection results in existing tunnel defect detection methods based on machine vision and deep learning. Summary of the invention
[0003] In order to solve the problems of high detection cost and poor detection result accuracy of existing tunnel disease detection methods based on machine vision and deep learning, the present invention provides a lightweight and robust tunnel disease detection method based on a pulsed large model.
[0004] The present invention is achieved by adopting the following technical solutions:
[0005] A lightweight and robust tunnel disease detection method based on a pulsed large model is implemented in the following steps:
[0006] Step 1: Collect the degraded image of the tunnel and input the degraded image into the U-Net network to generate the overexposure coefficient and the restored image;
[0007] Step 2: The overexposure coefficient and the restored image are synthesized and input into the generator of the adversarial network to generate a fake image; then, on the one hand, the fake image is input into the discriminator of the adversarial network for training, and on the other hand, the normal brightness image is input into the discriminator of the adversarial network as the real image for training, and the parameters of the discriminator and the generator are iteratively updated during the training process, so that the generator continuously fits the real data, thereby generating a restored image close to the real image;
[0008] Step 3: Construct three independent branches: the first branch is the high-frequency feature generator, the second branch is the patch embedding layer, and the third branch is the Couple Net network; then, the restored image close to the real image is input into the three branches respectively, thereby generating three feature embeddings respectively; the specific steps are as follows:
[0009] Step 3.1: Input the restored image close to the real image into the first branch, thereby generating the first feature embedding, which contains the location information of the target object;
[0010] Step 3.2: Input the restored image close to the real image into the second branch, thereby generating a second feature embedding, which contains rich contextual information;
[0011] Step 3.3: Input the restored image close to the real image into the third branch, thereby generating a third feature embedding, which contains more explicit context information;
[0012] Step 4: The three feature embeddings are fed into the feature refiner to generate a high-quality embedding that contains contextual information and the location of the target object. The high-quality embedding is then fed into the criss-cross attention layer of the query refiner and the high-quality embedding is combined with the second feature embedding and fed into the image encoder to generate an embedding.
[0013] Step 5: Input the query into the query refiner to obtain a refined query;
[0014] Step 6: The refined query and the embedding generated by the image encoder are input into the box decoder, thereby outputting the bounding box prediction result of the tunnel disease; at the same time, the refined query and the embedding generated by the image encoder are input into the category decoder, thereby outputting the category prediction result of the tunnel disease.
[0015] Furthermore, in step 3.1, the first branch first uses a high-pass filter to filter out low-frequency components of the image, and then uses a high-frequency feature embedding layer to reconstruct the image, thereby generating a first feature embedding.
[0016] Furthermore, in step 3.2, the second branch uses large kernel convolution and increased dilation rate to quickly expand the receptive field, thereby generating a second feature embedding.
[0017] Furthermore, in step 3.3, the third branch makes the small object features of the image clearer by enlarging the feature map of the image, thereby generating a third feature embedding.
[0018] Furthermore, in the step four, the feature refiner first uses the pulse cross attention layer to merge the three feature embeddings, and then uses the batch normalization layer and the fully connected layer to normalize the three feature embeddings and integrate and reorganize the features, thereby generating a high-quality embedding containing context information and target object location information.
[0019] Furthermore, in step five, the query refiner first normalizes the query using a batch normalization layer, and then refines the query using a pulse cross attention layer and high-quality embedding, thereby obtaining a refined query.
[0020] Furthermore, in step six, the box decoder updates all embeddings by using the pulse self-attention mechanism and the pulse cross-attention mechanism in two directions, thereby outputting the bounding box prediction result of the tunnel disease; the category decoder updates all embeddings by using the pulse self-attention mechanism and the pulse cross-attention mechanism in only one direction, thereby outputting the category prediction result of the tunnel disease.
[0021] Furthermore, the working process of the pulse cross attention layer is:
[0022] First, extract the query Q from input sequence 1, and extract the key K and value V from input sequence 2;
[0023] Then, the query Q, key K, and value V are converted into the query Q in the form of a spike sequence through the spike neuron layer. s , pulse sequence form key K s , the value V in the form of a pulse train s ;
[0024] Then, the similarity matrix S between the query Q and the key K is calculated; the specific calculation formula is:
[0025]
[0026] Where: d represents the dimension of the key vector;
[0027] Then, the similarity matrix S is normalized using the softmax function to obtain the attention weight matrix A; specifically, it is expressed as:
[0028]
[0029] Finally, the attention weight matrix A is combined with the value V in the form of a spike train s Multiply them together to get the final output.
[0030] Furthermore, the first feature embedding is used as the input sequence 1 of the pulse cross attention layer in the feature refiner; the second feature embedding and the third feature embedding are combined as the input sequence 2 of the pulse cross attention layer in the feature refiner; the high-quality embedding is used as the input sequence 1 of the pulse cross attention layer in the query refiner; and the normalized query is used as the input sequence 2 of the pulse cross attention layer in the query refiner.
[0031] Compared with the existing tunnel disease detection methods based on machine vision and deep learning, the pulsed large model adopted by the present invention has the characteristics of consuming energy only when emitting pulses, and being able to process large amounts of data, thus having the advantages of low power consumption and strong generalization ability, thereby making the detection cost of the present invention lower and the detection result more accurate.
[0032] The present invention effectively solves the problems of high detection cost and poor detection result accuracy of existing tunnel disease detection methods based on machine vision and deep learning, and is suitable for tunnel disease detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a schematic diagram of the principle of the present invention.
[0034] Figure 2 It is a schematic diagram of the working process of the pulse cross attention layer in the present invention. DETAILED DESCRIPTION
[0035] A lightweight and robust tunnel disease detection method based on a pulsed large model is implemented in the following steps:
[0036] Step 1: Collect the degraded image of the tunnel and input the degraded image into the U-Net network to generate the overexposure coefficient and the restored image;
[0037] Step 2: The overexposure coefficient and the restored image are synthesized and input into the generator of the adversarial network to generate a fake image; then, on the one hand, the fake image is input into the discriminator of the adversarial network for training, and on the other hand, the normal brightness image is input into the discriminator of the adversarial network as the real image for training, and the parameters of the discriminator and the generator are iteratively updated during the training process, so that the generator continuously fits the real data, thereby generating a restored image close to the real image;
[0038] Step 3: Construct three independent branches: the first branch is the high-frequency feature generator, the second branch is the patch embedding layer, and the third branch is the Couple Net network; then, the restored image close to the real image is input into the three branches respectively, thereby generating three feature embeddings respectively; the specific steps are as follows:
[0039] Step 3.1: Input the restored image close to the real image into the first branch, thereby generating the first feature embedding, which contains the location information of the target object;
[0040] Step 3.2: Input the restored image close to the real image into the second branch, thereby generating a second feature embedding, which contains rich contextual information;
[0041] Step 3.3: Input the restored image close to the real image into the third branch, thereby generating a third feature embedding, which contains more explicit context information;
[0042] Step 4: The three feature embeddings are fed into the feature refiner to generate a high-quality embedding that contains contextual information and the location of the target object. The high-quality embedding is then fed into the criss-cross attention layer of the query refiner and the high-quality embedding is combined with the second feature embedding and fed into the image encoder to generate an embedding.
[0043] Step 5: Input the query into the query refiner to obtain a refined query;
[0044] Step 6: The refined query and the embedding generated by the image encoder are input into the box decoder, thereby outputting the bounding box prediction result of the tunnel disease; at the same time, the refined query and the embedding generated by the image encoder are input into the category decoder, thereby outputting the category prediction result of the tunnel disease.
[0045] In step 3.1, the first branch first uses a high-pass filter to filter out the low-frequency components of the image, and then uses a high-frequency feature embedding layer to reconstruct the image, thereby generating a first feature embedding.
[0046] In step 3.2, the second branch uses large kernel convolution and increased dilation rate to quickly expand the receptive field, thereby generating a second feature embedding.
[0047] In step 3.3, the third branch makes the small object features of the image clearer by expanding the feature map of the image, thereby generating a third feature embedding.
[0048] In the step 4, the feature refiner first uses the pulse cross attention layer to merge the three feature embeddings, and then uses the batch normalization layer and the fully connected layer to normalize the three feature embeddings and integrate and reorganize the features, thereby generating a high-quality embedding containing context information and target object location information.
[0049] In the step five, the query refiner first normalizes the query using a batch normalization layer, and then refines the query using a pulse cross attention layer and high-quality embedding, thereby obtaining a refined query.
[0050] In step six, the box decoder updates all embeddings by using the pulse self-attention mechanism and the pulse cross-attention mechanism in two directions, thereby outputting the bounding box prediction result of the tunnel disease; the category decoder updates all embeddings by using the pulse self-attention mechanism and the pulse cross-attention mechanism in only one direction, thereby outputting the category prediction result of the tunnel disease.
[0051] The working process of the pulse cross attention layer is:
[0052] First, extract the query Q from input sequence 1, and extract the key K and value V from input sequence 2;
[0053] Then, the query Q, key K, and value V are converted into the query Q in the form of a spike sequence through the spike neuron layer. s , pulse sequence form key K s , the value V in the form of a pulse train s ;
[0054] Then, the similarity matrix S between the query Q and the key K is calculated; the specific calculation formula is:
[0055]
[0056] Where: d represents the dimension of the key vector;
[0057] Then, the similarity matrix S is normalized using the softmax function to obtain the attention weight matrix A; specifically, it is expressed as:
[0058]
[0059] Finally, the attention weight matrix A is combined with the value V in the form of a spike train s Multiply them together to get the final output.
[0060] The first feature embedding is used as the input sequence 1 of the pulse cross attention layer in the feature refiner; the second feature embedding and the third feature embedding are combined as the input sequence 2 of the pulse cross attention layer in the feature refiner; the high-quality embedding is used as the input sequence 1 of the pulse cross attention layer in the query refiner; the normalized query is used as the input sequence 2 of the pulse cross attention layer in the query refiner.
[0061] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.
Claims
1. A lightweight and robust tunnel disease detection method based on a pulsed large model, characterized by: This method is implemented by the following steps: Step 1: Collect the degraded image of the tunnel and input the degraded image into the U-Net network to generate the overexposure coefficient and the restored image; Step 2: The overexposure coefficient and the restored image are synthesized and input into the generator of the adversarial network to generate a fake image; then, on the one hand, the fake image is input into the discriminator of the adversarial network for training, and on the other hand, the normal brightness image is input into the discriminator of the adversarial network as the real image for training, and the parameters of the discriminator and the generator are iteratively updated during the training process, so that the generator continuously fits the real data, thereby generating a restored image close to the real image; Step 3: Construct three independent branches: the first branch is the high-frequency feature generator, the second branch is the patch embedding layer, and the third branch is the Couple Net network; then, the restored image close to the real image is input into the three branches respectively, thereby generating three feature embeddings respectively; The specific steps are as follows: Step 3.1: Input the restored image close to the real image into the first branch, thereby generating the first feature embedding, which contains the location information of the target object; Step 3.2: Input the restored image close to the real image into the second branch, thereby generating a second feature embedding, which contains rich contextual information; Step 3.3: Input the restored image close to the real image into the third branch, thereby generating a third feature embedding, which contains more explicit context information; Step 4: The three feature embeddings are fed into the feature refiner to generate a high-quality embedding that contains contextual information and the location of the target object. The high-quality embedding is then fed into the criss-cross attention layer of the query refiner and the high-quality embedding is combined with the second feature embedding and fed into the image encoder to generate an embedding. Step 5: Input the query into the query refiner to obtain a refined query; Step 6: The refined query and the embedding generated by the image encoder are input into the box decoder, thereby outputting the bounding box prediction result of the tunnel disease; at the same time, the refined query and the embedding generated by the image encoder are input into the category decoder, thereby outputting the category prediction result of the tunnel disease.
2. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In step 3.1, the first branch first uses a high-pass filter to filter out the low-frequency components of the image, and then uses a high-frequency feature embedding layer to reconstruct the image, thereby generating a first feature embedding.
3. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In step 3.2, the second branch uses large kernel convolution and increased dilation rate to quickly expand the receptive field, thereby generating a second feature embedding.
4. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In step 3.3, the third branch makes the small object features of the image clearer by expanding the feature map of the image, thereby generating a third feature embedding.
5. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In the step 4, the feature refiner first uses the pulse cross attention layer to merge the three feature embeddings, and then uses the batch normalization layer and the fully connected layer to normalize the three feature embeddings and integrate and reorganize the features, thereby generating a high-quality embedding containing context information and target object location information.
6. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In the step five, the query refiner first normalizes the query using a batch normalization layer, and then refines the query using a pulse cross attention layer and high-quality embedding, thereby obtaining a refined query.
7. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: In step 6, the box decoder updates all embeddings by using a pulse self-attention mechanism and a pulse cross-attention mechanism in two directions, thereby outputting a bounding box prediction result of the tunnel disease; The category decoder uses spiking self-attention and spiking cross-attention in one direction only to update all embeddings, thereby outputting the category prediction result of tunnel disease.
8. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 1 is characterized by: The working process of the pulse cross attention layer is: First, extract the query Q from input sequence 1, and extract the key K and value V from input sequence 2; Then, the query Q, key K, and value V are converted into the query Q in the form of a spike sequence through the spike neuron layer. s , pulse sequence form key K s , the value V in the form of a pulse train s ; Then, the similarity matrix S between the query Q and the key K is calculated; the specific calculation formula is: Where: d represents the dimension of the key vector; Then, the similarity matrix S is normalized using the softmax function to obtain the attention weight matrix A; specifically, it is expressed as: Finally, the attention weight matrix A is combined with the value V in the form of a spike train s Multiply them together to get the final output.
9. The lightweight robust tunnel disease detection method based on pulsed large model according to claim 8 is characterized by: The first feature embedding is used as the input sequence 1 of the pulse cross attention layer in the feature refiner; the second feature embedding and the third feature embedding are combined as the input sequence 2 of the pulse cross attention layer in the feature refiner; the high-quality embedding is used as the input sequence 1 of the pulse cross attention layer in the query refiner; the normalized query is used as the input sequence 2 of the pulse cross attention layer in the query refiner.