Tunnel smoke and fire detection method based on mutual attention mechanism and domain self-adaption

Abstract

The invention discloses a tunnel smoke and fire detection method based on a mutual attention mechanism and field adaptation, and belongs to the technical field of deep learning. According to the method, tunnel smoke and fire are detected according to problems existing in an actual application scene by using a mutual attention mechanism and fine granularity field self-adaption-based smoke and fire area positioning detection method, and firstly, a large amount of simulated tunnel smoke and fire image data are generated by using an image generation technology; measuring the correlation between the feature vectors of the simulated tunnel smoke and fire image and the real tunnel smoke and fire image through a mutual attention mechanism, preferentially aligning the feature distribution with higher correlation, aligning the image-level feature distribution and the instance-level feature distribution by using a fine-grained field adaptive method, and finally realizing fine-grained field adaptive tunnel smoke and fire positioning detection. According to the method provided by the invention, when tunnel smoke and fire detection is carried out, the distribution of the simulated tunnel smoke and fire image and the distribution of the real tunnel smoke and fire image can be effectively aligned, the tunnel smoke and fire area positioning detection capability is improved by utilizing feature knowledge in the simulated tunnel smoke and fire image, and the accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically designs a tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation. Background Art

[0002] Power cables are mainly used to transmit and distribute electric energy in power systems. As cables have excellent insulation properties, they do not require the erection of transmission towers and can be laid in underground spaces, without occupying ground space and without affecting the beauty of the city. Therefore, cables have become the main component of urban power grids. In order to meet the growing demand for urban power supply grids, the use of power cables has increased year by year. With the increasing density of high-voltage cable tunnels, the risk level of operation in the channel has increased accordingly. There are many flammable materials in the cable channel, and fire is currently the focus of identifying and preventing hidden dangers in the channel. The cross-linked polyethylene of the cable body, the polyethylene sheath of the cable, and the waterproof filling glue in the cable joint are mostly flammable materials. Once the cable body or the joint fails and breaks down, it will cause a continuous fire until the entire cable channel section is burned, which is extremely harmful.

[0003] In order to avoid the safety hazards caused by tunnel fireworks, a large number of visual monitoring devices have been installed in the channel. The current mainstream fireworks detection is mainly divided into traditional detection methods and visual detection methods. Traditional fireworks detection methods have limited human resources and low detection efficiency; sensor detection methods have limited detection distance and high layout and maintenance costs. In recent years, the method of tunnel fireworks detection using visual images has received more and more attention due to its unique advantages. For example, fireworks visual detection based on image processing and fireworks visual detection methods based on deep learning can improve the accuracy of fireworks detection to a certain extent. However, due to the scarcity of tunnel fireworks data and the non-rigidity of fireworks, tunnel fireworks detection still has problems such as weak algorithm generalization and poor recognition effect.

[0004] Due to the difference in feature distribution between simulated images and real images, the feature knowledge of simulated tunnel fireworks cannot be directly used for the detection of real tunnel fireworks. Therefore, it is necessary to use domain adaptation methods to achieve knowledge transfer. The domain adaptation technology used in the research of tunnel fireworks detection methods has shown its effectiveness in many fields. Domain adaptation technology has been widely used in image classification, target detection, semantic segmentation, face recognition, natural language processing and other fields. By utilizing the knowledge and information of the source domain, the domain adaptation method can help the model achieve better performance in the new domain and improve the generalization ability and application scope of the model. Summary of the invention

[0005] In order to solve the problems of the scarcity of tunnel fireworks data and the difficulty of accurate identification of fireworks in the current tunnel fireworks detection, the present invention provides a tunnel fireworks detection method based on mutual attention mechanism and domain adaptation; the present invention detects and identifies tunnel fireworks according to the problems existing in the actual application scenario by using the fireworks area positioning detection method based on mutual attention mechanism and fine-grained domain adaptation, wherein the mutual attention mechanism targets the problem of feature distribution deviation between the simulated tunnel fireworks image and the real tunnel fireworks image, realizes the measurement of the correlation between the features of the simulated tunnel fireworks image and the features of the real tunnel fireworks image, and uses the fine-grained domain adaptation method to align the image-level features and instance-level features of the single-stage target detection, which can achieve faster and better tunnel fireworks detection. The technical problem proposed by the present invention is solved as follows:

[0006] Specific content:

[0007] A tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation includes the following steps:

[0008] Step 1, obtaining the required tunnel fireworks image dataset, including simulated tunnel fireworks images and real tunnel fireworks images, and preliminarily processing the dataset to divide the source domain data and the target domain data;

[0009] Step 2: Input the source domain image and the target domain image into the convolutional feature extraction network to obtain the source domain feature map and the target domain feature map;

[0010] Step 3: Use the mutual attention mechanism to calculate the correlation between the source domain feature vector and the target domain feature vector obtained in step 2, and obtain the mutual attention weighted source domain feature map and target domain feature map;

[0011] Step 4: For each feature vector obtained in step 3, a domain discriminator is designed separately to obtain the probability that the feature vector at the corresponding position predicted by each domain discriminator comes from the target domain or the source domain, and the loss of the image-level domain is calculated;

[0012] Step 5: extract instance-level target domain features and source domain features, and use the domain discriminator and instance-level loss function to calculate the instance-level domain loss;

[0013] Step 6: Calculate the final detection loss through the objective function and update the network parameters;

[0014] Step 7: Train the tunnel fireworks detection network constructed in steps 2, 3, 4, 5, and 6 on the tunnel fireworks dataset.

[0015] Step 1 of the present invention comprises:

[0016] Step 1-1: Collect and select high-quality real tunnel fireworks images on the Internet as the target domain dataset;

[0017] Step 1-2: Generate simulated tunnel fireworks images as source domain datasets through large-scale image generation algorithms such as Diffusion;

[0018] Step 1-3: Perform preliminary processing on the data set, delete irrelevant images, and crop the images to a uniform size;

[0019] Step 2 of the present invention comprises:

[0020] Step 2-1, select YOLOv5 as the basic positioning and detection framework for tunnel fireworks, and use the feature extraction network to extract feature maps;

[0021] Step 2-2, using mosaic image enhancement, slice channel expansion and pyramid feature fusion algorithm to greatly enhance the image feature extraction capability;

[0022] Step 3 of the present invention comprises:

[0023] Step 3-1: Input the source domain image and the target domain image in step 2 into the feature extraction network to obtain the source domain feature map and the target domain feature map, use the source domain feature map to obtain Q, use the target domain feature map to obtain K, and use the following formula to calculate the correlation between the source domain feature vector and the target domain feature vector:

[0024]

[0025] Obtain the mutual attention weighted source domain feature map or target domain feature map, and finally use multiple domain discriminators to align the weighted feature distribution to achieve fine-grained domain adaptation;

[0026] Step 3-2, taking the calculation of the mutual attention weighted features of the target domain as an example, first use the fully connected neural network W Q The target domain feature vector For linear transformation, the formula is as follows:

[0027]

[0028] So as to obtain the target domain feature vector The query vector At the same time, the fully connected neural network W K For the source domain feature vector, For linear transformation, the formula is as follows:

[0029]

[0030] Thus, the keyword vector of the target domain feature vector is obtained During the optimization of the network, the query vector and keyword vector By WQ and W K The parameters of the target domain features and the source domain features are obtained. Then the target domain feature vector can be calculated by the following formula Mutual attention weight with source domain feature vector

[0031]

[0032] Step 3-3, calculate After the attention weights, each target domain feature vector The corresponding mutual attention weights Multiplying them together will give us the feature vector weighted by mutual attention;

[0033] Step 3-4: In order to improve the reliability of the domain adaptation network in aligning the feature distribution of different domains, a residual mechanism is introduced to reduce the impact of incorrect mutual attention weights. The formula for calculating the feature vector of mutual attention weight is as follows:

[0034]

[0035] Step 3-5: According to steps 3-1 to 3-4, perform mutual attention weighting on the target domain and the source domain, and finally obtain the mutual attention weighted source domain feature map and target domain feature map;

[0036] Step 4 of the present invention comprises:

[0037] Step 4-1, design a domain discriminator for each feature vector. Each domain discriminator consists of a three-layer fully connected neural network, and a total of H×W domain discriminators are designed;

[0038] Step 4-2, domain discriminator Responsible for predicting the probability that the feature vector at position (i, j) comes from the target domain or the source domain. The predicted probability and domain label are used to calculate the loss, and then the gradient reversal layer GRL and the feature extractor G f Relative confrontation, thus achieving fine-grained domain adaptation;

[0039] Step 4-3, calculate the domain discrimination loss of the k-th scale feature layer through the image-level objective function, the objective function is as follows:

[0040]

[0041]

[0042] Step 5 of the present invention comprises:

[0043] Step 5-1, similar to step 4-1, designs a separate domain discriminator for each feature vector of the target domain and source domain features, and performs instance-level fine-grained domain adaptation on instance-level features at multiple scales simultaneously;

[0044] Step 5-2, through instance-level multi-scale fine-grained domain adaptation objective function:

[0045]

[0046]

[0047] Get the domain discriminant loss of the instance-level features of the k-th scale of the input sample image.

[0048] Step 6 of the present invention comprises:

[0049] Step 6-1, the objective function of the detection task is mainly composed of three parts: bounding box regression loss, confidence loss and classification loss. By designing the objective function of the detection task:

[0050]

[0051] Calculate the loss of the detection task;

[0052] Step 6-2, the fine-grained domain adaptation objective functions at the image level and instance level obtained in steps 4 and 5 are responsible for constraining the domain discriminator and feature extractor, prompting the domain discriminator to learn the ability to determine which data domain each feature vector comes from, and at the same time, the gradient reversal layer enables the feature extractor to extract the common features between the two data domains. The purpose of fine-grained alignment of the two data and feature distributions is achieved, thereby improving the generalization performance of the network model.

[0053] Step 6-3, through the final objective function:

[0054]

[0055] The final network loss is calculated, and the network’s detection capability and domain adaptation capability are optimized to improve the network’s cross-domain target detection capability. f (x s ) and the task inference network calculate the location coordinates O and category C of the tunnel fireworks area.

[0056] The present invention proposes a tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation, which has the following advantages compared with the prior art:

[0057] The present invention combines the mutual attention mechanism and the domain adaptation method, which can effectively improve the accuracy of tunnel smoke and fire detection. Compared with the existing methods, the method proposed in the present invention can effectively align the distribution of simulated tunnel smoke and fire images and real tunnel smoke and fire images when performing tunnel smoke and fire detection, so as to solve the problems of lack of tunnel smoke and fire data and difficulty in extracting smoke and fire features, and improve the accuracy of tunnel smoke and fire detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is the overall flow chart of the method of the present invention;

[0059] Figure 2 This is a fine-grained domain adaptive tunnel fire area positioning detection framework diagram based on mutual attention weighting;

[0060] Figure 3 It is a schematic diagram of the mutual attention mechanism;

[0061] Figure 4 It is the mutual attention weighted source domain feature map and target domain feature map;

[0062] Figure 5 It is a schematic diagram of the domain discriminator;

[0063] Figure 6 is a schematic diagram of instance-level mutual attention weighted feature alignment network;

[0064] Figure 7 This is the YOLOv5 algorithm framework diagram;

[0065] Figure 8 This is an example of tunnel fireworks; DETAILED DESCRIPTION

[0066] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0067] The present invention is implemented including:

[0068] The overall process of the method proposed by the present invention is as follows Figure 1 As shown in the figure, the fine-grained domain adaptive tunnel fire area positioning detection framework based on mutual attention weighting is as follows Figure 2 As shown, the specific construction steps of tunnel smoke and fire detection in the embodiment of the present invention are as follows:

[0069] Step 1: obtain the required tunnel fireworks image dataset, including 5000 simulated tunnel fireworks images and 2000 real tunnel fireworks images, and perform preliminary processing on the dataset to divide the source domain data and target domain data;

[0070] Step 2: Input the source domain image and the target domain image into the convolutional feature extraction network to obtain the source domain feature map and the target domain feature map;

[0071] Step 3: Use the mutual attention mechanism to calculate the correlation between the source domain feature vector and the target domain feature vector obtained in step 2. The schematic diagram of the mutual attention mechanism is as follows: Figure 3 As shown, the mutual attention weighted source domain feature map and target domain feature map are obtained, as shown in Figure 4 As shown;

[0072] Step 4: Design a domain discriminator for each feature vector obtained in step 3. The domain discriminator is as follows: Figure 5 As shown, the probability that the feature vector at the corresponding position predicted by each domain discriminator comes from the target domain or the source domain is obtained, and the loss of the image-level domain is calculated;

[0073] Step 5: Extract instance-level target domain features and source domain features. The schematic diagram of instance-level mutual attention weighted feature alignment network is as follows: Figure 6 As shown, the domain discriminator and instance-level loss function are used to calculate the instance-level domain loss;

[0074] Step 6: Calculate the final detection loss through the objective function and update the network parameters;

[0075] Step 7: Train the tunnel fireworks detection network built in steps 2, 3, 4, 5, and 6 on the tunnel fireworks dataset. The tunnel fireworks examples in the dataset are as follows: Figure 8 shown.

[0076] Step 1 of the present invention comprises:

[0077] Step 1-1: Collect and select 2,000 high-quality real tunnel fireworks images on the Internet as the target domain dataset;

[0078] Step 1-2: Generate 5,000 simulated tunnel fireworks images as the source domain dataset through large-scale image generation algorithms such as Diffusion;

[0079] Step 1-3: Perform preliminary processing on the data set, delete irrelevant pictures, adjust the length and width of the pictures to (640, 640), and divide them into training set, validation set and test set, where the data volume in the training set accounts for 70% of the total data volume, the validation set accounts for 10% of the total data volume, and the test set accounts for 20% of the total data volume;

[0080] Step 2 of the present invention comprises:

[0081] Step 2-1, select YOLOv5 as the basic positioning and detection framework for tunnel fireworks. Its algorithm framework is as follows: Figure 7 As shown, a feature extraction network is used to extract feature maps;

[0082] Step 2-2, using mosaic image enhancement, slice channel expansion and pyramid feature fusion algorithm to greatly enhance the image feature extraction capability;

[0083] Step 3 of the present invention comprises:

[0084] Step 3-1: Input the source domain image and the target domain image in step 2 into the feature extraction network to obtain the source domain feature map and the target domain feature map, use the source domain feature map to obtain Q, use the target domain feature map to obtain K, and use the following formula to calculate the correlation between the source domain feature vector and the target domain feature vector:

[0085]

[0086] In the formula, Softm represents the Softmax activation function, d k Denotes the dimension of each eigenvector of matrices Q, K.

[0087] Thus, the source domain feature map or target domain feature map with mutual attention weighting is obtained. The schematic diagram of the mutual attention mechanism is as follows: Figure 3 Finally, multiple domain discriminators are used to align the weighted feature distribution to achieve fine-grained domain adaptation;

[0088] Step 3-2, taking the calculation of the mutual attention weighted features of the target domain as an example, first use the fully connected neural network W Q The target domain feature vector For linear transformation, the formula is as follows:

[0089]

[0090] So as to obtain the target domain feature vector The query vector At the same time, the fully connected neural network W K For the source domain feature vector, For linear transformation, the formula is as follows:

[0091]

[0092] Thus, the keyword vector of the target domain feature vector is obtained During the optimization of the network, the query vector and keyword vector By W Q and W K The parameters of the target domain features and the source domain features are obtained. Then the target domain feature vector can be calculated by the following formula Mutual attention weight with source domain feature vector

[0093]

[0094] Where d represents the query vector and keyword vector The dimension of represents the target domain (i, j) and (l, k) Degree of relevance.

[0095] Step 3-3, calculate After the attention weights, each target domain feature vector The corresponding mutual attention weights Multiplying them together will give us the feature vector weighted by mutual attention;

[0096] Step 3-4: In order to improve the reliability of the domain adaptation network in aligning the feature distribution of different domains, a residual mechanism is introduced to reduce the impact of incorrect mutual attention weights. The formula for calculating the feature vector of mutual attention weight is as follows:

[0097]

[0098] Step 3-5: According to steps 3-1 to 3-4, the target domain and the source domain are weighted for mutual attention, and finally the mutual attention weighted source domain feature map and target domain feature map are obtained, such as Figure 4 As shown;

[0099] Step 4 of the present invention comprises:

[0100] Step 4-1, design a domain discriminator for each feature vector. Each domain discriminator consists of a three-layer fully connected neural network. A total of H×W domain discriminators are designed. The domain discriminators are as follows: Figure 5 As shown;

[0101] Step 4-2, domain discriminator Responsible for predicting the probability that the feature vector at position (i, j) comes from the target domain or the source domain. The predicted probability and domain label are used to calculate the loss, and then the gradient reversal layer GRL and the feature extractor G f Relative confrontation, thus achieving fine-grained domain adaptation;

[0102] Step 4-3, calculate the domain discrimination loss of the k-th scale feature layer through the image-level objective function, the objective function is as follows:

[0103]

[0104]

[0105] Step 5 of the present invention comprises:

[0106] Step 5-1, similar to step 4-1, designs a separate domain discriminator for each feature vector of the target domain and source domain features, and simultaneously performs instance-level fine-grained domain adaptation on instance-level features at multiple scales. The instance-level mutual attention weighted feature alignment network is as follows: Figure 6 As shown;

[0107] Step 5-2, through instance-level multi-scale fine-grained domain adaptation objective function:

[0108]

[0109]

[0110] in is the input sample image x m The instance-level mutual attention weighted feature vector of . M = h × w represents the total number of instance-level feature vectors. is the domain discrimination loss of instance-level features at the kth scale, μ k is the balance coefficient of domain adaptation of the k-th scale instance-level feature layer.

[0111] Thereby obtaining the domain discrimination loss of the instance-level features of the k-th scale of the input sample image.

[0112] Step 6 of the present invention comprises:

[0113] Step 6-1, the objective function of the detection task is mainly composed of three parts: bounding box regression loss, confidence loss and classification loss. By designing the objective function of the detection task:

[0114]

[0115] t pre and t gt Represents the prediction and label of the detection task branch of the network, including the location information and category information of the bounding box. s Indicates that the detection task is trained in the source domain data, n s Represents the number of source domain image samples per iteration. Indicates the prediction of bounding boxes and categories on the feature layers of three scales. * Respectively represent the weights of each loss function. 2 is the size of the feature map and B is the total number of bounding boxes predicted for each feature. Indicates whether there is an object in the jth bounding box predicted by the i-th feature point in the k-th feature map. If so, it is 1, otherwise it is 0.

[0116] Thus, the detection task loss is calculated according to the formula;

[0117] Step 6-2, the fine-grained domain adaptation objective functions at the image level and instance level obtained in steps 4 and 5 are responsible for constraining the domain discriminator and feature extractor, prompting the domain discriminator to learn the ability to determine which data domain each feature vector comes from, and at the same time, the gradient reversal layer enables the feature extractor to extract the common features between the two data domains. The purpose of fine-grained alignment of the two data and feature distributions is achieved, thereby improving the generalization performance of the network model.

[0118] Step 6-3, through the final objective function:

[0119]

[0120] The final network loss is calculated, and the network’s detection capability and domain adaptation capability are optimized to improve the network’s cross-domain target detection capability. f (x s ) and the task inference network calculate the location coordinates O and category C of the tunnel fireworks area.

[0121] The present invention proposes a tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation, which has the following advantages compared with the prior art:

[0122] The present invention combines the mutual attention mechanism and the domain adaptation method, which can effectively improve the accuracy of tunnel smoke and fire detection. Compared with the existing methods, the method proposed in the present invention can effectively align the distribution of simulated tunnel smoke and fire images and real tunnel smoke and fire images when performing tunnel smoke and fire detection, so as to solve the problems of lack of tunnel smoke and fire data and difficulty in extracting smoke and fire features, and improve the accuracy of tunnel smoke and fire detection.

[0123] The present invention provides a tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation. The above description is only an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tunnel smoke and fire detection method based on mutual attention mechanism and domain adaptation, characterized in that: The following steps are involved: Step 1, obtaining the required tunnel fireworks image dataset, including simulated tunnel fireworks images and real tunnel fireworks images, and preliminarily processing the dataset to divide the source domain data and the target domain data; Step 2: Input the source domain image and the target domain image into the convolutional feature extraction network to obtain the source domain feature map and the target domain feature map; Step 3: Use the mutual attention mechanism to calculate the correlation between the source domain feature vector and the target domain feature vector obtained in step 2, and obtain the mutual attention weighted source domain feature map and target domain feature map; Step 4: For each feature vector obtained in step 3, a domain discriminator is designed separately to obtain the probability that the feature vector at the corresponding position predicted by each domain discriminator comes from the target domain or the source domain, and the loss of the image-level domain is calculated; Step 5: extract instance-level target domain features and source domain features, and use the domain discriminator and instance-level loss function to calculate the instance-level domain loss; Step 6: Calculate the final detection loss through the objective function and update the network parameters; Step 7: Train the tunnel fireworks detection network constructed in steps 2, 3, 4, 5, and 6 on the tunnel fireworks dataset.

2. The method according to claim 1, characterized in that Step 1 includes: Step 1-1: Collect and select high-quality real tunnel fireworks images on the Internet as the target domain dataset; Step 1-2: Generate simulated tunnel fireworks images as source domain datasets through large-scale image generation algorithms such as Diffusion; Step 1-3: Perform preliminary processing on the data set, delete irrelevant images, and crop the images to a uniform size.

3. The method according to claim 1, characterized in that Step 2 includes: Step 2-1, select YOLOv5 as the basic positioning and detection framework for tunnel fireworks, and use the feature extraction network to extract feature maps; Step 2-2, using mosaic image enhancement, slice channel expansion and pyramid feature fusion algorithm to greatly enhance the image feature extraction capability.

4. The method according to claim 1, characterized in that Step 3 includes: Step 3-1: Input the source domain image and the target domain image in step 2 into the feature extraction network to obtain the source domain feature map and the target domain feature map, use the source domain feature map to obtain Q, use the target domain feature map to obtain K, and use the following formula to calculate the correlation between the source domain feature vector and the target domain feature vector: Obtain the mutual attention weighted source domain feature map or target domain feature map, and finally use multiple domain discriminators to align the weighted feature distribution to achieve fine-grained domain adaptation; Step 3-2, taking the calculation of the mutual attention weighted features of the target domain as an example, first use the fully connected neural network W Q The target domain feature vector For linear transformation, the formula is as follows: So as to obtain the target domain feature vector The query vector At the same time, the fully connected neural network W K For the source domain feature vector, For linear transformation, the formula is as follows: Thus, the keyword vector of the target domain feature vector is obtained During the optimization of the network, the query vector and keyword vector By W Q and W K The parameters of the target domain features and the source domain features are obtained. Then the target domain feature vector can be calculated by the following formula Mutual attention weight with source domain feature vector Step 3-3, calculate After the attention weights, each target domain feature vector The corresponding mutual attention weights Multiplying them together will give us the feature vector weighted by mutual attention; Step 3-4: In order to improve the reliability of the domain adaptation network in aligning the feature distribution of different domains, a residual mechanism is introduced to reduce the impact of incorrect mutual attention weights. The formula for calculating the feature vector of mutual attention weight is as follows: Step 3-5: According to steps 3-1 to 3-4, perform mutual attention weighting on the target domain and the source domain, and finally obtain the mutual attention weighted source domain feature map and target domain feature map.

5. The method according to claim 1, characterized in that Step 4 includes: Step 4-1, design a domain discriminator for each feature vector. Each domain discriminator consists of a three-layer fully connected neural network, and a total of H×W domain discriminators are designed; Step 4-2, domain discriminator Responsible for predicting the probability that the feature vector at position (i, j) comes from the target domain or the source domain. The predicted probability and domain label are used to calculate the loss, and then the gradient reversal layer GRL and the feature extractor G f Relative confrontation, thus achieving fine-grained domain adaptation; Step 4-3, calculate the domain discrimination loss of the k-th scale feature layer through the image-level objective function, the objective function is as follows:

6. The method according to claim 1, characterized in that Step 5 includes: Step 5-1, similar to step 4-1, designs a separate domain discriminator for each feature vector of the target domain and source domain features, and performs instance-level fine-grained domain adaptation on instance-level features at multiple scales simultaneously; Step 5-2, through instance-level multi-scale fine-grained domain adaptation objective function: Get the domain discrimination loss of the instance-level features of the k-th scale of the input sample image.

7. The method according to claim 1, characterized in that Step 6 includes: Step 6-1, the objective function of the detection task is mainly composed of three parts: bounding box regression loss, confidence loss and classification loss. By designing the objective function of the detection task: Calculate the detection task loss; Step 6-2, the fine-grained domain adaptation objective functions at the image level and instance level obtained in steps 4 and 5 are responsible for constraining the domain discriminator and feature extractor, prompting the domain discriminator to learn the ability to determine which data domain each feature vector comes from, and at the same time, the gradient reversal layer enables the feature extractor to extract the common features between the two data domains. The purpose of fine-grained alignment of the two data and feature distributions is achieved, thereby improving the generalization performance of the network model. Step 6-3, through the final objective function: The final network loss is calculated, and the network’s detection capability and domain adaptation capability are optimized to improve the network’s cross-domain target detection capability. f (x s ) and the task inference network calculate the location coordinates O and category C of the tunnel fireworks area.

Images