Abnormal facility detection method, device and electronic equipment based on deep learning

Abstract

The present application discloses a method for detecting abnormal facilities based on deep learning, a device for detecting abnormal facilities based on deep learning, an electronic device and a computer-readable storage medium. The detection method is implemented based on a pre-trained facility detection model. The facility detection model sets an MSDSF module after the feature extraction module corresponding to the last scale of the backbone network, and fuses local target information and non-local background information at different scales, thereby enriching the features of small targets and enhancing the distinction between small targets and backgrounds. An ARFA submodule is set in the feature extraction module corresponding to each scale, and different regions of the input features at a single scale are fused based on different weights; the attention level of different regions of the input features is dynamically adjusted based on the weight learning mechanism, so that the input features can be processed more finely and adaptively, the expression ability of key features is improved, and the detection accuracy of the facility detection model for different abnormal urban facilities under complex and changing backgrounds is improved.

Description

Technical Field

[0001] The present application belongs to the field of target detection technology, and in particular, relates to a method for detecting abnormal facilities based on deep learning, a device for detecting abnormal facilities based on deep learning, an electronic device, and a computer-readable storage medium. Background Art

[0002] As the global urbanization process continues to accelerate, the scale and complexity of urban infrastructure are growing, which requires urban management to have a higher level of intelligence to ensure the efficient operation and sustainable development of the city. Various facility anomalies are inevitable in the daily operation of the city, such as damage to various public safety facilities. If the urban facility anomalies caused by these events cannot be quickly detected and effectively responded to, it will not only reduce the efficiency of urban management, but may also directly threaten the safety and quality of life of citizens.

[0003] Abnormal urban facility detection often faces the challenge of small target detection. For example, small scattered garbage targets are small in size, which leads to unclear features and less information, thus increasing the difficulty of detecting abnormal urban facilities. Summary of the invention

[0004] The present application provides a method for detecting abnormal facilities, an abnormal facility detection device, an electronic device and a computer-readable storage medium. The semantic fusion module can fuse local target information and non-local background information at different scales to provide more contextual information for small target detection, thereby reducing the difficulty of detecting abnormal urban facilities.

[0005] In the first aspect, the present application provides a method for detecting abnormal facilities based on deep learning. The detection method is implemented based on a pre-trained facility detection model. The facility detection model includes a backbone network, a neck network, and a detection head. The backbone network is provided with a multi-scale semantic fusion module, and the semantic fusion module is used to fuse local target information and non-local background information at different scales. The detection method includes:

[0006] Perform feature extraction operations on the processed image based on the backbone network to obtain image features;

[0007] Perform feature fusion operation on image features based on the neck network to obtain fusion features;

[0008] The fused features are detected based on the detection head to obtain the detection results of abnormal urban facilities in the image to be processed.

[0009] In a second aspect, the present application provides a detection device for abnormal facilities based on deep learning, the detection device is configured with a pre-trained facility detection model, the facility detection model includes a backbone network, a neck network and a detection head; the backbone network is provided with a multi-scale semantic fusion module for fusing local target information and non-local background information at different scales; the detection device includes:

[0010] An extraction module, used for performing feature extraction operation on the image to be processed based on the backbone network to obtain image features;

[0011] A fusion module, used for performing a feature fusion operation on image features based on the neck network to obtain fused features;

[0012] The detection module is used to detect the fused features based on the detection head to obtain the detection results of abnormal urban facilities in the image to be processed.

[0013] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.

[0015] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it implements the steps of the method of the first aspect.

[0016] Compared with the prior art, the present application has the following beneficial effects: a multi-scale semantic fusion module is set in the backbone network to fuse local target information and non-local background information at different scales, thereby enriching the features of small targets and enhancing the distinction between small targets and backgrounds. Through such a backbone network, image features containing more contextual information can be extracted, and then the fusion features obtained by image fusion based on image features also contain more contextual information of small targets. Detecting the fusion features can improve the accuracy of target detection, thereby reducing the difficulty of detecting urban facilities.

[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a schematic diagram of classification examples of 11 types of abnormal urban facilities provided in an embodiment of the present application;

[0020] Figure 2 It is a quantitative statistical histogram of 11 types of abnormal urban facilities provided in the embodiment of the present application;

[0021] Figure 3 is a schematic flow chart of a method for detecting abnormal facilities based on deep learning provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the network structure of the MSDSF module provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the network structure of the ARFA submodule provided in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the network structure of the facility detection model MSF-YOLOv8s provided in an embodiment of the present application;

[0025] Figure 7 It is a data comparison diagram of key evaluation indicators of the two models MSF-YOLOv8s and YOLO v8s provided in the embodiments of the present application;

[0026] Figure 8 It is a schematic diagram of the detection results of three abnormal urban facilities by MSF-YOLOv8s and YOLOv8s provided in the embodiment of the present application;

[0027] Fig. 9 It is a structural schematic diagram of a detection device for abnormal facilities based on deep learning provided in an embodiment of the present application;

[0028] Fig.10 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0030] Urban facilities may include one or more of roads, manhole covers, guardrails, walls, telescopic devices, drainage facilities, isolation fences, crash barriers, and vehicle stoppers.

[0031] In order to enable the facility detection model to more accurately understand and detect abnormal urban facilities, this application creates a professional dataset called Abnormal City Facilities (ACF), which focuses on collecting and analyzing abnormal urban facility instances. The uniqueness of this dataset is that all image data comes from on-site collection in a real vehicle environment, which ensures the authenticity and diversity of the data.

[0032] It can be understood that the more images there are in the dataset, the better the performance of the trained facility detection model. Assuming that the entire dataset covers 40,000 high-quality images, in order to ensure the effectiveness and generalization ability of the algorithm model, the dataset is divided into two parts according to the classic 8:2 ratio: a training set containing 32,000 images for training and optimizing the facility detection model; a validation set containing 8,000 images for verifying the detection ability, accuracy and generalization of the facility detection model, and ensuring the performance of the facility detection model on unknown data.

[0033] Optionally, in order to provide a higher quality dataset, in addition to the number of images, the images can also cover as many abnormal urban facilities commonly found in urban environments as possible, so as to comprehensively cover and accurately focus on multiple challenges in urban management.

[0034] For example, this application roughly divides abnormal urban facilities into 11 categories, named from the first to the eleventh category, and the corresponding diagrams of each category can be found in Figure 1 Of course, different classifications can be adjusted according to actual needs, and are not limited in this application.

[0035] Furthermore, in order to clearly show the prevalence and distribution characteristics of various types of abnormal urban facilities, the number of images in each category can be counted separately. Based on the above classification example, in order to intuitively display the statistical results, refer to Figure 2, the statistical results can be presented in the form of charts. This chart can intuitively reflect the number of different abnormal urban equipment, and then provide data support for subsequent priority sorting, resource allocation and even the formulation of prevention strategies. It is an indispensable part of studying the optimization of urban facility management. Through data visualization, we can more systematically understand which problems are high-frequency pain points in urban management, so as to effectively optimize and deploy corresponding resources and countermeasures for high-frequency pain points.

[0036] from Figure 2 It is not difficult to see that the first and third types of facilities are high-frequency pain points in the urban management process. However, the size of these two types of targets is usually small, which means that the detection of abnormal urban facilities faces the challenge of small target detection. The features of these targets are not obvious and the amount of information is small, which increases the difficulty of detecting abnormal urban facilities.

[0037] In order to solve this problem, this application proposes a facility detection model, which includes a backbone network, a neck network and a detection head. Specifically, the backbone network is used to extract features from the input image to obtain image features; the neck network is used to fuse the image features to obtain fused features; the detection head is used to detect the fused features to obtain the detection results of abnormal urban facilities in the image to be processed.

[0038] Among them, in order to enhance the characteristics of small targets, improve the accuracy of small target detection, and thus reduce the difficulty of detecting abnormal urban facilities, the present application sets a multi-scale semantic fusion module (multi-scale detailed semantic fusion module, MSDSF module) in the backbone network. The MSDSF module can be used to fuse local target information and non-local background information at different scales, thereby enriching the characteristics of small targets and enhancing the distinction between small targets and backgrounds. The improved backbone network can extract image features containing more contextual information, and then the fusion features obtained by image fusion based on the image features also contain more contextual information of small targets. By detecting the fusion features, the accuracy of target detection can be improved, thereby reducing the difficulty of detecting urban facilities.

[0039] In some embodiments, the backbone network includes m feature extraction modules of different scales, where m is a positive integer and m≥2; the MSDSF module is arranged after the mth feature extraction module. Assuming that the facility detection model is improved based on the YOLO series model, for example, for YOLOv5s, the efficient Spatial Pyramid Pooling – Fast (SPPF) structure in YOLOv5s will be replaced by the MSDSF module; for YOLOv8s, the SPPF structure in YOLOv8s will be replaced by the MSDSF module.

[0040] In some embodiments, the MSDSF module may include at least two single-scale fusion sub-modules, wherein different single-scale sub-fusion modules include dilated convolution layers with different dilation rates, and the single-scale fusion sub-modules are sequentially connected. Based on the MSDSF module of this structure, a multi-scale semantic fusion operation can be performed on the first input feature of the MSDSF module, thereby fusing local target information and non-local background information at different scales.

[0041] In some embodiments, the single-scale fusion submodule may include a preset convolution layer, a hole convolution layer corresponding to a preset hole rate, and a splicing layer. The preset convolution layer is used to perform a convolution operation on the second input feature input to the single-scale fusion submodule to integrate the features and adjust the number of channels of the features. The hole convolution layer with a preset hole rate is used to perform a deep convolution on the convolution result of the preset convolution layer to extract local target information and non-local background information at the current scale. The splicing layer can splice the second input feature with the convolution result of the deep convolution in the channel dimension to obtain a single-scale fusion feature corresponding to the single-scale fusion submodule.

[0042] It can be understood that based on the connection relationship of each single-scale fusion sub-module, the second input feature corresponding to the single-scale fusion sub-module that is first in the connection order is the first input feature, and the second input feature corresponding to the single-scale fusion sub-module that is not first in the connection order is the single-scale fusion feature corresponding to the previous single-scale fusion sub-module; the semantic fusion feature is obtained based on the single-scale fusion feature corresponding to the single-scale fusion sub-module that is last in the connection order.

[0043] The complex and ever-changing urban environment also poses considerable challenges to the detection of abnormal urban facilities. In related technologies, facility detection lacks sufficient flexibility and adaptability, making it difficult to accurately capture and adapt to dynamic changes in the environment. For example, abnormal facilities of the third, fifth, and tenth categories are easily affected by the environment and are prone to problems such as unclear category definition, complexity in context or semantic understanding, which makes it difficult for the facility detection model to accurately classify them into a specific category, resulting in problems such as missed detection or false alarms.

[0044] In some embodiments, in order to solve this problem, based on the above embodiments, the present application sets an adaptive receptive field attention mechanism (ARFA) submodule in the feature extraction module corresponding to each scale, which is used to fuse different regions of the input features at a single scale based on different weights. Based on the weight learning mechanism, the attention level (i.e., receptive field) of different regions of the input features is dynamically adjusted, and the input features can be processed more finely and adaptively, and the expression ability of key features can be improved, thereby improving the detection accuracy of the facility detection model for different abnormal facilities under complex and changing backgrounds to a certain extent, and improving the detection effect of the facility detection model on complex tasks.

[0045] In some embodiments, the ARFA submodule includes a feature extraction unit, an adaptive receptive field attention unit, and a fusion unit. Among them, the feature extraction unit can be used to perform convolution operations of at least 3 convolution scales on the third input feature of the ARFA submodule. Different convolution operations correspond to different convolution kernel sizes, and can focus on at least 3 areas at the current scale, that is, obtain at least 3 receptive fields. In order to dynamically adjust the attention of different areas, the feature maps corresponding to each area can be fused, reshaped, and spliced ​​through the adaptive receptive field attention unit to calculate the weights corresponding to each area. Finally, the fusion unit performs a weighted sum operation on each feature map and the corresponding weight to obtain the fusion feature corresponding to the third input feature. This fusion feature emphasizes the effective integration of information from different regions and channels and the enhancement of key features, and its expression ability is stronger.

[0046] In some embodiments, the adaptive receptive field attention unit includes a summation layer, a global average pooling layer, a global fully connected layer, a fully connected layer corresponding to each convolution scale, and a normalization layer. Among them, the size of at least three feature maps obtained by the feature extraction unit performing more than three convolution operations on the third feature is the same. Based on this, the summation layer can perform element-wise sum operations on each feature map of the same size to obtain an initial fused feature; the global average pooling layer can perform global average pooling on the initial fused feature to compress feature vectors with the same number of channels and capture global information; the compressed initial fused features are fused through the global fully connected layer to obtain a global fused feature map; the global fused features are reshaped and stacked through the fully connected layers corresponding to each convolution scale to obtain a stacked result; the stacked result is normalized through the normalization layer to determine the weights corresponding to each convolution scale.

[0047] In some embodiments, the loss function of the facility detection model is mainly composed of regression loss and classification loss. Among them, the classification loss is the binary cross entropy loss BCE Loss, which is used to determine the specific category in the anchor box. The regression loss is CIoU Loss and DFL loss, which are used to calculate the error between the predicted bounding box and the ground truth box. At the same time, the matching strategy between positive and negative samples adopts the TAL dynamic matching method.

[0048] Specifically, the formula of BCE Loss is as follows:

[0049]

[0050] in, L BCE is the binary cross entropy loss, n is the number of image samples of all abnormal urban facilities, y i For the i The true category of the image samples, is the predicted category of the i-th image sample.

[0051] The regression loss is used to calculate the error between the predicted bound box and the ground truth box. The regression loss uses the CIOU loss (Complete Intersection over Union loss, L CIoU ), which is an improved version of commonly used loss functions such as L1, L2, Intersection over Union (IoU) and GIoU. L CIoU The penalty term for aspect ratio is added. When the center points of the predicted box and the labeled box coincide, the errors in different situations can be better distinguished and the scale is invariant.

[0052]

[0053] in, b and b gt They are the center points of the prediction box and the annotation box respectively; ρ is the Euclidean distance between the predicted box and the labeled box; c is the distance between the diagonal of the closed area of ​​the prediction box and the annotation box; v To ensure the consistency of the relative proportions of the predicted box and the labeled box, IoU is the intersection-over-union ratio of the predicted box and the labeled box; αis the weight coefficient; w and w gt They are the width of the prediction box and the annotation box respectively; h and h gt They are the heights of the predicted box and the labeled box respectively.

[0054] The formula for DFL loss is as follows:

[0055]

[0056] in, S i is the cross entropy loss between the target true value and the predicted value on the left, S i+1 It is the cross entropy loss between the true target value and the predicted value on the right.

[0057] Total loss function of the facility detection model Loss The formula is as follows:

[0058]

[0059] Among them, λ1 and λ1 are balance coefficients.

[0060] The facility detection model described in any of the foregoing embodiments is trained based on the total loss function of the facility detection model. By continuously iterating the facility detection model so that the facility detection model has better detection effect on various abnormal urban facilities and tends to be stable, a robust version can be selected from multiple converged versions as the trained facility detection model.

[0061] In some embodiments, in order to comprehensively and accurately measure the performance of each version of the facility detection device, it can be evaluated through a validation set. Specifically, the performance of the facility detection device on the validation set can be evaluated according to preset conditions. For example, these preset conditions may include performance indicators such as the intersection-over-union ratio, detection accuracy, and recall rate of the facility detection device for road defects.

[0062] IoU is an indicator for evaluating the degree of overlap between the predicted box and the annotated box. It calculates the ratio of the intersection and union of the predicted box and the annotated box. It plays a key role in determining whether it is a correct detection. The calculation formula of IoU can be written as:

[0063]

[0064] Precision (p), also known as the precision rate, refers to the proportion of correct positive predictions to all positive predictions, as shown in the formula:

[0065]

[0066] Recall (R), also known as recall rate, refers to the proportion of correctly predicted positives to all actual positives, as shown in the formula:

[0067]

[0068] Average Precision (AP) is calculated from precision and recall. A line graph of precision is drawn based on the recall value, and the area under the line is calculated, as shown in the formula:

[0069]

[0070] Mean Average Precision (mAP) refers to the average value of the average precision AP of C different road disease categories, as shown in the formula:

[0071]

[0072] That is to say, after verifying each version of the facility detection model through the verification set, the facility detection model of each version can be comprehensively evaluated based on the above-mentioned indicators, so as to determine the facility detection model with the best performance from each version as the trained facility detection model.

[0073] Based on the facility detection model of the previous embodiment, the present application proposes a method for detecting abnormal facilities based on deep learning.

[0074] The abnormal facility detection method based on deep learning provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, vehicle-mounted equipment, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc., and the electronic device is equipped with a trained facility detection module. The embodiment of the present application does not impose any restrictions on the specific type of electronic devices.

[0075] In order to illustrate the technical solution proposed in this application, each embodiment will be described below with an electronic device as the execution subject.

[0076] Figure 3 A schematic flow chart of a method for detecting abnormal facilities based on deep learning provided by the present application is shown, and the detection method includes:

[0077] Step 310: The electronic device performs a feature extraction operation on the image to be processed based on the backbone network to obtain image features.

[0078] Step 320: The electronic device performs a feature fusion operation on the image features based on the neck network to obtain fused features.

[0079] Step 330: The electronic device detects the fused features based on the detection head to obtain the detection result of the abnormal urban facilities in the image to be processed.

[0080] In this embodiment, the improved backbone network benefits from the introduction of the MSDSF module, and the electronic device can use the MSDSF module to fuse local target information and non-local background information at different scales, thereby enriching the features of small targets and enhancing the distinction between small targets and backgrounds. Through such a backbone network, image features containing more contextual information can be extracted, and then the fusion features obtained by image fusion based on the image features also contain more contextual information of small targets. By detecting the fusion features, the accuracy of abnormal urban facility detection can be improved, thereby reducing the difficulty of urban facility detection.

[0081] In some embodiments, step 320 may include:

[0082] Step A1: The electronic device inputs the first input feature of the MSDSF module into the sequentially connected single-scale fusion sub-modules to perform a multi-scale semantic fusion operation to obtain a semantic fusion feature.

[0083] The first input feature will be processed sequentially by the sequentially connected single-scale fusion submodules to finally obtain a semantic fusion feature. The semantic fusion feature contains more contextual information of small targets. By detecting the fusion feature, the accuracy of target detection can be improved, thereby reducing the difficulty of detecting urban facilities.

[0084] It can be understood that step A1 is only part of the operation in step 320, and is a step performed by the improved structure MSDSF module. The steps performed by other modules in the backbone network are not the focus of this application, so they are not repeated.

[0085] In some embodiments, for each single-scale fusion submodule, step A1 specifically includes:

[0086] Step A110: The electronic device performs a convolution operation on the second input feature based on a preset convolution layer to obtain a first convolution result.

[0087] Step A120: The electronic device performs depth convolution on the first convolution result based on the dilated convolution layer to obtain a second convolution result.

[0088] Step A130: The electronic device concatenates the second input feature with the second convolution result based on the concatenation layer to obtain a single-scale fusion feature corresponding to the single-scale fusion submodule.

[0089] The second input features are different for different single-scale fusion submodules. Among them, the second input feature corresponding to the single-scale fusion submodule that is in the first connection order is the first input feature, and the second input feature corresponding to the single-scale fusion submodule that is not in the first connection order is the single-scale fusion feature corresponding to the previous single-scale fusion submodule. The semantic fusion feature is obtained based on the single-scale fusion feature corresponding to the single-scale fusion submodule that is in the last connection order.

[0090] Although the second input features corresponding to each single-scale fusion sub-module are different, the operations performed on the second input features are the same. They all first perform a convolution operation to obtain the first convolution result, then perform a deep convolution operation to obtain the second convolution result, and finally concatenate the second convolution result with the second input feature to fuse the features in a single pass but at a different scale to obtain a single-scale fusion feature.

[0091] That is to say, the above steps A110-A130 are only operations performed by a single-scale fusion sub-module. No matter how many single-scale fusion sub-modules are set, each single-scale fusion sub-module can obtain the corresponding single-scale fusion feature according to the above steps A110-A130 after receiving the corresponding second input feature.

[0092] In some embodiments, the semantic fusion module includes three single-scale fusion sub-modules, and the hole rates of the hole convolution layers corresponding to the 1st to 3rd single-scale fusion sub-modules connected sequentially are 2, 4, and 8, respectively. Based on each single-scale fusion feature, step A1 may include:

[0093] Step A111: The electronic device obtains a first single-scale fusion feature based on the first single-scale fusion submodule.

[0094] Step A112: The electronic device inputs the first single-scale fusion feature into the second single-scale fusion submodule to obtain a second single-scale fusion feature.

[0095] Step A113: The electronic device inputs the second single-scale fusion feature into the third single-scale fusion submodule to obtain a third single-scale fusion feature.

[0096] Step A114: The electronic device adjusts the number of channels of the third single-scale fusion feature based on the number of channels of the first input feature to obtain a semantic fusion feature.

[0097] For example, when the semantic fusion module includes three single-scale fusion submodules, the hole rates of the hole convolution layers corresponding to the sequentially connected 1st to 3rd single-scale fusion submodules are 2, 4, and 8 respectively, and the preset convolution layer is a ×1 convolution layer, the structure diagram of the MSDSF module can refer to 4. Among them, the three single-scale fusion submodule groups can respectively extract feature maps at different scales, that is, Figure 4 The feature map in F 1. F 2 and F 3. Finally, the feature maps of these three scales are fused through 1×1 convolution to obtain the final semantic fusion feature, which is the output of the second input feature after being processed by the MSDSF module.

[0098] In order to understand how to obtain semantic fusion features, combined Figure 4 Let's illustrate this with a specific example. Assume that the first input feature is X ∈ R H×W×C , each single-scale fusion sub-module in the MSDSF module can be expressed as:

[0099]

[0100] in, Y 1~ Y 3 are the single-scale fusion features corresponding to the 1st to 3rd single-scale fusion sub-modules, O is X The semantic fusion features obtained after being processed by the MSDSF module.

[0101] Among them, for the first single-scale fusion submodule, Indicates that the convolution kernel size is 1x1 and the output channel is C / 2 convolution is used to integrate features and adjust the number of channels; It means that the convolution kernel size is 3×3, the void ratio is 2, and its output channel is C / 2 depth convolution, we get F 1; Y 1 indicates the original input X and F 1 is concatenated in the channel dimension.

[0102] For the second single-scale fusion submodule, Indicates that the convolution kernel size is 1x1 and the output channel is C / 4 convolution is used to integrate features and adjust the number of channels; it means that the convolution kernel size is 3×3, the void ratio is 4, and its output channel is C / 4 depth convolution, we get F 2; Y 2 represents the original inputY 1 and F 2 are concatenated in the channel dimension.

[0103] For the third single-scale fusion submodule, Indicates that the convolution kernel size is 1x1 and the output channel is C / 4 convolution is used to integrate features and adjust the number of channels; It means that the convolution kernel size is 3×3, the void ratio is 8, and its output channel is C / 4 depth convolution, we get F 3; Y 3 represents the original input Y 2 and F 3 are spliced ​​in the channel dimension.

[0104] For the output O, through a 1×1 convolution Y 3 is obtained by feature fusion. The MSDSF module combines contextual information of different scales and can provide richer feature representation, which helps to solve the problem of missing local semantic information of small targets. At the same time, the MSDSF module effectively balances the capture of local details and non-local background information through dilated convolutions with different void rates, improves the model's ability to understand complex scenes, and can improve the accuracy of the facility detection model in complex scenes for different abnormal urban facilities to a certain extent.

[0105] In some embodiments, in order to further improve the accuracy of the facility detection model for different abnormal urban facilities under complex backgrounds, in the feature extraction process at each scale, the ARFA submodule fuses different regions of the input features at the current scale based on different weights. Based on the weight learning mechanism, the attention level (i.e., receptive field) of different regions of the input features is dynamically adjusted, and the input features can be processed more finely and adaptively, improving the expression ability of key features, and thus, to a certain extent, improving the detection accuracy of the facility detection model for different abnormal facilities under complex and changing backgrounds, and improving the detection effect of the facility detection model on complex tasks.

[0106] In some embodiments, based on the units included in the ARFA submodule, the electronic device may perform the following operations for the third input feature input to the ARFA submodule:

[0107] Step B1: The electronic device performs convolution operations of at least three convolution scales on the third input feature through a feature extraction unit to obtain a feature map corresponding to each convolution scale.

[0108] Different convolution operations correspond to different convolution kernel sizes. To facilitate feature fusion in the scale dimension, the sizes of the feature maps obtained after each convolution operation are the same. Exemplarily, the electronic device can set the padding parameter of the convolution so that convolution operations with different convolution kernel sizes can output feature maps of the same size.

[0109] Step B2: The electronic device fuses, reshapes, and concatenates the feature maps through an adaptive receptive field attention unit to calculate the weight corresponding to each convolution scale.

[0110] To facilitate the feature fusion operation in step B3, the feature map is reshaped to have the same size as the third input feature.

[0111] Step B3: The electronic device performs a weighted sum operation on the feature maps corresponding to each convolution scale and the corresponding weights through a fusion unit to obtain a fusion feature corresponding to the third input feature.

[0112] In some embodiments, the adaptive receptive field attention unit includes a summation layer, a global average pooling layer, a global fully connected layer, a fully connected layer corresponding to each convolution scale, and a normalization layer. Step B2 specifically includes:

[0113] Step B21: The electronic device sums the feature maps based on the summation layer to obtain an initial fused feature map.

[0114] Step B22: The electronic device compresses the initial fused feature map based on the global average pooling layer, and fuses the compressed initial fused features through the global fully connected layer to obtain a global fused feature map.

[0115] Step B23: The electronic device reshapes and stacks the global fusion features through the fully connected layers corresponding to each convolution scale to obtain a stacking result.

[0116] Step B24: The electronic device performs a normalization operation on the stacking result through a normalization layer to determine the weight corresponding to each convolution scale.

[0117] For example, it is assumed that there are four convolution operations, and the convolution kernel sizes are 1×1, 3×3, 5×5, and 7×7 respectively. The network structure of the ARFA submodule can be found in Figure 5 The ARFA submodule combines multi-scale convolution, feature fusion and attention mechanism to achieve more refined and adaptive processing of input features. It emphasizes the effective integration of information between different feature scales and channels, as well as the enhancement of key features, which is conducive to improving the feature expression ability of the facility detection model.

[0118] First, the electronic device performs CBR operations on the input features based on the four convolution kernels of different sizes of the ARFA submodule, that is, a set of operations including convolution (Convolution, Conv), batch normalization (Batch Normalization, BN) and ReLU activation, to obtain feature maps of four scales; then, the electronic device sums the different feature maps in the scale dimension to obtain the initial fused feature map; the initial fused feature map is globally averaged pooled and compressed into a feature vector with the same number of channels to capture global information, and the compressed initial fused features are fused through the global fully connected layer to obtain the global fused feature map. Then, the fully connected layer corresponding to each convolution scale of the electronic device reshapes the global fused feature map to the same size as the input, and the reshaped feature vectors are stacked according to the 0th dimension (scale dimension) to obtain the stacking result, that is, to form a new tensor; the weights corresponding to each convolution scale are normalized by the softmax function so that the sum of the weights corresponding to each convolution scale is 1. Finally, the weight of each convolution scale is weighted and summed with the corresponding result after the previous convolution to obtain the fused feature corresponding to the third input feature.

[0119] It should be noted that the convolution scale here is a further refinement of any scale in the previous article, so that the effect of multi-scale fusion can be achieved at a single scale.

[0120] Assume that the third input feature is X ∈ R B×H×W×C , for each convolution kernel of different sizes k i ( i represents the convolution kernel size), performs CBR operations, and obtains feature maps of different scales F i Based on this, the corresponding operations for the four convolution kernels of different sizes can be described by the following formula:

[0121]

[0122] Furthermore, for each feature map, the sum can be performed on the scale dimension to form an initial fusion feature map F m ∈ R N ×B×H×W×C :

[0123]

[0124] Then F m Perform global average pooling (GAP) to obtain the global description vector Vglobal , through the fully connected layer FC, Fully connected at each scale FC i Realize feature integration and concatenate in the scale dimension to obtain the channel description vector V desc ∈ R N×B×1×1×C , and then normalized ( Softmax ) operation to obtain the normalized weight W s ∈ R N×B×1×1×C .

[0125]

[0126] Eventually W s Applied to the original multi-scale features for weighted summation to obtain the fusion feature corresponding to the third input feature F o .

[0127]

[0128] By adding the ARFA sub-module to the feature extraction modules of three different scales (1 / 8, 1 / 16 and 1 / 32) of the backbone network, the effect of multi-scale fusion can be achieved at a single scale by dynamically adjusting the network's attention to different areas of the input features. Through the multi-receptive field and weight learning mechanism, the facility detection model can, to a certain extent, solve the complexity of semantic understanding in abnormal urban facility detection, which helps to improve the detection effect of the facility detection model on complex tasks.

[0129] In some embodiments, based on YOLOv8s, YOLOv8s is improved by MSDSF module and ARFA submodule to obtain a facility detection model, which is recorded as MSF-YOLOv8s. The structure of MSF-YOLOv8s is as follows Figure 6 It is worth noting that the basic model here is only used as an example. In fact, other series of YOLO models can also be used as improved basic models, such as YOLOv5s, YOLOv6s, etc.

[0130] The electronic device is equipped with an Intel(R) Xeon(R) Platinum 8255C CPU, 314GB memory, and an NVIDIA Tesla V100 32GB graphics card as an accelerator, running on a CentOS 8.5.2 64-bit system platform. The MFS-YOLOv8s model was constructed using the PyTorch framework, and was programmed using PyCharm as the development environment.

[0131] MSF-YOLOv8s and YOLOv8s are trained using the ACF-training set customized by this application, and the performance of the two models is evaluated based on the ACF-validation set. Figure 7 , Figure 7 A data comparison of the key evaluation indicators of the two models is shown.

[0132] By comparing the data of evaluation indicators, we can find that:

[0133] 1. Compared with YOLOv8s, MFS-YOLOv8s has shown certain performance gains in the task of detecting anomalies in urban facilities. Specifically, MFS-YOLOv8s achieved a 2.2% increase in precision, a 0.7% increase in recall, and a 0.7% increase in mean average precision (mAP). These improvements demonstrate the superiority of MFS-YOLOv8s in capturing and distinguishing target anomalies, and further enhance the accuracy and comprehensiveness of abnormal urban facility detection.

[0134] 2. In the analysis of specific categories, the YOLOv8s network encountered bottlenecks in the detection of three types of abnormal urban facilities: the first, third, and ninth categories, which were manifested as poor detection accuracy. In contrast, the MFS-YOLO network showed stronger detection capabilities on these complex categories:

[0135] (1) Precision. Compared with YOLOv8s, the precision of MFS-YOLOv8s in the first category increased by 1%, the precision in the third category jumped by 6.3%, and the precision in the ninth category also increased by 4.7%.

[0136] (2) Recall rate. Compared with YOLOv8s, the recall rate of MFS-YOLOv8s in the first category increased by 0.8%, the recall rate in the third category increased significantly by 5.8%, and the recall rate in the ninth category also increased by 2.6%.

[0137] (3) Average Precision. In terms of average precision (mAP), compared with YOLOv8s, MFS-YOLOv8s improves the average precision of the first category by 1.3%, the average precision of the third category by 1.9%, and the average precision of the ninth category by 3.2%.

[0138] By comparing the performance of the two models, it is not difficult to see that MFS-YOLOv8s can effectively cope with the detection problems of small-sized targets, complexity in semantic understanding, and abnormal urban facilities with various sizes through the improvement of network structure, thereby achieving better detection results in difficult categories, thereby improving the robustness and generalization of the facility detection model.

[0139] To more intuitively present the difference in performance between the two models, please refer to Figure 8 ,exist Figure 8 In the figure, a, b, and c represent images containing three types of abnormal urban facilities, where a corresponds to images containing the first type of abnormal urban facilities; b corresponds to images containing the third type of abnormal urban facilities; c corresponds to images containing the fifth type of abnormal urban facilities. The three images corresponding to each letter are, from left to right, the original image, the output image of YOLOv8s based on the original image, and the output image of MFS-YOLOv8s based on the original image.

[0140] from Figure 8 It can be seen that:

[0141] (1) In a, for the first type of semantically complex detection targets, YOLOv8s made a wrong detection, while MFS-YOLOv8s detected it accurately. This shows that MFS-YOLOv8s can more accurately grasp the complex details in the detection of abnormal urban facilities through enhanced context understanding ability, thereby avoiding the impact caused by semantic confusion and improving the detection accuracy.

[0142] (2) In b, for the third type of detection targets with small size and easily confusing semantics, YOLOv8s again made false detections, while MFS-YOLO showed excellent small target distinction ability and effectively avoided false detections. This highlights the optimization of MFS-YOLO in capturing subtle target features in network design.

[0143] (3) In c, YOLOv8s missed the detection of the fifth type of abnormal urban facility target, while MFS-YOLOv8s successfully detected it. This comparison shows that the adjustment of MFS-YOLOv8s in the network architecture makes the model more balanced in handling the detection of targets of different sizes, reflecting its powerful multi-scale detection capability.

[0144] In summary, the above visualization example visualization analysis not only verifies the superior performance of MFS-YOLOv8s on specific challenging categories, but also indirectly reflects the effectiveness of the model in improving detection accuracy, enhancing small target discrimination, and handling scale change problems.

[0145] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0146] Corresponding to the abnormal facility detection method based on deep learning in the above embodiment, Fig. 9A structural block diagram of a deep learning-based abnormal facility detection device 9 provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0147] The detection device 9 is configured with a pre-trained facility detection model, which includes a backbone network, a neck network and a detection head; the backbone network is provided with a multi-scale semantic fusion module for fusing local target information and non-local background information at different scales.

[0148] Reference Fig. 9 , the abnormal facility detection device 9 based on deep learning includes:

[0149] An extraction module 91 is used to perform a feature extraction operation on the image to be processed based on the backbone network to obtain image features;

[0150] A fusion module 92, configured to perform a feature fusion operation on the image features based on the neck network to obtain fused features;

[0151] The detection module 93 is used to detect the fused features based on the detection head to obtain the detection results of abnormal urban facilities in the image to be processed.

[0152] Optionally, the backbone network includes m feature extraction modules of different scales, where m is a positive integer and m≥2; the semantic fusion module is arranged after the mth feature extraction module; the semantic fusion module includes at least 2 single-scale fusion sub-modules, and different single-scale fusion sub-modules include hole convolution layers with different hole rates; the extraction module 91 can be used to: input the first input feature of the semantic fusion module into the sequentially connected single-scale fusion sub-modules to perform a multi-scale semantic fusion operation to obtain a semantic fusion feature.

[0153] Optionally, the single-scale fusion submodule includes a preset convolution layer, a hole convolution layer corresponding to a preset hole rate, and a splicing layer; more specifically, the extraction module 91 may include an extraction submodule, which is used to:

[0154] For the second input feature input into each single-scale fusion submodule, a convolution operation is performed on the second input feature based on a preset convolution layer to obtain a first convolution result;

[0155] Performing depth convolution on the first convolution result based on the hole convolution layer to obtain a second convolution result;

[0156] The second input feature is concatenated with the second convolution result based on the concatenation layer to obtain a single-scale fusion feature corresponding to the single-scale fusion submodule;

[0157] Among them, the second input feature corresponding to the single-scale fusion submodule that is in the first place in the connection order is the first input feature, and the second input feature corresponding to the single-scale fusion submodule that is not in the first place in the connection order is the single-scale fusion feature corresponding to the previous single-scale fusion submodule; the semantic fusion feature is obtained based on the single-scale fusion feature corresponding to the single-scale fusion submodule that is in the last place in the connection order.

[0158] Optionally, the semantic fusion module includes three single-scale fusion sub-modules, and the hole rates of the hole convolution layers corresponding to the first to third single-scale fusion sub-modules connected sequentially are 2, 4, and 8 respectively; the extraction sub-module is specifically used for:

[0159] The first single-scale fusion feature is obtained based on the first single-scale fusion submodule;

[0160] Input the first single-scale fusion feature into the second single-scale fusion submodule to obtain the second single-scale fusion feature;

[0161] Input the second single-scale fusion feature into the third single-scale fusion submodule to obtain the third single-scale fusion feature;

[0162] The number of channels of the third single-scale fusion feature is adjusted based on the number of channels of the first input feature to obtain a semantic fusion feature.

[0163] Optionally, an ARFA submodule is provided in the feature extraction module corresponding to each scale, and the ARFA submodule is used to fuse different regions of the input features at a single scale based on different weights.

[0164] Optionally,

[0165] The ARFA submodule includes a feature extraction unit, an adaptive receptive field attention unit, and a fusion unit. For the third input feature input to the ARFA submodule, the following operations are performed:

[0166] Performing convolution operations of at least three convolution scales on the third input feature through the feature extraction unit to obtain a feature map corresponding to each convolution scale; different convolution operations correspond to different convolution kernel sizes, and the sizes of the feature maps corresponding to each convolution scale are the same;

[0167] The feature maps are fused, reshaped, and concatenated through adaptive receptive field attention units to calculate the weights corresponding to each convolution scale;

[0168] The fusion unit performs a weighted sum operation on the feature maps corresponding to each convolution scale and the corresponding weights to obtain a fusion feature corresponding to the third input feature.

[0169] Optionally, the adaptive receptive field attention unit includes a summation layer, a global average pooling layer, a global fully connected layer, a fully connected layer corresponding to each convolution scale, and a normalization layer. The extraction module 91 is more specifically used for:

[0170] Based on the summation layer, each feature map is summed to obtain an initial fused feature map;

[0171] Based on the global average pooling layer, the initial fusion feature map is compressed, and the compressed initial fusion features are fused through the global fully connected layer to obtain the global fusion feature map;

[0172] The global fusion features are reshaped and stacked through the fully connected layers corresponding to each convolution scale to obtain the stacked result;

[0173] The stacking result is normalized through the normalization layer to determine the weight corresponding to each convolution scale.

[0174] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0175] Fig.10 This is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present application. Fig.10 As shown, the electronic device 10 of this embodiment includes: at least one processor 100 ( Fig.10 Only one processor is shown in the figure), a memory 101, and a computer program 102 stored in the memory 101 and executable on at least one processor 100. When the processor 100 executes the computer program 102, the steps in any of the above-mentioned abnormal facility detection method embodiments based on deep learning are implemented, for example Figure 3 Steps 310-330 are shown.

[0176] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0177] In some embodiments, the memory 101 may be an internal storage unit of the electronic device 10, such as a hard disk or memory of the electronic device 10. In other embodiments, the memory 101 may also be an external storage device of the electronic device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 10.

[0178] Furthermore, the memory 101 may also include both an internal storage unit of the electronic device 10 and an external storage device. The memory 101 is used to store operating devices, applications, boot loaders, data, and other programs, such as program codes of computer programs, etc. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0179] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0180] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0181] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0183] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0184] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0185] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0186] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting abnormal facilities based on deep learning, characterized in that: The detection method is implemented based on a pre-trained facility detection model, which includes a backbone network, a neck network, and a detection head; the backbone network is provided with a multi-scale semantic fusion module, and the semantic fusion module is used to fuse local target information and non-local background information at different scales; the detection method includes: Performing feature extraction operation on the image to be processed based on the backbone network to obtain image features; Performing a feature fusion operation on the image features based on the neck network to obtain fused features; Detecting the fused features based on the detection head to obtain a detection result of abnormal urban facilities in the image to be processed; The backbone network includes m feature extraction modules of different scales, where m is a positive integer and m≥2; the semantic fusion module is arranged after the mth feature extraction module; the semantic fusion module includes at least two single-scale fusion submodules, and different single-scale fusion submodules include hole convolution layers with different hole rates; the image feature includes a semantic fusion feature, and the feature extraction operation is performed on the processed image based on the backbone network to obtain the image feature, including: Inputting the first input feature of the semantic fusion module into the sequentially connected single-scale fusion sub-modules to perform a multi-scale semantic fusion operation to obtain the semantic fusion feature; The single-scale fusion submodule includes a preset convolution layer, a hole convolution layer corresponding to a preset hole rate, and a splicing layer; For the second input feature input to each of the single-scale fusion submodules, the following steps are performed: Performing a convolution operation on the second input feature based on the preset convolution layer to obtain a first convolution result; Performing depth convolution on the first convolution result based on the hole convolution layer to obtain a second convolution result; Based on the splicing layer, the second input feature and the second convolution result are spliced ​​to obtain a single-scale fusion feature corresponding to the single-scale fusion submodule; Among them, the second input feature corresponding to the single-scale fusion sub-module that is first in the connection order is the first input feature, and the second input feature corresponding to the single-scale fusion sub-module that is not first in the connection order is the single-scale fusion feature corresponding to the previous single-scale fusion sub-module; the semantic fusion feature is obtained based on the single-scale fusion feature corresponding to the single-scale fusion sub-module that is last in the connection order.

2. The detection method according to claim 1, characterized in that The semantic fusion module includes three single-scale fusion sub-modules, and the hole rates of the hole convolution layers corresponding to the first to third single-scale fusion sub-modules connected in sequence are 2, 4, and 8 respectively; the first input feature of the semantic fusion module is input into the sequentially connected single-scale fusion sub-modules to perform a multi-scale semantic fusion operation to obtain the semantic fusion feature, including: Obtaining a first single-scale fusion feature based on the first single-scale fusion submodule; Inputting the first single-scale fusion feature into the second single-scale fusion submodule to obtain a second single-scale fusion feature; Inputting the second single-scale fusion feature into the third single-scale fusion submodule to obtain a third single-scale fusion feature; The number of channels of the third single-scale fusion feature is adjusted based on the number of channels of the first input feature to obtain the semantic fusion feature.

3. The detection method according to claim 1 or 2, characterized in that The feature extraction module corresponding to each scale is provided with an ARFA submodule, and the ARFA submodule is used to fuse different regions of input features at a single scale based on different weights.

4. The detection method according to claim 3, characterized in that The ARFA submodule includes a feature extraction unit, an adaptive receptive field attention unit, and a fusion unit; for the third input feature input to the ARFA submodule, the following operations are performed: Performing convolution operations of at least three convolution scales on the third input feature through a feature extraction unit to obtain a feature map corresponding to each convolution scale; different convolution operations correspond to different convolution kernel sizes, and the sizes of the feature maps corresponding to each convolution scale are the same; The feature maps are fused, reshaped and concatenated through the adaptive receptive field attention unit to calculate the weight corresponding to each convolution scale; A fusion unit performs a weighted sum operation on the feature map corresponding to each convolution scale and the corresponding weight to obtain a fusion feature corresponding to the third input feature.

5. The detection method according to claim 4, characterized in that: The adaptive receptive field attention unit includes a summation layer, a global average pooling layer, a global fully connected layer, a fully connected layer corresponding to each convolution scale, and a normalization layer. The adaptive receptive field attention unit fuses, reshapes, and splices the feature maps to calculate the weight corresponding to each convolution scale, including: Based on the summation layer, each of the feature maps is summed to obtain an initial fused feature map; Compressing the initial fused feature map based on the global average pooling layer, and fusing the compressed initial fused features through the global fully connected layer to obtain a global fused feature map; Reshape and stack the global fusion features through the fully connected layers corresponding to each convolution scale to obtain a stacking result; A normalization operation is performed on the stacking result through the normalization layer to determine the weight corresponding to each convolution scale.

6. A detection device for abnormal facilities based on deep learning, characterized in that: The detection device is configured with a pre-trained facility detection model, which includes a backbone network, a neck network, and a detection head; the backbone network is provided with a multi-scale semantic fusion module, and the semantic fusion module is used to fuse local target information and non-local background information at different scales; The detection device includes: An extraction module, used for performing a feature extraction operation on the image to be processed based on the backbone network to obtain image features; A fusion module, used for performing a feature fusion operation on the image features based on the neck network to obtain a fusion feature; A detection module, used for detecting the fusion feature based on the detection head to obtain a detection result of abnormal urban facilities in the image to be processed; The backbone network includes m feature extraction modules of different scales, where m is a positive integer and m≥2; the semantic fusion module is arranged after the mth feature extraction module; the semantic fusion module includes at least two single-scale fusion submodules, and different single-scale fusion submodules include hole convolution layers with different hole rates; the extraction module is used to: input the first input feature of the semantic fusion module into the sequentially connected single-scale fusion submodules to perform multi-scale semantic fusion operations to obtain semantic fusion features; The single-scale fusion submodule includes a preset convolution layer, a hole convolution layer corresponding to a preset hole rate, and a splicing layer; the extraction module includes an extraction submodule, and the extraction submodule is used to: For a second input feature input into each of the single-scale fusion submodules, a convolution operation is performed on the second input feature based on the preset convolution layer to obtain a first convolution result; Performing depth convolution on the first convolution result based on the hole convolution layer to obtain a second convolution result; Based on the splicing layer, the second input feature and the second convolution result are spliced ​​to obtain a single-scale fusion feature corresponding to the single-scale fusion submodule; Among them, the second input feature corresponding to the single-scale fusion sub-module that is first in the connection order is the first input feature, and the second input feature corresponding to the single-scale fusion sub-module that is not first in the connection order is the single-scale fusion feature corresponding to the previous single-scale fusion sub-module; the semantic fusion feature is obtained based on the single-scale fusion feature corresponding to the single-scale fusion sub-module that is last in the connection order.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the abnormal facility detection method based on deep learning as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting abnormal facilities based on deep learning as described in any one of claims 1 to 5 is implemented.

Images