Unmanned aerial vehicle-based urban management inspection method and device, electronic equipment and program product

Abstract

The application discloses an unmanned aerial vehicle (UAV)-based urban management inspection method and device, an electronic device and a program product. The method performs feature extraction, feature fusion and detection on the images collected by the UAV to generate an inspection result. During the feature extraction process, an integrated strategy of "state space-based continuous context modeling + global scale adaptive modulation + residual information reservation" is adopted to improve the identification sensitivity and robustness of low-contrast and multi-scale targets without increasing the inference delay. During the feature fusion process, multi-scale global pooling is used to explicitly model the height, width and global statistical features to capture the distribution of large-scale abnormal areas. A lightweight windowed local self-attention mechanism is introduced to mine the structure correlation and relative position information between local pixels and enhance the perception ability of scattered or occluded targets. Meanwhile, a nonlinear class weight distribution strategy based on the median reference is proposed to finely regulate the classification loss.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method for urban management inspection based on drones, an urban management inspection device based on drones, electronic equipment and computer program products. Background Technology

[0002] With the improvement of smart city construction and refined urban management, intelligent urban management based on drone remote sensing vision is gradually becoming an important technological means. By using drones to conduct high-altitude inspections of urban public spaces and combining them with deep learning algorithms to identify and locate targets such as construction waste, illegally dumped materials, domestic waste, and abnormal urban order, more efficient urban governance can be achieved.

[0003] In urban management inspection scenarios, inspection targets typically exhibit either local discreteness or overall continuity in space. This limits the feature representation capabilities of existing methods in complex scenarios, thus affecting the overall accuracy and robustness of identification. Summary of the Invention

[0004] This application provides a method, device, electronic equipment and computer program product for urban management inspection based on drones, which can simultaneously extract fine local features and uniformly model global context information across regions, thereby improving the accuracy and robustness of target identification in complex scenarios.

[0005] Firstly, this application provides a method for urban management inspection based on unmanned aerial vehicles (UAVs), including:

[0006] Feature extraction is performed on the input image to be detected to obtain image features; the image to be detected is a video frame or still image containing urban space captured by a drone.

[0007] Image features are fused to obtain target fused features;

[0008] Output the inspection results corresponding to the image to be detected based on the target fusion features;

[0009] During feature extraction, the following operations are performed:

[0010] The first input features are transformed into block-level representations, and positional information is introduced to obtain the feature sequence;

[0011] Perform continuous context modeling based on state space on the feature sequence to obtain global features;

[0012] The global features are enhanced to obtain the target global features;

[0013] The target global features are fused with the first input features to obtain the first output features corresponding to the first input features;

[0014] The first input feature is extracted from the image to be detected; the image feature is obtained from the first output feature.

[0015] Furthermore, continuous context modeling based on state space is performed on the feature sequence to obtain global features, including:

[0016] Based on the state-space model, the following operations are performed iteratively to update the feature sequence position by position, so as to determine the corresponding output feature based on the current hidden state at each position, and then fuse the output features of all positions to obtain the global feature:

[0017] For the current position, obtain the corresponding current input features and the historical hidden states passed from the previous position;

[0018] Learnable parameters adapted to the current input features are generated based on a preset selective scanning mechanism;

[0019] A structure-aware gating operation is performed based on the degree of structural change of the current input features to obtain gating weights; the gating weights are used to adjust the fusion ratio between historical hidden states and current input features;

[0020] The state update relationship is adjusted based on learnable parameters, and the historical hidden state and the current input features are weighted and fused together by gating weights to obtain the current hidden state.

[0021] Furthermore, the global features are multi-scale features; the global features are enhanced to obtain the target global features, including:

[0022] Global average pooling is performed on the global features at each scale to obtain the global statistical features corresponding to each scale;

[0023] By concatenating the global statistical features at each scale, a multi-scale aggregated feature is obtained.

[0024] A lightweight mapping operation is performed on the multi-scale aggregated features to obtain a multi-scale association representation;

[0025] The multi-scale correlation representation is adjusted based on a preset learnable temperature coefficient, and the normalized weights are obtained through normalization operations.

[0026] At each scale, the global features of the current scale are weighted based on the normalized weights corresponding to the current scale to obtain the enhanced global features of the current scale.

[0027] The enhanced global features corresponding to each scale are fused to obtain the target global features.

[0028] Furthermore, during the fusion process, the following operations are performed:

[0029] Multi-scale modeling and interaction enhancement processing are performed on the second input features to obtain the first attention representation in the width dimension and the first attention representation in the height dimension.

[0030] Local self-attention structure-aware enhancement processing is performed on the second input features to obtain a high-dimensional second attention representation and a width-dimensional second attention representation.

[0031] Based on the content-adaptive gating fusion mechanism, the first attention representation in the width dimension and the second attention representation in the width dimension are weighted and fused, and the first attention representation in the height dimension and the second attention representation in the height dimension are weighted and fused to obtain the fused attention representation in the height dimension and the fused attention representation in the width dimension.

[0032] The second input feature is enhanced based on the fusion attention representation of the height dimension and the fusion attention representation of the width dimension to obtain the second output feature corresponding to the first input feature;

[0033] The second input feature is obtained based on image features, and the target fusion feature is obtained based on the second output feature.

[0034] Furthermore, multi-scale modeling and interaction enhancement processing are performed on the second input features to obtain the width-dimension first attention representation and the height-dimension first attention representation, including:

[0035] Under any preset dimension, adaptive average pooling, channel compression, and batch normalization are performed sequentially. An activation operation is performed on the normalized context compression representation of the current dimension. The normalized context compression representation of the current dimension is weighted based on the attention weight of the current dimension. The enhanced representation of the current dimension is projected and scaled to obtain the target feature of the current dimension. The preset dimensions include width, height, and global dimensions.

[0036] The target features of all preset dimensions are fused, and the fusion result is subjected to dimension-related feature recalibration processing to obtain the first attention representation of the width dimension and the first attention representation of the height dimension.

[0037] Furthermore, when the preset dimensions are height and width, the enhanced representation of the current dimension is projected and scaled, including:

[0038] Based on the learnable projection matrix, a bidirectional cross projection is performed on the enhanced representation of the current dimension to obtain the projection result of the current dimension; the projection result is the feature semantic interaction result of the width dimension and the height dimension.

[0039] A scaling operation is performed on the projection result of the current dimension based on the global dimension to obtain the target feature of the current dimension.

[0040] Furthermore, local self-attention structure-aware enhancement processing is performed on the second input features to obtain a high-dimensional second attention representation and a width-dimensional second attention representation, including:

[0041] The second input feature is divided into multiple regular and non-overlapping local regions;

[0042] For each local region, a linear projection is performed to generate the query, key, and value of the current local region, and relative position encoding is introduced to calculate the attention response within the current local region;

[0043] The attention responses corresponding to each local region are aggregated and projected to obtain structure perception enhancement features;

[0044] The structure-aware enhancement features are sequentially subjected to average pooling, channel compression, and batch normalization to obtain batch-normalized intermediate representations. Linear activation is then performed on the batch-normalized intermediate representations, and weighting is applied to the batch-normalized intermediate representations based on the obtained attention weights. Finally, the enhanced representations are projected and scaled to obtain the target enhancement features.

[0045] Based on the target enhancement features, dimension-related feature recalibration processing is performed to obtain the second attention representation in the height dimension and the second attention representation in the width dimension.

[0046] Furthermore, the urban management inspection method is implemented based on a pre-trained urban management inspection model. The classification loss function for training the urban management inspection model is a binary cross-entropy loss based on nonlinear class weight allocation using median reference. :

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053] in, Indicates the total number of categories; Indicates the first The number of samples in the class, and ; A vector representing the number of samples in each category; Representing vectors the median; Indicates the first Class balance factor; and Let be a nonlinear mapping parameter, and satisfy . ; Indicates intermediate mapping variables; and Representing all categories The minimum and maximum values; and These represent the lower and upper bounds of the category weights, respectively. Indicates the first The weight corresponding to the class; Indicates classification loss; Indicates the total number of samples; The model predicts the number of... The probability of a class; This represents the corresponding real label; and is the numerical stability constant.

[0054] Secondly, this application provides an urban management inspection device based on unmanned aerial vehicles (UAVs), comprising:

[0055] The feature extraction structure is used to extract features from the input image to be detected, thereby obtaining image features; the image to be detected is a video frame or still image containing urban space, acquired by a drone.

[0056] Feature fusion structure is used to fuse image features to obtain target fused features;

[0057] The detection structure is used to output the inspection results corresponding to the image to be detected based on the target fusion features.

[0058] Specifically, the feature extraction structure is used for:

[0059] During feature extraction, perform the following operations:

[0060] The first input features are transformed into block-level representations, and positional information is introduced to obtain the feature sequence;

[0061] Perform continuous context modeling based on state space on the feature sequence to obtain global features;

[0062] The global features are enhanced to obtain the target global features;

[0063] The target global features are fused with the first input features to obtain the first output features corresponding to the first input features;

[0064] The first input feature is extracted from the image to be detected; the image feature is obtained from the first output feature.

[0065] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0066] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0067] Fifthly, this application provides a computer program product comprising a computer program that, when executed by one or more processors, implements the steps of the method described in the first aspect.

[0068] The advantages of this application compared to existing technologies are as follows: By performing feature extraction, feature fusion, and detection operations on images to be inspected collected by a UAV, inspection results are generated. In the feature extraction process, the first input features are first converted into block-level representations and location information is introduced to form a feature sequence, providing clear spatial structure information. Subsequently, continuous context modeling based on state space is performed on the feature sequence to achieve cross-regional contextual association modeling and enhance spatial dependency expression capabilities. Based on this, global features are enhanced to strengthen the feature representation of key regions. Finally, the enhanced global features are fused with the first input features, retaining fine local features while introducing global contextual information. This enables the model to identify both locally discrete inspection targets and globally continuous inspection targets, thereby significantly improving the accuracy and robustness of inspection target recognition in complex scenarios.

[0069] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1This is a flowchart illustrating the urban management inspection method based on unmanned aerial vehicles (UAVs) provided in an embodiment of this application.

[0072] Figure 2 This is a schematic diagram of the network structure of the SAVM-AFC module provided in the embodiments of this application;

[0073] Figure 3 This is a schematic diagram of the network structure of the TF-MSCA module provided in the embodiments of this application;

[0074] Figure 4 This is a schematic diagram of the network structure of the urban management inspection model provided in the embodiments of this application;

[0075] Figure 5 This is a schematic diagram of the structure of the urban management inspection device based on unmanned aerial vehicles provided in the embodiments of this application;

[0076] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0077] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0078] In urban management inspection scenarios, inspection targets typically exhibit local discreteness or overall continuity in space. This limits the feature representation capabilities of existing methods in complex scenarios, thereby affecting the accuracy and robustness of target recognition.

[0079] This study found that the representation ability of inspection targets in complex scenarios is limited, mainly because existing methods have difficulty simultaneously taking into account local detail characterization and global context modeling during feature extraction, resulting in low accuracy and robustness in the identification of locally discrete or globally continuous targets.

[0080] Based on this, in order to solve the identification problem, this application proposes an urban management inspection method based on UAVs. This method constructs a feature sequence by block-level feature representation and location information encoding, combines continuous context modeling of state space to achieve cross-regional context association modeling, and then enhances the global features and fuses them with the input features. This not only preserves fine local features but also models the global context in a unified manner, so as to achieve high-precision and high-robustness identification of inspection targets in complex scenarios.

[0081] The urban management inspection method based on drones provided in this application can be applied to electronic devices such as drones, mobile phones, tablets, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and edge computing devices. This application does not impose any restrictions on the specific type of electronic device.

[0082] To illustrate the technical solutions proposed in this application, the following description will use an electronic device as the execution subject to illustrate various embodiments.

[0083] Figure 1 A schematic flowchart of the UAV-based urban management inspection method provided in this application is shown. The UAV-based urban management inspection method includes:

[0084] Step 110: The electronic device extracts features from the input image to be detected to obtain the image feature image to be detected.

[0085] Step 120: The electronic device performs fusion processing on the image features to obtain the target fused features.

[0086] Step 130: The electronic device outputs the inspection results corresponding to the image to be detected based on the target fusion features.

[0087] The image to be inspected is a video frame or still image containing urban space captured by a drone. The electronic device first extracts features from the image to obtain image features; then it performs fusion processing on the image features to generate target fusion features; finally, it outputs the inspection results based on the target fusion features, thereby determining whether there are inspection targets in the image and labeling their category and location information.

[0088] Considering that inspection targets typically possess either local discreteness or overall continuity, in order to accurately identify these targets, the electronic device also performs the following operations during feature extraction:

[0089] Step A1: The electronic device converts the first input feature into a block-level representation and introduces position information to obtain a feature sequence.

[0090] To effectively capture the overall morphological features of the inspection target and the long-range semantic dependencies between different regions through subsequent state modeling processes, the first input features can be converted into a block-level representation. Specifically, the electronic device can divide the image to be detected into multiple image patches using patch embedding, and map each image patch to a corresponding feature vector, thereby converting the original two-dimensional image data into a serialized representation composed of multiple image patch features. In this way, the image can be divided into multiple basic feature units containing local semantic information, enabling the subsequent state modeling process to uniformly model the semantic relationships between different image regions at a higher level. This is more conducive to capturing the overall structural features and long-range dependencies of the inspection target in the image.

[0091] However, after dividing the first input features into multiple image patches and constructing a feature sequence, the two-dimensional spatial adjacency relationship between adjacent regions in the original first input features is no longer explicitly expressed in the sequence representation. This may lead to the model struggling to directly perceive the spatial distribution relationship between different image patches and the continuity of local structures when performing sequence modeling. Especially for structures like inspection targets that are elongated and directionally continuous, relying solely on sequence features may be insufficient to fully characterize their spatial extension morphology. Therefore, after obtaining the block-level feature representation, the electronic device can introduce positional information encoding into the feature vector corresponding to each image patch to represent the spatial positional relationship of each image patch in the original image. This allows the feature sequence to simultaneously include both semantic and spatial positional information of the image patches during subsequent state modeling.

[0092] Based on the introduced location information encoding, when modeling the state of feature sequences, the model can not only learn the semantic relationships between different image patches, but also perceive the spatial distribution of each image patch in the image, thereby establishing a more reasonable semantic relationship structure globally. In this way, the spatial extension shape and overall distribution characteristics of the inspection target in the image can be captured more effectively, thus obtaining a feature representation containing global semantic information. This provides a more sufficient feature foundation for the subsequent generation of crack segmentation results, thereby improving the accuracy and stability of crack segmentation.

[0093] Step A2: The electronic device performs continuous context modeling based on state space on the feature sequence to obtain global features.

[0094] State modeling can be achieved using a State Space Model (SSM). A State Space Model is a modeling approach used to describe the dynamic evolution of sequential data. It constructs latent state variables and recursively updates these latent states based on the input sequence, thereby gradually aggregating historical information along the sequence dimension. This allows the model to establish dependencies between features over a longer period. In this way, long-range dependencies between image patch features can be effectively modeled, leading to a global semantic understanding of the overall image structure and obtaining global features.

[0095] For example, a sequence modeling structure based on Visual Mamba can be used to model the state of feature sequences. Visual Mamba introduces a selective state space mechanism to dynamically update the state of the input feature sequences, enabling the model to effectively capture long-distance dependencies while maintaining low computational complexity. During state modeling of the feature sequences, the feature vectors corresponding to each image patch can be sequentially input into the state space model. Through the state update process, sequence context information is gradually fused to obtain a feature representation containing global semantic information. This state modeling process enhances the model's ability to express the overall morphological structure of the inspected target and cross-regional correlation features, thus providing more comprehensive semantic feature support for subsequent crack segmentation.

[0096] Step A3: The electronic device enhances the global features to obtain the target global features.

[0097] To enhance the response to key contextual information and suppress interference from redundant background information, electronic devices can perform enhancement processing on global features. This is achieved by enhancing features related to the inspection target and suppressing irrelevant background features, thereby obtaining the target's global features. The enhancement processing includes, but is not limited to, weighted modulation based on multi-scale information, attention enhancement based on channel or spatial dimensions, and feature recalibration based on normalized weights.

[0098] Step A4: The electronic device fuses the target global features with the first input features to obtain the first output features corresponding to the first input features.

[0099] Finally, the first input feature is fused with the target global feature in a residual manner, that is, the element-wise addition operation is performed to obtain the first output feature. This way, while introducing global context information, the original fine-grained structural information is preserved, and its attenuation during deep modeling is avoided.

[0100] In this embodiment, the electronic device generates inspection results by performing feature extraction, feature fusion, and detection operations on the images to be inspected collected by the UAV. During feature extraction, the first input feature is first converted into a block-level representation and location information is introduced to form a feature sequence, providing spatial structure information for subsequent modeling. Then, continuous context modeling based on state space is performed on the feature sequence to achieve cross-regional contextual association modeling and enhance spatial dependency expression capabilities. The obtained global features are enhanced to strengthen key region features. Finally, the enhanced global features are fused with the first input feature, retaining both fine local features for identifying locally discrete inspection targets, such as scattered household waste or gravel, and incorporating global context information to determine overall continuous inspection targets, such as large piles of slag or continuous material stacks, thereby significantly improving the accuracy and robustness of inspection target identification in complex scenarios.

[0101] In some embodiments, to enable accurate modeling, the electronic device, based on a state-space model, iteratively updates the feature sequence position by position to determine the corresponding output feature based on the current hidden state at each position, and fuses the output features from all positions to obtain the global feature:

[0102] Step A21: For the current position, the electronic device acquires the corresponding current input features and the historical hidden states passed from the previous position.

[0103] For the current position, the electronic device can acquire the current input features and the historical hidden states passed from previous positions. The historical hidden states are used to represent the contextual information of the processed positions. The electronic device associates the local information of the current position with the contextual information accumulated from previous positions, which can provide a basis for subsequent state updates.

[0104] Step A22: The electronic device generates learnable parameters that are adapted to the current input features based on a preset selective scanning mechanism.

[0105] Selective scanning mechanisms can be two-dimensional, such as the SS2D scanning mechanism, used to adaptively determine the state propagation path and propagation intensity based on input features. Under the action of selective scanning mechanisms, corresponding learnable parameters are generated for the current input features, enabling the model to dynamically adjust the state update method according to the current feature content, thereby improving its adaptability to different spatial structures.

[0106] Step A23: The electronic device performs a structure-aware gating operation based on the degree of structural change of the current input features to obtain the gating weights.

[0107] The degree of structural change can be characterized by local feature differences or continuity. Electronic devices perform structure-aware gating based on the degree of structural change of the current input features to determine gating weights used to adjust the fusion ratio between historical hidden states and current input features. By adjusting the gating weights, the influence of the current input features in state updates can be enhanced when spatial structure changes are drastic or the target scale is small; when the spatial structure is relatively continuous or the regional scale is large, the cumulative effect of historical hidden states can be enhanced. This enables adaptive modeling of different spatial structures and achieves unified representation of multi-scale structural information within the same modeling framework.

[0108] Step A24: The electronic device adjusts the state update relationship based on learnable parameters, and combines the gating weights to perform weighted fusion of historical hidden states and current input features to obtain the current hidden state.

[0109] Electronic devices adjust state update relationships based on learnable parameters to adaptively control the propagation mode and update intensity of historical hidden states in the spatial dimension. Simultaneously, they combine gating weights to weightedly fuse historical hidden states and current input features, dynamically allocating their contribution ratios in state updates to obtain the current hidden state. In this update process, the learned parameters characterize the dynamic properties of state propagation, while the gating weights adjust the weight allocation for information fusion. These two elements work synergistically in the same update process, enabling unified modeling and adaptive balance of local and contextual information.

[0110] In this embodiment, the electronic device combines the continuous accumulation of historical hidden states with the dynamic modulation of current input features through the above-described position-by-position recursive update process. It also incorporates a selective scanning mechanism and a structure-aware gating mechanism to adaptively adjust the state update process. This allows the model to fully utilize the contextual information passed from previous positions to achieve global modeling across regions within the same modeling framework, while also preserving key local details based on current structural changes. This effectively enhances feature representation capabilities, enabling a unified characterization of locally discrete and globally continuous targets in complex scenes, thereby improving the accuracy and robustness of inspection target recognition.

[0111] In some embodiments, the global features are multi-scale features. In order to maintain key local information while sustaining long-range information transmission, the electronic device may perform the following operations:

[0112] Step A31: The electronic device performs global average pooling on the global features at each scale to obtain the global statistical features corresponding to each scale.

[0113] Step A32: The electronic device splices together the global statistical features at each scale to obtain multi-scale aggregated features.

[0114] Global features typically comprise feature representations from different scales. Global features at different scales can be obtained step-by-step through the same or similar processing methods. For example, global features from a previous scale can be used as input for global features at a next scale, and the resulting global features can be obtained. Furthermore, to achieve effective fusion of global features at different scales, adaptive upsampling operations can be performed on global features at each scale to unify the spatial dimensions of global features at each scale.

[0115] Building upon this foundation, to preserve key local information while maintaining long-range information transmission, global average pooling is first performed on the global features at each scale. This compresses spatial information into compact statistical representations, thereby extracting global semantic information corresponding to each scale, such as reflecting large-scale regional distribution characteristics and fine-grained local response intensity. Subsequently, the global statistical features at each scale are concatenated to form a unified multi-scale aggregated feature, enabling the alignment and integration of information from different scales within the same feature space, thus providing a foundation for subsequent cross-scale relationship modeling.

[0116] Step A33: The electronic device performs a lightweight mapping operation on the multi-scale aggregated features to obtain a multi-scale association representation.

[0117] Step A34: The electronic device adjusts the multi-scale correlation representation based on a preset learnable temperature coefficient and obtains the normalized weights through a normalization operation.

[0118] Lightweight mapping operations, such as linear transformations based on channel-wise mapping or lightweight MLP (two 1×1 convolutions + ReLU), are performed on multi-scale aggregated features to obtain multi-scale correlation representations. This explicitly models the correlations between features at different scales, enabling the model to perceive the complementary relationships between information at each scale. Furthermore, to enhance the sharpness of the attention distribution and improve the selectivity for key branches, a pre-defined learnable temperature coefficient is introduced to adjust the multi-scale correlation representations. Normalized weights are generated through normalization operations, where different scales correspond to different weights to characterize the differences in importance of features at each scale under the current input conditions.

[0119] Step A35: At each scale, the electronic device assigns weights to the global features of the current scale based on the normalized weights corresponding to the current scale, thereby obtaining the enhanced global features of the current scale.

[0120] Step A36: The electronic device fuses the enhanced global features corresponding to each scale to obtain the target global features.

[0121] Based on the aforementioned normalized weights, adaptive weighting is applied to the global features at each scale, strengthening scale features that contribute more significantly while suppressing redundant or interfering scale features, thus obtaining enhanced global features for each scale. Finally, the enhanced global features at each scale are summed element-wise to obtain the target global features, which retain the discriminative ability of each scale while fusing multi-scale information.

[0122] In this embodiment, the electronic device maintains the effective transmission of long-range semantic information through global statistics and cross-scale correlation modeling. On the other hand, it strengthens different scales by normalizing weights to avoid weakening key local information during the fusion process. Thus, it takes into account both global consistency and local sensitivity in the same feature representation. The combination of the two aspects can realize the adaptive modeling and dynamic adjustment of multi-scale features, thereby improving the representation ability of inspection targets in complex scenarios and the accuracy and robustness of identification.

[0123] In the scenario of intelligent drone inspection for refined urban management, the inspected targets often exhibit characteristics such as low contrast, irregular shape, and complex distribution due to the influence of multiple heterogeneous factors such as surface texture interference, dynamic occlusion, lighting changes, and reflections. These targets are easily submerged by the background or confused with normal scenes, leading to a decrease in the model's discrimination ability and robustness. At the same time, existing methods have limitations in feature modeling: traditional convolutional networks are difficult to adaptively focus on key regions, common attention mechanisms do not adequately model spatial information or have coarse granularity, and lack effective characterization of spatial structural interactions, making it difficult to accurately locate complexly distributed targets, thus further limiting the recognition performance in complex scenes.

[0124] In some embodiments, to address the above-mentioned problems, the electronic device may perform the following operations during the feature fusion process:

[0125] In step B1, the electronic device performs multi-scale modeling and interaction enhancement processing on the second input features to obtain the first attention representation in the width dimension and the first attention representation in the height dimension.

[0126] When performing multi-scale modeling and interactive enhancement on the second input features, contextual information can be extracted along both the width and height dimensions, and cross-directional interactive modeling can be introduced, enabling the features to perceive long-range dependencies in both spatial directions. For example, the width dimension can reflect the horizontally continuous distribution of regional structures, while the height dimension can characterize the vertically extended spatial relationships. Through the collaborative modeling of both, a preliminary characterization of the overall outline and distribution pattern of the target can be formed, thus obtaining the first attention representations in the width and height dimensions, which are used to highlight large-scale structural information.

[0127] In step B2, the electronic device performs local self-attention structure perception enhancement processing on the second input features to obtain a high-dimensional second attention representation and a width-dimensional second attention representation.

[0128] Building upon this, the electronic device performs local self-attention structure-aware enhancement processing on the second input features. By establishing correlations between pixels within a local window and introducing relative positional information to characterize neighborhood structural relationships, the model is able to capture fine-grained local differences. For example, for edge regions, partially occluded areas, or regions with abrupt texture changes, their responses can be enhanced through local self-attention, thereby forming second attention representations in the width and height dimensions, respectively, to supplement fine structural information.

[0129] Step B3: The electronic device performs weighted fusion of the first attention representation in the width dimension and the second attention representation in the width dimension based on the content adaptive gating fusion mechanism, and performs weighted fusion of the first attention representation in the height dimension and the second attention representation in the height dimension to obtain the fused attention representation in the height dimension and the fused attention representation in the width dimension.

[0130] Subsequently, a content-adaptive gating fusion mechanism is introduced to perform weighted fusion of two types of attention representations on the same dimension. The gating weights are dynamically generated based on the current feature content and are used to adjust the contribution ratio between multi-scale modeling results and local structure enhancement results. Through this fusion method, the role of global modeling results can be enhanced in structurally continuous regions, while the influence of local self-attention is increased in regions with significant structural changes or rich details, thus obtaining fused attention representations in both width and height dimensions.

[0131] Step B4: The electronic device enhances the second input feature based on the fusion attention representation of the height dimension and the fusion attention representation of the width dimension to obtain the second output feature corresponding to the first input feature.

[0132] Finally, the second input feature is modulated using the fused width and height dimension attention representations. By applying element-wise attention, key region responses are highlighted and redundant background information is suppressed, resulting in an enhanced second output feature, which is then used to generate the subsequent target fusion feature. The second input feature is derived from image features, and the target fusion feature is derived from the second output feature.

[0133] In this embodiment, the electronic device achieves a unified expression of global structural information and local detailed information within the same framework through the synergistic effect of multi-scale modeling and local structural perception, combined with a content-adaptive gating fusion mechanism. This enables the features to not only perceive large-scale continuous areas but also accurately depict fine-grained structural changes, thereby effectively improving the ability to distinguish target features in complex scenarios and thus enhancing the accuracy and robustness of inspection target identification.

[0134] In some embodiments, multi-scale modeling and interaction enhancement processing is performed on the second input features to obtain a first attention representation in the width dimension and a first attention representation in the height dimension, including:

[0135] Step B11: Under any preset dimension, the electronic device sequentially performs adaptive average pooling, channel compression, and batch normalization operations. It then performs an activation operation on the normalized context compression representation of the current dimension, assigns weights to the normalized context compression representation of the current dimension based on the attention weights of the current dimension, and projects and scales the enhanced representation of the current dimension to obtain the target feature of the current dimension.

[0136] When processing any preset dimension, the input features are first compressed along that dimension using adaptive average pooling. This allows spatial information to converge into a compact statistical representation across the height, width, or global dimensions, thereby highlighting the overall response of the target region and suppressing redundant information. For example, pooling in the width dimension can summarize the lateral structural distribution, making it easier to capture the continuous features of stacked or long strip materials; pooling in the height dimension can reflect vertical stacking or changes in stack height; and global dimension pooling can extract the overall semantic distribution. The representations after pooling in these three dimensions complement each other in terms of information emphasis, encoding row-level, column-level, and global semantics respectively, thereby strengthening the local lateral, vertical, and overall global responses and providing complementary perspectives for subsequent fusion.

[0137] Subsequently, channel compression reduces the number of pooled representation channels to the latent space dimension, and batch normalization stabilizes the feature distribution. Then, a non-linear mapping using an activation function amplifies key features, enhancing the responsiveness to salient regions. Next, the normalized context-compressed representation is weighted according to the attention weights of the current dimension, selectively emphasizing important regions while suppressing background interference. After weighting, projection and scaling operations map the enhanced features back to the original dimension, forming the target features for that dimension and achieving a coordinated expression of local details and cross-regional context.

[0138] Step B12: The electronic device fuses all target features of preset dimensions and performs dimension-related feature recalibration processing on the fusion result to obtain the first attention representation of the width dimension and the first attention representation of the height dimension.

[0139] The target features obtained from each preset dimension (width, height, and global dimension) are fused to allow horizontal, vertical, and global contextual information to fully interact, complement, and integrate within the same feature space, thereby strengthening key information. The fused features are then subjected to dimension-related recalibration processing. By adjusting the contribution ratio of different dimensions, the response of key structures is strengthened and the feature scale is unified, thereby generating first attention representations for the width dimension and the height dimension, providing a foundation for subsequent multi-scale interaction and fusion.

[0140] In this embodiment, the electronic device enhances local features and integrates global context through cross-dimensional adaptive pooling, compression, activation and weighting operations. Multi-dimensional fusion and recalibration ensure the collaborative expression of each spatial direction, enabling the model to capture local details and global structural information simultaneously in complex scenarios, thereby significantly improving the accuracy and robustness of inspection target identification.

[0141] In some embodiments, the second input feature is enhanced based on the fusion attention representation in the height dimension and the fusion attention representation in the width dimension to obtain the second output feature corresponding to the first input feature, including:

[0142] Step B41: The electronic device divides the second input feature into multiple regular and non-overlapping local regions.

[0143] The electronic device first divides the second input feature into multiple regular and non-overlapping local regions, such as non-overlapping 3×3, 5×5, or 7×7 windows. This division method helps the model capture local anomalies (such as occlusion) or irregularly distributed target information at a fine-grained scale, such as gravel piled up in a corner, pipes placed at an angle, or scattered household waste.

[0144] Step B42: The electronic device performs a linear projection on each local region, generates the query, key and value of the current local region, and introduces relative position encoding to calculate the attention response in the current local region.

[0145] Subsequently, a linear projection is performed on each local region to generate queries, keys, and values, and a learnable relative position encoding is introduced. This allows attention computation to consider not only the content relationships between pixels but also spatial location information, thereby enhancing the ability to perceive local structure. For example, within a windowed 7×7 local region, the model can accurately distinguish the differences between a shadowed garbage heap and the surrounding ground texture.

[0146] Step B43: The electronic device aggregates and projects the attention responses corresponding to each local region to obtain structure perception enhancement features.

[0147] Next, the attention responses of each local region are aggregated and projected to form an overall structure perception enhancement feature, enabling the model to retain local fine information while integrating the contextual relationships between regions.

[0148] Step B44: The electronic device sequentially performs average pooling, channel compression, and batch normalization operations on the structure-aware enhancement features to obtain batch-normalized intermediate representations; performs linear activation operations on the batch-normalized intermediate representations, performs weighting operations on the batch-normalized intermediate representations based on the obtained attention weights, and projects and scales the obtained enhancement representations to obtain target enhancement features.

[0149] After obtaining the structure-aware enhanced features, spatial information can be further compressed using average pooling. This allows significant responses within local regions to be aggregated into more compact statistical representations, thereby reducing noise interference and highlighting key areas. Subsequently, channel compression reduces redundant channel information, and batch normalization stabilizes the feature distribution, making subsequent attention calculations more stable. After linear activation, corresponding attention weights are obtained, and these weights are used to assign weights to the batch-normalized intermediate representations, enhancing responses relevant to the target region while suppressing responses from background or irrelevant regions. Finally, through projection and scale adjustment, the enhanced features are mapped to the same dimensional range as the original features, resulting in target enhanced features that balance local detail representation with overall structural consistency.

[0150] Step B45: The electronic device performs dimension-related feature recalibration processing based on the target enhancement features to obtain the second attention representation in the height dimension and the second attention representation in the width dimension.

[0151] Finally, based on the enhanced features of this target, dimension-related recalibration is performed, enabling the model to learn the importance distribution in different directions along both the height and width dimensions. For example, for horizontally extended, obstructing stacked targets, the response in the width direction can be enhanced; for vertically stacked materials or vertically occluded areas, the response in the height direction can be enhanced. This yields second attention representations in the height and width dimensions, providing a foundation for subsequent attention fusion in different directions.

[0152] In this embodiment, the electronic device captures local salient structures and local contextual information to enhance its perception of scattered and irregularly distributed targets, while maintaining spatial consistency of features in the height and width dimensions, providing a reliable foundation for accurately locating and identifying inspection targets in complex scenarios.

[0153] In some embodiments, when the preset dimensions are height and width, projecting and scaling the input representation may include:

[0154] Step B111: The electronic device performs bidirectional cross-projection on the enhanced representation of the current dimension based on the learnable projection matrix to obtain the projection result of the current dimension; the projection result is the feature semantic interaction result of the width dimension and the height dimension.

[0155] Step B112: The electronic device performs a scaling operation on the projection result of the current dimension based on the global dimension to obtain the target features of the current dimension.

[0156] By performing bidirectional cross-projection on the input representations in either the height or width dimensions using a learnable projection matrix, semantic interaction between horizontal and vertical features is achieved, thereby capturing key target structures that are not axially aligned or obliquely distributed. Subsequently, the projection results are scaled based on the global dimension to align the input representations with the original feature space, ensuring complete spatial mapping of information. Simultaneously, the global dimension itself does not need to participate in the interaction or expansion, avoiding redundant computation. By introducing bidirectional cross-projection to achieve semantic interaction between horizontal and vertical features, cross-directional local structure awareness can be enhanced while maintaining global consistency, providing a precise and efficient feature foundation for subsequent attention fusion. The input representation can be an enhanced representation of the current dimension or an enhanced representation obtained based on structure-aware enhanced feature processing.

[0157] In some embodiments, dimension-dependent feature recalibration processing operations may include a combination of nonlinear activation operations, mean operations, and channel recovery operations.

[0158] Among these methods, nonlinear activation enhances feature representation, making important responses across different dimensions more prominent; averaging provides an overall statistical analysis of responses in the current dimension, reducing interference from local noise or outliers; and channel restoration remaps compressed features back to the original channel space, ensuring the recalibrated features retain their complete semantic expressive power. This combination of operations highlights salient features in key directions while maintaining the stability and integrity of feature distribution, thereby improving the accuracy of attention representations in both height and width dimensions.

[0159] In some embodiments, the above-described urban management inspection method can be implemented using a pre-trained deep learning detection model. This detection model may include a backbone network for extracting features from the input image to be inspected, a neck network for performing feature fusion on the extracted image features, and a detection head for detecting the fused target features, thereby outputting corresponding inspection results. The backbone network includes a Structure-Aware Visual Mamba (SAVM)-based Adaptive Context Fusion (SAVM-AFC) module. The SAVM-AFC module may include an encoding submodule, a modeling submodule, a feature enhancement submodule, and a fusion submodule, each submodule being used for:

[0160] The encoding submodule is used to perform block encoding and positional encoding on the corresponding first input features to obtain the feature sequence.

[0161] The modeling submodule is used to perform continuous context modeling based on state space on the feature sequence to obtain global features.

[0162] The feature enhancement submodule is used to enhance global features to obtain the target global features.

[0163] The fusion submodule is used to fuse the target global features with the first input features to obtain the corresponding first output features.

[0164] In this embodiment, the SAVM-AFC module is incorporated into the backbone network of the detection model, enabling the model to balance local detail preservation with global context modeling. Specifically, the encoding submodule generates feature sequences through block encoding and positional encoding, providing spatial structure information for subsequent modeling; the modeling submodule performs continuous context modeling on the feature sequences based on a state-space model to establish long-range dependencies across regions; the feature enhancement submodule enhances global features, strengthening responses in key regions and suppressing background interference; and the fusion submodule fuses the target's global features with the first input features, thereby introducing global context information while preserving local details. This improves the accuracy and robustness of target recognition in complex scenarios.

[0165] In some embodiments, the modeling submodule introduces an adaptive scaling mechanism based on global context to jointly model the multi-level state space features obtained by structure-aware visual Mamba modeling, so as to obtain global features.

[0166] Assume the first input feature is The encoding module first divides the two-dimensional spatial features into several local perceptual units using Patch Embedding and maps them to a unified embedding space to enhance the expression of local structures and reduce redundant information. Then, Position Embedding introduces positional encoding information to explicitly inject spatial location information, enabling the model to distinguish the relative relationships between different spatial locations during subsequent sequence modeling. After embedding and positional encoding are completed, the features are rearranged into a one-dimensional sequence and input into the modeling submodule for contextual modeling.

[0167] The modeling submodule can be built based on the Selective State-Space Model (SAVM). Its core lies in continuously modeling the spatial sequence through hidden state recursion and dynamically adjusting the state update behavior according to the structural complexity of the input features. For the t-th position in the sequence, its state update process can be represented as:

[0168]

[0169] in, Given the input sequence, Controlling the propagation of hidden states across spatial dimensions, The input shortcut connection matrix, Indicates the first The hidden state at each time step; and Generated by the SS2D selective scanning mechanism, corresponding to spatial dimensions With the time dimension All of these are learnable parameters. Output This is the update result for the current time step.

[0170] To enhance the model's ability to perceive differences in spatial structure, the SAVM module introduces a structure-aware gating mechanism. This mechanism adaptively adjusts the fusion ratio between historical states and the current input based on the continuity and degree of change of local features. When spatial structure changes drastically or the target scale is small, the model tends to strengthen the current local input; when the spatial structure is continuous and the regional scale is large, it gradually enhances the accumulation of long-range context states, thereby achieving a unified representation of multi-scale structural information within the same modeling framework.

[0171] For example, multi-level SAVM modules can be constructed, and multi-scale contextual feature representations can be formed through different state evolution depths and modeling spans. Specifically, the input features pass through multiple SAVM blocks sequentially. Each layer performs state space modeling on the spatial sequence, and after modeling, it restores the spatial resolution to the same level as the input through an upsampling operation, thereby forming four-way multi-scale global features. The global features at each scale gradually expand from local to global in terms of perception range, which can cover the structural features of targets inspected at different scales.

[0172] In some embodiments, for the multi-level state space features obtained by structurally-aware visual Mamba modeling, i.e. global features corresponding to each scale, under the action of the feature enhancement submodule, the fusion weights corresponding to each scale can be generated first through a lightweight MLP, so that the model can adaptively enhance key context responses and suppress redundant background information according to the spatial structural complexity.

[0173] Specifically, see Figure 2 , Figure 2 A schematic diagram of the network structure of an SAVM-AFC module is shown. In order to identify which scales of global features are more discriminative, the feature enhancement submodule introduces a dynamic gating mechanism based on global context.

[0174] Using the four-scale global feature example, the feature enhancement submodule will perform global average pooling (GAP) on the global features at each scale to obtain the global statistical features at each scale:

[0175]

[0176] Global statistical features at each scale are concatenated into a joint context representation to obtain multi-scale aggregated features:

[0177]

[0178] Subsequently, it is compressed and mapped to branch importance logits using a lightweight MLP (two layers of 1×1 convolution + ReLU) to obtain a multi-scale association representation:

[0179]

[0180] To enhance the sharpness of attention distribution and improve selectivity for key branches, a learnable temperature coefficient is introduced. Calculate the normalized weights:

[0181]

[0182] Among them, temperature Through backpropagation and automatic learning, the initial level approaches 1 (soft attention) and later approaches 0 (approximately hard selection), achieving an adaptive transition from exploration to utilization.

[0183] Finally, the spatial dimension is recovered for feature weighting:

[0184]

[0185] The four features are fused using the aforementioned weights with channel alignment. Specifically, at each scale, the global features at that scale are weighted based on the normalized weights corresponding to that scale, and the weighting results at each scale are fused to obtain the target global features.

[0186]

[0187] Finally, the target global features are compared with the original input. The input features are summed and then passed through a 1×1 projection layer to restore channel consistency and enhance nonlinear expressive power, resulting in the first input feature:

[0188]

[0189] In some embodiments, the neck network includes a Transformer-Fused Multi-Scale Cross-Attention (TF-MSCA) module. The TF-MSCA module comprises two branches: one branch performs multi-scale modeling and interaction enhancement processing on the second input feature to obtain a width-dimensional first attention representation and a height-dimensional first attention representation; the other branch performs local self-attention structure-aware enhancement processing on the second input feature to obtain a height-dimensional second attention representation and a width-dimensional second attention representation. Following these two branches, a fusion branch is provided to perform weighted fusion of the width-dimensional first attention representation and the width-dimensional second attention representation based on a content-adaptive gating fusion mechanism, and to perform weighted fusion of the height-dimensional first attention representation and the height-dimensional second attention representation to obtain a height-dimensional fused attention representation and a width-dimensional fused attention representation. Based on the height-dimensional fused attention representation and the width-dimensional fused attention representation, the second input feature is enhanced to obtain the second output feature corresponding to the first input feature.

[0190] The TF-MSCA module constructs a dual-path collaborative perception architecture. On one hand, it uses multi-scale global pooling to explicitly model features of different dimensions to capture the overall distribution of large-scale anomaly regions. On the other hand, it uses a local self-attention mechanism to mine structural relationships and relative positional information between local pixels, enhancing the ability to perceive details of scattered and occluded targets. More importantly, through a learnable dynamic gating fusion strategy, it adaptively weights the attention responses generated by the two paths according to the input content, achieving an organic unity between global semantic guidance and local structural enhancement.

[0191] In some embodiments, in the first branch, the height dimension, width dimension, and global dimension can be explicitly modeled.

[0192] For example, see Figure 3 The multi-scale branch diagram illustrates the network structure of the first branch. Based on this network structure diagram, for the second input feature... We can first perform adaptive average pooling along the height, width, and global dimensions respectively to generate three complementary context representations:

[0193] , ,

[0194] in, , , Separately encode row-level, column-level, and global semantics.

[0195] To reduce computational overhead, a shared 1×1 convolution is used to compress the number of channels to the latent space dimension. D = C / r ( r (for compression ratio), and apply batch normalization and activation operations (e.g., activation via hard-swish):

[0196] , ,

[0197] in, It is a learnable convolutional kernel. This represents the hard-swish activation function.

[0198] Then projection and scaling operations can be performed to obtain the target features in each dimension.

[0199] Optionally, to enable full semantic interaction between horizontal and vertical features, when the preset dimensions are height and width, bidirectional cross-projection can be used.

[0200]

[0201]

[0202] in, Learnable projection matrix. Broadcast mechanism will be used to... and Expand to the full image size and integrate global context (target features across all dimensions):

[0203]

[0204] For the fusion result, dimension-related feature recalibration processing can be performed, such as obtaining multi-scale interactive features through nonlinear activation, and performing a mean operation on the multi-scale interactive features. Finally, the number of channels of the mean operation result is restored through convolution operation to obtain the corresponding width dimension first attention representation and height dimension first attention representation.

[0205] In some embodiments, in the second branch, the self-attention mechanism can be a lightweight windowed local self-attention mechanism.

[0206] For example, see Figure 3 The self-attention branch in the diagram illustrates the network structure of the second branch. Based on this network structure diagram, for the second input feature, it can first be divided into non-overlapping 7×7 local regions, and within each local region, a query (Q), key (K), and value (V) are generated through linear projection; subsequently, a learnable relative position encoding is introduced to enhance spatial awareness, and the attention response of the scaled dot product within the local region is calculated;

[0207]

[0208] in, Queries generated within a 7x7 local area, key and value. ; For learnable relative position encoding; This indicates that the localized output will be recombined into a complete feature map. After that, there is a convolutional projection layer with C output channels.

[0209] Finally, through attention operations, aggregation operations, and projection operations at output, the structure-aware enhanced features are obtained. By performing the same operations as in the first branch on the structure-aware features—namely, average pooling, channel compression, batch normalization, linear activation, weighting, projection, scaling, mean, and channel restoration—a second attention representation with both a high-dimensional and a wide-dimensional dimension is obtained.

[0210] In some embodiments, see Figure 3 For the outputs of the two branches, content adaptive fusion gating can be used:

[0211]

[0212] in, This gating is used to perform weighted fusion of the attention maps of the two paths:

[0213]

[0214]

[0215] Finally, the original features are recalibrated using the fused coordinate attention map:

[0216]

[0217] in, This represents element-wise multiplication (Hadamard product).

[0218] Figure 3 The network structure design of the TF-MSCA module shown can simultaneously utilize global statistical regularities and local structural details: multi-scale paths effectively capture the overall outline and spatial distribution of large-scale material accumulation, while local self-attention paths accurately activate scattered targets that are occluded, have low contrast, or are irregularly shaped; the dynamic gating mechanism further ensures adaptive selection of the optimal perception strategy under different scenarios. Experiments show that TF-MSCA can significantly improve the localization accuracy and robustness of diverse urban anomalies in complex urban scenarios, while maintaining lightweight computational overhead, making it suitable for real-time deployment on edge drone platforms.

[0219] In some embodiments, given the detection advantages of the YOLO series models, deep learning detection models can be derived by improving upon the YOLO series models. YOLOv8s is preferred.

[0220] For example, if the YOLOv8s-based network is improved by using the SAVM-AFC and TF-MSCA modules, the network structure of the detection model can be found in [reference needed]. Figure 4 .

[0221] based on Figure 4 The detection model, for the input image to be detected, uses a backbone network to perform multi-scale feature extraction on the image through sequentially connected GConv-4×[GConv+SAVM-AFC]-SPPF. GConv is a grouped convolution, used to divide the input channels into multiple groups and perform convolutions separately to reduce computational cost and parameter count. The backbone network has three feature extraction modules at three scales, the latter three being [GConv+SAVM-AFC] modules, which can be numbered as feature extraction modules 1, 2, and 3 along the data transmission direction.

[0222] Based on feature extraction at various scales within the backbone network, the neck network incorporates feature fusion modules corresponding to three scales. Each of these modules includes a TF-MSCA module. Each scale's feature fusion module first unifies features from the other two scales to the current scale, then concatenates them, and finally outputs the results through the c2f and TF-MSCA modules to the corresponding decoupled detection head for detection. During scale alignment, downsampling is used to align larger scales to smaller scales, while upsampling is used to align smaller scales to larger scales.

[0223] By analyzing the target fusion features at each scale, the corresponding detection network can detect them and output the detection results.

[0224] During training, in addition to employing a 4×reg_max distributed regression strategy to make the bounding box regression more detailed and stable, thereby improving localization accuracy, the total number of categories (nc) is also set to guide the output layer to generate a corresponding number of category prediction results, ensuring that the detection task can cover all target categories. 4×reg_max represents the number of output channels for the predicted bounding boxes, and 4 represents the four regression components of the bounding box. reg_max This represents the maximum value of the discrete regression interval corresponding to each regression component; n c This indicates the predicted number of categories for clothes drying along the street.

[0225] Once the trained road defect detection model is put into application, redundant detection box removal (such as non-maximum suppression, NMS) and confidence filtering operations can be further introduced to eliminate overlapping detection boxes and select high-confidence prediction results, thereby improving the accuracy and reliability of the final detection results.

[0226] In this embodiment, an improvement based on the SAVM-AFC module is achieved by introducing an adaptive scale modulation mechanism based on global context to jointly model multi-level state space features obtained from structure-aware visual Mamba modeling. A lightweight MLP generates fusion weights corresponding to each scale, enabling the model to adaptively enhance key context responses and suppress redundant background information based on spatial structural complexity. Finally, the fused features are reinjected into the backbone network in a residual manner to avoid the attenuation of fine-grained structural information during deep modeling. This design abandons feature extraction methods that rely on explicit multi-scale convolution or deformation modeling, adopting an integrated strategy of "continuous context modeling based on state space + global scale adaptive modulation + residual information preservation." This significantly improves the sensitivity and robustness of recognizing low-contrast, multi-scale targets with almost no increase in inference latency.

[0227] Based on improvements to the TF-MSCA module, it can simultaneously utilize global statistical regularities and local structural details: multi-scale paths effectively capture the overall outline and spatial distribution of large-scale material accumulations, while local self-attention paths accurately activate scattered targets that are occluded, have low contrast, or are irregularly shaped; the dynamic gating mechanism further ensures the optimal perception strategy is adaptively selected under different scenarios. Experiments show that TF-MSCA can significantly improve the localization accuracy and robustness of identifying diverse urban anomalies in complex urban scenarios, while maintaining lightweight computational overhead. It is particularly suitable for real-time deployment on edge drone platforms, facilitating integrated drone patrol and detection.

[0228] In some embodiments, for drone-based intelligent inspection tasks aimed at refined urban management, the training data exhibits a significant long-tail distribution due to the large number of common abnormal target samples such as household waste and scattered construction debris, while the number of high-risk abnormal target samples such as hazardous waste piles, large discarded furniture, and illegally occupied construction fences is relatively small. Traditional binary cross-entropy loss assigns equal weights to all categories, which can easily lead to the model favoring high-frequency categories during training, while failing to adequately learn about low-frequency but high-value abnormal targets, resulting in missed or false detections.

[0229] To reduce the risk of false negatives or false negatives, this embodiment proposes a non-linear class weight allocation strategy based on median reference to dynamically adjust the classification loss in the detection head. Let there be C classes in the training set, and the sample count vector for each class be:

[0230]

[0231] Furthermore, using the median number of samples in each category as a reference metric, the sample scarcity of category c is defined as:

[0232]

[0233] in, is the numerical stability constant.

[0234] Subsequently, based on the target weight boundary , and hyperparameters , Construct the category weight mapping relationship:

[0235]

[0236] in, , .

[0237] The final category weight vector is obtained as follows:

[0238]

[0239] In this way, categories with fewer samples can obtain higher loss weights, while high-frequency categories have relatively lower loss weights. This allows the model to pay more attention to low-frequency, high-risk targets such as hazardous waste and large road obstructions during training, reducing the probability of them being misclassified as ordinary debris or background areas.

[0240] In the classification loss calculation process, class weights are introduced into the binary cross-entropy loss to obtain a non-linear class-weighted BCE loss based on the median reference:

[0241]

[0242] in, Indicates the total number of categories. This represents the predicted probability of class c. This represents the corresponding real label.

[0243] In the regression loss section, this embodiment uses a combination of CIoU loss and DFL loss to constrain the deviation between the predicted bounding box and the ground truth bounding box. The CIoU loss not only considers the overlap between the predicted and ground truth boxes but also further introduces center point distance and aspect ratio consistency constraints. This allows it to distinguish errors of different scales and shapes even when the target centers coincide. Its expression is:

[0244]

[0245] in, and These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This represents the Euclidean distance between two center points. This represents the length of the diagonal of the smallest closure region between the predicted bounding box and the ground truth bounding box. (Parameter) The expression used to measure the consistency between the predicted bounding box and the ground truth bounding box in terms of aspect ratio is:

[0246]

[0247]

[0248] in, and This indicates the center point of the two rectangles. This represents the Euclidean distance between two rectangles. This represents the distance between the diagonals of the enclosing regions of the two rectangles.

[0249] Through the above design, CIoU loss can more accurately constrain the position, size, and shape consistency of the prediction box, thereby improving the positioning accuracy of irregular targets, long strips of material, and large-scale obstructions.

[0250] Furthermore, to further improve the precision of boundary regression, this embodiment introduces DFL loss, the expression of which is:

[0251]

[0252] in, This represents the predicted probability corresponding to the discrete position on the left. This represents the predicted probability corresponding to the discrete position on the right. and These represent adjacent discrete positions. This represents the true location of the target. By discretizing and modeling the true boundary location, the model's ability to represent boundary details can be improved, making the edges of the predicted box fit the true target contour more closely.

[0253] Ultimately, the total loss function is composed of both the classification loss and the regression loss:

[0254]

[0255] in, and This is a balancing coefficient used to adjust the contribution ratio of classification loss and regression loss in the overall training process. Through this loss design, while ensuring positioning accuracy, the model's sensitivity and recall rate for low-frequency, high-risk urban anomalies can be improved, thereby enhancing the early warning capability and law enforcement support reliability of the drone inspection system in complex urban scenarios.

[0256] In some embodiments, to enhance target detection capabilities in urban refined governance tasks from the perspective of drones, a professional dataset for urban inspection scenarios is constructed, named UDIFG. This dataset mainly targets typical urban appearance violations in public spaces, covering five categories of targets: construction waste, illegally dumped materials, domestic waste, unlicensed street vendors, and abnormal urban appearance. Construction waste includes exposed slag and discarded building materials; illegally dumped materials include temporary sand and gravel piles, damaged packaging materials, and disorderly stacked pipes and boards; domestic waste includes scattered plastic bags, kitchen waste, and discarded items from green belts; unlicensed street vendors include mobile vendors and vehicles occupying roads for sales; and abnormal urban appearance includes illegal billboards, discarded furniture, and piles of debris in undesignated areas. These targets generally exhibit irregular shapes, large scale ranges, and high degree of indistinguishability from the background, thus posing significant identification challenges and application value in smart city visual supervision.

[0257] Furthermore, the images in the dataset are primarily from drone aerial perspectives, accounting for over 85% of all samples, supplemented by a small number of fixed high-point monitoring perspectives to enhance the model's adaptability to different viewpoints. All samples were collected from real urban management scenarios in multiple first-tier and new first-tier cities in China, covering typical public areas such as sidewalks, street corners, community entrances and exits, commercial outdoor areas, and urban-rural fringe areas, thus ensuring high data authenticity, governance relevance, and deployment practicality.

[0258] In terms of spatial distribution, the dataset covers a variety of typical urban functional areas, including narrow alleys in old city districts, modern commercial streets, high-density residential communities, and urban-rural transition zones. In terms of ground materials, it covers a variety of substrate types, including permeable bricks, cement, asphalt, lawn edges, and dirt surfaces. In terms of environmental conditions, it covers complex weather and lighting scenarios, such as low light at dawn and dusk, strong light at noon, diffused light on cloudy days, slippery conditions in light rain, and hazy conditions in light fog. In terms of target characteristics, it takes into account scattered garbage at the sub-meter level, large-area piles of materials covering tens of square meters, rigid building materials, flexible fabrics, and target forms obscured by vegetation, vehicles, or shadows, in order to comprehensively reflect the complex visual challenges in real urban governance.

[0259] In terms of data scale, UDIFG contains 3000 high-quality annotated images, each with either bounding box-level or pixel-level annotations to adapt to different visual tasks such as object detection, instance segmentation, or semantic segmentation. Furthermore, all images are divided into training, validation, and test sets in an 8:1:1 ratio, with the training set containing 2400 images, the validation set containing 300 images, and the test set containing 300 images. The training set is used for model parameter learning, the validation set for hyperparameter tuning, and the test set for independent model performance evaluation. This dataset construction method not only ensures the stability of the model training process and the reproducibility of experimental results but also provides a unified and reliable evaluation benchmark for urban governance models in UAV intelligent inspection scenarios.

[0260] In some embodiments, to comprehensively evaluate the detection performance of the model, this experiment selected four metrics: F1-Score, mean average precision (mAP), number of parameters (params), and gross computational cost (GFLOPs). The F1-Score combines precision and recall to measure the overall performance and stability of the model. Mean precision (AP) is calculated by covering the area under the precision-recall curve, while mAP is the average of AP across all classes. The number of parameters reflects the complexity of the model, and gross computational cost measures the computational complexity of the model. The calculation process of these metrics will be described in detail below.

[0261]

[0262]

[0263]

[0264]

[0265]

[0266] TP, FP, and FN represent the number of correctly predicted, misjudged, and missed positive samples, respectively.

[0267] In some embodiments, the model runtime environment includes an Intel Xeon Platinum 8255C processor, 314 GB of memory, an NVIDIA Tesla V100 32GB graphics card, and a CentOS 8.5.2 (64-bit) operating system. The deep neural network is built on the PyTorch framework, with an input image size of [640, 640], and employs a multi-scale training strategy. The experimental setup is a batch size of 64, training for 200 epochs, using the SDG optimizer with an initial learning rate of 0.01, and optimization using a cosine decay strategy.

[0268] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0269] Corresponding to the drone-based urban management inspection method in the above embodiment, Figure 5 The diagram shows a structural block diagram of the UAV-based urban management inspection device 5 provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0270] Reference Figure 5 The drone-based urban management inspection device 5 includes:

[0271] The feature extraction structure 51 is used to extract features from the input image to be detected to obtain image features; the image to be detected is a video frame or still image containing urban space collected by a drone.

[0272] Feature fusion structure 52 is used to fuse image features to obtain target fused features;

[0273] Detection structure 53 is used to output the inspection results corresponding to the image to be detected based on the target fusion features;

[0274] Specifically, feature extraction structure 51 is used for:

[0275] During feature extraction, perform the following operations:

[0276] The first input features are transformed into block-level representations, and positional information is introduced to obtain the feature sequence;

[0277] Perform continuous context modeling based on state space on the feature sequence to obtain global features;

[0278] The global features are enhanced to obtain the target global features;

[0279] The target global features are fused with the first input features to obtain the first output features corresponding to the first input features;

[0280] The first input feature is extracted from the image to be detected; the image feature is obtained from the first output feature.

[0281] Optionally, the feature extraction structure 51 is specifically used for:

[0282] Based on the state-space model, the following operations are performed iteratively to update the feature sequence position by position, so as to determine the corresponding output feature based on the current hidden state at each position, and then fuse the output features of all positions to obtain the global feature:

[0283] For the current position, obtain the corresponding current input features and the historical hidden states passed from the previous position;

[0284] Learnable parameters adapted to the current input features are generated based on a preset selective scanning mechanism;

[0285] A structure-aware gating operation is performed based on the degree of structural change of the current input features to obtain gating weights; the gating weights are used to adjust the fusion ratio between historical hidden states and current input features;

[0286] The state update relationship is adjusted based on learnable parameters, and the historical hidden state and the current input features are weighted and fused together by gating weights to obtain the current hidden state.

[0287] Optionally, the global features are multi-scale features; the feature extraction structure 51 is specifically used for:

[0288] Global average pooling is performed on the global features at each scale to obtain the global statistical features corresponding to each scale;

[0289] By concatenating the global statistical features at each scale, a multi-scale aggregated feature is obtained.

[0290] A lightweight mapping operation is performed on the multi-scale aggregated features to obtain a multi-scale association representation;

[0291] The multi-scale correlation representation is adjusted based on a preset learnable temperature coefficient, and the normalized weights are obtained through normalization operations.

[0292] At each scale, the global features of the current scale are weighted based on the normalized weights corresponding to the current scale to obtain the enhanced global features of the current scale.

[0293] The enhanced global features corresponding to each scale are fused to obtain the target global features.

[0294] Optionally, during the fusion process, the feature fusion structure 52 is specifically used for:

[0295] Multi-scale modeling and interaction enhancement processing are performed on the second input features to obtain the first attention representation in the width dimension and the first attention representation in the height dimension.

[0296] Local self-attention structure-aware enhancement processing is performed on the second input features to obtain a high-dimensional second attention representation and a width-dimensional second attention representation.

[0297] Based on the content-adaptive gating fusion mechanism, the first attention representation in the width dimension and the second attention representation in the width dimension are weighted and fused, and the first attention representation in the height dimension and the second attention representation in the height dimension are weighted and fused to obtain the fused attention representation in the height dimension and the fused attention representation in the width dimension.

[0298] The second input feature is enhanced based on the fusion attention representation of the height dimension and the fusion attention representation of the width dimension to obtain the second output feature corresponding to the first input feature;

[0299] The second input feature is obtained based on image features, and the target fusion feature is obtained based on the second output feature.

[0300] Optionally, the feature fusion structure 52 is specifically used for:

[0301] Under any preset dimension, adaptive average pooling, channel compression, and batch normalization are performed sequentially. An activation operation is performed on the normalized context compression representation of the current dimension. The normalized context compression representation of the current dimension is weighted based on the attention weight of the current dimension. The enhanced representation of the current dimension is projected and scaled to obtain the target feature of the current dimension. The preset dimensions include width, height, and global dimensions.

[0302] The target features of all preset dimensions are fused, and the fusion result is subjected to dimension-related feature recalibration processing to obtain the first attention representation of the width dimension and the first attention representation of the height dimension.

[0303] Optionally, the feature fusion structure 52 is specifically used for:

[0304] The second input feature is divided into multiple regular and non-overlapping local regions;

[0305] For each local region, a linear projection is performed to generate the query, key, and value of the current local region, and relative position encoding is introduced to calculate the attention response within the current local region;

[0306] The attention responses corresponding to each local region are aggregated and projected to obtain structure perception enhancement features;

[0307] The structure-aware enhancement features are sequentially subjected to average pooling, channel compression, and batch normalization to obtain batch-normalized intermediate representations. Linear activation is then performed on the batch-normalized intermediate representations, and weighting is applied to the batch-normalized intermediate representations based on the obtained attention weights. Finally, the enhanced representations are projected and scaled to obtain the target enhancement features.

[0308] Based on the target enhancement features, dimension-related feature recalibration processing is performed to obtain the second attention representation in the height dimension and the second attention representation in the width dimension.

[0309] Optionally, the urban management inspection method is implemented based on a pre-trained urban management inspection model. The classification loss function for training the urban management inspection model is a binary cross-entropy loss based on nonlinear class weight allocation using median reference. :

[0310]

[0311]

[0312]

[0313]

[0314]

[0315]

[0316] in, Indicates the total number of categories; Indicates the first The number of samples in the class, and ; A vector representing the number of samples in each category; Representing vectors the median; Indicates the first Class balance factor; and Let be a nonlinear mapping parameter, and satisfy . ; Indicates intermediate mapping variables; and Representing all categories The minimum and maximum values; and These represent the lower and upper bounds of the category weights, respectively. Indicates the first The weight corresponding to the class; Indicates classification loss; Indicates the total number of samples; The model predicts the number of... The probability of a class; This represents the corresponding real label; and is the numerical stability constant.

[0317] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0318] Figure 6 This is a schematic diagram of the physical layer structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 The diagram shows only one processor, memory 61, and a computer program 62 stored in memory 61 that can run on at least one processor 60. When processor 60 executes computer program 62, it implements the steps in any of the above embodiments of the UAV-based urban management inspection method. Figure 1 Steps 110-130 are shown.

[0319] The processor 60 may be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0320] In some embodiments, memory 61 may be an internal storage unit of electronic device 6, such as a hard disk or memory of electronic device 6. In other embodiments, memory 61 may also be an external storage device of electronic device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 6.

[0321] Furthermore, memory 61 may include both internal storage units and external storage devices of electronic device 6. Memory 61 is used to store operating devices, application programs, bootloaders, data, and other programs, such as program code for computer programs. Memory 61 can also be used to temporarily store data that has been output or will be output.

[0322] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0323] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0324] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0325] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0326] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0327] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0328] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0329] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0330] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for urban management inspection based on unmanned aerial vehicles (UAVs), characterized in that, include: Feature extraction is performed on the input image to be detected to obtain image features; The image to be detected is a video frame or still image containing urban space, collected by a drone. The image features are fused to obtain the target fused features; Based on the target fusion features, the inspection result corresponding to the image to be detected is output; During feature extraction, the following operations are performed: The first input features are transformed into block-level representations, and positional information is introduced to obtain the feature sequence; Perform continuous context modeling based on state space on the feature sequence to obtain global features; The global features are enhanced to obtain the target global features; The target global feature is fused with the first input feature to obtain the first output feature corresponding to the first input feature; Wherein, the first input feature is extracted based on the image to be detected; the image feature is obtained based on the first output feature; The step of performing continuous context modeling based on state space on the feature sequence to obtain global features includes: Based on the state-space model, the following operations are performed cyclically to recursively update the feature sequence position by position, so as to determine the corresponding output feature based on the current hidden state at each position, and to fuse the output features at all positions to obtain the global feature: For the current position, obtain the corresponding current input features and the historical hidden states passed from the previous position; Learnable parameters adapted to the current input features are generated based on a preset selective scanning mechanism; A structure-aware gating operation is performed based on the degree of structural change of the current input feature to obtain gating weights; the gating weights are used to adjust the fusion ratio between the historical hidden state and the current input feature. The state update relationship is adjusted based on the learnable parameters, and the historical hidden state and the current input feature are weighted and fused together with the gating weights to obtain the current hidden state.

2. The urban management inspection method as described in claim 1, characterized in that, The global features are multi-scale features; The enhancement process for the global features to obtain the target global features includes: Global average pooling is performed on the global features at each scale to obtain the global statistical features corresponding to each scale; The global statistical features at each scale are fused to obtain multi-scale aggregated features; A lightweight mapping operation is performed on the multi-scale aggregated features to obtain a multi-scale association representation; The multi-scale correlation representation is adjusted based on a preset learnable temperature coefficient, and normalized weights are obtained through normalization operations. At each scale, the global features of the current scale are weighted based on the normalized weights corresponding to the current scale to obtain the enhanced global features of the current scale. The enhanced global features corresponding to each scale are fused to obtain the target global features.

3. The urban management inspection method as described in claim 1 or 2, characterized in that, During the fusion process, the following operations are performed: Multi-scale modeling and interaction enhancement processing are performed on the second input features to obtain the first attention representation in the width dimension and the first attention representation in the height dimension. Local self-attention structure-aware enhancement processing is performed on the second input features to obtain a high-dimensional second attention representation and a width-dimensional second attention representation. Based on the content-adaptive gating fusion mechanism, the first attention representation in the width dimension and the second attention representation in the width dimension are weighted and fused, and the first attention representation in the height dimension and the second attention representation in the height dimension are weighted and fused to obtain the fused attention representation in the height dimension and the fused attention representation in the width dimension. The second input feature is enhanced based on the fusion attention representation of the height dimension and the fusion attention representation of the width dimension to obtain the second output feature corresponding to the first input feature; The second input feature is obtained based on the image features, and the target fusion feature is obtained based on the second output feature.

4. The urban management inspection method as described in claim 3, characterized in that, The process of performing multi-scale modeling and interaction enhancement on the second input features yields a width-dimension first attention representation and a height-dimension first attention representation, including: Under any preset dimension, adaptive average pooling, channel compression, and batch normalization operations are performed sequentially. An activation operation is performed on the normalized context compression representation of the current dimension. The normalized context compression representation of the current dimension is weighted based on the attention weight of the current dimension. The enhanced representation of the current dimension is projected and scaled to obtain the target feature of the current dimension. The preset dimension includes width dimension, height dimension, and global dimension. The target features of all preset dimensions are fused, and the fusion result is subjected to dimension-related feature recalibration processing to obtain the first attention representation of the width dimension and the first attention representation of the height dimension.

5. The urban management inspection method as described in claim 3, characterized in that, The process of performing local self-attention structure-aware enhancement on the second input features to obtain a second attention representation in both height and width dimensions includes: The second input feature is divided into multiple regular and non-overlapping local regions; For each local region, a linear projection is performed to generate the query, key, and value of the current local region, and relative position encoding is introduced to calculate the attention response within the current local region; The attention responses corresponding to each local region are aggregated and projected to obtain structure perception enhancement features; The structure-aware enhancement features are sequentially subjected to average pooling, channel compression, and batch normalization to obtain batch-normalized intermediate representations. Linear activation is then performed on the batch-normalized intermediate representations, and weighting is applied to the batch-normalized intermediate representations based on the obtained attention weights. Finally, the enhanced representations are projected and scaled to obtain the target enhancement features. Based on the target enhancement features, a dimension-related feature recalibration process is performed to obtain a second attention representation in the height dimension and a second attention representation in the width dimension.

6. The urban management inspection method as described in claim 1 or 2, characterized in that, The urban management inspection method is implemented based on a pre-trained urban management inspection model. The classification loss function used to train the urban management inspection model is a binary cross-entropy loss based on a non-linear class weight allocation with median reference. : in, Indicates the total number of categories; Indicates the first The number of samples in the class, and ; A vector representing the number of samples in each category; Representing vectors the median; Indicates the first Class balance factor; and Let be a nonlinear mapping parameter, and satisfy . ; Indicates intermediate mapping variables; and Representing all categories The minimum and maximum values; and These represent the lower and upper bounds of the category weights, respectively. Indicates the first The weight corresponding to the class; Indicates classification loss; Indicates the total number of samples; The model predicts the number of... The probability of a class; This represents the corresponding real label; and is the numerical stability constant.

7. A drone-based urban management inspection device, characterized in that, include: The feature extraction structure is used to extract features from the input image to be detected, thereby obtaining image features; The image to be detected is a video frame or still image containing urban space, collected by a drone. A feature fusion structure is used to fuse the image features to obtain target fused features; A detection structure is used to output the inspection result corresponding to the image to be detected based on the target fusion features; Specifically, the feature extraction structure is used for: During feature extraction, perform the following operations: The first input features are transformed into block-level representations, and positional information is introduced to obtain the feature sequence; Perform continuous context modeling based on state space on the feature sequence to obtain global features; The global features are enhanced to obtain the target global features; The target global feature is fused with the first input feature to obtain the first output feature corresponding to the first input feature; Wherein, the first input feature is extracted based on the image to be detected; the image feature is obtained based on the first output feature; The step of performing continuous context modeling based on state space on the feature sequence to obtain global features includes: Based on the state-space model, the following operations are performed cyclically to recursively update the feature sequence position by position, so as to determine the corresponding output feature based on the current hidden state at each position, and to fuse the output features at all positions to obtain the global feature: For the current position, obtain the corresponding current input features and the historical hidden states passed from the previous position; Learnable parameters adapted to the current input features are generated based on a preset selective scanning mechanism; A structure-aware gating operation is performed based on the degree of structural change of the current input feature to obtain gating weights; the gating weights are used to adjust the fusion ratio between the historical hidden state and the current input feature. The state update relationship is adjusted based on the learnable parameters, and the historical hidden state and the current input feature are weighted and fused together with the gating weights to obtain the current hidden state.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the urban management inspection method based on unmanned aerial vehicles as described in any one of claims 1 to 6.

9. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the urban management inspection method based on unmanned aerial vehicles as described in any one of claims 1 to 6.

Images