A multi-target recognition method, system, device and medium

Abstract

The application discloses a kind of multi-target identification method, system, equipment and medium, belong to remote sensing image processing technical field, the method is: obtaining multi-target identification model and target area remote sensing image data;Remote sensing image data is input to the model, and the detection branch in the model detects transmission equipment and hangs and detects to obtain initial detection feature map, simultaneously, in the semantic segmentation branch, water body abnormal area is segmented to obtain initial segmentation feature map, according to the semantic mask of water body area information in initial segmentation feature map corresponding attention weight is determined, to feature map of detection branch is weighted, and according to object position information in initial detection feature map determines spatial constraint mask, to feature map of semantic segmentation branch is constrained processing, obtains target detection result and target segmentation result, therefore, by implementing the application, multi-target identification precision can be improved Effect.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a multi-target recognition method, system, device and medium. Background Technology

[0002] In recent years, due to the frequent occurrence of extreme weather events, the number of power transmission network outages and equipment damage caused by floods has increased significantly. After floods, abnormally inundated areas may form around transmission lines. Simultaneously, strong winds and heavy rains often cause debris such as plastic film, advertising cloth, and corrugated steel sheets to attach to or float near power equipment, potentially leading to short circuits, power outages, and even wildfires, causing widespread power outages and other serious risks to power grid operation. Therefore, to ensure the safe operation of the power grid and guide emergency repairs, there is an urgent need for a technology capable of identifying abnormally inundated areas and debris around power transmission networks.

[0003] Currently, single-target detection is mainly based on a combination of remote sensing imagery and deep learning models. For example, by training a semantic segmentation model, pixel-level classification can be performed on high-resolution optical satellite imagery to extract areas of abnormal water inundation. However, in real-world disaster scenarios, water bodies, floating debris, and electrical equipment often coexist and interfere with each other. Using separate models for detection and segmentation fails to achieve collaborative optimization at the feature level, resulting in low recognition accuracy. Summary of the Invention

[0004] This invention provides a multi-target recognition method, system, device, and medium that can solve the problem of low recognition accuracy.

[0005] This invention provides a multi-target recognition method, comprising: Acquire a multi-target recognition model and remote sensing image data of the target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The remote sensing image data is input into the multi-target recognition model. The remote sensing image data is encoded through the target detection branch to generate an initial detection feature map. Simultaneously, the remote sensing image data is encoded through the semantic segmentation branch to generate an initial segmentation feature map. A spatial constraint mask is determined based on the object position information during the decoding process of the initial detection feature map. The decoding process of the initial segmentation feature map is then constrained based on the spatial constraint mask to obtain a target segmentation result representing anomaly areas in the water body. An attention weight is determined based on the semantic mask corresponding to the anomaly areas in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weight to obtain a target detection result representing power transmission equipment and hanging objects.

[0006] This invention, through parallel setting of target detection and semantic segmentation branches, can simultaneously output device, floating object frames, and water body masks in a single forward pass, reducing inference latency and improving processing throughput. The target detection branch is dedicated to power transmission equipment and floating objects, improving the recall rate of small, elongated target objects and reducing the false negative rate. The semantic segmentation branch is dedicated to abnormal water body regions, improving the accuracy of locating planar water body boundaries and reducing edge errors. Attention weights are generated using the semantic mask of the water body, enhancing background suppression of the feature map of the target detection branch, improving the signal-to-noise ratio, and increasing detection confidence. A spatial constraint mask is generated using the detection box position, and the segmentation branch's weight is reduced in the device region, decreasing the probability of marking water body extensions as water bodies and improving segmentation accuracy. Simultaneous acquisition of detection and segmentation results enables the quantification of the spatial relationship between devices and water bodies within the same image frame, providing highly consistent data for subsequent risk assessment.

[0007] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

[0008] This skip connection between the encoding and decoding layers enables multi-scale feature fusion, balancing details and semantics, and improving the clarity of segmentation edges. The attention gating unit suppresses the weights of device regions in the feature map of the encoding layer, reducing interference from device regions received by the decoding layer and improving the continuity of the water mask. The hierarchical fusion output allows each layer to receive refined features, ultimately reducing the positioning error of the segmentation mask edges.

[0009] Further, the step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result specifically involves: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

[0010] In this way, edge gradient features are extracted through edge perception learning to obtain high-frequency information of the water body boundary; the edge optimization module uses gradient features to refine the edges of the initial mask, further reducing the boundary positioning error and improving the mask edge accuracy.

[0011] Furthermore, the multi-target recognition method further includes: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

[0012] By clustering the anchor box size, the matching degree between the anchor box and the target's true scale is improved, the peak value of the IoU distribution shifts upward, and the convergence speed is accelerated. Based on the clustered anchor box retraining, the candidate box regression residual is reduced, the localization error is reduced, and the detection accuracy is improved.

[0013] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

[0014] This process generates initial detection boxes, ensuring that all potential targets are identified and maintaining a high recall rate. Overlapping detection boxes are identified, and redundant boxes are accurately located. Then, weighted non-maximum suppression is performed, with the suppression weight calculated based on both confidence and box quality. The boxes with the highest localization accuracy are retained, and false detection boxes are eliminated, ultimately improving the accuracy of the detection results.

[0015] Further, the step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically involves: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

[0016] By extracting features in three dimensions—scattering, spatial structure, and temporal sequence—the information dimensions are expanded, and the feature discrimination power is enhanced. By combining cascaded channels and spatial attention, the weights of key channels and key spatial locations can be amplified, improving the signal-to-noise ratio of the feature map. By weightedly fusing the attention weight map with the initial feature map, the contrast of the target feature map is enhanced, simultaneously improving the classification and regression accuracy of subsequent detection heads.

[0017] Furthermore, the multi-target recognition method further includes: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

[0018] This method of acquiring an initial map provides a unified coordinate benchmark and background geographic information; through spatial alignment and vectorization, the detection box and segmentation mask are converted into unified projection coordinates, enhancing spatial consistency; and by generating a target map, equipment, floating objects, and water bodies can be accurately overlaid on the same vector layer, allowing for immediate measurement of risk distances and improving monitoring efficiency.

[0019] Another embodiment of the present invention provides a multi-target recognition system, including: an acquisition module and a recognition module; The acquisition module is used to acquire remote sensing image data of a multi-target recognition model and a target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The recognition module is used to input the remote sensing image data into the multi-target recognition model, encode the remote sensing image data through the target detection branch to generate an initial detection feature map, and simultaneously encode the remote sensing image data through the semantic segmentation branch to generate an initial segmentation feature map; determine a spatial constraint mask based on the object position information in the decoding process of the initial detection feature map, and perform constraint processing on the decoding process of the initial segmentation feature map based on the spatial constraint mask to obtain a target segmentation result for characterizing anomaly areas in water bodies; determine attention weights based on the semantic mask corresponding to the anomaly areas in the target segmentation result, and perform weighted processing on the decoding process of the initial detection feature map based on the attention weights to obtain target detection results for characterizing power transmission equipment and hanging objects.

[0020] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the multi-target recognition method of the present invention.

[0021] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the multi-target recognition method of the present invention. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a multi-target recognition method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a multi-target recognition system provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0026] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0029] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0030] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0031] See Figure 1 To address the problem of low recognition accuracy in existing technologies, an embodiment of the present invention provides a multi-target recognition method, comprising: Step 101: Obtain remote sensing image data of the target region and the multi-target recognition model, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch.

[0032] In the above steps, based on the requirements of the identification task, the geographical range (i.e., target area), timeliness, spatial resolution, and cloud cover constraints of the required image are clarified. Satellite observation tasks are formulated according to the requirements, and observation parameters such as observation area, observation time, and observation angle are determined. Remote sensing platforms (such as satellites and UAVs) are then scheduled to collect data to obtain raw data. The collected raw data is stored in single-view complex (SLC) or ground distance multiview (GRD) image format. Data processing is performed on the collected data, including denoising and correction, data format conversion and standardization, multiview processing and resampling, and data quality control, to obtain the remote sensing image data.

[0033] The multi-target recognition model is a pre-trained deep learning model. Its core structure includes a target detection branch and a semantic segmentation branch. The target detection branch identifies discrete targets such as power transmission equipment (e.g., towers, conductors, insulators) and hanging objects (e.g., dust nets, corrugated steel sheets). This branch can be built based on a two-stage detection framework (e.g., Faster R-CNN series) or a single-stage detection framework (e.g., YOLO series, SSD series), and is responsible for outputting the target's category, location, and confidence score. The semantic segmentation branch identifies continuous pixel regions such as abnormal water bodies. This branch can be built based on an encoder-decoder structure (e.g., U-Net series, DeepLab series), and is responsible for outputting the classification result of whether each pixel belongs to an abnormal water body region. A cross-task feature fusion module, as an interactive unit connecting the two branches, is used to achieve information complementarity and collaborative optimization between the detection and segmentation tasks at the feature level, improving the model's overall recognition performance in complex scenes.

[0034] It should be noted that the optimized multi-target recognition model can be deployed on UAV onboard computers, edge computing devices near power transmission towers, or other field computing nodes to achieve localized data processing and real-time recognition, reducing reliance on remote communication networks. To further improve the adaptability and accuracy of the recognition system under different environmental conditions, multi-source sensing data can be integrated. For example, infrared thermal imaging data can be combined to support recognition in low-visibility environments such as nighttime or foggy conditions, or LiDAR point cloud data can be introduced to obtain the three-dimensional structural information of the target. This enhances the robustness of the overall recognition through complementary multi-dimensional features.

[0035] Step 102: Input the remote sensing image data into the multi-target recognition model. Encode the remote sensing image data through the target detection branch to generate an initial detection feature map. Simultaneously, encode the remote sensing image data through the semantic segmentation branch to generate an initial segmentation feature map. Determine a spatial constraint mask based on the object position information during the decoding process of the initial detection feature map, and perform constraint processing on the decoding process of the initial segmentation feature map based on the spatial constraint mask to obtain target segmentation results for characterizing abnormal water body areas. Determine attention weights based on the semantic mask corresponding to the abnormal water body areas in the target segmentation results, and perform weighted processing on the decoding process of the initial detection feature map based on the attention weights to obtain target detection results for characterizing power transmission equipment and hanging objects.

[0036] In the above steps, after the remote sensing image data is input into the model, the target detection branch and the semantic segmentation branch within the model work in parallel. The target detection branch analyzes image features to locate and classify two discrete targets in the image: power transmission equipment and floating objects, outputting detection results including their location and category. The semantic segmentation branch performs pixel-level classification of the image, distinguishing abnormal water areas (such as flooded areas) from other backgrounds, and outputting a segmentation result that labels whether each pixel belongs to a water body. Two-way information interaction and optimization are performed. In the optimization process from segmentation to detection, this module uses the water body region information in the segmentation result output by the semantic segmentation branch to generate an attention weight map representing which image regions are water bodies. This weight is applied to the feature map in the decoding process of the target detection branch to suppress or reduce the interference of water background regions on feature expression, thereby improving the detection accuracy of power transmission equipment and floating objects in complex aquatic environments. In the optimization process from detection to segmentation, the semantic segmentation branch utilizes the object location information obtained from the initial detection feature map decoding process of the object detection branch to generate a spatial constraint mask representing which image locations have been identified as objects (non-water bodies). This mask is introduced into the semantic segmentation branch's processing to prevent clearly identified objects such as power transmission equipment from being incorrectly classified as water bodies, thereby optimizing the boundary accuracy of water body region segmentation and avoiding missegmentation. After the bidirectional optimization processing of the aforementioned cross-task feature fusion module, the model outputs the final results, including optimized object detection results and optimized object segmentation results. These two results have better consistency in semantics and space.

[0037] As an example of an embodiment of the present invention, the multi-target recognition method further includes: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

[0038] In this embodiment, for the long and regular shapes of power transmission equipment (iron towers, conductors, insulators) and the irregular shapes of hanging objects (sheds, corrugated steel tiles, dust nets), the K-means clustering algorithm is used to redesign the anchor frame size (such as adding slender anchor frames) to improve the target matching degree.

[0039] Specifically, a labeled dataset is obtained for training the object detection branch. This dataset contains a large number of labeled remote sensing image samples. In each sample, the power transmission equipment and hanging objects to be identified are precisely labeled using rectangular boxes. The labeling information includes at least the position coordinates of the rectangular box in the image (usually the coordinates of the upper left and lower right corners, or the coordinates of the center point plus width and height) and the category label. The width and height of the labeled rectangular boxes for the two target categories of power transmission equipment and hanging objects are extracted from all training samples to form a scale dataset. The above scale dataset is analyzed using a clustering algorithm to learn several typical sizes (width and height combinations) of the most frequently occurring power transmission equipment and hanging objects in the dataset, such as slender (corresponding to conductors), tall and large (corresponding to the main body of the tower), and wide and flat (corresponding to large-area hanging objects). The width and height values ​​of the K cluster centers obtained after clustering are used to determine multiple preset anchor box sizes. An object detection network is initialized as the initial detection branch. The preset anchor box size obtained from clustering is configured as the prior anchor box for this initial detection branch, replacing its original, general default anchor box size. The network is then trained using training samples. During training, the model performs target matching and position regression learning based on this optimized anchor box. After training, the target detection branch is obtained. Thus, because the detection prior (anchor box) of this branch closely matches the true scale distribution of the target, it exhibits higher initial matching efficiency, faster convergence speed, and better final detection accuracy in the detection tasks of power transmission equipment and hanging objects.

[0040] As an example of an embodiment of the present invention, the step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically includes: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

[0041] In this embodiment, feature extraction is performed on the input remote sensing image data to generate an initial feature map. Taking SAR imagery as an example, the core differences between easily confused categories are reflected in three dimensions: scattering characteristics, spatial structure, and temporal changes. A feature extraction module needs to be designed specifically to mine category-specific features. Therefore, scattering features are extracted by analyzing the backscattering coefficient of the remote sensing image data; spatial structure features are extracted by identifying the geometric shape, texture, spatial layout, and contextual relationships of the target; and temporal features are extracted by combining multi-temporal image data of the same area to analyze the changing patterns of the target (such as the water body area and the state of floating objects) over time, thereby enhancing the feature discrimination power by utilizing its dynamic characteristics.

[0042] Regarding scattering characteristics, calm water bodies exhibit specular scattering, appearing as strong dark spots in SAR images (low and uniform backscattering coefficient σ); flooded water bodies have a certain roughness, with σ slightly higher than calm water bodies, but still lower than surrounding landforms and hanging objects; hanging objects (greenhouses, corrugated steel sheets, dust nets) exhibit volume scattering or two-way scattering, with moderate σ and local inhomogeneity (e.g., the metal surface of corrugated steel sheets causes local bright spots, and the porous structure of dust nets causes alternating bright and dark areas); similar landforms (e.g., wetlands, bare land) exhibit surface scattering, with σ close to that of water bodies, but with a more irregular spatial distribution and lacking the continuous boundary characteristics of water bodies.

[0043] In terms of spatial structural characteristics, water bodies are continuous, irregular planar regions with smooth boundaries and connectivity; hanging objects are discrete, small blocky or strip-shaped regions with irregular boundaries but limited range (usually smaller than the distance between transmission towers); similar landforms are discontinuous regions without fixed shapes, with blurred boundaries and blending into the surrounding terrain.

[0044] Regarding temporal characteristics, under severe weather conditions, water bodies and floating objects exhibit dynamic changes, while landforms remain relatively stable. Differences can be identified through temporal SAR imagery: During floods, the water body gradually expands, and the σ value gradually decreases and tends to stabilize; after strong winds and heavy rains, the position and shape of floating objects may change (e.g., dust nets are blown apart, corrugated steel sheets are displaced), and the spatial distribution of the σ value will also change accordingly; similar landforms show relatively small changes in σ value and spatial morphology over time.

[0045] As an example of an embodiment of the present invention, the process of decoding the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

[0046] In this embodiment, based on the initial segmentation feature map enhanced by the attention mechanism obtained in the aforementioned steps, the detection heads (classification head and regression head) of the target detection branch output prediction results for potential targets in the image. The prediction results include, for each spatial location (or each preset anchor box) in the image, the bounding box offset, i.e., the adjustment parameter of the bounding box where a target may exist relative to the preset anchor box; and the class confidence, i.e., the probability score of the bounding box belonging to either power transmission equipment or a hanging object. By thresholding all predictions (e.g., retaining only predictions with confidence scores higher than a preset threshold), an initial detection box set including multiple candidate targets is obtained. Each detection box in the set contains its location coordinates (e.g., center point, width and height), predicted class, and corresponding confidence score. In the generated initial detection box set, because the model may generate multiple high-confidence predictions around the target or in different feature regions of the same target, multiple detection boxes may correspond to the same real object, resulting in overlap. To identify these redundant detection boxes, all detection boxes are traversed, and the degree of overlap between each pair of detection boxes is calculated. For example, the Intersection over Union (IoU) is used as a metric. The IoU is defined as the ratio of the overlapping area of ​​two detection boxes to their union area. All detection box pairs with an IoU exceeding a certain overlap threshold (e.g., 0.5 or 0.7) are identified as overlapping detection boxes belonging to the same target. Traditional Non-Maximum Suppression (NMS) algorithms typically retain only the box with the highest confidence when processing overlapping boxes, suppressing the rest. This method is prone to missed detections in scenes with dense targets or occlusion. This embodiment employs a weighted nonmaximum suppression algorithm (WeightedNMS) for optimization. All overlapping detection boxes belonging to the same target (i.e., the set of detection boxes with IoU exceeding a threshold) are treated as a single processing unit. For this unit, a weighted average is calculated based on the confidence score of each box. Specifically, the coordinate values ​​(such as center point, width, and height) of each overlapping detection box are weighted for confidence calculation. A new, optimized detection box is generated through weighted fusion. The coordinates of this box are the weighted average of the coordinates of the original overlapping boxes, and its confidence score is typically the highest confidence score or the weighted average confidence score of the original boxes. After processing all identified overlapping detection box groups, the remaining independent detection boxes that do not significantly overlap with other boxes, along with all optimized detection boxes generated after weighted NMS processing, constitute the target detection result.

[0047] As an example of an embodiment of the present invention, the process of decoding the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

[0048] In this embodiment, the semantic segmentation branch adopts an encoder-decoder architecture. The encoding layer is used to downsample and extract deep features from the input remote sensing image, outputting an encoded feature map with high-level semantic information. The encoded feature map output by the last encoding layer of the encoder is the initial segmentation feature map. The decoding part includes multiple sequentially connected decoding layers, responsible for progressively upsampling and restoring the spatial resolution of the feature map. Each decoding layer receives two types of input: the feature map of the corresponding level passed through skip connections from the encoding layer (i.e., the current encoded feature map), and the output feature map from the previous decoding layer (i.e., the previous decoded feature map). In the decoding layer with integrated attention gating units, the previous decoded feature map is used as the gating signal. This signal carries some context and spatial information that has been restored in the current decoding stage. The unit processes the gating signal through a small neural network (usually containing convolution, activation functions, and sigmoid functions) to generate an attention weight map with the same spatial size as the current encoded feature map. The value of this weight map ranges from 0 to 1. The generated attention weight map is multiplied (or weighted) element-wise with the current encoded feature map from the skip connection encoder. This is used to suppress feature responses in spatial regions in the current encoded feature map that are irrelevant to the current segmentation task (identifying abnormal water areas) according to the needs of the decoding process, resulting in a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output from the previous decoding layer (usually through channel concatenation or element-wise addition) to obtain the current decoded feature map. If the current decoding layer is the last layer, it is fed into the final classification convolutional layer to classify each pixel (water or non-water) to obtain the target segmentation result. Otherwise, the processed current decoded feature map is used as the new previous decoded feature map and input into the next decoding layer to continue the above decoding and optimization process.

[0049] As an example of an embodiment of the present invention, the step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result specifically involves: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

[0050] In this embodiment, a parallel edge-aware learning branch is introduced during the training and inference process of the semantic segmentation network. This branch takes the deep semantic features extracted by the encoder as input and learns the boundary gradient features of abnormal water areas through supervised learning (usually using edge maps of real water body masks as labels), thereby generating edge gradient feature maps that reflect the boundary strength and direction. After obtaining the preliminary intermediate segmentation results, the extracted edge gradient features are fused and optimized with the intermediate segmentation results. Through the guidance and constraints of edge information, coarse or blurry segmentation boundaries are refined and corrected, resulting in clearer, more continuous, and more accurate target segmentation results, effectively improving the geometric accuracy of water area recognition.

[0051] As an example of an embodiment of the present invention, the multi-target recognition method further includes: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

[0052] In this embodiment, an initial map (such as a power facility layout map or a basic geographic base map) with precise geographic coordinates of the target area is acquired. Then, the target segmentation results (water body pixel mask) and target detection results (equipment and hanging object bounding boxes) output by the model are uniformly converted to the same geographic coordinate system based on the geographic positioning parameters of the original remote sensing image. Vectorization processing is then used to generate vector datasets of areal features (water body area) and point / area features (equipment and hanging objects), respectively. Finally, this vector dataset is overlaid as a new layer on the initial map to generate a comprehensive target map that integrates real-time identification information. This map can be used for spatial querying, risk level assessment (such as hazard classification based on water depth and distance), and early warning work order dispatch, achieving seamless integration from automated identification to operational risk monitoring and decision support.

[0053] like Figure 2As shown, based on the above-described method embodiments, an embodiment of the present invention provides a multi-target recognition system 200, including: an acquisition module 201 and a recognition module 202; The acquisition module 201 is used to acquire remote sensing image data of a multi-target recognition model and a target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The recognition module 202 is used to input the remote sensing image data into the multi-target recognition model, encode the remote sensing image data through the target detection branch to generate an initial detection feature map, and simultaneously encode the remote sensing image data through the semantic segmentation branch to generate an initial segmentation feature map. Based on the object position information obtained during the decoding process of the initial detection feature map, a spatial constraint mask is determined. This spatial constraint mask is then used to constrain the decoding process of the initial segmentation feature map, resulting in a target segmentation result characterizing anomaly regions in the water body. Attention weights are determined based on the semantic mask corresponding to the abnormal water area in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weights to obtain the target detection result used to characterize the power transmission equipment and the hanging objects.

[0054] Acquire a multi-target recognition model and remote sensing image data of the target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The remote sensing image data is input into the multi-target recognition model. The remote sensing image data is encoded through the target detection branch to generate an initial detection feature map. Simultaneously, the remote sensing image data is encoded through the semantic segmentation branch to generate an initial segmentation feature map. A spatial constraint mask is determined based on the object position information during the decoding process of the initial detection feature map. The decoding process of the initial segmentation feature map is then constrained based on the spatial constraint mask to obtain a target segmentation result representing anomaly areas in the water body. An attention weight is determined based on the semantic mask corresponding to the anomaly areas in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weight to obtain a target detection result representing power transmission equipment and hanging objects.

[0055] This invention, through parallel setting of target detection and semantic segmentation branches, can simultaneously output device, floating object frames, and water body masks in a single forward pass, reducing inference latency and improving processing throughput. The target detection branch is dedicated to power transmission equipment and floating objects, improving the recall rate of small, elongated target objects and reducing the false negative rate. The semantic segmentation branch is dedicated to abnormal water body regions, improving the accuracy of locating planar water body boundaries and reducing edge errors. Attention weights are generated using the semantic mask of the water body, enhancing background suppression of the feature map of the target detection branch, improving the signal-to-noise ratio, and increasing detection confidence. A spatial constraint mask is generated using the detection box position, and the segmentation branch's weight is reduced in the device region, decreasing the probability of marking water body extensions as water bodies and improving segmentation accuracy. Simultaneous acquisition of detection and segmentation results enables the quantification of the spatial relationship between devices and water bodies within the same image frame, providing highly consistent data for subsequent risk assessment.

[0056] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

[0057] This skip connection between the encoding and decoding layers enables multi-scale feature fusion, balancing details and semantics, and improving the clarity of segmentation edges. The attention gating unit suppresses the weights of device regions in the feature map of the encoding layer, reducing interference from device regions received by the decoding layer and improving the continuity of the water mask. The hierarchical fusion output allows each layer to receive refined features, ultimately reducing the positioning error of the segmentation mask edges.

[0058] Further, the step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result specifically involves: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

[0059] In this way, edge gradient features are extracted through edge perception learning to obtain high-frequency information of the water body boundary; the edge optimization module uses gradient features to refine the edges of the initial mask, further reducing the boundary positioning error and improving the mask edge accuracy.

[0060] Furthermore, the multi-target recognition system is also used for: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

[0061] By clustering the anchor box size, the matching degree between the anchor box and the target's true scale is improved, the peak value of the IoU distribution shifts upward, and the convergence speed is accelerated. Based on the clustered anchor box retraining, the candidate box regression residual is reduced, the localization error is reduced, and the detection accuracy is improved.

[0062] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

[0063] This process generates initial detection boxes, ensuring that all potential targets are identified and maintaining a high recall rate. Overlapping detection boxes are identified, and redundant boxes are accurately located. Then, weighted non-maximum suppression is performed, with the suppression weight calculated based on both confidence and box quality. The boxes with the highest localization accuracy are retained, and false detection boxes are eliminated, ultimately improving the accuracy of the detection results.

[0064] Further, the step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically involves: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

[0065] By extracting features in three dimensions—scattering, spatial structure, and temporal sequence—the information dimensions are expanded, and the feature discrimination power is enhanced. By combining cascaded channels and spatial attention, the weights of key channels and key spatial locations can be amplified, improving the signal-to-noise ratio of the feature map. By weightedly fusing the attention weight map with the initial feature map, the contrast of the target feature map is enhanced, simultaneously improving the classification and regression accuracy of subsequent detection heads.

[0066] Furthermore, the multi-target recognition system is also used for: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

[0067] This method of acquiring an initial map provides a unified coordinate benchmark and background geographic information; through spatial alignment and vectorization, the detection box and segmentation mask are converted into unified projection coordinates, enhancing spatial consistency; and by generating a target map, equipment, floating objects, and water bodies can be accurately overlaid on the same vector layer, allowing for immediate measurement of risk distances and improving monitoring efficiency.

[0068] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can implement the multi-target recognition method provided by any of the above method embodiments of the present invention.

[0069] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0070] For ease of description and brevity, the system embodiments of the present invention include all the implementation methods described in the above multi-target recognition method embodiments, and will not be repeated here.

[0071] Based on the above embodiments of the multi-target recognition method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it performs the following steps: Acquire a multi-target recognition model and remote sensing image data of the target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The remote sensing image data is input into the multi-target recognition model. The remote sensing image data is encoded through the target detection branch to generate an initial detection feature map. Simultaneously, the remote sensing image data is encoded through the semantic segmentation branch to generate an initial segmentation feature map. A spatial constraint mask is determined based on the object position information during the decoding process of the initial detection feature map. The decoding process of the initial segmentation feature map is then constrained based on the spatial constraint mask to obtain a target segmentation result representing anomaly areas in the water body. An attention weight is determined based on the semantic mask corresponding to the anomaly areas in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weight to obtain a target detection result representing power transmission equipment and hanging objects.

[0072] This invention, through parallel setting of target detection and semantic segmentation branches, can simultaneously output device, floating object frames, and water body masks in a single forward pass, reducing inference latency and improving processing throughput. The target detection branch is dedicated to power transmission equipment and floating objects, improving the recall rate of small, elongated target objects and reducing the false negative rate. The semantic segmentation branch is dedicated to abnormal water body regions, improving the accuracy of locating planar water body boundaries and reducing edge errors. Attention weights are generated using the semantic mask of the water body, enhancing background suppression of the feature map of the target detection branch, improving the signal-to-noise ratio, and increasing detection confidence. A spatial constraint mask is generated using the detection box position, and the segmentation branch's weight is reduced in the device region, decreasing the probability of marking water body extensions as water bodies and improving segmentation accuracy. Simultaneous acquisition of detection and segmentation results enables the quantification of the spatial relationship between devices and water bodies within the same image frame, providing highly consistent data for subsequent risk assessment.

[0073] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

[0074] This skip connection between the encoding and decoding layers enables multi-scale feature fusion, balancing details and semantics, and improving the clarity of segmentation edges. The attention gating unit suppresses the weights of device regions in the feature map of the encoding layer, reducing interference from device regions received by the decoding layer and improving the continuity of the water mask. The hierarchical fusion output allows each layer to receive refined features, ultimately reducing the positioning error of the segmentation mask edges.

[0075] Further, the step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result specifically involves: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

[0076] In this way, edge gradient features are extracted through edge perception learning to obtain high-frequency information of the water body boundary; the edge optimization module uses gradient features to refine the edges of the initial mask, further reducing the boundary positioning error and improving the mask edge accuracy.

[0077] Furthermore, when the processor executes the computer program, it also performs the following steps: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

[0078] By clustering the anchor box size, the matching degree between the anchor box and the target's true scale is improved, the peak value of the IoU distribution shifts upward, and the convergence speed is accelerated. Based on the clustered anchor box retraining, the candidate box regression residual is reduced, the localization error is reduced, and the detection accuracy is improved.

[0079] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

[0080] This process generates initial detection boxes, ensuring that all potential targets are identified and maintaining a high recall rate. Overlapping detection boxes are identified, and redundant boxes are accurately located. Then, weighted non-maximum suppression is performed, with the suppression weight calculated based on both confidence and box quality. The boxes with the highest localization accuracy are retained, and false detection boxes are eliminated, ultimately improving the accuracy of the detection results.

[0081] Further, the step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically involves: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

[0082] By extracting features in three dimensions—scattering, spatial structure, and temporal sequence—the information dimensions are expanded, and the feature discrimination power is enhanced. By combining cascaded channels and spatial attention, the weights of key channels and key spatial locations can be amplified, improving the signal-to-noise ratio of the feature map. By weightedly fusing the attention weight map with the initial feature map, the contrast of the target feature map is enhanced, simultaneously improving the classification and regression accuracy of subsequent detection heads.

[0083] Furthermore, when the processor executes the computer program, it also performs the following steps: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

[0084] This method of acquiring an initial map provides a unified coordinate benchmark and background geographic information; through spatial alignment and vectorization, the detection box and segmentation mask are converted into unified projection coordinates, enhancing spatial consistency; and by generating a target map, equipment, floating objects, and water bodies can be accurately overlaid on the same vector layer, allowing for immediate measurement of risk distances and improving monitoring efficiency.

[0085] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0086] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0087] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0088] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the following steps: Acquire a multi-target recognition model and remote sensing image data of the target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The remote sensing image data is input into the multi-target recognition model. The remote sensing image data is encoded through the target detection branch to generate an initial detection feature map. Simultaneously, the remote sensing image data is encoded through the semantic segmentation branch to generate an initial segmentation feature map. A spatial constraint mask is determined based on the object position information during the decoding process of the initial detection feature map. The decoding process of the initial segmentation feature map is then constrained based on the spatial constraint mask to obtain a target segmentation result representing anomaly areas in the water body. An attention weight is determined based on the semantic mask corresponding to the anomaly areas in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weight to obtain a target detection result representing power transmission equipment and hanging objects.

[0089] This invention, through parallel setting of target detection and semantic segmentation branches, can simultaneously output device, floating object frames, and water body masks in a single forward pass, reducing inference latency and improving processing throughput. The target detection branch is dedicated to power transmission equipment and floating objects, improving the recall rate of small, elongated target objects and reducing the false negative rate. The semantic segmentation branch is dedicated to abnormal water body regions, improving the accuracy of locating planar water body boundaries and reducing edge errors. Attention weights are generated using the semantic mask of the water body, enhancing background suppression of the feature map of the target detection branch, improving the signal-to-noise ratio, and increasing detection confidence. A spatial constraint mask is generated using the detection box position, and the segmentation branch's weight is reduced in the device region, decreasing the probability of marking water body extensions as water bodies and improving segmentation accuracy. Simultaneous acquisition of detection and segmentation results enables the quantification of the spatial relationship between devices and water bodies within the same image frame, providing highly consistent data for subsequent risk assessment.

[0090] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

[0091] This skip connection between the encoding and decoding layers enables multi-scale feature fusion, balancing details and semantics, and improving the clarity of segmentation edges. The attention gating unit suppresses the weights of device regions in the feature map of the encoding layer, reducing interference from device regions received by the decoding layer and improving the continuity of the water mask. The hierarchical fusion output allows each layer to receive refined features, ultimately reducing the positioning error of the segmentation mask edges.

[0092] Further, the step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result specifically involves: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

[0093] In this way, edge gradient features are extracted through edge perception learning to obtain high-frequency information of the water body boundary; the edge optimization module uses gradient features to refine the edges of the initial mask, further reducing the boundary positioning error and improving the mask edge accuracy.

[0094] Furthermore, during the execution of the computer program, the device containing the computer-readable storage medium also performs the following steps: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

[0095] By clustering the anchor box size, the matching degree between the anchor box and the target's true scale is improved, the peak value of the IoU distribution shifts upward, and the convergence speed is accelerated. Based on the clustered anchor box retraining, the candidate box regression residual is reduced, the localization error is reduced, and the detection accuracy is improved.

[0096] Furthermore, the decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

[0097] This process generates initial detection boxes, ensuring that all potential targets are identified and maintaining a high recall rate. Overlapping detection boxes are identified, and redundant boxes are accurately located. Then, weighted non-maximum suppression is performed, with the suppression weight calculated based on both confidence and box quality. The boxes with the highest localization accuracy are retained, and false detection boxes are eliminated, ultimately improving the accuracy of the detection results.

[0098] Further, the step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically involves: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

[0099] By extracting features in three dimensions—scattering, spatial structure, and temporal sequence—the information dimensions are expanded, and the feature discrimination power is enhanced. By combining cascaded channels and spatial attention, the weights of key channels and key spatial locations can be amplified, improving the signal-to-noise ratio of the feature map. By weightedly fusing the attention weight map with the initial feature map, the contrast of the target feature map is enhanced, simultaneously improving the classification and regression accuracy of subsequent detection heads.

[0100] Furthermore, during the execution of the computer program, the device containing the computer-readable storage medium also performs the following steps: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

[0101] This method of acquiring an initial map provides a unified coordinate benchmark and background geographic information; through spatial alignment and vectorization, the detection box and segmentation mask are converted into unified projection coordinates, enhancing spatial consistency; and by generating a target map, equipment, floating objects, and water bodies can be accurately overlaid on the same vector layer, allowing for immediate measurement of risk distances and improving monitoring efficiency.

[0102] Based on the above-described method embodiments, this invention also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of any of the above-described method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0103] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0104] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-target recognition method, characterized in that, include: Acquire a multi-target recognition model and remote sensing image data of the target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The remote sensing image data is input into the multi-target recognition model. The remote sensing image data is encoded through the target detection branch to generate an initial detection feature map. Simultaneously, the remote sensing image data is encoded through the semantic segmentation branch to generate an initial segmentation feature map. A spatial constraint mask is determined based on the object position information during the decoding process of the initial detection feature map. The decoding process of the initial segmentation feature map is then constrained based on the spatial constraint mask to obtain a target segmentation result representing anomaly areas in the water body. An attention weight is determined based on the semantic mask corresponding to the anomaly areas in the target segmentation result. The decoding process of the initial detection feature map is then weighted based on the attention weight to obtain a target detection result representing power transmission equipment and hanging objects.

2. The multi-target recognition method as described in claim 1, characterized in that, The decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: The initial segmentation feature map is decoded layer by layer. During this decoding process, if the current decoding layer has an attention gating unit, the spatial location corresponding to the power transmission equipment in the current encoded feature map is weighted and suppressed based on the spatial constraint mask to obtain a weighted encoded feature map. The weighted encoded feature map is then fused with the previous decoded feature map output by the previous decoding layer to obtain the current decoded feature map. The current encoded feature map is the encoded feature map output by the current encoding layer that is skip-connected to the current decoding layer. If the current decoding layer does not have an attention gating unit, then the previous decoding feature map and the current encoding feature map are fused to obtain the current decoding feature map. If the current decoding layer is the last layer, semantic segmentation is performed on the current decoding feature map to obtain the target segmentation result; otherwise, the current decoding feature map is input into the next decoding layer.

3. The multi-target recognition method as described in claim 2, characterized in that, The step of performing semantic segmentation on the current decoded feature map to obtain the target segmentation result is specifically as follows: Edge gradient features are obtained by performing edge-aware learning on the remote sensing image data; Perform semantic segmentation on the current decoded feature map to obtain intermediate segmentation results; The intermediate segmentation result is optimized based on the edge gradient features to obtain the target segmentation result.

4. The multi-target recognition method as described in claim 1, characterized in that, The multi-target recognition method further includes: Obtain training samples; Clustering algorithms are used to perform cluster analysis on the dimensions of the annotation frames of the power transmission equipment and the hanging objects in the training samples to obtain multiple preset anchor frame dimensions; Based on the preset anchor frame size, the initial detection branch is trained by anchor frame matching optimization to obtain the target detection branch.

5. The multi-target recognition method as described in claim 1, characterized in that, The decoding process of the initial segmentation feature map based on the spatial constraint mask is constrained to obtain the target segmentation result used to characterize the abnormal water body region, specifically as follows: Target detection is performed on the power transmission equipment and the hanging objects in the initial segmentation feature map to obtain several initial detection boxes; The overlapping portions of all the initial detection boxes are identified to obtain overlapping detection boxes; The overlapping detection boxes are processed using a weighted nonmaximum suppression algorithm to obtain the target detection result.

6. The method according to claim 1, characterized in that, The step of encoding the remote sensing image data through the target detection branch to generate an initial detection feature map specifically involves: The initial detection feature map is obtained by extracting and fusing the scattering features, spatial structure features, and temporal features in the remote sensing image data.

7. The multi-target recognition method as described in claim 1, characterized in that, The multi-target recognition method further includes: Obtain the initial map of the target area; The target segmentation results and the target detection results are spatially aligned and vectorized to obtain a vector dataset. By combining the vector dataset and the initial map, a target map for risk monitoring is obtained.

8. A multi-target recognition system, characterized in that, include: Acquisition module and recognition module; The acquisition module is used to acquire remote sensing image data of a multi-target recognition model and a target region, wherein the multi-target recognition model includes a target detection branch and a semantic segmentation branch; The recognition module is used to input the remote sensing image data into the multi-target recognition model, encode the remote sensing image data through the target detection branch to generate an initial detection feature map, and simultaneously encode the remote sensing image data through the semantic segmentation branch to generate an initial segmentation feature map; determine a spatial constraint mask based on the object position information in the decoding process of the initial detection feature map, and perform constraint processing on the decoding process of the initial segmentation feature map based on the spatial constraint mask to obtain a target segmentation result for characterizing anomaly areas in water bodies; determine attention weights based on the semantic mask corresponding to the anomaly areas in the target segmentation result, and perform weighted processing on the decoding process of the initial detection feature map based on the attention weights to obtain target detection results for characterizing power transmission equipment and hanging objects.

9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the multi-target recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the multi-target recognition method as described in any one of claims 1-7.

Images