A multi-modal target image recognition method for a drone warning system

Abstract

The application discloses a multi-modal target image recognition method of a UAV warning system, relates to the technical field of image recognition, and aims to solve the technical problem that the edge gradient energy of a target and a background in an image collected by a visible light camera of a UAV in a low-light environment at night decreases and is submerged by noise, resulting in the inability to trigger a warning, the method comprising: performing edge-preserving smoothing processing on a monitoring image collected by the UAV, extracting a plurality of fragments representing local areas of potential targets based on the gradient direction distribution of each pixel in the smoothed image, identifying a fragment set belonging to the same presumed entity as a candidate target fragment set based on the physical space correlation between the fragments, and using the known boundaries of the fragments in the candidate target fragment set as anchor points to constrain the provision of external guidance by a boundary continuity probability graph to infer and complete the missing boundary regions between the fragments and generate a complete closed candidate target region.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a multimodal target image recognition method for a drone warning system. Background Technology

[0002] When drone warning systems perform security monitoring tasks, they typically need to process the surveillance images captured by the drones in real time. By analyzing pixel brightness and texture differences, areas inconsistent with the background are identified as potential targets. These areas are then subjected to contour extraction and size determination to identify specific targets (such as suspicious persons) and issue warnings. However, in low-light outdoor park patrol scenarios at night, drones face significant technical challenges when operating solely with visible light cameras. Summary of the Invention

[0003] This application provides a multimodal target image recognition method for a drone warning system, which aims to solve the technical problem that in low-light environments at night, the gradient energy of the target and background edges in images captured by the visible light camera of a drone decreases and is submerged by noise, resulting in the inability to obtain a complete closed contour in the target boundary extraction stage, thus causing personnel targets to be missed and warnings to be not triggered.

[0004] Firstly, to address the aforementioned technical problems, this invention provides a multimodal target image recognition method for a drone warning system. This method includes: performing edge-preserving smoothing processing on monitoring images acquired by the drone; constructing a boundary continuity probability map to characterize the possibility of weak physical boundaries based on the gradient direction distribution of each pixel in the smoothed image; weak physical boundaries refer to boundaries between the target and the background where the gradient magnitude is lower than a preset strong edge threshold due to similar reflectivity; extracting multiple fragments representing potential target local regions based on the boundary continuity probability map; and identifying fragment sets belonging to the same presumed entity as candidate target fragment sets based on the physical spatial correlation between fragments; potential target local regions refer to low-probability internal regions surrounded or semi-surrounded by high-probability boundaries in the boundary continuity probability map; and, under the constraint of known boundaries of fragments in the candidate target fragment set as anchor points, using external guidance provided by the boundary continuity probability map, inferring and completing missing boundary regions between fragments to generate complete closed candidate target regions for subsequent target recognition and warning determination.

[0005] Secondly, this application provides a target image recognition system for a drone warning system. The system includes: a construction unit, used to perform edge-preserving smoothing processing on monitoring images acquired by the drone, and based on the gradient direction distribution of each pixel in the smoothed image, construct a boundary continuity probability map to characterize the possibility of weak physical boundaries. Weak physical boundaries refer to boundaries between the target and the background where the gradient magnitude is lower than a preset strong edge threshold due to similar reflectivity; an extraction unit, used to extract multiple fragments representing potential target local regions based on the boundary continuity probability map, and based on the physical spatial correlation between fragments, identify fragment sets belonging to the same presumed entity as candidate target fragment sets. Potential target local regions refer to low-probability internal regions surrounded or semi-surrounded by high-probability boundaries in the boundary continuity probability map; and a processing unit, used to infer and complete missing boundary regions between fragments under the constraint of known boundaries of fragments in the candidate target fragment set as anchor points, utilizing external guidance provided by the boundary continuity probability map, to generate complete closed candidate target regions for subsequent target recognition and warning determination.

[0006] This application has at least the following beneficial effects: The target image recognition method for the UAV warning system disclosed in this application performs edge-preserving smoothing processing on the monitoring images collected by the UAV, and constructs a boundary continuity probability map to characterize the possibility of weak physical boundaries based on the gradient direction distribution of each pixel in the smoothed image. This effectively solves the problem that traditional edge detection methods cannot accurately identify weak boundaries in low-light environments at night because the gradient magnitude between the target and the background is lower than the preset strong edge threshold due to similar reflectivity. Furthermore, this method extracts multiple fragments representing local regions of potential targets based on the boundary continuity probability map, and identifies fragment sets belonging to the same presumed entity as candidate target fragment sets based on the physical spatial correlation between fragments. This overcomes the problem of fragmented recognition caused by incomplete target contours in existing technologies. Finally, under the known boundary of the fragments in the candidate target fragment set as anchor point constraints, the missing boundary regions between fragments are inferred and completed using the external guidance provided by the boundary continuity probability map, generating complete closed candidate target regions. This solves the technical problem in existing technologies where the target boundary extraction stage cannot obtain complete closed contours, resulting in the omission of personnel targets and failure to trigger warnings. Through the above technical solution, this application can significantly improve the target recognition accuracy and robustness of the UAV warning system in complex low-light environments, effectively avoid target miss detection, and enhance the system's early warning capability. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating a multimodal target image recognition method for an unmanned aerial vehicle (UAV) warning system provided in this application. Figure 2This is a flowchart illustrating another method for multimodal target image recognition in a drone warning system provided in this application; Figure 3 This is a schematic diagram of the structure of a target image recognition system for a drone warning system provided in this application. Detailed Implementation

[0008] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0009] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0010] When a drone warning system performs security monitoring tasks, traditional methods fail to identify personnel targets in low-light outdoor park patrol scenarios at night. When a stationary person approaches dark bushes with similar reflectivity, the edge gradient energy of the target and background objects decreases and is submerged by noise, resulting in the target boundary extraction process failing to obtain a complete closed contour. Consequently, the personnel target is missed and the warning cannot be triggered.

[0011] In view of the above problems, this application provides a multimodal target image recognition method for UAV warning systems. This application introduces steps such as edge-preserving smoothing processing, boundary continuity probability map construction, fragment extraction and association, and missing boundary inference and completion. It aims to effectively address the problem of unclear target and background boundaries in low-light and low-contrast environments, thereby improving the target recognition accuracy and robustness of UAV warning systems in complex environments.

[0012] The target image recognition method for the drone warning system provided in this application will be described in detail and explained through the following specific embodiments.

[0013] Reference Figure 1 This application provides a method for multimodal target image recognition in an unmanned aerial vehicle (UAV) warning system, which may include the following steps: S101. Perform edge-preserving smoothing processing on the monitoring images collected by the UAV, and construct a boundary continuity probability map to characterize the possibility of the existence of weak physical boundaries based on the gradient direction distribution of each pixel in the smoothed image.

[0014] Among them, weak physical boundaries refer to the boundaries between the target and the background where the gradient magnitude is lower than the preset strong edge threshold due to similar reflectivity.

[0015] S102. Based on the boundary continuity probability map, extract multiple fragments representing the local region of the potential target, and based on the physical spatial correlation between fragments, identify the set of fragments belonging to the same presumed entity as the set of candidate target fragments.

[0016] Among them, the potential target local region refers to the low-probability internal region that is surrounded or partially surrounded by high-probability boundaries in the boundary continuity probability map.

[0017] S103. Under the constraint of the known boundaries of the fragments in the candidate target fragment set as anchor points, the missing boundary regions between fragments are inferred and filled in using the external guidance provided by the boundary continuity probability map, so as to generate complete closed candidate target regions for subsequent target identification and warning judgment.

[0018] To better understand the method proposed in this application, some key terms involved will be explained first.

[0019] "Edge-preserving smoothing" refers to preserving edge information in an image as much as possible while performing noise reduction, avoiding edge blurring. This is crucial for subsequent boundary detection, especially in low-contrast scenes where subtle edge information is easily lost during the smoothing process.

[0020] A "boundary continuity probability map" is an image representation where the value of each pixel represents the probability that a weak physical boundary exists at that location. This probability map is constructed considering the gradient direction distribution of pixels, aiming to capture boundaries that have similar reflectivity, low gradient magnitudes, but exhibit a degree of continuity.

[0021] "Weak physical boundaries" specifically refer to boundaries between a target and the background where the gradient magnitude is lower than a preset strong edge threshold due to similar reflectivity. These types of boundaries are often difficult to identify effectively in traditional edge detection methods.

[0022] "Potential target local regions" refer to low-probability interior regions in a boundary continuity probability map that are surrounded or partially surrounded by high-probability boundaries. These regions typically correspond to the interior of a target, with low boundary continuity probability, while the exterior is composed of high-probability boundaries.

[0023] "Fragments" refer to image patches extracted from local areas of a potential target that represent a portion of the target region. These fragments are the basic units that constitute the complete target region.

[0024] "Physical spatial correlation" refers to the relative positions, distances, and directional relationships between fragments in actual physical space. By analyzing this correlation, fragments belonging to the same entity can be effectively aggregated.

[0025] "Anchor point constraint" refers to using the boundaries of identified fragments as fixed reference points during the boundary inference and completion process to guide the evolution and convergence of missing boundaries.

[0026] "External guidance" refers to using external information provided by the boundary continuity probability map to guide the active contour model to evolve towards high-probability boundary regions, thereby achieving accurate inference of missing boundaries.

[0027] The core of the target image recognition method for UAV warning systems proposed in this application lies in the effective perception, extraction, and completion of weak target boundaries in low-light and low-contrast environments.

[0028] Specifically, various filtering mechanisms can be employed when smoothing surveillance images captured by drones to preserve edges. For example, Gaussian filtering or mean filtering can be used for initial noise reduction. However, these traditional filtering methods often blur image edges while smoothing the image, leading to the loss of subtle boundary information. As another approach, nonlocal mean filtering can be used, which smooths each pixel by calculating a weighted average of all pixels in the image, where the weights depend on the similarity between pixels. This method can preserve edges to some extent, but its computational complexity is high.

[0029] When constructing a boundary continuity probability map to characterize the likelihood of weak physical boundaries, the first step is to calculate the gradient of the smoothed image to obtain the gradient magnitude and direction of each pixel. For example, the Sobel or Prewitt operators can be used to calculate the gradient information. These operators, through convolution operations, can effectively extract the edge strength and direction of the image. Alternatively, the Canny operator can be used, which performs non-maximum suppression and double thresholding while calculating the gradient to obtain finer edges. After obtaining the gradient information, a local sliding window can be set for each pixel, and the angular distribution of the gradient directions of each pixel within the window can be statistically analyzed to form a gradient direction distribution histogram. For example, the 360-degree gradient direction can be divided into multiple discrete intervals (such as 8 or 16), and the number of pixels in each interval can be counted. Based on the proportion of pixels with the dominant gradient direction within the window and the gradient magnitude of the central pixel, the boundary continuity probability value at the location of the central pixel is calculated. For example, a function can be designed that takes the percentage of pixels along the dominant gradient direction and the gradient magnitude of the center pixel as input and outputs a probability value between 0 and 1. The higher the percentage of pixels along the dominant gradient direction and the larger the gradient magnitude of the center pixel, the greater the probability value of boundary continuity at that position.

[0030] When extracting multiple fragments representing local regions of potential targets based on a boundary continuity probability map, a region growing algorithm can be used. For example, pixels with probability values ​​below a preset growth threshold can be manually selected as seed points. Then, pixels adjacent to the seed points and satisfying certain similarity criteria (such as probability values ​​below the growth threshold) are added to the region until no more new pixels can be added. Alternatively, a watershed algorithm can be used, which treats the image as a topographic map where pixel grayscale values ​​represent height, segmenting the image by simulating the convergence of water flow. After generating multiple fragment regions, internal features can be extracted from each fragment region. For example, the centroid coordinates, area, principal axis direction, bounding rectangle, grayscale mean, and grayscale co-occurrence matrix features of the fragment can be calculated. These features can be used for subsequent fragment association and target recognition.

[0031] When identifying a set of fragments belonging to the same presumed entity as a candidate target fragment set based on the physical spatial correlation between fragments, all extracted fragment regions can be constructed as an undirected weighted graph, where each node represents a fragment region. For example, an adjacency matrix can be used to represent the graph structure, where the elements of the matrix represent the connection weights between nodes. The connection weight is calculated for each pair of nodes in the graph, taking into account spatial distance constraints, orientation alignment constraints, and internal feature similarity. For example, spatial distance constraints can be simply measured by calculating the Euclidean distance between the centroids of fragments; orientation alignment constraints can be determined by comparing the angles between the principal axes of the fragments; and internal feature similarity can be measured by calculating the Bachian distance or chi-square distance of the fragment grayscale histograms. After calculating the connection weights, a pre-defined graph clustering algorithm can be used to merge fragment nodes with connection weights greater than a dynamic threshold into the same subgraph, and the merged fragment set is determined as the candidate target fragment set. For example, spectral clustering or hierarchical clustering algorithms can be used to perform fragment clustering.

[0032] When inferring and completing missing boundary regions between fragments in a candidate target fragment set, using the known boundaries of fragments as anchor points and external guidance provided by the boundary continuity probability map, the initial contour can be determined based on the convex hull of the global bounding polygon of the candidate target fragment set. For example, the convex hull of all fragment vertices can be calculated and used as the initial contour. Then, an active contour energy function composed of internal and external energy terms is constructed. The internal energy term can be used to control the smoothness of the initial contour; for example, the curvature and length of the contour can be used as the internal energy term. The external energy term can be defined by the boundary continuity probability map and used to attract the initial contour to evolve towards regions with high probability values; for example, the negative values ​​of the boundary continuity probability map can be used as the external energy term. The actual boundaries of all fragments in the candidate target fragment set are set as immovable anchor points. Under the constraint of the anchor points, the initial contour is driven to evolve and converge in the missing regions between fragments by iteratively minimizing the active contour energy function, forming a completed closed candidate target region. For example, gradient descent or finite difference methods can be used to minimize the energy function, thereby realizing the evolution of the contour.

[0033] In some of the above embodiments, edge-preserving smoothing of the surveillance images acquired by the UAV is a crucial step in constructing the boundary continuity probability map. However, in practical applications, images acquired by UAVs under different lighting and environmental conditions often contain complex noise, and there may be weak boundaries between the target and the background due to similar reflectivity. If traditional edge-preserving smoothing methods are used, they may not be able to effectively distinguish between real textures, weak boundaries, and noise in the image, resulting in over-smoothing or under-smoothing, which in turn affects the accuracy of the subsequent boundary continuity probability map.

[0034] In this regard, refer to Figure 2 This application further proposes the following steps for edge-preserving and smoothing the surveillance images acquired by the UAV: S201. Obtain the camera gain value at the time of acquisition corresponding to the monitoring image.

[0035] The camera gain value is used to determine the global noise variance estimate of the current image based on a pre-calibrated noise model.

[0036] S202. A bilateral filtering mechanism with a local texture variance adjustment factor is adopted to calculate the weight of each pixel in the image.

[0037] Specifically, for a central pixel whose absolute grayscale difference with neighboring pixels is less than the global noise variance estimate, the local texture variance of the central pixel is further determined: if the local texture variance is lower than a preset flatness threshold, the central pixel is determined to be in a pure noise region and is given a first smoothing weight; if the local texture variance is not lower than the preset flatness threshold and exhibits a specific directionality, the central pixel is determined to be in a potential boundary region and is given a second smoothing weight, the value of the second smoothing weight being less than the first smoothing weight.

[0038] Specifically, the camera gain value refers to the amplification factor used by the drone camera when acquiring images, reflecting the intensity of the image sensor's response to light signals. This gain value, combined with a pre-calibrated noise model, can accurately estimate the global noise variance of the current image. The global noise variance estimate characterizes the overall noise level in the image, providing a quantitative basis for subsequent adaptive smoothing. The bilateral filtering mechanism, which introduces a local texture variance adjustment factor, is an improved filtering method. Based on traditional bilateral filtering, it dynamically adjusts the smoothing weights by considering the local texture characteristics of pixels. For each pixel in the image, the weight calculation considers not only spatial distance and gray-level similarity but also introduces local texture variance as an adjustment factor. Specifically, when the absolute value of the gray-level difference between the center pixel and its neighboring pixels is less than the global noise variance estimate, it indicates that the area may be affected by noise, requiring further assessment of its local texture characteristics. In this case, if the local texture variance of the center pixel is lower than a preset flatness threshold, the center pixel is determined to be in a pure noise region. Pure noise regions typically exhibit minimal grayscale variations and lack clear directionality. In such cases, a larger initial smoothing weight should be assigned to achieve powerful denoising. Conversely, if the local texture variance is not lower than a preset flatness threshold and displays a specific directionality, the central pixel is determined to be within a potential boundary region. Potential boundary regions may contain weak edges between the target and the background. In these cases, a smaller second smoothing weight should be assigned to better preserve edge details while denoising, preventing over-smoothing from causing boundary blurring. The value of the second smoothing weight is smaller than the first smoothing weight, ensuring that the smoothness in potential boundary regions is lower than in pure noise regions, thereby effectively protecting weak boundaries.

[0039] Through the above technical solution, this application can achieve more refined and adaptive edge-preserving smoothing processing based on the actual noise level and local texture characteristics of the image. This not only effectively suppresses the complex noise commonly found in UAV images, but also avoids the loss of boundary information caused by over-smoothing when dealing with weak boundaries caused by similar reflectivity between the target and the background, thus significantly improving the quality of the smoothed image. Therefore, it provides a more accurate and reliable input for subsequently constructing a boundary continuity probability map to characterize the possibility of weak physical boundaries, thereby improving the robustness and recognition accuracy of the entire UAV warning system target image recognition method.

[0040] In some preferred embodiments, suppose a drone captures a surveillance image in a low-light environment. This image contains strong random noise and a faint target with a color similar to the background. First, the system obtains the camera gain value at the time of image capture. For example, a high gain value indicates that the image may have a high noise level. Based on a pre-calibrated noise model, the system calculates the global noise variance estimate of the current image based on the gain value, for example, 15. Next, when performing bilateral filtering on the image, for a certain center pixel, if the absolute value of its grayscale difference with neighboring pixels is less than 15, the system further checks the local texture variance of the center pixel. If the local texture variance is very small, for example, below a preset flatness threshold of 5, the pixel is determined to be located in a pure noise region and is assigned a large first smoothing weight (e.g., 0.8) to strongly remove noise. If the local texture variance is not less than 5, but exhibits a specific directionality—for example, a gradual change in grayscale in one direction but a slight change in the vertical direction—then the pixel is determined to be located in a potential boundary region and given a small second smoothing weight (e.g., 0.3) to preserve the weak boundary while smoothing noise. In this way, most of the noise in the image is effectively suppressed, while the weak boundary between the target and the background is well preserved, laying a good foundation for subsequent boundary detection and target recognition.

[0041] The above-mentioned construction of a boundary continuity probability graph to characterize the possibility of the existence of weak physical boundaries includes: At each pixel location of the smoothed image, the gradient magnitude and gradient direction are calculated; A local sliding window is set for each pixel, and the angular distribution of gradient directions of each pixel within the window is statistically analyzed to form a gradient direction distribution histogram. The gradient direction distribution histogram is used to determine whether there is a dominant gradient direction within the window and the percentage of pixels with that dominant gradient direction. Based on the proportion of pixels in the dominant gradient direction within the window and the gradient magnitude of the center pixel, calculate the boundary continuity probability value at the location of the center pixel. The higher the proportion of pixels in the dominant gradient direction and the larger the gradient magnitude of the center pixel, the greater the boundary continuity probability value at that location. Based on the flight altitude and camera focal length transmitted in real time by the UAV, the theoretical pixel size of the ground target in the monitored image is determined, a multi-scale image pyramid is constructed, the boundary continuity probability response is calculated at different scale layers, and the probability responses at multiple scales are fused into a unified boundary continuity probability map by a cross-scale maximum preservation strategy.

[0042] Specifically, at each pixel location in the smoothed image, its gradient magnitude and gradient direction are first calculated. The gradient magnitude reflects the intensity of the pixel's grayscale change, while the gradient direction indicates the direction of the grayscale change. This information forms the basis for subsequent determination of the likelihood of a boundary. The gradient magnitude can be calculated using edge detection algorithms such as the Sobel operator, Prewitt operator, or Canny operator, while the gradient direction is typically calculated using the arctangent function.

[0043] Furthermore, a local sliding window is defined for each pixel, such as a square or circular region of 3x3, 5x5, or larger. Within this window, the angular distribution of gradient directions for all pixels is statistically analyzed, forming a gradient direction distribution histogram. This histogram quantifies the concentration of gradient directions within the window, thereby determining whether one or more dominant gradient directions exist, and the proportion of pixels corresponding to these dominant gradient directions within the window. For example, the 360-degree gradient direction can be divided into several angular intervals (such as 8 or 16 intervals), and the number of pixels in each interval can be counted.

[0044] Based on this, the probability value of boundary continuity at the location of the center pixel is calculated according to the pixel proportion of the dominant gradient direction within the window and the gradient magnitude of the center pixel. A higher pixel proportion of the dominant gradient direction indicates a more consistent gradient direction within the local region, making a continuous boundary more likely. A larger gradient magnitude of the center pixel indicates a more pronounced grayscale change at that location. Therefore, a higher pixel proportion of the dominant gradient direction and a larger gradient magnitude of the center pixel result in a higher probability value of boundary continuity at that location, and vice versa. This calculation method effectively distinguishes between isolated gradients caused by noise and continuous gradients caused by true boundaries.

[0045] Furthermore, considering that the pixel size of ground targets in monitoring images varies significantly depending on the UAV's flight altitude and the use of cameras with different focal lengths, this application introduces a multi-scale processing mechanism. Specifically, based on the UAV's real-time downlinked flight altitude and camera focal length, the theoretical pixel size range of the ground target in the monitoring image can be determined. Based on this, a multi-scale image pyramid is constructed, for example, by generating a series of image layers with different resolutions through Gaussian filtering and downsampling operations. At each scale layer, the aforementioned gradient calculation, gradient direction distribution histogram construction, and boundary continuity probability value calculation are performed respectively, thereby obtaining the boundary continuity probability response of the corresponding scale layer. Finally, through a cross-scale maximum preservation strategy, the probability responses at multiple scales are fused into a unified boundary continuity probability map. This means that for each spatial location in the image, its final boundary continuity probability value is taken from the maximum probability response at that location across all scale layers, thus ensuring that weak boundaries can be effectively captured regardless of the target size.

[0046] Through the above technical solution, this application can generate a more accurate and robust boundary continuity probability map. In particular, by combining the UAV flight altitude and camera focal length information, and employing multi-scale image pyramid processing, the system can adaptively process targets of different sizes and distances, effectively capturing weak physical boundaries with low gradient magnitudes due to similar reflectivity. This significantly improves the accuracy of subsequent target recognition and warning determination, reduces the risk of missed and false alarms, and is particularly suitable for complex and ever-changing UAV monitoring environments.

[0047] In some preferred embodiments, it is assumed that a drone monitors the ground at different altitudes. When the drone flies at a higher altitude, ground targets appear as smaller pixels in the monitored image; when the drone flies at a lower altitude, the targets appear as larger pixels. Traditional single-scale boundary detection methods may perform well at one altitude but poorly at another. The solution in this application accurately calculates the theoretical pixel size range of ground targets in the current image by acquiring the drone's flight altitude and camera focal length in real time. Based on this, an image pyramid containing multiple scale layers is constructed; for example, a three-layer pyramid containing the original image, a half-scaled image, and a quarter-scaled image can be constructed. At each layer, the gradient magnitude and gradient direction are calculated, and a gradient direction distribution histogram is statistically analyzed using a local sliding window (e.g., the window size can be adjusted according to the current scale layer to adapt to features at different scales), thereby calculating the boundary continuity probability value. Finally, by comparing the probability values ​​of the same spatial location at different scale layers and retaining the maximum value, a boundary continuity probability map that integrates multi-scale information is generated. For example, for a small target in the original image, its boundary may exhibit a higher continuity probability at a smaller scale layer of the pyramid; while for a larger target, its boundary may exhibit a higher continuity probability at the original scale layer. This cross-scale maximum preservation strategy ensures that weak boundaries can be effectively enhanced and accurately represented in the final probability map regardless of the target size.

[0048] Specifically, the extraction of multiple fragments representing local regions of a potential target includes: Using a preset region growth algorithm or watershed algorithm, pixels with probability values ​​below a preset growth threshold in the boundary continuity probability map are used as seed points for growth. Growth stops when the growth front touches a pixel with a probability value above the preset boundary threshold, thereby generating multiple fragment regions. For each generated fragment region, its internal features are extracted. The internal features include the centroid coordinates, area, principal axis direction, circumscribed rectangle, gray-level mean, and gray-level co-occurrence matrix features of the fragment.

[0049] Among these, the region growing algorithm or watershed algorithm are commonly used methods for image segmentation. The region growing algorithm merges adjacent pixels that satisfy specific similarity criteria into the same region, starting from one or more seed points. The watershed algorithm treats the image as a topographic map, where gray values ​​represent height, and segments regions by simulating the convergence of water flows. The purpose of choosing these two algorithms is to effectively identify connected regions with similar characteristics from the boundary continuity probability map; these regions are considered local fragments of potential targets.

[0050] The preset growth threshold is used to determine which pixels can serve as starting points for region growth; that is, pixels with lower boundary continuity probability values, indicating that they are more likely to be uniform regions within the target. The preset boundary threshold is used to define the stopping condition for region growth; that is, when the growth front encounters a pixel with a probability value higher than the threshold, it indicates that a potential boundary has been reached, and growth should stop to avoid crossing the target boundary. Therefore, the generated fragmented regions can more accurately reflect the shape and extent of the local target region.

[0051] Furthermore, for each generated fragment region, its internal features are extracted. These internal features include the fragment's centroid coordinates, area, principal axis direction, circumscribed rectangle, grayscale mean, and grayscale co-occurrence matrix (GLCM) features. Specifically, the centroid coordinates represent the geometric center position of the fragment, used for spatial positioning and distance calculation; the area represents the number of pixels occupied by the fragment, reflecting its size; the principal axis direction represents the main extension direction of the fragment, which is important for identifying target fragments with specific shapes; the circumscribed rectangle describes the minimum bounding rectangle of the fragment, providing information on its size and approximate shape; the grayscale mean represents the average grayscale value of the pixels within the fragment, reflecting the overall brightness characteristics of the fragment; and the GLCM features are a set of statistical features used to describe image texture, such as energy, contrast, correlation, and entropy. By analyzing the GLCM, the roughness, directionality, and regularity of the fragment's internal texture can be quantified, thus more precisely characterizing the visual properties of the fragment. The purpose of extracting these features is to provide rich information for subsequent fragment association and identification.

[0052] The above technical solution enables precise extraction of local regions of potential targets and provides a comprehensive internal feature description for each fragment. This method not only effectively addresses the weak boundary problem caused by the similar reflectivity of the target and background, ensuring the accuracy of fragment extraction by utilizing the guidance of boundary continuity probability maps, but also greatly enhances the richness of fragment representation by extracting multi-dimensional internal features. This lays a solid foundation for subsequent fragment association and identification of candidate target fragment sets, thereby improving the robustness and accuracy of target image recognition in UAV warning systems.

[0053] In response, this application further proposes a method for identifying a set of fragments belonging to the same presumed entity as a set of candidate target fragments based on the physical spatial correlation between fragments, specifically including: All extracted fragment regions are constructed into an undirected weighted graph, where each node represents a fragment region; For each pair of nodes in the graph, a connection weight is calculated. The calculation of the connection weight integrates spatial distance constraints, orientation alignment constraints, and internal feature similarity. The spatial distance constraint refers to the constraint based on the UAV flight altitude and camera intrinsic parameters, which converts the pixel distance between fragments into actual physical distance. The internal feature similarity refers to the measurement of the Mahalanobis distance between fragment feature vectors by using the texture covariance matrix of the background region. A preset graph clustering algorithm is used to merge fragment nodes with connection weights greater than a dynamic threshold into the same subgraph, and the merged fragment set is determined as the candidate target fragment set. The dynamic threshold is adaptively determined based on the overall contrast and noise level of the current image.

[0054] Specifically, when constructing an undirected weighted graph, each fragment region generated by a preset region growing algorithm or watershed algorithm is considered an independent node in the graph. This graph structure can intuitively represent the potential relationships between fragments.

[0055] Among these, calculating the connection weight is crucial for evaluating the strength of the association between two fragments. This weight comprehensively considers multiple dimensions: First, there's the spatial distance constraint, designed to ensure that only fragments that are physically close enough can be considered part of the same entity. To eliminate the impact of drone altitude variations on pixel distance, the pixel distances between fragments are precisely converted into actual physical distances using the drone's real-time downlinked flight altitude and pre-calibrated camera intrinsics. For example, a camera projection model can be used to convert image coordinates to world coordinates, thus obtaining more accurate physical distances.

[0056] Secondly, there is the orientation alignment constraint, which aims to evaluate whether the principal axis directions or dominant texture directions of two fragments are consistent or approximately parallel. For example, if two fragments belong to different parts of a slender target, their principal axis directions tend to be consistent.

[0057] Furthermore, there is the internal feature similarity, which measures the degree of similarity in features such as texture and color within fragments. To more robustly measure this similarity, this application utilizes the texture covariance matrix of the background region to calculate the Mahalanobis distance between fragment feature vectors. The Mahalanobis distance can take into account the correlation between features and normalize the scale of different features, thus providing a more accurate similarity measure than Euclidean distance in the presence of noise or uneven illumination. The texture covariance matrix of the background region can be obtained by sampling and statistically analyzing non-target regions in the image, reflecting the statistical characteristics of the image background texture.

[0058] In practical applications, after calculating the connection weights between all fragment pairs, a pre-defined graph clustering algorithm is used to process the graph. This graph clustering algorithm can be spectral clustering, hierarchical clustering, or density-based clustering, etc., and its goal is to merge fragment nodes with connection weights greater than a dynamic threshold into the same subgraph. Each subgraph represents a potential set of candidate target fragments.

[0059] The dynamic threshold is adaptively determined based on the overall contrast and noise level of the current image. For example, when the image contrast is low or the noise level is high, the dynamic threshold can be appropriately lowered to allow weaker connections to be identified, thereby preventing target fragments from being incorrectly segmented due to poor image quality; conversely, when the image quality is good, the threshold can be appropriately increased to reduce false clustering. This adaptive mechanism enhances the algorithm's robustness to different environmental conditions.

[0060] Through the above technical solutions, this application significantly improves the ability to accurately identify and aggregate candidate target regions from weak boundary fragments under complex backgrounds and variable environmental conditions. Specifically, the multi-dimensional connection weight calculation, especially the spatial distance constraint that converts pixel distance into physical distance and the Mahalanobis distance based on the background texture covariance matrix, makes the correlation judgment between fragments more accurate and robust, effectively overcoming the shortcomings of traditional methods when dealing with challenges such as similar reflectivity between the target and the background, uneven illumination, or partial occlusion. In addition, the introduction of dynamic thresholds allows the graph clustering process to adaptively adjust its sensitivity, thereby achieving optimal fragment merging results under different image quality and noise levels, reducing the risk of false recognition and missed recognition. Ultimately, the solution of this application can generate more complete and accurate candidate target regions, providing high-quality input for subsequent target recognition and warning determination, thereby improving the overall performance and reliability of the UAV warning system.

[0061] In some preferred embodiments, a specific example is given below. Suppose a drone is monitoring a person wearing clothing similar in color to the ground in low-visibility conditions, such as fog or dusk. In this situation, because the reflectivity of the target and the background are similar, and the lighting conditions are poor, traditional edge detection methods may only be able to detect a weak portion of the target's boundary and segment it into multiple discontinuous fragments.

[0062] Based on the above method, fragments representing local areas of the human body, as well as some fragments that may be caused by background noise or irrelevant objects, are first extracted. These fragments are then constructed into an undirected weighted graph. For each pair of fragment nodes in the graph, their connection weights are calculated. For example, if two fragments represent the human torso and head respectively, they will be relatively close in physical space, and their principal axes may exhibit some alignment. Simultaneously, by utilizing the texture covariance matrix of the background region (e.g., ground, vegetation), the Mahalanobis distance between the internal features (such as gray-level co-occurrence matrix features) of these two fragments is calculated, revealing that they have high similarity, while their similarity to background fragments (such as fog, leaves) is low.

[0063] Furthermore, considering that the overall contrast of the current image may be low due to fog, the system adaptively determines a relatively lenient dynamic threshold. Then, a graph clustering algorithm is used to merge fragment nodes with connection weights greater than this dynamic threshold into the same subgraph. For example, fragments belonging to the human body, such as torso fragments, head fragments, and arm fragments, will be successfully clustered into one subgraph, forming a complete set of candidate target fragments, while fragments related to the background will be excluded. In this way, even under harsh monitoring conditions, the proposed solution can effectively aggregate scattered and weak fragments into highly reliable candidate target regions, laying a solid foundation for subsequent warning determination.

[0064] In some of the embodiments described above in this application, the calculation of internal feature similarity is a crucial step in identifying a set of fragments belonging to the same presumed entity based on the physical spatial correlation between fragments. However, for fragments with complex or anisotropic textures, traditional gray-level co-occurrence matrix feature extraction methods may not be able to fully capture their unique texture orientation information. This could lead to insufficient accuracy in calculating the internal feature similarity for fragments with specific texture orientations during the analysis of physical spatial correlation between fragments, thereby affecting the accuracy of identifying a set of fragments belonging to the same presumed entity.

[0065] In response, this application further proposes to optimize the calculation of internal feature similarity in order to improve the accuracy of fragment identification.

[0066] The calculation of the internal feature similarity includes: Principal component analysis is performed on the pixels within each fragment region to determine its dominant texture direction; When calculating the gray-level co-occurrence matrix features of the fragment region, feature extraction is performed only along the dominant texture direction and in directions perpendicular to the dominant texture direction.

[0067] Specifically, when performing principal component analysis (PCA) on pixels within each fragment region, statistical analysis can be performed on the grayscale values ​​or local gradient information of all pixels within the fragment region to identify the direction of the most significant pixel distribution or texture change within that region, i.e., the dominant texture direction. Principal component analysis (PCA) is a commonly used dimensionality reduction technique that can effectively extract key features from high-dimensional data and determine the dominant direction of data change. Through PCA, the anisotropy of texture within fragment regions can be quantified, and its main directional features can be determined.

[0068] Gray-Level Co-occurrence Matrix (GLCM) is a statistical texture analysis method that describes texture by calculating the frequency of gray-level values ​​between two pixels in an image that have a specific spatial relationship (e.g., distance and angle). Traditional GLCM feature extraction typically considers multiple directions (e.g., 0°, 45°, 90°, 135°) and averages or combines features from these directions. However, when a fragment region has a clear dominant texture direction, extracting features only along this dominant texture direction and its perpendicular direction can more effectively capture the key texture information of the fragment region, avoiding the dilution or interference of texture information from irrelevant directions on the feature representation. For example, for a long, thin target fragment, its texture often exhibits stronger continuity or repetition along the long axis, while it may show different variation patterns along the short axis. By focusing on these key directions, more representative and discriminative texture features can be obtained.

[0069] Through the above technical solution, this application can significantly improve the accuracy and robustness of fragment internal feature similarity calculation. Because the feature extraction process focuses more on the key texture directions of the fragment region, the obtained gray-level co-occurrence matrix features can more accurately characterize the intrinsic texture attributes of the fragments, especially for fragments with specific directional textures or low-contrast targets, where the discriminative power of their texture features is enhanced. Therefore, when identifying candidate target fragment sets based on the physical spatial correlation between fragments, fragments belonging to the same presumed entity can be merged more accurately, effectively reducing false or missed clustering of fragments due to inaccurate texture feature descriptions. This lays a more solid foundation for the subsequent generation of complete closed candidate target regions, improving the overall performance and reliability of the target image recognition method for the entire UAV warning system.

[0070] In some preferred embodiments, it is assumed that there is a slender, low-contrast humanoid target in the drone surveillance image, whose body is segmented into multiple fragmented regions. One of these fragmented regions may correspond to the target's torso, whose internal texture (e.g., clothing folds) exhibits a generally vertical orientation.

[0071] First, principal component analysis was performed on the pixels within the torso fragment region. By analyzing the pixel grayscale distribution, it was determined that the dominant texture direction in this fragment region is approximately vertical.

[0072] Next, when calculating the gray-level co-occurrence matrix features of the fragment region, instead of averaging across multiple directions such as 0°, 45°, 90°, and 135°, feature extraction is performed only along the dominant vertical texture direction (e.g., 90°) and its perpendicular direction (e.g., 0°). For example, gray-level co-occurrence matrix features such as energy, contrast, and entropy in the 90° direction, as well as the corresponding features in the 0° direction, can be calculated.

[0073] In this way, the extracted gray-level co-occurrence matrix features can more accurately capture the texture continuity along the vertical direction and the texture variation along the horizontal direction of the torso fragments, thus forming a more representative feature vector. When comparing the internal feature similarity with other fragments (such as head or leg fragments), this precise texture feature can more accurately reflect whether they belong to the same target entity. Even when the target and background reflectance are similar and the boundaries are weak, it can effectively distinguish and associate fragments, ultimately helping to form a more complete and accurate candidate target region.

[0074] In some of the embodiments described above in this application, although multiple fragments representing local regions of potential targets can be extracted based on the boundary continuity probability map, and a set of candidate target fragments can be identified based on the physical spatial correlation between the fragments, when inferring and completing the missing boundary regions between fragments, it may be difficult to accurately reconstruct the complete outline of the target if only simple connection or interpolation methods are used, especially when the contrast between the target and the background is low, the boundary is blurred, or there is a complex shape, which can easily lead to inaccurate or incomplete candidate target regions after completion. If the above problems are not solved, the accuracy of subsequent target recognition and warning determination will be affected. In this regard, this application further proposes a boundary inference and completion method based on an active contour model and anchor point constraints to generate a more complete and accurate closed candidate target region.

[0075] In response, this application further proposes a method for inferring and completing missing boundary regions between fragments, using the known boundaries of fragments in the candidate target fragment set as anchor constraints and external guidance provided by the boundary continuity probability map. Specifically, this method includes: The initial contour is determined based on the convex hull of the global bounding polygon of the candidate target fragment set; An active contour energy function is constructed, consisting of an internal energy term and an external energy term. The internal energy term is used to control the smoothness of the initial contour, and the external energy term is defined by the boundary continuity probability map and is used to attract the initial contour to evolve towards the region with a high probability value. The actual boundaries of all fragments in the candidate target fragment set are set as immovable anchor points. Under the constraint of the anchor points, the active contour energy function is minimized iteratively to drive the initial contour to evolve and converge in the missing region between fragments, forming a closed candidate target region after completion.

[0076] Specifically, when inferring and completing the missing boundary regions between fragments, firstly, based on the identified set of candidate target fragments, the convex hull of its global bounding polygon is calculated, and this convex hull is used as the initial contour for active contour evolution. This initial contour provides a coarse boundary estimate encompassing all fragments. Secondly, an active contour energy function is constructed, consisting of an internal energy term and an external energy term. The internal energy term aims to maintain the smoothness and continuity of the contour, avoiding unnatural sharpness or breaks, and controls the geometric characteristics of the initial contour. The external energy term utilizes information provided by the boundary continuity probability map to guide the contour to move towards regions with high boundary probabilities in the image, i.e., attracting the contour closer to the real object boundaries. Furthermore, known and actual fragment boundaries in the candidate target fragment set are set as immovable anchor points. These anchor points act as strong constraints, ensuring that the active contour does not deviate from the determined target portion during evolution. Finally, under the strict constraints of these anchor points, the initial contour is driven to evolve in the missing regions between fragments by iteratively minimizing the constructed active contour energy function. As the iterations continue, the outline will gradually converge to the real, complete object boundary, eventually forming a completed closed candidate target region.

[0077] Through the above technical solution, this application can significantly improve the accuracy and robustness of target boundary inference and completion in UAV surveillance images. Especially for targets with weak or discontinuous boundaries due to similar reflectivity, low contrast, or partial occlusion, this method can effectively reconstruct complete and accurate closed target regions by utilizing global guidance from the boundary continuity probability map and local constraints from fragment anchor points. This provides high-quality input for subsequent target recognition and warning determination, thereby improving the performance and reliability of the entire UAV warning system.

[0078] In some preferred embodiments, suppose a UAV acquires a surveillance image in a low-visibility environment, where the boundary of a ground target (e.g., a person wearing clothing similar in color to the ground) appears as multiple discontinuous, low-contrast fragments. First, the system constructs a convex hull of a globally bounding polygon as an initial contour based on these fragments. Then, a boundary continuity probability map (enhanced with edge-preserving smoothing and gradient direction analysis to improve weak boundary information) is used as an external guide, while the identified fragment boundaries are fixed as anchor points. During active contour evolution, internal energy terms ensure contour smoothness, while external energy terms attract the contour to move towards high-probability regions in the boundary continuity probability map. Due to the anchor points, the contour strictly follows known fragment boundaries during evolution, inferring and completing only in the gaps between fragments. For example, if the target's torso and head are identified as two fragments, the active contour smoothly connects and completes the missing neck and shoulder regions between these two fragments along the path indicated by the boundary continuity probability map, ultimately forming a complete, closed human contour.

[0079] In some embodiments described above, this application proposes using the known boundaries of fragments in a candidate target fragment set as anchor points, and leveraging external guidance provided by a boundary continuity probability map to infer and complete missing boundary regions between fragments, thereby generating a fully closed candidate target region. However, in practical applications, especially in complex backgrounds or low-contrast scenes, if there are strong interference edges (such as tree branches, fences, etc.) unrelated to the target near the missing boundary region, the active contour may be attracted by these external strong interference edges during the evolution process, causing the contour to deviate from the true target boundary, thus affecting the accuracy and completeness of the final candidate target region.

[0080] In response, this application further proposes that after inferring and completing the missing boundary regions between fragments, the method also includes: When there is a missing area between two anchor points in the candidate target fragment set that is not covered by fragments, the two anchor points are identified as a pair of anchor points that need to be inferred from the boundary. A dynamic evolution corridor is constructed for the pair of anchor points. The process of constructing the dynamic evolution corridor is as follows: the centroid line connecting the pair of anchor points is used as the central axis of the corridor. The lateral boundary of the corridor is determined according to the average width of the pair of anchor points and the preset human body structure ratio. The inner area of ​​the corridor is defined as the legal evolution zone, and the outer area of ​​the corridor is defined as the strong constraint zone. Before active contour evolution, a corridor mask is used to spatially modulate the original boundary continuity probability map, forcibly attenuating the probability value of pixels located in the strongly constrained region to zero, so as to shield the attraction of external strong interference edges. The active contour evolution is a process of driving the evolution of the initial contour by iteratively minimizing the active contour energy function. The initial contour evolves in the spatially modulated boundary continuity probability map, so that the initial contour converges to the true boundary within the legal evolution region.

[0081] Specifically, when there is an area uncovered by fragments between two anchor points in the candidate target fragment set, these two anchor points are identified as a pair of anchor points requiring boundary inference. For example, if a target (such as a human body) is partially occluded, different parts of its body may be identified as independent fragments, and the connection points between these fragments are the missing boundary areas, with the edges of these fragments serving as anchor points. For such a pair of anchor points, a dynamic evolution corridor is constructed. The construction process of this corridor is as follows: First, the line connecting the centroids of the pair of anchor points is used as the central axis of the corridor, providing basic directional guidance. Second, the lateral boundary of the corridor is determined based on the average width of the pair of anchor points and a preset proportion of the human body structure. For example, if the target is a human body, the width of the corridor can be set according to the typical width proportions of different parts of the human body to ensure that the corridor can reasonably cover the missing boundary areas of the human body. Thus, the area inside the corridor is designated as a legal evolution zone, meaning that the evolution of active contours within this area is as expected; while the area outside the corridor is designated as a strongly constrained zone, indicating that pixels within this area should not attract active contours.

[0082] Before active contour evolution, a corridor mask is used to spatially modulate the original boundary continuity probability map. Specifically, the probability values ​​of pixels located within the strongly constrained region are forcibly decayed to zero. This spatial modulation effectively shields the attractive force of external strong interference edges that may exist within the strongly constrained region. The active contour evolution process, which drives the initial contour evolution by iteratively minimizing the active contour energy function, takes place on the spatially modulated boundary continuity probability map. This means that the evolution of the active contour will be primarily guided by the true boundary within the legal evolution region, allowing the initial contour to converge more accurately to the true boundary within the legal evolution region.

[0083] The above technical solution significantly improves the accuracy and robustness of candidate target region boundary inference and completion in complex backgrounds or low-contrast scenes. By introducing a dynamic evolution corridor and spatial modulation mechanism, this solution effectively suppresses the influence of strong external interference edges on the active contour evolution, ensuring that the active contour converges more accurately to the true target boundary and avoiding contour deviation or distortion. As a result, the generated candidate target regions are more complete and accurate, providing high-quality input for subsequent target recognition and warning determination, thereby improving the overall performance and reliability of the UAV warning system.

[0084] In some preferred embodiments, assuming the drone is monitoring ground personnel at night or in foggy weather, the contrast between the personnel target and the background is extremely low due to insufficient light or low visibility, and the background may contain strong interfering edges such as tree and building edges. In this case, after identifying multiple fragments of the person using the above method, if there are missing areas not covered by fragments between the person's torso or limbs, the system will identify the anchor points corresponding to these missing areas. For example, if a person's arm and torso are identified as two fragments, the connection point between them is the missing area, and the edges of the arm and torso serve as anchor points. For this pair of anchor points, the system constructs a dynamically evolving corridor. This corridor uses the line connecting the centroids of the anchor points as its central axis and determines its lateral boundary based on a preset connection width ratio between the human arm and torso. Before active contour evolution, the boundary continuity probability value of the outer region of the corridor (strong constraint region) is forcibly decayed to zero, thereby shielding it from the attraction of tree branches or building edges in the background. In this way, the active contour evolves only within the internal area of ​​the corridor (the legal evolution zone) and eventually converges to the real boundary between the person's arm and torso. Even if this boundary is very weak, it can be accurately inferred and completed to form a completely closed target area for the person.

[0085] While some of the embodiments described above in this application can generate completely closed candidate target regions, in practical applications, relying solely on the region recognition results of a single frame image for warning determination may lead to false alarms or missed alarms, especially in complex backgrounds or when the target is briefly occluded. The robustness and reliability of the system need further improvement. If these problems are not addressed, the warning system may frequently trigger invalid alarms or fail to respond promptly to real threats, thus affecting the actual effectiveness of the UAV warning system. Therefore, this application further proposes a scheme for multi-dimensional evaluation and inter-frame verification of candidate target regions to improve the accuracy of target recognition and the reliability of warning determination.

[0086] The above methods also include: The candidate target region is evaluated intra-frame, and the overall grayscale distribution statistical difference between the pixels inside the candidate target region and the adjacent external background pixels is compared. A first confidence score is calculated based on the shape compactness and aspect ratio of the candidate target region. The first confidence score is used as prior information input into the inter-frame verification process. In the inter-frame verification process, the candidate target region generated in the current frame is associated with the expected position predicted by the target state in the historical frame through a Kalman filter. If the association is successful, the second confidence score of the current frame is updated by combining the historical confidence score with the first confidence score. If the association fails for multiple consecutive frames, the target is determined to be lost and the trajectory is terminated. When the second confidence score exceeds the preset warning threshold, target information including target identifier, bounding box coordinates and geographic coordinates is output to trigger a warning.

[0087] Specifically, intra-frame overall evaluation refers to the real-time analysis of candidate target regions identified within a single image frame. This evaluation determines whether the region possesses visually distinct characteristics from the background by comparing the overall grayscale distribution statistical differences between pixels within the candidate target region and its adjacent background pixels. For example, statistical measures such as grayscale mean, variance, or histogram differences can be calculated. Simultaneously, a first confidence score is calculated based on geometric features such as the shape compactness and aspect ratio of the candidate target region. Shape compactness measures the regularity of the region, while aspect ratio determines whether the region conforms to the typical proportions of a specific target; for example, for human targets, the aspect ratio typically falls within a specific range. This first confidence score reflects the initial probability that the candidate target region in the current frame is a real target and is passed as prior information to subsequent inter-frame verification stages.

[0088] The inter-frame verification stage aims to further verify and enhance the reliability of target recognition using time-series information. In this stage, candidate target regions generated in the current frame are attempted to be correlated with the expected positions predicted by a Kalman filter based on the target states in historical frames. The Kalman filter is a commonly used state estimation algorithm that can predict the future position of a target based on its motion model and observation data, thus achieving smoothing and prediction of the target trajectory. If the candidate target region in the current frame is successfully correlated with the predicted position, it indicates that the region is likely the same entity as the target in the historical frame. At this point, the system combines the historical confidence score with the first confidence score of the current frame, using a weighted average or other fusion strategy to update the second confidence score of the current frame. This second confidence score integrates the target's continuity in the time dimension and the visual features of a single frame, exhibiting higher robustness. If the target fails to be correlated successfully for multiple consecutive frames, the system determines that the target has been lost and terminates its trajectory to avoid invalid tracking of a disappeared target.

[0089] In practical applications, when the second confidence score exceeds a preset warning threshold, it indicates that the system has a high degree of confidence in identifying the target. At this time, the system will output target information including the target identifier, bounding box coordinates, and geographic coordinates. The target identifier is used to uniquely identify the target, the bounding box coordinates provide the target's precise location and size in the image, and the geographic coordinates provide the target's spatial location in the real world. This information can be used to trigger the warning system, such as issuing an alarm to the operator, automatically adjusting the drone's attitude, or initiating other countermeasures.

[0090] Through the above technical solutions, this application can significantly improve the accuracy of target image recognition and the reliability of warning determination in UAV warning systems. The strategy of combining intra-frame overall evaluation with inter-frame verification enables the system to effectively filter out noise and interference that may exist in a single frame image, reducing the false alarm rate. Simultaneously, through Kalman filter prediction of target trajectories and data association, the system's ability to determine the continuous presence of targets is enhanced. Even in complex environments or when targets are briefly occluded, stable tracking and recognition can be maintained, thereby reducing missed alarms. Therefore, this application provides a more robust and intelligent target recognition and warning determination mechanism, enabling UAV warning systems to respond to potential threats more accurately and promptly, greatly improving the system's practical value and reliability.

[0091] In some preferred embodiments, it is assumed that the drone is monitoring an area and has detected a potential target.

[0092] At a certain time T1, the image acquired by the UAV, after edge-preserving smoothing, boundary continuity probability map construction, fragment extraction and completion, generates a candidate target region. At this time, the system performs an intra-frame overall evaluation of the candidate target region, calculates the difference in gray-level distribution between its internal pixels and external background pixels, and combines its shape compactness and aspect ratio to obtain a first confidence score, for example, 0.6.

[0093] At the next time step T2, the system generates another candidate target region. Simultaneously, based on the target state at time T1, the Kalman filter predicts the target's expected location at time T2. If the candidate target region at time T2 successfully correlates with the predicted location, the system combines the historical confidence score at time T1 (e.g., assuming the second confidence score at time T1 is also 0.6, or initially set to 0.5) and the first confidence score at time T2 (e.g., 0.7), updating the second confidence score at time T2, for example, to 0.65.

[0094] At time T3, the system continues the above process. If the first confidence score at time T3 further increases to 0.8 and is successfully associated with the predicted position, the second confidence score may be updated to 0.75.

[0095] If the preset warning threshold is 0.7, then at time T3, since the second confidence score of 0.75 exceeds the threshold, the system will immediately output target information including target identifier, bounding box coordinates and geographic coordinates, and trigger a warning.

[0096] Conversely, if at any given moment, due to a sudden change in lighting or a brief occlusion, the first confidence score of a frame is low (e.g., 0.3), and it fails to be successfully associated with the trajectory predicted by the Kalman filter, or fails to be successfully associated with multiple consecutive frames, the system will determine that the target is lost and stop issuing warnings. This mechanism effectively avoids false alarms caused by poor image quality in a single frame or transient interference, ensuring the accuracy and timeliness of warnings.

[0097] In some embodiments, after generating a completely closed candidate target region, the target image recognition method for the UAV warning system provided in this application further includes the following steps: constructing a multi-dimensional feature description model of the target through cross-modal correlation analysis of visible light, thermal infrared and multispectral data; and combining a lightweight Transformer network and a dynamic threshold optimization algorithm to perform target recognition and warning determination for target scenes with environmental complexity greater than a preset threshold.

[0098] Specifically, after the drone warning system generates a completely closed candidate target area, the system simultaneously acquires visible light, thermal infrared, and multispectral images of the same scene using an onboard multimode sensor. The multimode sensor has completed time synchronization and optical axis registration at the hardware level, ensuring that the three image data streams are strictly aligned in the spatiotemporal dimensions.

[0099] In the cross-modal correlation analysis stage, the system uses the completed candidate target region as the region of interest. It extracts gradient direction histogram features and local binary mode features from the visible light image of this region, temperature distribution statistical features and thermal radiation gradient features from the thermal infrared image, and reflectance ratio features from the near-infrared and short-wave infrared bands from the multispectral image. These three types of features together constitute the first feature vector of the target multi-dimensional feature description model. This feature vector jointly represents the target from three dimensions: texture, temperature, and material properties.

[0100] Cross-modal correlation analysis also includes feature-level complementarity evaluation. The system calculates the mutual information between visible light features and thermal infrared features. When the confidence of texture features in the visible light channel decreases due to low illumination, the weight of the temperature feature in the thermal infrared channel is automatically increased; when the contrast of the thermal infrared channel decreases due to ambient temperature saturation, the weight of the material reflectance ratio feature in the multispectral channels is automatically increased. Through adaptive weight allocation, the system generates a second feature vector after cross-modal fusion, which serves as the input to the subsequent network model.

[0101] The lightweight Transformer network employs an encoder-decoder architecture. The encoder consists of three cascaded lightweight attention modules, each containing a multi-head self-attention layer and a feedforward neural network layer. Unlike the standard Transformer, the multi-head self-attention layer in this network uses a local window attention mechanism. Each query vector only performs attention calculations with the key vector within a preset-sized spatial window, reducing the computational complexity of self-attention calculations from being proportional to the square of the feature map size to being linearly related to the feature map size.

[0102] In the inference process, the system combines the cross-modal fusion feature vector generated in Example 1 with the shape compactness, aspect ratio, and spatial coordinates of the candidate target region to form a unified input embedding vector. This input embedding vector is then positionally encoded and input to the encoder of the lightweight Transformer network. The encoder outputs a target feature representation with context awareness. The decoder receives the encoder's output and outputs a confidence score indicating whether the target belongs to the real person category through a fully connected classification head.

[0103] To further reduce computational overhead, the feedforward neural network layer uses depthwise separable convolutions instead of fully connected operations, compressing the number of parameters to less than one-third of that of a standard Transformer. This lightweight design enables the network to achieve real-time inference on an embedded computing unit on a UAV, without relying on ground stations or cloud computing power.

[0104] Environmental complexity assessment is a prerequisite for dynamic threshold optimization. The system defines environmental complexity as a weighted composite index across three dimensions: the first dimension is the noise level of the image, quantified by extracting the energy value of high-frequency subbands through wavelet transform; the second dimension is the background texture density, quantified by calculating the entropy value of the gray-level co-occurrence matrix of the background region; and the third dimension is the target-background contrast, quantified by calculating the Bach distance between the candidate target region and its immediate external background. These three dimensions are normalized and then summed according to preset weights to obtain the environmental complexity value for the current frame.

[0105] When the environmental complexity value exceeds a preset environmental complexity threshold, the system determines the current target scene as a high-complexity scene and triggers a dynamic threshold optimization mechanism. Specifically, the system lowers the preset warning threshold described in claim 9 from the baseline value, with the reduction amount being positively correlated with the extent to which the environmental complexity value exceeds the preset environmental complexity threshold. Simultaneously, the system compares the confidence score output by the lightweight Transformer network with the lowered warning threshold; if the confidence score is greater than the lowered warning threshold, an alert is triggered.

[0106] When the environmental complexity value is lower than the preset environmental complexity threshold, the system maintains the baseline warning threshold unchanged. This dynamic threshold optimization mechanism enables the system to ensure the detection rate by appropriately lowering the judgment threshold in complex scenarios, and to control the false alarm rate by maintaining a higher threshold in simple scenarios, thus achieving an adaptive balance under different environmental conditions.

[0107] Understandably, in environmental monitoring scenarios, drones need to quickly identify and classify complex targets such as water pollutants and forest fires. Traditional pattern recognition methods based on single-source data are prone to false detections under dynamic background interference, and their stability in multi-target tracking is insufficient. This technology proposes a multimodal feature fusion and spatiotemporal context modeling method. Through cross-modal correlation analysis of visible light, thermal infrared, and multispectral data, a multi-dimensional feature description model of the target is constructed. Combined with a lightweight Transformer network and a dynamic threshold optimization algorithm, highly robust target detection and continuous tracking are achieved in complex environments, effectively solving the recognition challenges of traditional methods in scenarios with target deformation and partial occlusion.

[0108] In some embodiments, reference is made to Figure 3 This application proposes a target image recognition system for a drone warning system, comprising: a construction unit 301, used to perform edge-preserving smoothing processing on the monitoring image acquired by the drone, and based on the gradient direction distribution of each pixel in the smoothed image, constructing a boundary continuity probability map to characterize the possibility of the existence of weak physical boundaries. Weak physical boundaries refer to the boundaries between the target and the background where the gradient magnitude is lower than a preset strong edge threshold due to similar reflectivity; an extraction unit 302, used to extract multiple fragments representing potential target local regions according to the boundary continuity probability map, and based on the physical spatial correlation between fragments, identify the set of fragments belonging to the same presumed entity as a set of candidate target fragments. Potential target local regions refer to low-probability internal regions surrounded or semi-enclosed by high-probability boundaries in the boundary continuity probability map; and a processing unit 303, used to infer and complete the missing boundary regions between fragments under the known boundary of the fragments in the candidate target fragment set as anchor point constraints, and using the external guidance provided by the boundary continuity probability map, to generate complete closed candidate target regions for subsequent target recognition and warning determination.

[0109] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multimodal target image recognition method for an unmanned aerial vehicle (UAV) warning system, characterized in that, include: The monitoring images collected by the UAV are processed to preserve the edges and smooth the edges. Based on the gradient direction distribution of each pixel in the smoothed image, a boundary continuity probability map is constructed to characterize the possibility of the existence of weak physical boundaries. The weak physical boundaries refer to the boundaries between the target and the background where the gradient magnitude is lower than the preset strong edge threshold due to similar reflectivity. Based on the boundary continuity probability map, multiple fragments representing the local region of a potential target are extracted, and based on the physical spatial correlation between the fragments, the set of fragments belonging to the same presumed entity is identified as the set of candidate target fragments. The local region of a potential target refers to the low-probability internal region surrounded or semi-surrounded by a high-probability boundary in the boundary continuity probability map. With the known boundaries of the fragments in the candidate target fragment set as anchor points, the missing boundary regions between fragments are inferred and filled in using the external guidance provided by the boundary continuity probability map, generating complete and closed candidate target regions for subsequent target identification and warning determination.

2. The method according to claim 1, characterized in that, The edge-preserving and smoothing processing of the monitoring images acquired by the drone includes: The camera gain value at the acquisition time corresponding to the monitoring image is obtained. The camera gain value is used to determine the global noise variance estimate of the current image based on a pre-calibrated noise model. A bilateral filtering mechanism that introduces a local texture variance adjustment factor is adopted to calculate the weight of each pixel in the image. For the center pixel whose absolute value of gray level difference with neighboring pixels is less than the global noise variance estimate, the local texture variance of the center pixel is further determined: if the local texture variance is lower than a preset flatness threshold, the center pixel is determined to be in a pure noise region and is given a first smoothing weight; if the local texture variance is not lower than the preset flatness threshold and exhibits a specific directionality, the center pixel is determined to be in a potential boundary region and is given a second smoothing weight, the value of the second smoothing weight being less than the first smoothing weight.

3. The method according to claim 1, characterized in that, The construction of the boundary continuity probability graph used to characterize the possibility of the existence of weak physical boundaries includes: At each pixel location of the smoothed image, the gradient magnitude and gradient direction are calculated; A local sliding window is set for each pixel, and the angular distribution of gradient directions of each pixel within the window is statistically analyzed to form a gradient direction distribution histogram. The gradient direction distribution histogram is used to determine whether there is a dominant gradient direction within the window and the percentage of pixels with that dominant gradient direction. Based on the proportion of pixels in the dominant gradient direction within the window and the gradient magnitude of the center pixel, calculate the boundary continuity probability value at the location of the center pixel. The higher the proportion of pixels in the dominant gradient direction and the larger the gradient magnitude of the center pixel, the greater the boundary continuity probability value at that location. Based on the flight altitude and camera focal length transmitted in real time by the UAV, the theoretical pixel size of the ground target in the monitored image is determined, a multi-scale image pyramid is constructed, the boundary continuity probability response is calculated at different scale layers, and the probability responses at multiple scales are fused into a unified boundary continuity probability map by a cross-scale maximum preservation strategy.

4. The method according to claim 1, characterized in that, The extraction of multiple fragments representing local regions of a potential target includes: Using a preset region growth algorithm or watershed algorithm, the pixels with probability values ​​lower than a preset growth threshold in the boundary continuity probability map are used as seed points for growth. The growth stops when the growth front touches a pixel with a probability value higher than the preset boundary threshold, thereby generating multiple fragmented regions. For each generated fragment region, its internal features are extracted. These internal features include the fragment's centroid coordinates, area, principal axis direction, circumscribed rectangle, grayscale mean, and grayscale co-occurrence matrix features.

5. The method according to claim 4, characterized in that, The method of identifying a set of fragments belonging to the same presumed entity as a candidate target fragment set based on the physical spatial correlation between fragments includes: All extracted fragment regions are constructed into an undirected weighted graph, where each node represents a fragment region; For each pair of nodes in the graph, a connection weight is calculated. The calculation of the connection weight integrates spatial distance constraints, orientation alignment constraints, and internal feature similarity. The spatial distance constraint refers to the constraint based on the UAV flight altitude and camera intrinsic parameters, which converts the pixel distance between fragments into actual physical distance. The internal feature similarity refers to the measurement of the Mahalanobis distance between fragment feature vectors by using the texture covariance matrix of the background region. A preset graph clustering algorithm is used to merge fragment nodes with connection weights greater than a dynamic threshold into the same subgraph, and the merged fragment set is determined as the candidate target fragment set. The dynamic threshold is adaptively determined based on the overall contrast and noise level of the current image.

6. The method according to claim 5, characterized in that, The calculation of the internal feature similarity includes: Principal component analysis is performed on the pixels within each fragment region to determine its dominant texture direction; When calculating the gray-level co-occurrence matrix features of the fragment region, feature extraction is performed only along the dominant texture direction and in directions perpendicular to the dominant texture direction.

7. The method according to claim 1, characterized in that, The process of inferring and completing missing boundary regions between fragments, using the known boundaries of fragments in the candidate target fragment set as anchor constraints and the external guidance provided by the boundary continuity probability map, includes: The initial contour is determined based on the convex hull of the global bounding polygon of the candidate target fragment set; An active contour energy function is constructed, consisting of an internal energy term and an external energy term. The internal energy term is used to control the smoothness of the initial contour, and the external energy term is defined by the boundary continuity probability map and is used to attract the initial contour to evolve towards the region with a high probability value. The actual boundaries of all fragments in the candidate target fragment set are set as immovable anchor points. Under the constraint of the anchor points, the active contour energy function is minimized iteratively to drive the initial contour to evolve and converge in the missing region between fragments, forming a closed candidate target region after completion.

8. The method according to claim 7, characterized in that, After inferring and completing the missing boundary regions between fragments, the method further includes: When there is a missing area between two anchor points in the candidate target fragment set that is not covered by fragments, the two anchor points are identified as a pair of anchor points that need to be inferred from the boundary. A dynamic evolution corridor is constructed for the pair of anchor points. The process of constructing the dynamic evolution corridor is as follows: the centroid line connecting the pair of anchor points is used as the central axis of the corridor. The lateral boundary of the corridor is determined according to the average width of the pair of anchor points and the preset human body structure ratio. The inner area of ​​the corridor is defined as the legal evolution zone, and the outer area of ​​the corridor is defined as the strong constraint zone. Before active contour evolution, a corridor mask is used to spatially modulate the original boundary continuity probability map, forcibly attenuating the probability value of pixels located in the strongly constrained region to zero, so as to shield the attraction of external strong interference edges. The active contour evolution is a process of driving the initial contour evolution by iteratively minimizing the active contour energy function. The initial contour evolves in the spatially modulated boundary continuity probability map, so that the initial contour converges to the true boundary within the legal evolution region.

9. The method according to claim 1, characterized in that, The method further includes: The candidate target region is evaluated intra-frame, and the overall grayscale distribution statistical difference between the pixels inside the candidate target region and the adjacent external background pixels is compared. A first confidence score is calculated based on the shape compactness and aspect ratio of the candidate target region. The first confidence score is used as prior information input into the inter-frame verification process. In the inter-frame verification process, the candidate target region generated in the current frame is associated with the expected position predicted by the target state in the historical frame through a Kalman filter. If the association is successful, the second confidence score of the current frame is updated by combining the historical confidence score with the first confidence score. If the association fails for multiple consecutive frames, the target is determined to be lost and the trajectory is terminated. When the second confidence score exceeds the preset warning threshold, target information including target identifier, bounding box coordinates and geographic coordinates is output to trigger a warning.

10. The method according to claim 1, characterized in that, After generating a fully closed candidate target region, the method further includes: A multi-dimensional feature description model of the target is constructed through cross-modal correlation analysis of visible light, thermal infrared and multispectral data. By combining a lightweight Transformer network with a dynamic threshold optimization algorithm, target recognition and warning determination are performed on target scenes with environmental complexity exceeding a preset threshold.

Images