An edge-cloud collaborative analysis and learning method and system for drone views

Abstract

This invention discloses an edge-cloud collaborative analysis learning method and system for UAV views, belonging to the field of collaborative analysis technology. This invention effectively avoids the problem of fragmented strongly correlated pixel blocks caused by traditional fixed segmentation, significantly reducing the computational burden of cross-segment feature association for edge nodes. Combining dynamic routing with semantic type labels and edge node load awareness, it achieves accurate matching of computational resources and view content, improving the overall processing efficiency and stability of the system. Furthermore, through master fusion node election, multi-view geometric correction, and pixel-level weighted fusion, it generates a high-quality virtual synthetic data stream. The cloud performs statistical analysis on the time alignment residual and pixel fusion residual, and adaptively adjusts the front-end segment clustering, feature matching, and fusion weight parameters online, forming an end-to-end closed-loop optimization system. This continuously improves the accuracy, real-time performance, and robustness of collaborative analysis in dynamic scenarios.

Description

Technical Field

[0001] This invention relates to the field of collaborative analysis technology, and in particular to an edge-cloud collaborative analysis learning method and system for drone views. Background Technology

[0002] With the widespread application of unmanned aerial vehicle (UAV) platforms in environmental perception, target monitoring, and emergency response, the limited field of view and singular observation perspective of a single UAV are becoming increasingly prominent. Multi-UAV collaborative view analysis has become an important direction for improving perception capabilities. Currently, multi-UAV view processing technology is developing along two main paths: First, a centralized processing architecture based on cloud-centricity, where each UAV collects view data, which is then aggregated at edge nodes and uploaded to the cloud, where feature extraction, spatiotemporal correlation, and scene fusion are performed. Second, a distributed edge processing scheme based on fixed rules, where the UAV slices the view at fixed time intervals or with fixed data volumes, and edge nodes independently process their respective slices and selectively upload the results to the cloud. In recent years, with the improvement of edge computing capabilities, some research has attempted to introduce lightweight feature extraction and matching algorithms at edge nodes to reduce the load on the cloud.

[0003] However, existing technologies generally suffer from the following shortcomings: First, the fragmentation strategy at the drone end ignores the spatiotemporal correlation characteristics of the view content itself, resulting in pixel blocks with high motion consistency being fragmented into different fragments, increasing computational redundancy in cross-fragment feature association; at the same time, fragments lack semantic description information, making it impossible to achieve content-based adaptive routing, causing a mismatch between edge node computing resources and view types. Second, feature association between cross-node views heavily relies on global aggregation in the cloud, and there is a lack of direct feature peer discovery and linkage mechanisms between edge nodes, causing the real-time performance of collaborative fusion to be limited by backbone network bandwidth and cloud processing bottlenecks. Third, although existing distributed collaborative solutions attempt to exchange data between edge nodes, they lack a unified linkage trigger identifier and fusion node election strategy, and the spatiotemporal alignment and geometric correction parameters in the fusion process are mostly static presets, making it difficult to adapt to dynamically changing scenarios and task requirements.

[0004] To address the aforementioned issues, some improved solutions have attempted to introduce semantic segmentation based on deep learning, but these solutions have still failed to effectively solve the core challenges of maintaining consistent feature identifiers across edge nodes and adaptively adjusting parameters online. Summary of the Invention

[0005] In view of the problems existing in the above-mentioned background art, the present invention is proposed.

[0006] Therefore, the problem to be solved by this invention is how to provide a closed-loop adaptive UAV view collaborative analysis method that can form a closed loop from data acquisition, edge processing to collaborative fusion, so as to solve the technical problems of fragmentation, lack of linkage and static parameters in the prior art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an edge-cloud collaborative analysis and learning method for UAV view, comprising: the UAV dynamically packages the collected view data into logical data fragments carrying unified metadata based on semantic and spatiotemporal information, and sends them to associated edge nodes according to semantic type; after receiving the logical data fragments, the edge nodes perform feature extraction and cross-node association identification generation based on semantic and spatiotemporal information, and publish them to a shared joint feature pool; the edge nodes establish linkage between the identifications based on the features, perform geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generate a virtual synthetic data stream; the virtual synthetic data stream is analyzed and uploaded back to the cloud; the cloud generates online adaptive parameters and distributes them in real time to optimize subsequent processing.

[0008] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the generation of logical data fragments includes: the UAV acquiring the current frame and its corresponding high-precision timestamp and UAV pose data, and extracting the motion vector field between the current frame and the keyframes in the previous sliding window, and calculating the neighboring block association strength value of each pixel block based on the motion vector field; the UAV clustering adjacent pixel blocks with neighboring block association strength values ​​higher than a preset association threshold into visual object units, and classifying and identifying each visual object unit to generate semantic type labels, forming a structured description of visual object unit—semantic type—internal pixel association relationship; generating logical data fragments for each visual object unit, including: a header carrying semantic type labels, high-precision timestamp, UAV pose, and internal pixel association relationship; and a body carrying the corresponding view data block.

[0009] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the motion vector includes a horizontal motion component, a vertical motion component, a motion direction, and a motion amplitude; the motion vector field is extracted using optical flow or block matching methods.

[0010] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the feature extraction includes: edge nodes guiding feature extraction based on the internal pixel association relationship, extracting key point features and descriptors in the view data block, and combining the semantic type label and the UAV pose to associate feature points that satisfy spatial geometric consistency and have the same semantics in different logical data slices received from the edge nodes into a set of feature pairs; the judgment conditions for spatial geometric consistency include: the ratio of the nearest distance to the second nearest distance of the descriptor is less than a preset matching threshold, the reprojection error is less than a preset geometric verification threshold, and the semantic labels are consistent.

[0011] As a preferred embodiment of the edge and cloud collaborative analysis and learning method for UAV view described in this invention, the cross-node association identifier generation includes: edge nodes assigning globally unique feature pair identifiers to each group of feature pairs, and combining the feature pairs and the feature pair identifiers to form a pre-association result; edge nodes writing the pre-association result into a shared joint feature pool, and publishing it with the feature pair identifiers as an index for other edge nodes to query and call.

[0012] As a preferred embodiment of the edge and cloud collaborative analysis and learning method for UAV view described in this invention, the step of establishing linkage based on feature pair identifiers includes: edge nodes periodically querying the joint feature pool, and when they detect pre-association results published by other edge nodes that have the same feature pair identifiers as their own published pre-association results, they extract the node addresses and corresponding sharding spatiotemporal stamp information of the other edge nodes.

[0013] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the generation of virtual synthetic data stream includes: edge nodes establishing point-to-point communication connections with other edge nodes based on the node address, exchanging the UAV pose and the high-precision timestamp in their respective logical data slices carrying feature pairs, and generating a linkage link recording the corresponding relationship between nodes based on the pose relative relationship and time alignment window; each edge node participating in the linkage determines a master fusion node through an election mechanism, and the remaining edge nodes act as subordinate nodes; the master fusion node obtains the corresponding logical data slices from itself and its subordinate nodes based on the high-precision timestamp in the linkage link, performs time alignment on each obtained logical data slice, and performs geometric correction on each view data block based on the UAV pose using a multi-view geometric method; the master fusion node performs pixel-level weighted fusion of the corrected image blocks according to spatial position to generate a virtual synthetic data stream, stores the generated virtual synthetic data stream in a local cache, and synchronizes the access address and data stream description to the joint feature pool.

[0014] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the analysis of the virtual synthetic data stream includes: the main fusion node performing target detection and scene parsing on the virtual synthetic data stream, generating structured analysis results, and recording the time alignment residual and pixel fusion residual during this collaborative fusion process as correlation biases; the main fusion node packaging the structured analysis results and the correlation biases into an upload data packet and sending it to the cloud; the cloud generating online adaptive parameters includes: the cloud receiving the correlation biases uploaded by multiple edge nodes, performing statistical analysis on each correlation bias, and generating collaborative learning parameters.

[0015] As a preferred embodiment of the edge-cloud collaborative analysis and learning method for UAV view described in this invention, the following steps are taken: Statistical analysis of the various correlation deviations to generate collaborative learning parameters includes: receiving the correlation deviations uploaded by multiple edge nodes in the cloud, aggregating the correlation deviations according to a preset time window, and calculating the statistics of the correlation deviations, including the mean deviation and the deviation change trend; generating a pre-correlation threshold correction amount for updating the preset correlation threshold based on the difference between the mean deviation and a preset target value; generating a geometric verification threshold correction amount for updating the preset matching threshold and the preset geometric verification threshold based on the comparison result of the deviation change trend and a preset time tolerance; and generating a fusion weight adjustment amount for updating the pixel-level weighted fusion weights based on the comparison result of the pixel fusion residuals and the average residuals corresponding to each source image.

[0016] Secondly, this invention provides an edge-cloud collaborative analysis and learning system for drone views, comprising: The view semantic sharding module dynamically packages the collected view data into logical data shards carrying unified metadata based on semantic and spatiotemporal information, and sends them to the associated edge nodes according to the semantic type. The feature association and publishing module is used to extract features and generate cross-node association identifiers based on semantic and spatiotemporal information after the edge nodes receive logical data fragments, and then publish them to the shared joint feature pool. The collaborative fusion generation module establishes linkage between edge nodes based on features, performs geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generates a virtual synthetic data stream. The cloud-based learning optimization module analyzes the virtual synthetic data stream and uploads the results to the cloud; the cloud generates online adaptive parameters and distributes them in real time to optimize subsequent processing.

[0017] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the edge and cloud collaborative analysis and learning method for UAV view as described in the first aspect of the present invention are implemented.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the edge and cloud collaborative analysis and learning method for UAV views as described in the first aspect of the present invention are implemented.

[0019] The beneficial effects of this invention are as follows: This invention effectively avoids the problem of strong-related pixel block fragmentation caused by traditional fixed fragmentation by dynamically packaging logical data fragments based on semantic and spatiotemporal information on the UAV end, and significantly reduces the computational burden of cross-fragment feature association of edge nodes; combined with dynamic routing of semantic type labels and edge node load perception, it achieves accurate matching of computing resources and view content, and improves the overall processing efficiency and stability of the system.

[0020] Furthermore, by electing the master fusion node, performing multi-view geometric correction and pixel-level weighted fusion, a high-quality virtual synthetic data stream is generated. The cloud performs statistical analysis on the time alignment residual and pixel fusion residual, and adaptively adjusts the front-end segmentation clustering, feature matching and fusion weight parameters online to form an end-to-end closed-loop optimization system. This continuously improves the accuracy, real-time performance and robustness of collaborative analysis in dynamic scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart for edge and cloud collaborative analysis learning methods for drone views.

[0023] Figure 2 A flowchart for the generation and clustering of logic fragments on the drone side.

[0024] Figure 3 This is a flowchart illustrating the cross-node linkage and collaborative integration process of edge nodes.

[0025] Figure 4 This is a structural diagram of an edge-cloud collaborative analysis and learning system based on drone views. Detailed Implementation

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0028] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0029] like Figures 1-3 As shown, the edge-cloud collaborative analysis and learning method for drone views includes: S1: Based on semantic and spatiotemporal information, the drone dynamically packages the collected view data into logical data fragments carrying unified metadata, and sends them to the associated edge nodes according to the semantic type.

[0030] In existing technologies, drones typically segment the collected view data at fixed time intervals or with fixed data volumes, and then directly upload the segments to edge nodes or the cloud. This approach ignores the spatiotemporal correlation characteristics of the view content itself, resulting in strongly correlated pixel blocks being fragmented into different segments. This increases the computational burden on edge nodes for cross-segment feature association, and the segments lack semantic description information, making content-based intelligent routing impossible and wasting network transmission and computing resources. To address these issues, the present invention operates as follows: S1.1: The UAV acquires the current frame and the corresponding high-precision timestamp and UAV pose data. The high-precision timestamp is synchronized at the microsecond level through PTP (Precise Time Protocol) or 1588v2 protocol. The UAV pose data is obtained by fusion calculation of GPS and IMU to obtain six-degree-of-freedom pose information, including three-dimensional spatial coordinates and three-axis attitude angles.

[0031] Furthermore, the drone maintains several previous keyframes using a sliding window approach. The selection of keyframes is dynamically determined based on the inter-frame motion amplitude and the degree of scene change. For the current frame and each keyframe within the sliding window, the motion vector field between them is extracted. The motion vector field represents the motion state of each pixel block in the image between consecutive frames, and its components include horizontal motion components, vertical motion components, motion direction, and motion amplitude.

[0032] The motion vector field is extracted using either optical flow or block matching. When optical flow is used, the dense motion vector field is obtained by calculating the spatiotemporal gradient of pixel intensity between adjacent frames and solving the optical flow constraint equation. When block matching is used, the current frame is divided into several pixel blocks, and the optimal matching block is found within the search window of the reference frame. The displacement of the matching block is used as the motion vector of that pixel block.

[0033] It is important to note that the drone simultaneously acquires and maintains the intrinsic parameter matrix of its own camera (including focal length, principal point coordinates, and distortion coefficients). The intrinsic parameters are obtained through offline calibration and are stored in the configuration file of the drone's onboard system. The logical data fragment header must carry the camera intrinsic parameters or intrinsic parameter identifier (if the intrinsic parameters of all drones are the same, this can be agreed upon uniformly; otherwise, they must be explicitly passed).

[0034] S1.2: After obtaining the motion vector field, the UAV calculates the neighbor association strength value of each pixel block on a pixel-by-pixel basis. This is achieved through the following steps: First, the current frame is divided into N×N pixel blocks, where N is chosen as 8, 16, or 32 depending on the view resolution and computing resources. Each pixel block serves as the basic unit for calculating the neighbor block association strength value. For each pixel block, the motion vector of that block is extracted from its corresponding motion vector field, including the horizontal component dx, the vertical component dy, and the motion amplitude. and direction of movement .

[0035] Secondly, for the current pixel block and its neighboring pixel blocks, the neighboring pixel blocks are defined using either a four-neighbor or eight-neighbor method. The difference in motion direction between the current pixel block and each of its neighboring pixel blocks is calculated separately. and motion amplitude difference .

[0036] The difference in motion direction is taken as the absolute value of the angle difference and normalized to the interval [0, π]. The difference in motion amplitude is taken as the absolute value and divided by the preset maximum motion amplitude. (Dynamically calculate the maximum motion amplitude of all pixel blocks within the current frame) and normalize it. If the maximum motion amplitude is 0, the normalized difference is directly set to 0. Summate the normalized direction difference with the amplitude difference to obtain the original spatial similarity value of the adjacent block pair. The value ranges from [0, 2]. The smaller the value, the more consistent the two pixel blocks are in spatial motion properties.

[0037] Next, for the current pixel block and each of its neighboring pixel blocks, obtain the motion vector sequences in multiple consecutive frames, taking the first three frames. Each sequence consists of the motion vectors (horizontal and vertical components) of that block in each frame. The similarity between two sequences can be calculated using the inverse of the Dynamic Time Warped (DTW) distance, or by calculating the average of the cosine similarities of the motion vectors in each frame between the sequences. The obtained similarity value is used as the original temporal similarity value for the adjacent block pair. The value ranges from [0,1]. The closer the value is to 1, the more consistent the motion patterns of the two pixel blocks are on the time axis.

[0038] Finally, for the current pixel block and each of its neighboring pixel blocks, the original spatial similarity and original temporal similarity values ​​are calculated separately, and then weighted and fused to obtain the neighboring block association strength value for that pair. Specifically, for the current pixel block and its neighboring pixel blocks, the spatial and temporal values ​​are weighted and fused to obtain the neighboring block association strength value: ; Among them, spatial weight coefficient The default value is 0.5, which can be adjusted according to the scenario.

[0039] S1.3: The UAV clusters adjacent pixel blocks whose neighboring block association strength value is higher than the preset association threshold (the initial value is set to 0.7, and it is dynamically adjusted according to the pre-association threshold correction amount issued by the cloud) into visual object units, and classifies and identifies each visual object unit through an embedded lightweight object detection model, generates semantic type labels, and forms a structured description of the visual object unit-semantic type-internal pixel association relationship.

[0040] The clustering process employs a region growing algorithm, using pixel blocks whose neighboring block association strength values ​​exceed a preset association threshold as seed points. It recursively merges neighboring pixel blocks that meet the strength conditions until it can no longer be expanded, thus forming a complete visual object unit.

[0041] Furthermore, for each clustered visual object unit, the UAV uses an embedded lightweight object detection model for identification and classification. The embedded lightweight object detection model is selected from YOLO-Nano, MobileNet-SSD, or Tiny-YOLO. These models undergo pruning and quantization to adapt to the limited computing resources of the UAV's onboard embedded platform. The model takes an image patch within the visual object unit as input. Before input, the image patch is scaled or cropped to a fixed size required by the model (e.g., 320×320), and the scaling factor is recorded for subsequent mapping back to the original coordinates. The output is a semantic type label for the unit, which includes, but is not limited to, categories such as car, pedestrian, cyclist, building, and vegetableation.

[0042] The association graph records the spatial adjacency relationships and motion consistency measures of each pixel block within a unit in a graph structure, providing guidance for subsequent feature extraction of edge nodes.

[0043] Finally, the UAV generates an independent logical data slice for each visual object unit. The logical data slice adopts a structured encapsulation format, specifically including two parts: a header and a body. The header carries semantic type labels, high-precision timestamps, UAV pose (six-DOF pose), camera intrinsic parameter matrix, and internal pixel relationships. The body carries the corresponding view data block, where the view data block is the raw pixel data or compressed image block within the coverage area of ​​the visual object unit.

[0044] Optionally, the main body may also include a motion vector field (sparse or dense) corresponding to the visual object unit. The motion vector field is obtained by cropping the result extracted in S1.1, retaining only the motion vectors of the pixel blocks within the unit, which are used for subsequent time alignment compensation of edge nodes. When compression encoding is used, intra-frame coding blocks of H.265 / HEVC or JPEG compression blocks can be selected to ensure compatibility with mainstream video coding standards.

[0045] After logical data fragments are generated, the drone sends them according to the semantic type label carried in the fragment header. The drone maintains an edge node service registry, which is pre-reported by each edge node and records the set of semantic types that each edge node can handle and its current load status (such as CPU utilization, memory utilization, and network bandwidth availability). The drone queries the service registry for each logical data fragment, selects the edge node that can handle that semantic type and has the lowest load, and sends the fragment to the selected edge node.

[0046] As can be seen, this invention achieves precise matching of computing resources and view content through a dynamic routing mechanism based on semantic types. Different edge nodes can handle shards of specific semantic types according to their computing power configuration and algorithmic expertise. For example, nodes skilled in object detection handle car and pedestrian shards, while nodes skilled in scene understanding handle building and vegetation shards, avoiding the resource waste caused by homogenizing all shards. Simultaneously, routing based on load status achieves load balancing among edge nodes, improving the overall system's processing efficiency and stability.

[0047] S2: After receiving logical data fragments, edge nodes perform feature extraction and cross-node association identification generation based on semantic and spatiotemporal information, and publish them to the shared joint feature pool.

[0048] S2.1: The edge node guides feature extraction based on the internal pixel association relationship, extracts key point features and descriptors in the view data block, and combines the semantic type label and the UAV pose to associate feature points that satisfy spatial geometric consistency and have the same semantics in different logical data slices received from the edge node into a set of feature pairs.

[0049] The internal pixel association relationships are recorded in the form of a graph structure, recording the spatial adjacency relationships and motion consistency measures between pixel blocks within a visual object unit. Specifically, it is represented by an undirected graph G=(V,E), where vertices V correspond to pixel blocks, and the weights of edges E represent the neighbor association strength values ​​between adjacent pixel blocks. Edge nodes determine the priority and sampling density of feature extraction based on this association graph: for regions with high association strength, the feature point extraction density is increased; for regions with low association strength, the extraction density is decreased or extraction is skipped.

[0050] Keypoint feature extraction employs a scale-invariant feature transform algorithm or its improved versions, such as SuperPoint or SIFT, to extract representative keypoints within each pixel block of the view data and generate a descriptor vector for each keypoint. The descriptor vector characterizes the gradient distribution or texture features of the keypoint's local neighborhood, and is represented as a floating-point vector or binary string with 128 or 256 dimensions.

[0051] Guided by the internal pixel correlation graph, edge nodes reduce the extraction of redundant feature points while ensuring feature representativeness, thus improving the computational efficiency of feature extraction and making the distribution of extracted feature points match the internal structure of the visual object unit.

[0052] S2.2: Edge nodes receive multiple logical data slices from the same or multiple drones, each slice corresponding to a different visual object unit. Edge nodes use the semantic type labels carried in the slice header as the initial screening condition for feature matching, and only perform cross-slice association on feature points with the same semantic type to reduce the matching search space.

[0053] For two segments with the same semantic type, the edge nodes extract their respective key point feature sets, denoted as... and Each feature contains keypoint coordinates, a descriptor vector, and the UAV pose information for its corresponding slice. The feature matching process consists of three stages: The first stage involves preliminary matching based on the ratio of the nearest distance to the second nearest distance of the descriptors. Each feature point in, The search function finds the feature point whose ratio of nearest distance to second nearest distance is less than the preset matching threshold (initial value is set to 0.7). If the ratio of nearest distance to second nearest distance is less than the preset matching threshold, then the feature point pair is considered as a candidate matching pair.

[0054] The second stage involves verification based on spatial geometric consistency. For each candidate matching pair, the edge nodes use the UAV poses corresponding to the slices to which the two feature points belong to calculate the reprojection error using a triangulation method. Specifically, based on the feature point coordinates and their camera poses in the first slice and the feature point coordinates and their camera poses in the second slice, the 3D spatial point coordinates are solved using the Direct Linear Transform (DLT) method or the midpoint method. After obtaining the 3D point, it is reprojected back onto the image planes of the two slices respectively, resulting in two projected coordinates. The Euclidean distance between the projected coordinates and the actual feature point coordinates is calculated, and the sum (or the maximum value) of the two is taken as the total reprojection error. If the total reprojection error is less than the preset geometric verification threshold (initial value set to 3 pixels), the candidate matching pair passes the geometric verification. This method does not rely on single-view depth priors; it only requires two-view matching point pairs and their corresponding poses to complete the geometric consistency check.

[0055] The third stage is semantic consistency verification. Feature pairs that have passed the verification in the first two stages must also meet the condition of semantic type label consistency. Since the initial screening in the first stage was based on semantic type, this stage is a secondary confirmation to ensure that feature points associated across segments belong to visual object units of the same semantic category.

[0056] Feature point pairs that have passed the three-stage verification process are grouped into a set of feature pairs. Each set of feature pairs records complete information about two feature points from different segments, including their respective keypoint coordinates, descriptor vectors, segment identifiers, and reprojection error values ​​when verification is successful.

[0057] S2.3: The edge node assigns a globally unique feature pair identifier to each group of feature pairs, and combines the feature pairs and the feature pair identifiers to form a pre-association result.

[0058] The feature pair identifiers are generated using a structured encoding method, with the format: <Semantic Type><Spatial Hash Value><Time Window ID>. The semantic type is directly derived from the semantic label of the visual object unit to which the feature pair belongs. The spatial hash value is obtained by spatial grid encoding the 3D spatial coordinates of the feature points, where the 3D spatial coordinates are calculated by triangulation in the S2.2 geometric verification stage. The 3D space is divided into a cubic grid with side length L, and the hash index of the grid containing the feature point is calculated. For each feature pair that passes geometric verification, the 3D point coordinates obtained from triangulation (or the average if both feature points are triangulated separately) are used as the basis for spatial hash calculation. The time window ID is determined based on the average of the high-precision timestamps of the two segments in the feature pair. The time axis is divided into fixed-length time windows (e.g., 100 milliseconds), the average of the two timestamps is calculated, and the sequence number of the window to which the average value belongs is taken as the time window ID. If the difference between the two timestamps exceeds one time window, no identifier is assigned to this feature pair (indicating excessive asynchrony, unsuitable for association).

[0059] Feature pairs are globally unique within the same collaborative analysis system, meaning that feature pairs generated by different edge nodes for the same target or scene maintain consistent identifiers. The spatial hash value calculation method ensures that targets near the same spatial location obtain similar hash values, and the introduction of time window IDs ensures that identifiers are naturally updated as targets move and over time.

[0060] The data structure of the pre-association results includes: feature pair identifiers, information on the two feature points in the feature pair (including coordinates, descriptors, and the identifier of the segment to which they belong), the reprojection error value when the verification is successful, and the semantic type labels, timestamps, and UAV poses of the two segments respectively.

[0061] S2.4: The edge nodes write the pre-association results into the shared joint feature pool and publish them with the feature pair identifier as the index for other edge nodes to query and call.

[0062] The joint feature pool is implemented using a distributed key-value storage system deployed on an edge node cluster, providing highly available and low-latency read and write services. The joint feature pool stores feature pairs as keys and pre-association results as values, supporting other edge nodes to perform precise queries based on feature pairs.

[0063] When writing pre-association results, edge nodes simultaneously set the data's validity period. The validity period is determined based on the timestamp of the feature-paired shard and a preset time window length, typically set to twice the time window length to ensure that other shards arriving within that time window can query the corresponding pre-association results. Pre-association results exceeding their validity period are automatically cleaned up by the joint feature pool to prevent unlimited storage expansion.

[0064] The joint feature pool also supports fuzzy queries based on spatial hash value prefixes. Edge nodes can query pre-association results in neighboring spatial grids based on the spatial location of their own feature points, which can be used to discover other potentially interconnected edge nodes.

[0065] Compared to traditional solutions that rely on the cloud for global feature aggregation, this method distributes feature association discovery and matching to edge nodes. Each node proactively discovers collaborating peer nodes by querying the joint feature pool, reducing the processing burden and network transmission overhead on the cloud. The distributed key-value storage architecture ensures the system's scalability and high availability, while the expiration date controls storage costs, enabling the system to operate stably for a long time.

[0066] S3: Edge nodes establish linkages with identifiers based on features, perform geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generate a virtual synthetic data stream.

[0067] It should be noted that in existing technologies, collaborative fusion of multiple UAV views typically employs a centralized architecture. This means that each edge node uploads its original view or preliminary processing results to the cloud, where spatiotemporal alignment, geometric correction, and image fusion are performed. This approach has the following drawbacks: First, the cloud becomes a processing bottleneck; as the number of UAVs increases, the computing pressure on the cloud grows exponentially. Second, a large amount of original view data needs to be transmitted to the cloud, consuming backbone network bandwidth and introducing high transmission latency. Third, the cloud-based fusion results need to be redistributed to each edge node, further increasing network overhead. While a few distributed collaborative solutions attempt to directly exchange data between edge nodes, they lack a unified linkage triggering mechanism and fusion node election strategy, resulting in low fusion efficiency or unstable fusion quality.

[0068] To address the aforementioned issues, this invention enables each edge node to autonomously discover collaborative counterparts with identical feature pairs by querying a shared joint feature pool, establishing point-to-point communication connections, and performing geometric correction and collaborative fusion based on a unified spatiotemporal benchmark to generate a virtual synthetic data stream. This mechanism decentralizes fusion computation to the edge node cluster, achieving decentralized collaborative processing and significantly reducing cloud load and network transmission overhead. The specific operation is as follows: S3.1: Edge nodes periodically (e.g., every 100 milliseconds) query the joint feature pool. When they detect a pre-association result published by another edge node that has the same feature pair identifier as the pre-association result published by itself, they extract the node address and corresponding sharding spatiotemporal stamp information of the other edge node.

[0069] Specifically, edge nodes extract one or more feature pairs that they have published, initiate a key-based exact query to the joint feature pool, and obtain all pre-association results stored with that feature pair as an index.

[0070] The query results returned by the joint feature pool include the node addresses of other edge nodes that published the pre-association results, the sharding spatiotemporal stamp information recorded in the pre-association results (i.e., the high-precision timestamp carried in the logical data shard header), and the feature pair information contained in the pre-association results.

[0071] Edge nodes extract node addresses and sharding time stamp information from the query results, which serve as the basis for establishing subsequent communication connections.

[0072] To avoid the access pressure on the joint feature pool caused by frequent queries, edge nodes can adopt a long polling mechanism or a publish-subscribe pattern to register the feature pairs they are interested in with the joint feature pool. The joint feature pool will then proactively push notifications when matching results are available, reducing the overhead of invalid queries.

[0073] S3.2: Based on the node address, the edge node establishes a point-to-point communication connection with other edge nodes, exchanges the UAV pose and the high-precision timestamp in the logical data slices carrying feature pairs, and generates a linkage link that records the corresponding relationship of the nodes according to the relative pose relationship and time alignment window.

[0074] The point-to-point communication connection uses WebRTC, QUIC, or TCP long connection protocol, which is adaptively selected according to the network environment to ensure low latency and high reliability data transmission.

[0075] After the communication connection is established, the participating edge nodes exchange key metadata from their respective logical data slices carrying the feature pair identifiers. This metadata includes: UAV pose (six-DOF pose data), high-precision timestamps, and feature point coordinates corresponding to the feature pair identifiers. After the exchange, each edge node parses the obtained data to obtain the following information: its own UAV pose matrix. timestamps for logical data sharding The UAV pose matrix corresponding to the peer edge node The timestamp of the logical data fragment corresponding to the peer edge node The coordinates of feature points and their corresponding 3D spatial points in the edge node view of the other end (obtained by the feature matching process).

[0076] The data structure of the linkage link includes: a list of edge nodes participating in the linkage (including the address identifier and drone identifier of each node), the timestamp information corresponding to each node, the pose matrix corresponding to each node, and feature pair identifiers.

[0077] The relative positional relationship is calculated using the opposite node as an example. Using the pose as the reference coordinate system, calculate the local node. to the peer node pose transformation matrix This yields the rotation matrix and translation vector between two frames. The time alignment window is determined by calculating the timestamp difference between the two frames. ,like If the time alignment window is less than a preset threshold (e.g., 50 milliseconds), then the two frames are deemed to meet the time alignment condition and can be directly used for fusion; if If the threshold is exceeded, it needs to be processed by interpolation in subsequent steps.

[0078] After the linkage link is generated, each participating node stores the link information in its local cache to guide subsequent time alignment and geometric correction operations.

[0079] S3.3: Each participating edge node determines a master fusion node through an election mechanism, and the remaining edge nodes act as subordinate nodes.

[0080] Specifically, the design of the election mechanism needs to balance computational efficiency and load balancing, and can adopt one of the following two methods: The first method is election based on hash modulo of feature pair identifiers. Each participating node performs a hash operation on the feature pair identifier to obtain a hash value H, and calculates H mod M, where M is the total number of participating nodes. The node whose calculation result equals a preset value (such as 0) is elected as the main fusion node. This method does not require communication between nodes, and each node calculates the result independently and consistently, achieving election without negotiation.

[0081] The second method is based on pose center selection. Each participating node calculates its position coordinates in 3D space according to its own UAV pose. The node closest to the geometric center of the scene or with the most centered pose is selected as the master fusion node. Specifically, the average of the position coordinates of each node is taken as the scene center, and the Euclidean distance between each node's position and the scene center is calculated. The node with the smallest distance is selected as the master fusion node. This method requires the nodes to exchange pose information, but it can select the node with the best fusion perspective as the master fusion node.

[0082] After the election is completed, the master fusion node sends a confirmation message to the other slave nodes, declaring its master node status and informing each slave node of the address and port for subsequent data uploads.

[0083] The two methods can be flexibly selected or combined according to the application scenario.

[0084] S3.4: The main fusion node obtains the corresponding logical data fragments from itself and its subordinate nodes based on the high-precision timestamp in the linkage link, performs time alignment on each obtained logical data fragment, and performs geometric correction on each view data block based on the UAV pose using a multi-view geometric method.

[0085] In this embodiment of the invention, the master fusion node sends data requests to each subordinate node based on the node list recorded in the linkage link, requesting to obtain the logical data fragments stored by each node that carry the identifier of the feature pair. The subordinate nodes respond to the requests and transmit the corresponding logical data fragments (including header metadata and main view data blocks) to the master fusion node.

[0086] After receiving all logical data fragments, the main fusion node performs time alignment based on the high-precision timestamp information in the linkage link. The time alignment adopts the following strategy: If the timestamp difference between each segment is less than the preset time alignment window threshold (e.g., 50 milliseconds), the original image frames of each segment are directly used for fusion without interpolation.

[0087] If there are segments with timestamp differences exceeding a threshold, virtual frames aligned with the reference timestamp must be generated through motion compensation or frame interpolation. Specifically, the segment with the centered timestamp is selected as the reference frame. For segments with large timestamp deviations, forward or backward optical flow compensation is performed based on the motion vector field to generate an interpolated image aligned with the reference frame's time.

[0088] Optical flow compensation employs a bidirectional optical flow method, which uses motion vectors from consecutive frames to predict the image content at intermediate moments, ensuring the spatiotemporal consistency of the interpolated image.

[0089] Furthermore, the purpose of geometric correction is to transform images taken from different perspectives to a unified reference plane, eliminate geometric distortion caused by differences in perspective, and provide pixel-to-pixel correspondence for pixel-level fusion.

[0090] The geometric correction employs a multi-view geometric method, specifically including the following steps: First, select a reference coordinate system. The reference coordinate system can be the drone camera coordinate system corresponding to the main fusion node, or the world coordinate system at the center of the scene.

[0091] Next, the homography matrix between each view and the reference view is calculated. For ground scenes or near-planar scenes, perspective transformation is performed using the homography matrix; for non-planar scenes, epipolar correction is performed using a fundamental matrix or essential matrix combined with stereo correction methods. The homography matrix is ​​calculated based on the matching feature point pairs in each view, and is solved using the least squares method or the RANSAC algorithm to obtain a 3×3 transformation matrix.

[0092] Next, a perspective transformation is applied to each view data block. The coordinates (u, v) of each pixel in the original image are transformed to coordinates in the reference view coordinate system using a homography matrix, generating a corrected image block. During the transformation, bilinear or bicubic interpolation is used to resample the pixel values ​​to ensure the smoothness of the corrected image.

[0093] S3.5: The main fusion node performs pixel-level weighted fusion of the corrected image blocks according to their spatial location to generate a virtual composite data stream, stores the generated virtual composite data stream in a local cache, and synchronizes the access address and data stream description to the joint feature pool.

[0094] The fusion weights are determined based on the following factors: the resolution of each source image and the distance between the center of each source image and the center of the scene.

[0095] The initial fusion weights use preset empirical values, such as a weighted formula based on distance and resolution: ; in, For the first The resolution weight of each source image is defined as the ratio of the total number of pixels in the image to the baseline resolution (e.g., 1920×1080), or can be set according to actual needs. The distance from the scene center is defined as the geometric center of all drone poses (3D spatial coordinates) participating in the fusion, i.e., the arithmetic mean of the position coordinates of each drone. For each source image, the distance from the scene center is the Euclidean distance between the corresponding drone position coordinates and the scene center. and These represent the maximum and minimum distances from the drone to the center of the scene among all the source images involved in the fusion.

[0096] Pixel-level fusion is performed pixel-by-pixel, for each pixel position. fused pixel values ,in, For the first The coordinates of the source image after correction The pixel value at that location, if the source image is at the corrected coordinates If there are no valid pixels at a location (i.e., it is outside the image boundary), then the source image will not participate in the fusion at that location.

[0097] Furthermore, the main fusion node stores the generated virtual composite data stream in a local cache. The virtual composite data stream is organized in the form of a video frame sequence, with each fusion frame accompanied by its corresponding fusion weight configuration, a list of source images participating in the fusion, and spatiotemporal alignment residuals and pixel fusion residuals from the fusion process.

[0098] After storage is complete, the main fusion node synchronizes the access address and data stream description of the virtual synthesized data stream to the joint feature pool. The data stream description includes: the feature pair identifiers corresponding to the virtual synthesized data stream, the timestamp range of the fusion frame, the spatial area covered (the projection area calculated from the UAV pose), and the size and encoding format of the fused image.

[0099] Other edge nodes can obtain the access address of the virtual synthetic data stream by querying the joint feature pool, and call the data stream for subsequent analysis as needed, avoiding repeated fusion calculations.

[0100] S4: Analyze the virtual synthetic data stream and upload the results to the cloud; the cloud generates online adaptive parameters and sends them down in real time to optimize subsequent processing.

[0101] S4.1: The main fusion node performs target detection and scene parsing on the virtual synthetic data stream, generates structured analysis results, and records the time alignment residual and pixel fusion residual during this collaborative fusion process as correlation bias. The main fusion node packages the structured analysis results and the correlation bias into an upload data packet and sends it to the cloud.

[0102] In this embodiment of the invention, the main fusion node performs target detection and scene parsing on each frame of the virtual synthetic data stream. Target detection employs a deep learning-based target detection model, such as YOLOv8, RT-DETR, or Faster R-CNN (these models are well-known existing technologies in the field; in practice, any suitable model can be selected based on computational resources and accuracy requirements. Specific network structures and parameter configurations can be found in relevant literature and will not be elaborated here). The input is the fused frame image, and the output is the category labels of various targets present in the image, the coordinates of the detection boxes (top left and bottom right corners), and the detection confidence score. The detection results are stored in a structured format, including target identifier, category, location, size, and confidence score.

[0103] Scene parsing employs a deep learning-based semantic segmentation model, such as DeepLabV3+, SegFormer, or Mask2Former (these are merely examples; in practical applications, any known semantic segmentation model in the field can be used. Specific implementation details are not the focus of this invention and will not be elaborated upon). The input is a fused frame image, and the output is a semantic segmentation map of the same size as the input image. Each pixel location corresponds to a semantic category label (e.g., road, sky, building, vegetation, vehicle, pedestrian, etc.). The semantic segmentation results are stored in a graph structure, containing the boundaries of each semantic region and the pixel affiliation.

[0104] The main fusion node generates structured analysis results based on the target detection results and scene parsing results. The structured analysis results include: frame number, timestamp, scene semantic composition (percentage of each semantic category), target list (including category, location, size, and confidence), and annotation information of regions of interest (dense target regions or semantically complex regions).

[0105] Furthermore, while completing the analysis, the main fusion node records two types of correlation deviations during this collaborative fusion process: time alignment residual and pixel fusion residual.

[0106] The time alignment residual records the timestamp difference between the source images participating in the fusion process. Specifically, in the time alignment operation of step S3, the main fusion node records the absolute value of the deviation between the timestamp of each source image and the timestamp of the reference frame, and takes the maximum value of each deviation as the time alignment residual for this fusion. The reference frame timestamp is typically selected from the source image with the timestamp centered.

[0107] The time alignment residual reflects the synchronization accuracy of multiple source views in the time dimension. The larger the residual, the higher the degree of asynchrony between source images.

[0108] In the pixel-level fusion operation of step S3, the main fusion node calculates the structural similarity index (SSIM) between each source image and the fused image. The SSIM measures the similarity between two images from three dimensions: brightness, contrast, and structure, and its value ranges from [0,1], with values ​​closer to 1 indicating higher similarity. Pixel fusion residuals... Defined as: ; in, For the merged image, For the first There are 1 source image. The average of the pixel fusion residuals of each source image is taken as the pixel fusion residual for this fusion. The pixel fusion residual reflects the information fidelity of the fused image relative to the source images. The larger the residual, the more severe the distortion or information loss introduced during the fusion process.

[0109] The main fusion node encapsulates the temporal alignment residual and pixel fusion residual into an associated bias data structure. The associated bias also includes the feature pair identifier corresponding to this fusion, the number of source images participating in the fusion, the timestamp of the fusion frame, and the fusion weight configuration of each source image.

[0110] S4.2: In this embodiment of the invention, generating online adaptive parameters in the cloud includes the following steps: The cloud receives the association deviations uploaded by multiple edge nodes, groups and aggregates the association deviation data according to feature pair identifiers or semantic types, and calculates the following statistics for each group of data: First, for the time alignment residuals, calculate the average time alignment residuals of all fusion operations within the time window. For pixel fusion residuals, the average value of pixel fusion residuals for all fusion operations within the time window is calculated. Secondly, a linear regression method is used, with the time sequence of fusion operations as the independent variable x and the correlation deviation as the dependent variable y, to fit a regression line y = kx + b, where the slope k represents the trend of deviation change. A positive k indicates an upward trend in deviation, suggesting a decrease in system performance; a negative k indicates a downward trend in deviation, suggesting an improvement in system performance. b is the intercept.

[0111] Cloud-based time-aligned residual average Compared with the preset target value The difference is used to generate the pre-association threshold correction amount. Preset target value The value is set according to the system's real-time requirements, typically 30 milliseconds or 50 milliseconds. The formula for calculating the pre-association threshold correction is as follows: ; in, This is a preset proportionality coefficient, with a value ranging from 0.1 to 0.5. For a fixed time base value (e.g., 100 milliseconds), the time residual is converted into a dimensionless relative deviation before scaling. Exceed hour, A positive value indicates that the time asynchrony problem is quite serious in the current fusion process, requiring an increase in the preset association threshold in S1.3 to make the piecewise clustering more rigorous, reduce the number of source images participating in the fusion, and reduce the difficulty of time alignment; when Below hour, A negative value indicates that the time synchronization is good, and the preset association threshold can be appropriately reduced to make the segmentation and clustering more lenient, increase the number of source images participating in the fusion, and improve the fusion coverage.

[0112] Furthermore, when generating the correction amount, the cloud takes into account the deviation change trend k. If k is positive and large, it indicates that the time asynchrony problem is worsening, and the correction amount should be increased appropriately; if k is negative, it indicates that the problem is improving, and the correction amount can be decreased appropriately.

[0113] Furthermore, based on the deviation trend k of the time alignment residual and the preset time tolerance... The comparison results generate a geometric verification threshold correction. Preset time tolerance Characterizes the maximum degree of time asynchrony allowed by the system, typically 100 milliseconds.

[0114] Specifically, the strategy for generating the geometric verification threshold correction is as follows: If k>0 and > This indicates that the time asynchrony problem has continued to worsen and has exceeded the tolerance range, requiring an increase in the geometric verification threshold. That is, relaxing the preset matching threshold and preset geometric verification threshold in S2.2 to tolerate greater spatiotemporal deviations and improve the success rate of cross-node feature association. Take a positive value, when Exceeding Within 30 milliseconds, Take +0.1; for values ​​exceeding 30 to 70 milliseconds, take +0.2; for values ​​exceeding 70 milliseconds, take +0.3. If k < 0 and < This indicates that time synchronization is in good condition and is improving, which allows for a reduction in the geometric verification threshold, i.e., tightening the matching conditions and improving the accuracy of feature association. When Below Within 10 milliseconds, Use 0; for values ​​between 10 and 30 milliseconds, use -0.1; for values ​​above 30 milliseconds, use -0.2. Otherwise, Take zero.

[0115] The geometric verification threshold correction value generated in the cloud applies to both the preset matching threshold and the preset geometric verification threshold, maintaining the relative relationship of the matching conditions.

[0116] Finally, the cloud extracts the fusion weights of each source image from the correlation deviation database for each fusion operation. and its corresponding pixel fusion residual : For each semantic type or feature pair identifier corresponding to the fusion operation, the average pixel fusion residual of all source images is calculated in the cloud. For the first A source image, if its pixel fusion residual Greater than This indicates that the source image contributed significantly to the distortion during the fusion process, and the current weight setting is too high, requiring a reduction in its weight; if Less than This indicates that the source image is of good quality and its weight can be appropriately increased.

[0117] The formula for calculating the fusion weight adjustment is as follows: ; in The learning rate, ranging from 0.01 to 0.1, controls the step size for weight adjustment. The adjusted weights... Normalization is required to ensure that the sum of the weights of all source images is 1.

[0118] The cloud can smooth the adjustment amount of multiple fusion operations and take the moving average as the final fusion weight adjustment amount, avoiding drastic weight fluctuations caused by a single outlier.

[0119] As can be seen, through statistical analysis of the correlation deviation, this invention achieves quantitative evaluation and precise control of the system's processing performance. The mean deviation reflects the current overall performance level of the system, while the deviation trend reflects the direction of performance evolution. The combination of these two factors allows parameter correction to consider both the current state and the trend, achieving predictive adjustment. The three types of correction quantities correspond to the three key stages of front-end segmentation and clustering, feature matching, and pixel fusion, forming an end-to-end closed-loop optimization system. Each correction quantity is independent yet cooperative, jointly improving the overall processing quality of the system.

[0120] Finally, the cloud encapsulates the generated collaborative learning parameters into a delivery message, which is then sent to the corresponding edge nodes and drones via the control channel. The delivery message includes the following fields: node identifier, parameter type (pre-association threshold correction, geometric verification threshold correction, fusion weight adjustment), correction value, and effective time. The cloud broadcasts the pre-association threshold correction to all drones and unicasts the geometric verification threshold correction and fusion weight adjustment to the corresponding edge nodes.

[0121] After receiving the collaborative learning parameters from the cloud, the edge nodes perform corresponding update operations based on the parameter type: For geometric verification threshold correction The edge nodes update the preset matching threshold and preset geometric verification threshold used in S2.2. The update method is: new preset matching threshold = old preset matching threshold + × New preset geometric verification threshold = old preset geometric verification threshold + × ,in, and The preset scaling factor, in this embodiment of the invention, takes values ​​of 0.2 and 1.0 respectively, to adapt to threshold parameters with different dimensions. The updated threshold takes effect immediately and is used for feature matching operations of subsequent received logical data fragments.

[0122] For the adjustment of fusion weight Edge nodes (especially those that may become the primary fusion node later) update the pixel-level weighted fusion weights used in S3.3. The updated weights are stored in the local configuration of the edge nodes for subsequent collaborative fusion operations.

[0123] The drone receives the pre-association threshold correction amount sent from the cloud. Then, update the preset association threshold used in S1.3. The update method is: new preset association threshold = old preset association threshold + The updated threshold takes effect immediately and is used for subsequent sharding and clustering operations on the collected view data.

[0124] The drone can also adaptively adjust the spatiotemporal weight coefficients in the neighbor block association strength value calculation based on the pre-association threshold correction amount. For example, three fixed weight levels are preset: spatial priority (higher spatial weight, lower temporal weight, suitable for scenarios with dense moving targets), balanced (equal spatial and temporal weights, suitable for general scenarios), and temporal priority (lower spatial weight, higher temporal weight, suitable for scenarios with intense drone movement). The specific weight values ​​of the three levels are preset during system initialization and do not change during operation.

[0125] The pre-association threshold correction amount distributed from the cloud is divided into four qualitative intervals based on its numerical value: positive large interval (pre-association threshold correction amount greater than +0.05), positive small interval (pre-association threshold correction amount greater than 0 and less than or equal to +0.05), negative small interval (pre-association threshold correction amount less than 0 and greater than or equal to -0.05), and negative large interval (pre-association threshold correction amount less than -0.05). The 0.05 value is a fixed empirical threshold, which does not require calculation and is only used as a comparison benchmark.

[0126] Upon receiving the pre-association threshold correction amount from the cloud, the drone will switch states according to the following rules based on its current weight level and the correction amount's range: If the correction amount is in a large positive range: regardless of the current weight level, the drone will switch to a space priority weight level. If the correction amount is in a large negative range: regardless of the current weight level, the drone will switch to a time priority weight level. If the correction amount is in a small positive range: based on the current weight level, the drone will switch to a balanced weight level if it is already in a time priority weight level; if it is already in a balanced weight level, it will switch to a space priority weight level; if it is already in a space priority weight level, it will remain unchanged. If the correction amount is in a small negative range: based on the current weight level, the drone will switch to a balanced weight level if it is already in a space priority weight level; if it is already in a balanced weight level, it will switch to a time priority weight level; if the correction amount is zero: the drone will maintain its current weight level.

[0127] Using the above rules, the drone only needs to perform simple comparisons and state transitions to achieve adaptive adjustment of the spatiotemporal weight coefficients. This adjustment mechanism works in conjunction with the update of the preset association threshold to jointly optimize the sensitivity of the front-end segmentation clustering.

[0128] As can be seen, the closed-loop mechanism for parameter distribution and updating in this invention enables the optimization effect to be transmitted from the cloud to the edge nodes, and then to the drone, spanning the entire data processing chain. The clustering sensitivity adjustment on the drone side, and the matching threshold and fusion weight adjustment on the edge nodes together constitute a complete adaptive optimization system from data acquisition to collaborative fusion.

[0129] Furthermore, such as Figure 4 As shown, this embodiment also provides an edge-cloud collaborative analysis and learning system for drone views, including, The view semantic sharding module dynamically packages the collected view data into logical data shards carrying unified metadata based on semantic and spatiotemporal information, and sends them to the associated edge nodes according to the semantic type. The feature association and publishing module is used to extract features and generate cross-node association identifiers based on semantic and spatiotemporal information after the edge nodes receive logical data fragments, and then publish them to the shared joint feature pool. The collaborative fusion generation module establishes linkage between edge nodes based on features, performs geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generates a virtual synthetic data stream. The cloud-based learning optimization module analyzes the virtual synthetic data stream and uploads the results to the cloud; the cloud generates online adaptive parameters and distributes them in real time to optimize subsequent processing.

[0130] This embodiment also provides a computer device suitable for edge-cloud collaborative analysis and learning methods of UAV view, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the edge-cloud collaborative analysis and learning method of UAV view as proposed in the above embodiment.

[0131] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0132] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the edge and cloud collaborative analysis and learning method for realizing UAV view as proposed in the above embodiments.

[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A collaborative analysis and learning method for edge and cloud-based UAV views, characterized in that: include: Based on semantic and spatiotemporal information, the drone dynamically packages the collected view data into logical data fragments carrying unified metadata and sends them to the associated edge nodes according to the semantic type. After receiving logical data fragments, the edge nodes perform feature extraction and cross-node association identification based on semantic and spatiotemporal information, and publish them to the shared joint feature pool. Edge nodes establish linkages with identifiers based on features, perform geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generate a virtual synthetic data stream; Analyze the virtual synthesis data stream and upload the results to the cloud; Online adaptive parameters are generated in the cloud and distributed in real time to optimize subsequent processing. The generation of the logical data fragments includes: The drone acquires the current frame and its corresponding high-precision timestamp and drone pose data, and extracts the motion vector field between the current frame and the key frames in the previous sliding window. Based on the motion vector field, the neighboring block association strength value of each pixel block is calculated. The drone clusters adjacent pixel blocks with a neighbor association strength value higher than a preset association threshold into visual object units, and classifies and identifies each visual object unit to generate semantic type labels, forming a structured description of the visual object unit-semantic type-internal pixel association relationship. Logical data fragments are generated for each visual object unit and routed according to the semantic type and load status registered by the edge nodes. The content includes: a header carrying a semantic type label, a high-precision timestamp, UAV pose, and internal pixel relationships; and a body carrying the corresponding view data block. Cloud-based online adaptive parameters are generated and distributed in real time to optimize subsequent processing, including: The cloud receives correlation deviations uploaded from multiple edge nodes, groups and aggregates the correlation deviation data according to feature, identifier or semantic type, and calculates residual statistics for each group of data. The cloud updates the segmentation clustering threshold, feature matching threshold, geometric verification threshold, and fusion weight based on the residual statistics, and sends them to the corresponding edge nodes.

2. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 1, characterized in that: The motion vector includes a horizontal motion component, a vertical motion component, a motion direction, and a motion amplitude; The motion vector field is extracted using optical flow or block matching methods.

3. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 2, characterized in that: The feature extraction process includes: The edge nodes guide feature extraction based on the internal pixel association relationship, extract key point features and descriptors in the view data block, and combine the semantic type label and the UAV pose to associate feature points that satisfy spatial geometric consistency and have the same semantics in different logical data slices received from the edge nodes into a set of feature pairs. The criteria for determining spatial geometric consistency include: the ratio of the nearest distance to the second nearest distance of the descriptor is less than a preset matching threshold, the reprojection error is less than a preset geometric verification threshold, and the semantic labels are consistent.

4. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 3, characterized in that: The generation of the cross-node association identifier includes: The edge nodes assign globally unique feature pair identifiers to each group of feature pairs, and combine the feature pairs and the feature pair identifiers to form a pre-association result; Edge nodes write the pre-association results into a shared joint feature pool and publish them using the feature pairs as indexes for other edge nodes to query and access.

5. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 4, characterized in that: The method of establishing linkage based on features includes: Edge nodes periodically query the joint feature pool. When they detect a pre-association result published by another edge node that has the same feature pair identifier as the pre-association result published by themselves, they extract the node address and corresponding sharding spatiotemporal stamp information of the other edge node.

6. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 5, characterized in that: The generation of the virtual synthetic data stream includes: Based on the node address, the edge node establishes a point-to-point communication connection with other edge nodes, exchanges the UAV pose and the high-precision timestamp in the logical data slices carrying feature pairs, and generates a linkage link that records the corresponding relationship of nodes according to the pose relative relationship and time alignment window. Each participating edge node determines a master fusion node through an election mechanism, and the remaining edge nodes act as subordinate nodes. The main fusion node obtains the corresponding logical data fragments from itself and its subordinate nodes based on the high-precision timestamp in the linkage link, performs time alignment on each obtained logical data fragment, and performs geometric correction on each view data block based on the UAV pose using a multi-view geometric method. The main fusion node performs pixel-level weighted fusion of the corrected image blocks according to their spatial location to generate a virtual composite data stream. The generated virtual composite data stream is stored in a local cache, and the access address and data stream description are synchronized to the joint feature pool.

7. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 6, characterized in that: The analysis of the virtual synthetic data stream includes: The main fusion node performs target detection and scene parsing on the virtual synthetic data stream, generates structured analysis results, and records the time alignment residual and pixel fusion residual during this collaborative fusion process as correlation bias. The main fusion node packages the structured analysis results and the correlation deviation into an upload data packet and sends it to the cloud; The cloud-generated online adaptive parameters include: The cloud receives the correlation deviations uploaded by multiple edge nodes, performs statistical analysis on each correlation deviation, and generates collaborative learning parameters.

8. The edge and cloud collaborative analysis and learning method for UAV views as described in claim 7, characterized in that: Statistical analysis was performed on the aforementioned correlation deviations to generate collaborative learning parameters, including: The cloud receives the correlation deviations uploaded by multiple edge nodes, aggregates each correlation deviation according to a preset time window, and calculates the statistics of the correlation deviations, including the mean deviation and the deviation change trend. The cloud generates a pre-association threshold correction amount for updating the preset association threshold based on the difference between the mean deviation and the preset target value; Based on the comparison results of the deviation change trend and the preset time tolerance, the cloud generates a geometric verification threshold correction amount for updating the preset matching threshold and the preset geometric verification threshold; Based on the comparison results of the pixel fusion residuals and the average residuals corresponding to each source image, the cloud generates a fusion weight adjustment amount for updating the pixel-level weighted fusion weights.

9. An edge-cloud collaborative analysis and learning system for UAV views, based on the edge-cloud collaborative analysis and learning method for UAV views according to any one of claims 1 to 8, characterized in that: Also includes: The view semantic sharding module dynamically packages the collected view data into logical data shards carrying unified metadata based on semantic and spatiotemporal information, and sends them to the associated edge nodes according to the semantic type. The feature association and publishing module is used to extract features and generate cross-node association identifiers based on semantic and spatiotemporal information after the edge nodes receive logical data fragments, and then publish them to the shared joint feature pool. The collaborative fusion generation module establishes linkage between edge nodes based on features, performs geometric correction and collaborative fusion based on a unified spatiotemporal benchmark, and generates a virtual synthetic data stream. The cloud-based learning optimization module analyzes the virtual synthetic data stream and uploads the results to the cloud. Online adaptive parameters are generated in the cloud and distributed in real time to optimize subsequent processing.

Images