A lightweight multi-view adaptive data fusion system and method for edge-cloud collaboration

Abstract

This invention discloses a lightweight multi-view data adaptive fusion system and method with edge-cloud collaboration, belonging to the fields of edge intelligence and multi-view learning technology. The edge side synchronously acquires images, video frames, time-series sensor data, and service tags from different observation angles; the edge side completes data synchronization and standardization, lightweight feature extraction, view quality assessment, cross-view attention and gating fusion, and dynamically adjusts view branches, channel retention rates, and early termination thresholds based on computing power, bandwidth, latency, and on-site interference; the cloud side utilizes a teacher model to perform global cross-view modeling, knowledge distillation, incremental aggregation, and model version distribution. This invention can dynamically allocate tasks between low-latency local response and high-precision cloud optimization, and maintain local inference, caching, retransmission, and re-diagnosis under weak network or network outage conditions, thereby reducing computational load and transmission overhead, and improving the real-time performance, robustness, and continuous adaptability of multi-view fusion in complex environments.

Description

Technical Field

[0001] This invention belongs to the fields of edge intelligence, industrial IoT, smart water conservancy informatization, automated monitoring and multi-view learning technology. Specifically, it relates to a lightweight multi-view data adaptive fusion system and method with edge-cloud collaboration, and in particular to a multi-view fusion technology that can dynamically adjust the model structure and task position according to computing power, bandwidth, latency and view quality under complex field conditions. Background Technology

[0002] With the development of the Industrial Internet of Things (IIoT), smart water conservancy, urban infrastructure monitoring, and automated inspection systems, field sensing devices are no longer limited to single sensors. Instead, they are typically composed of multiple cameras, various online sensors, data acquisition terminals installed at different angles, and historical data from business platforms. Multi-view data can characterize the same target object or monitoring scene from different perspectives, helping to reduce the impact of obstructions, local noise, single-point failures, and environmental disturbances on the identification results.

[0003] However, existing multi-view fusion methods still have significant shortcomings in engineering deployment. First, cloud-based deep learning models typically possess strong global modeling capabilities and high recognition accuracy, but they have a large number of parameters, high computational overhead, and rely on stable network transmission, making it difficult to meet the needs of low-latency monitoring, on-site alarms, and emergency response. Second, while lightweight edge models can reduce inference latency, they are prone to accuracy loss under conditions of strong occlusion, heterogeneous perspectives, lighting fluctuations, sensor drift, and changes in sample distribution. Third, existing solutions typically deploy the model structure fixed to edge devices, failing to dynamically tailor model channels or adjust task allocation based on real-time computing power, network bandwidth, and service latency requirements. Fourth, newly added samples and challenging on-site examples at the edge cannot be promptly incorporated into global optimization in the cloud, leading to decreased model adaptability after long-term operation.

[0004] Therefore, there is an urgent need for a lightweight multi-view data adaptive fusion system and method that enables edge-cloud collaboration, allowing it to quickly complete lightweight fusion of multiple perspectives and local response at the edge, complete high-precision global modeling and model updates in the cloud, and balance real-time performance, accuracy, bandwidth usage and continuous learning capabilities through dynamic pruning, task scheduling and incremental collaborative update mechanisms. Summary of the Invention

[0005] The purpose of this invention is to provide a lightweight multi-view data adaptive fusion system and method for edge-cloud collaboration, in order to solve the problems of existing multi-view fusion methods, such as difficulty in balancing accuracy and efficiency, insufficient generalization ability at the edge, high response latency in the cloud, inflexible task splitting, and inability to continuously update the edge model.

[0006] To achieve the above objectives, this invention constructs a three-tiered collaborative architecture consisting of edge-side sensing nodes, edge computing gateways, and a cloud-based management and control platform. Edge-side sensing nodes are responsible for synchronous multi-view data acquisition and initial labeling; edge computing gateways are responsible for data standardization, lightweight feature extraction, view quality assessment, cross-view adaptive fusion, dynamic pruning, early termination inference, and local emergency response; the cloud-based management and control platform is responsible for training a high-precision teacher model based on the full dataset, performing cross-regional or long-term analysis, completing knowledge distillation, aggregating incremental edge information, and distributing model versions and scheduling strategies.

[0007] Furthermore, the edge sensing node collects data from different observation angles. The collected data includes multi-view images, video frames, time-series sensing data, and service tags of the same target object or monitoring area, and timestamps, device numbers, and viewpoint identifiers are added to the collected data.

[0008] The edge computing gateway performs time synchronization, viewpoint calibration, noise suppression, missing viewpoint labeling, and scale normalization on the collected data to form a multi-view sample group. MobileNetV3-Small is used to extract lightweight features from each viewpoint data. Viewpoint reliability weights are generated based on occlusion ratio, blur level, signal-to-noise ratio, illumination stability, sensor status, and historical reliability. A fusion representation is generated through the cross-viewpoint attention fusion module and dynamic gating fusion module of DMVAF-Net. Furthermore, channel retention rate, viewpoint branch mask, and early warning are generated based on remaining computing power, network bandwidth, service latency requirements, and on-site interference intensity. A threshold is set to perform dynamic channel pruning, low-quality viewpoint branch closure, and early exit of intermediate inference heads. The edge computing gateway executes real-time inference, local alarms, and emergency response tasks on the edge side, and schedules global analysis, cross-regional sample mining, and model optimization tasks to the cloud management platform. The cloud management platform uses SwinTransformer-Base and Cross-ViewTransformer to form a cloud teacher model, performs cross-viewpoint modeling and anomaly sample verification on all samples, and performs knowledge distillation on the edge student model through soft labels, attention distribution, and intermediate features.

[0009] The cloud-based management platform receives local incremental sample summaries, hard example sample indexes, gradient summaries, or parameter summaries uploaded from the edge side, and completes aggregation, verification, and model version distribution.

[0010] The edge computing gateway outputs target identification results, state classification results, anomaly level, fusion confidence, view contribution, scheduling decision and model version information.

[0011] Furthermore, the edge sensing node includes one or more of the following: fixed camera, mobile inspection camera, drone camera, infrared or multispectral sensing device, online sensor and edge acquisition controller, with overlapping observation areas or temporal correlation between adjacent viewpoints for the same business object.

[0012] Furthermore, the edge computing gateway uses a sliding time window to match timestamps from different perspectives; when the difference between timestamps from multiple perspectives is less than the synchronization threshold, the corresponding data are grouped into the same candidate sample group; when there is slight asynchrony, alignment is achieved through linear interpolation, nearest neighbor frame interpolation, or short-term buffering; when a certain perspective is severely lagging, a delay flag is written for that perspective and its initial fusion weight is reduced.

[0013] Furthermore, the DMVAF-Net includes a multi-view lightweight feature extraction branch, a view quality evaluation branch, a cross-view attention fusion module, a dynamic gating fusion module, a dynamic channel pruning module, a complete inference output head, and at least one early termination inference head; the multi-view lightweight feature extraction branch shares or partially shares the parameters of the MobileNetV3-Small backbone network.

[0014] Furthermore, the view quality assessment branch aggregates the sharpness index, occlusion and disconnection degree, signal-to-noise ratio index, device health status and historical stability into a view quality score, and obtains the reliability weight of each view through softmax normalization; the cross-view attention fusion module calculates the correlation between different view features based on query, key and value mapping, and uses the view reliability weight to further weight the attention results.

[0015] Furthermore, the dynamic channel pruning module determines the channel retention rate based on the remaining computing power on the edge side, available network bandwidth, on-site interference intensity, and service latency constraint pressure, and determines the view branch mask based on the view reliability weight and view reliability threshold; when the confidence of the intermediate layer reaches the early termination threshold, the corresponding early termination inference head outputs the result, otherwise it enters the complete inference output head.

[0016] Furthermore, the cloud-based management platform performs classification distillation, feature distillation, and attention distillation on the cloud-based teacher model, and weights and aggregates the parameters or parameter summaries uploaded by multiple edge computing gateways according to the corresponding effective sample size; after the new model version is verified by accuracy regression, edge resource occupation regression, and key scenario sample replay, it is distributed in a grayscale manner, and a version rollback is performed when accuracy decreases or latency exceeds the limit.

[0017] Furthermore, when communication between the edge computing gateway and the cloud management platform is abnormal, the edge computing gateway continues to perform local inference and local alarms, and caches key samples, structured features, perspective weights, model versions and inference results according to business value levels; after communication is restored, the breakpoint resumes transmission, the cloud management platform performs supplementary diagnosis on the retransmitted data, and associates the supplementary diagnosis results with the real-time results on the edge side during the network outage to the same business object timeline.

[0018] Furthermore, the edge-cloud task scheduling adopts a set of candidate scheduling schemes for selection. The candidate scheduling schemes include at least edge independent execution, edge initial screening and cloud verification, cloud main judgment and edge supplementation, and cached delayed upload. The execution position, data type and rollback strategy of the current task are determined by comprehensively considering end-to-end latency, edge energy consumption, expected accuracy and network transmission overhead.

[0019] A lightweight multi-view data adaptive fusion method includes: synchronously acquiring multi-view data of the same target object or monitoring area at the edge, and recording timestamps, device numbers, and view identifiers; performing time synchronization, view calibration, noise suppression, missing view labeling, and scale normalization on the multi-view data at the edge to form a multi-view sample group; extracting lightweight features of each view using MobileNetV3-Small, and calculating view reliability weights based on image quality, sensor status, occlusion degree, signal-to-noise ratio, and historical reliability; generating a fusion representation using the cross-view attention fusion module and dynamic gating fusion module of DMVAF-Net; and determining the fusion representation based on edge computing power and network bandwidth. Adjust channel retention rate, view branch mask, and early termination threshold based on bandwidth, service latency requirements, and on-site interference intensity. When the reliability weight of a certain view is lower than the threshold or there is occlusion, disconnection, or strong noise, reduce the fusion contribution of that view or close the view branch. Keep real-time inference, local alarm, and emergency response tasks on the edge side, and schedule global analysis, long-term trend judgment, and model optimization tasks to the cloud. The cloud teacher model performs cross-view modeling on the full set of samples and sends distillation supervision or parameter updates to the edge student model. The edge side completes fine-tuning or hard case screening based on new data, uploads parameter summaries to the cloud for aggregation, and outputs fusion results, confidence, view contribution, and model version.

[0020] In this invention, the edge-side student model preferably employs a combination of MobileNetV3-Small and DMVAF-Net. MobileNetV3-Small is used to extract lightweight features on resource-constrained edge gateways; DMVAF-Net is used to generate dynamically fused representations based on the reliability and complementarity of different perspectives. DMVAF-Net includes a multi-view lightweight feature extraction branch, a perspective quality evaluation branch, a cross-view attention fusion module, a dynamic channel pruning module, and an early termination inference head.

[0021] The cloud-based teacher model preferably employs a combination of SwinTransformer-Base and Cross-ViewTransformer. SwinTransformer-Base is used to obtain hierarchical high-precision features, while Cross-ViewTransformer is used to learn long-distance associations and global complementarity relationships between different viewpoints. During the training phase, the cloud-based teacher model provides soft labels, attention distributions, intermediate features, and risk or state classification supervision to the edge student model, enabling the lightweight edge model to achieve better fusion accuracy while maintaining low computational cost.

[0022] The lightweight multi-view fusion model consists of multiple view branches, a quality assessment branch, a cross-view attention module, a dynamic gating fusion module, a dynamic channel pruning module, and an early termination inference head. The inputs from views 1 to N are processed by MobileNetV3-Small to extract lightweight features. A portion of these features is then fed into the view quality assessment module to generate reliability scores, while the other portion is fed into the cross-view attention module to calculate complementarity relationships. The outputs of both modules enter the dynamic gating fusion module to form a fused representation. Finally, dynamic channel pruning and an early termination inference mechanisms output classification, regression, or anomaly detection results. This figure emphasizes that this invention does not simply stitch together multi-view features, but rather achieves adaptive fusion through a joint structure of "quality assessment + attention interaction + gating fusion + dynamic pruning."

[0023] The method of this invention includes steps such as multi-view acquisition, data standardization, lightweight edge feature extraction, view quality assessment, adaptive cross-view fusion, dynamic pruning and early termination inference, edge-cloud task splitting, cloud-based teacher diagnosis and knowledge distillation, incremental update, and closed-loop output. This method can dynamically select view branches, channel retention rates, and task execution locations based on on-site computing power, network bandwidth, service latency requirements, and interference intensity. Attached Figure Description

[0024] Figure 1 This is a flowchart of the overall architecture of the three-level collaboration between the edge, cloud, and end in this invention.

[0025] Figure 2 This is a flowchart of the lightweight multi-view fusion model structure of the present invention.

[0026] Figure 3 This is the main flowchart of the multi-view data adaptive fusion method of the present invention.

[0027] Figure 4 This is a flowchart of the dynamic clipping and early termination reasoning process of this invention.

[0028] Figure 5 This is a flowchart of the dynamic splitting and scheduling process for edge-cloud tasks in this invention.

[0029] Figure 6 This is a flowchart of the cloud-edge knowledge distillation and incremental update process of this invention.

[0030] Figure 7 This is a flowchart of the multi-perspective quality assessment and weight fusion process of the present invention.

[0031] Figure 8 This is a flowchart of the local caching, retransmission, and diagnostic process under weak or disconnected network conditions.

[0032] Figure 9 This is a flowchart of a smart water conservancy pump and gate monitoring scenario according to Embodiment 2 of the present invention.

[0033] Figure 10 This is a flowchart of an industrial IoT production line inspection scenario according to Embodiment 3 of the present invention.

[0034] Figure 11 This is a flowchart of continuous operation and supplementary diagnosis under weak network conditions in Embodiment 4 of the present invention. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0036] Combination Figures 1 to 11 It can be seen that the technical chain of this invention includes seven aspects: edge-side multi-view data acquisition, edge-side lightweight fusion inference, edge-cloud task dynamic scheduling, cloud-based teacher model optimization, edge student model incremental update, weak network continuous operation guarantee, and closed-loop application in typical engineering scenarios. Among them, Figure 1 Used to illustrate the three-tier collaborative architecture of edge-cloud; Figure 2 Used to illustrate the structure of a lightweight multi-view fusion model; Figure 3 Used to illustrate the main process of the method; Figure 4 Used to illustrate the reasoning logic of dynamic clipping and early termination; Figure 5 Used to illustrate the task splitting and scheduling logic; Figure 6 Used to illustrate knowledge distillation and incremental update mechanisms; Figure 7 Used to illustrate the process of perspective quality assessment and weight fusion; Figure 8 This is used to explain the data caching and diagnostic mechanisms under weak network or network outage conditions; Figures 9 to 11 These documents illustrate implementation procedures for smart water conservancy pump and gate monitoring, industrial IoT production line inspection, and continuous operation scenarios with weak network conditions.

[0037] Example 1: Overall Operation Flow and Structure of the Edge-Cloud Collaborative System

[0038] 1. System composition.

[0039] In this embodiment, the edge sensing node can be deployed in industrial production lines, river and lake shorelines, pumping stations, construction sites, traffic intersections, or other distributed monitoring scenarios to collect multi-view images, video clips, infrared data, sensor time-series data, and service tags for the same target object or the same monitoring area. The edge sensing node can be installed in a fixed location, used for mobile inspections, or monitored by drones, and a time synchronization module adds a unified timestamp to each frame or segment of data. Preferably, the edge sensing node includes a data acquisition interface, a clock synchronization module, an edge buffer module, and a device self-test module. The data acquisition interface is used to access logs from cameras, infrared modules, vibration sensors, water level gauges, flow meters, or controllers. The self-test module periodically records states such as lens occlusion, sensor offline status, low battery, and clock drift, and uses these states as input factors for subsequent viewpoint quality assessment.

[0040] The edge computing gateway is deployed close to the edge sensing node and preferably includes a data access interface, a preprocessing acceleration module, a lightweight multi-view fusion model inference module, a local caching module, a resource monitoring module, and a task scheduling and execution module. The edge computing gateway is responsible for initial screening, synchronization, quality assessment, and local inference of multi-view data, and directly outputs local alarms when abnormal or high-priority events occur. The resource monitoring module continuously collects CPU, GPU or NPU utilization, memory availability, queue length, average inference latency, and network status, and refreshes scheduling parameters at a fixed decision cycle (e.g., 0.5 seconds to 5 seconds) to drive channel pruning, view shutdown, early inference termination, and cloud upload ratio adjustment.

[0041] The cloud-based management platform includes a full data storage module, a global training module, a teacher model inference module, a knowledge distillation module, an incremental aggregation module, a strategy generation module, and a model version management module. The cloud-based management platform performs cross-regional, cross-time, and cross-device analysis on data from multiple edge gateways to generate more stable global model parameters and scheduling strategies. Preferably, the cloud-based management platform also includes a sample deduplication module, a difficult example replay module, and a version canary release module. The sample deduplication module eliminates redundant training samples caused by the same object being uploaded multiple times. The difficult example replay module prioritizes retaining samples with low confidence, conflicting perspectives, and manually reviewed and corrected errors. The version canary release module first distributes the new model to some edge gateways to verify whether its accuracy, latency, and resource consumption meet threshold requirements before deciding whether to release it to the entire platform.

[0042] like Figure 1As shown, a closed-loop collaborative link is formed between the edge sensing nodes, the edge computing gateway, and the cloud management platform. The edge sensing nodes are responsible for synchronously collecting multi-view images, video clips, and sensor sequences of the same business object and attaching a unified timestamp; after receiving the data, the edge computing gateway completes standardized preprocessing, lightweight feature extraction, adaptive fusion, local inference, and emergency response; after receiving key samples, structured features, or task requests selected from the edge, the cloud management platform performs teacher model training, knowledge distillation, policy generation, and model version distribution. Figure 1 The "Fusion Result Output" node in the system simultaneously receives real-time results from the edge and verification results from the cloud, and writes the results back to the business platform according to the object number, time slice, and version number, thereby ensuring that real-time results and post-verification results can be traced on the same business object timeline.

[0043] 2. Model selection and model structure.

[0044] In this embodiment, the edge student model preferably adopts a combination of MobileNetV3-Small and the dynamic multi-view gating fusion network DMVAF-Net. MobileNetV3-Small has the characteristics of low parameter count, fast inference speed and suitability for edge device deployment, and can be used as a lightweight feature extraction backbone network for each view branch. DMVAF-Net is used to perform quality perception, attention interaction, gating fusion and dynamic cropping among multiple view features. Preferably, the input of each view branch can be an RGB image, an infrared image or a feature vector obtained by mapping from a temporal sensor; when the input is an image, it is first encoded into a lightweight feature map at a uniform scale of 224×224 or 320×320; when the input is a temporal feature, it can be mapped to an embedding space of the same dimension as the visual feature after one or more layers of one-dimensional convolution and fully connected projection, and then participate in subsequent cross-view fusion.

[0045] The DMVAF-Net model structure includes:

[0046] (1) Multi-view lightweight feature extraction branch, used to map each view input to lightweight features;

[0047] (2) Viewpoint quality assessment branch, used to generate quality scores based on image sharpness, occlusion ratio, signal-to-noise ratio, sensor status and historical reliability;

[0048] (3) Cross-view attention fusion module, used to calculate the complementary relationship between different viewpoints;

[0049] (4) Dynamic channel pruning module, used to generate channel mask and retention rate according to resource status;

[0050] (5) Early-retrieval inference head, used to output results in advance when the confidence level of the intermediate layer meets the requirements. Preferably, the cross-view attention fusion module first performs query, key and value mapping on the features of each view, and then re-weights the attention matrix through quality weights; the dynamic gating fusion module generates gating coefficients based on the quality score and attention context of each view. The effective features are accumulated into a fusion representation based on the contribution of the viewpoint; the dynamic pruning module generates a reserved mask between 0 and 1 for the fused channels or directly outputs the Top-K channel index, so as to reduce the computation of subsequent convolution and fully connected layers when edge computing power is tight.

[0051] Furthermore, the "task output head" described in the claims preferably includes a full inference output head and one or more early-retreat inference heads disposed in the intermediate layer. The full inference output head is used to output the final category, state, or anomaly level while retaining all computation paths; the early-retreat inference head is used to provide the result in advance when the confidence level in the intermediate layer has met the threshold, thereby reducing edge-side latency and computing power consumption together with the dynamic pruning mechanism.

[0052] Furthermore, the "dynamic pruning and early termination inference unit" described in the claims belongs to the system-level operation control unit. On the one hand, it calls the dynamic channel pruning module and task output header within the lightweight multi-view fusion model unit; on the other hand, it combines the remaining computing power, bandwidth, and latency constraints given by the resource monitoring module to generate the channel retention rate, view branch mask, and early termination threshold for the current period. Thus, a clear division of labor is formed between the internal structure of the model and the external scheduling control of the system, avoiding any ambiguity with the module division in claims 4 to 7.

[0053] The cloud-based teacher model preferably employs a combination of SwinTransformer-Base and Cross-ViewTransformer. SwinTransformer-Base is used to extract high-precision hierarchical visual features or multi-source encoded features, while Cross-ViewTransformer is used to capture long-range dependencies and global complementarity relationships between different viewpoints. The soft labels, attention matrix, and intermediate features output by the cloud-based teacher model are used to supervise the training of the edge student models, enabling them to approximate the fusion capabilities of the cloud-based teacher model as closely as possible under lightweight deployment conditions. Furthermore, the teacher model can additionally output intermediate-stage feature pyramids, inter-view similarity matrices, and error sample correction labels beyond the final layer. The edge student models receive supervision from classification distillation, feature distillation, and attention distillation during training to avoid insufficient learning of cross-view relationships due to reliance solely on the final soft labels.

[0054] 3. Data preprocessing and standardization.

[0055] After receiving data uploaded by the sensing nodes on the edge computing gateway, the system first performs time synchronization and viewpoint grouping, so that multiple observation data of the same target object or the same monitoring area within a similar time window constitute a sample group. For image or video data, the system can perform distortion correction, brightness normalization, noise reduction, cropping, and scale normalization; for sensor time-series data, the system can perform missing value marking, abnormal peak suppression, and sampling frequency unification. Preferably, time synchronization adopts a sliding time window matching strategy. When the difference between the timestamps of multiple views is less than a preset synchronization threshold, they are classified into the same candidate sample group; when there is slight asynchrony but still alignment is possible, alignment can be completed through linear interpolation, nearest neighbor frame interpolation, or short-term buffering waiting mechanisms; when a certain viewpoint is severely lagging, a delay mark is written to that viewpoint and its initial weight in this round of fusion is reduced.

[0056] When partial viewpoints are obstructed, interrupted, or subject to strong noise, the system does not discard the entire sample group directly. Instead, it reduces the weight of that viewpoint through a viewpoint quality assessment branch, or the missing viewpoint compensation submodule generates missing labels and historical mean features, enabling the model to continue using other valid viewpoints for inference. Preferably, the missing viewpoint compensation submodule does not directly forge complete images, but outputs a missing mask, a cache of the previous valid features, statistical mean features, or simplified prediction features, which are then input into the fusion network along with the current valid viewpoints. This ensures information consistency while reducing the interference of error compensation on the final judgment result.

[0057] like Figure 7 As shown, the viewing angle quality assessment and weighted fusion process first calculates quality factors such as sharpness, occlusion and disconnection, signal-to-noise ratio, and equipment status in parallel for the same sample group, and then aggregates these factors into a quality score. The weights for each viewpoint were then obtained through softmax normalization. The weights are then input into the weighted attention fusion module to generate the final fused representation. . Figure 7 Correspondingly, this invention explains how, when some viewpoints degrade, the invention suppresses the interference of low-reliability viewpoints on the overall judgment result through a dynamic weighting mechanism driven by quality scores. Preferably, the quality assessment module can be implemented using a combination of rules and learning: on the one hand, a basic score is generated from interpretable quality indicators; on the other hand, a lightweight quality assessment sub-network supplements and corrects complex factors such as scene interference, motion blur, and sensor drift. The final quality score is obtained by weighting the two together.

[0058] 4. Feature quantities and fusion formulas.

[0059] The formulas in this section illustrate the calculation methods for multi-view lightweight feature extraction, view reliability weighting, adaptive fusion, dynamic pruning, task scheduling, knowledge distillation, and incremental updates in this invention. For ease of explanation, the following formulas all describe the same multi-view sample group within the current inference period t, and the coefficients, thresholds, and weights in the formulas can be calibrated during the training or deployment phase according to the business scenario.

[0060] (1)

[0061] in, For the i-th viewpoint input, A lightweight feature extractor for edge-side MobileNetV3-Small. For student model parameters, Let N be the lightweight feature of the i-th viewpoint, and N be the number of viewpoints. Formula (1) means that the original input of each viewpoint is first encoded into a feature vector or feature map by a lightweight backbone network with shared or partially shared parameters, and then participates in subsequent quality assessment and cross-viewpoint fusion.

[0062]

[0063] (2)

[0064] in, The quality assessment score for the i-th viewpoint. The normalized viewpoint reliability weights, Here, j represents the temperature coefficient, and j represents the view index participating in the fusion. Indicates the clarity index, Indicates the occlusion ratio. This represents the signal-to-noise ratio (SNR) metric. This indicates the health status score of the equipment. This represents the stability score of the viewpoint within the historical window, with a1 to a5 being the weighting coefficients of the corresponding quality factors. Formula (2) illustrates that the system does not simply average the views, but rather integrates multiple quality factors to obtain the result. Then, the fusion weights are generated through softmax normalization.

[0065] (3)

[0066] in, For cross-perspective fusion representation, (·) represents a cross-view attention map, used to express the complementary relationship between the current viewpoint and other viewpoints. Let be the quality weight of the i-th perspective. Formula (3) means that each perspective feature is first calculated to be correlated with other perspective features, and then weighted and accumulated according to the quality weight, so as to retain the dominant information of the high-quality perspective and the auxiliary information of the low-quality but still supplementary perspective at the same time.

[0067] (4)

[0068] in, The current channel retention rate. Based on retention rate, For the remaining computing power at the edge, For available network bandwidth, The intensity of on-site interference, To account for business latency constraints, a, b, c, and d are weighting coefficients. The minimum retention rate is given by formula (4). Formula (4) shows that the system increases the channel retention rate when computing power and bandwidth are sufficient, and decreases the channel retention rate when interference intensity is high or delay pressure is high, so as to achieve a dynamic balance between accuracy and real-time performance.

[0069] (5)

[0070] in, For the i-th viewpoint branch mask, As the reliability threshold of the viewpoint, This represents the state of the i-th end-side device. When... When the value is 0, this perspective branch is closed or retains only a very small number of basic feature calculations in the current inference cycle; when... When the value is 1, the perspective continues to participate in the full fusion. Formula (5) is used to map the continuous quality weights to a discrete decision on whether to retain the perspective branch.

[0071] (6)

[0072] in, For edge-cloud task scheduling solutions, For the set of candidate scheduling schemes, For end-to-end delay, For energy consumption, To ensure the accuracy of the prediction, For network transmission overhead, to The scheduling weight is given by formula (6). Formula (6) indicates that the scheduler selects the task allocation method with the lowest cost from the candidate schemes after comprehensively considering latency, edge power consumption, network cost and expected accuracy.

[0073] (7)

[0074] in, For knowledge distillation loss, Output logits for the cloud-based teacher model. The marginal student model outputs logits, where T is the distillation temperature and KL represents the relative entropy. Formula (7) is used to constrain the output distribution of the student model to approximate the output distribution of the teacher model, so that the marginal student model can inherit the discrimination boundary information of the teacher model under the condition of low parameter quantity.

[0075] (8)

[0076] in, For total training losses, For business task losses, For intermediate feature alignment loss, For attention distribution alignment loss, to is the weight coefficient. Formula (8) shows that student model training does not rely solely on business supervision, but incorporates distillation loss, feature alignment loss and attention alignment loss into a unified optimization objective.

[0077] (9)

[0078] in, For the parameters of the (k+1)th round aggregation model in the cloud, For the k-th round of parameters or parameter summary uploaded by the e-th edge gateway, R represents the effective sample size on the corresponding edge side, and r is the index of the edge gateway participating in the aggregation. Formula (9) indicates that the cloud can perform weighted aggregation on the uploaded parameters according to the effective sample size of each edge gateway, so as to reduce the disturbance of a small number of abnormal samples or low-quality edge updates to the global model.

[0079] 5. Dynamic pruning and task scheduling.

[0080] During the operation of the edge computing gateway, the system continuously monitors the device's CPU, GPU or NPU load, memory usage, queue length, network bandwidth, and service latency requirements. When the service is in a high-priority state or the network is unstable, the system prioritizes maintaining the real-time inference link on the edge side and reduces the proportion of raw data uploaded; when the edge device load is too high, the system reduces the channel retention rate. The low-reliability perspective branch is closed, and the early-retreat inference head is enabled. Preferably, the scheduling decision is not completely recalculated every frame, but a periodic refresh strategy with a hysteresis interval is adopted: when the resource status change does not exceed the preset threshold, the decision of the previous cycle remains unchanged to avoid jitter caused by frequent switching; when the resource status deteriorates for several consecutive cycles, a more aggressive pruning and uploading strategy is triggered.

[0081] In practice, the dynamic pruning and early termination inference unit does not change the basic structure of the task output head, but adjusts the activation conditions of the complete inference output head and each early termination inference head: when resources are sufficient, the complete inference path is retained first; when latency pressure increases and the intermediate layer confidence meets the threshold, the corresponding early termination inference head is selected to output the result; when multiple perspectives degrade simultaneously, low-reliability branches are closed in conjunction with perspective branch masks. In this way, the hierarchical relationship between the task output head, the dynamic channel pruning module, and the system-level scheduling control remains consistent, facilitating the correspondence between the claims and the implementation methods.

[0082] The edge-cloud task scheduling unit calculates the cost of different task allocation schemes according to formula (6). For low-latency tasks such as alarms, control, and emergency response, priority is given to execution on the edge side; for cross-regional comparison, historical trend analysis, global model training, and strategy optimization tasks, priority is given to execution on the cloud; for medium-latency tasks, a collaborative approach of edge-side initial screening and cloud-side review can be adopted. Furthermore, the candidate scheduling schemes can include at least four modes: "independent edge execution", "edge initial screening + cloud-side review", "cloud-side main judgment + edge supplementation", and "delayed upload after caching". The system selects one of these modes based on the task level, remaining bandwidth, and business tolerance latency, and writes the final scheduling result along with the model version number into the structured log.

[0083] 6. Knowledge distillation and incremental collaborative updating.

[0084] The cloud-based teacher model is trained using full-sample, difficult-example, and cross-regional samples to obtain a high-precision cross-perspective global representation. After training, the cloud does not directly deploy the large model to all edge gateways. Instead, it transfers the teacher model's knowledge to the edge student model through soft-label distillation, feature distillation, and attention distillation. Preferably, distillation training can be divided into two stages: offline distillation and online incremental distillation. Offline distillation is used to generate a stable basic student model, while online incremental distillation is used to quickly correct the local capabilities of the student model using newly added difficult-example samples when new interferences, equipment aging, or changes in business distribution occur at the edge.

[0085] During operation, the edge computing gateway filters newly added local samples, samples with low confidence, samples with conflicting perspectives, and manually reviewed samples, and performs lightweight fine-tuning or generates gradient summaries based on these samples. After receiving parameter summaries uploaded by multiple edge gateways, the cloud performs weighted aggregation according to formula (9), and issues a new model version after verification. If the new version experiences a decrease in accuracy or exceeds latency limits in a certain edge scenario, the model version management module can perform a rollback. Preferably, the content uploaded by the edge side includes at least the sample index, scene label, model version number, local loss statistics, parameter summary, and resource usage summary, rather than uploading raw data indiscriminately, thereby balancing bandwidth saving, privacy protection, and continuous learning effects.

[0086] like Figure 3 As shown, the main flow of the method of this invention includes nine stages in sequence: multi-view acquisition, time synchronization and standardization, lightweight feature extraction, view quality assessment, adaptive cross-view fusion, dynamic pruning and early termination, edge-cloud task scheduling, cloud-based teacher diagnosis, and incremental update and closed-loop output. This diagram is used to illustrate how the method chain of this invention connects real-time edge fusion and continuous cloud optimization into a unified closed loop. The stages are not strictly executed sequentially, but are allowed to proceed in parallel in a pipelined manner during stable operation. For example, while the edge is processing the current cycle inference task, the cloud can simultaneously perform verification and distillation training on the difficult examples uploaded in the previous cycle.

[0087] like Figure 4 As shown, the dynamic pruning and early termination inference process starts by reading the edge resource status, first calculating the channel retention rate, and then determining whether the weight of a certain viewpoint is lower than a threshold. If it is lower than the threshold, the corresponding viewpoint branch is closed or reduced in weight; if it is higher than the threshold, the branch is retained to participate in subsequent inference. After pruning is completed, the system continues to determine whether the confidence of the intermediate layer meets the standard. If it does, early termination is executed; if it does not, the complete inference path is entered. Figure 4 The key point is how this invention adjusts the model's computational load synchronously based on real-time resources and viewpoint reliability. Preferably, the confidence level of the intermediate layer can be jointly determined by the maximum class probability, the difference between the probabilities of the first two classes, the stability of the outlier score, or the uncertainty after temperature calibration. Early termination is only triggered when the aforementioned indicators simultaneously meet the threshold conditions, thereby reducing the risk of missed judgments caused by false terminations.

[0088] like Figure 5 As shown, the dynamic task splitting and scheduling process between edge and cloud reads the current latency constraints, edge computing power, and network bandwidth based on the task request, and prioritizes determining whether it belongs to a low-latency task. For tasks such as alarms, control, and emergency response, the system executes them directly on the edge; for tasks requiring global analysis or long-term optimization, the system schedules them to the cloud for execution; for tasks in between, a collaborative processing mode of initial screening at the edge and review by the cloud is adopted. Figure 5 This is used to explain the execution logic of the scheduling decision corresponding to formula (6) in the project. Furthermore, the scheduling result can also be output as structured fields, including execution location, data type of data sent, expected latency, expected accuracy and rollback strategy, so that the business platform can audit and perform post-analysis on the task chain.

[0089] like Figure 8As shown, the data caching, retransmission, and supplementary diagnosis process under weak network or network outage conditions begins with network status monitoring. When the system determines that cloud-edge communication is normal, it maintains the real-time upload and cloud verification path; when it determines that communication is abnormal, it switches to edge local inference, local alarm, and key sample caching mode. After communication is restored, the system performs breakpoint resume transmission, the cloud completes supplementary diagnosis of cached samples, and associates the supplementary diagnosis results with the timeline of the same business object. Figure 8 This invention is used to illustrate its ability to maintain continuous service operation and correct post-hoc results even under unstable network conditions. Preferably, the key sample caching adopts a two-layer strategy: the first layer caches structured features, logs, and small-sized keyframes to ensure long-term sustainable operation, while the second layer caches a small number of high-value original fragments for high-precision verification after communication recovery.

[0090] like Figure 6 As shown, the cloud-edge knowledge distillation and incremental update process starts with the full set of samples in the cloud, constructs a high-precision teacher network using the SwinTransformer teacher model and Cross-ViewTransformer, and outputs soft labels, feature supervision, and attention supervision signals to perform distillation training on the edge student model. After training, the new model version is distributed to the edge gateway; the edge gateway performs incremental fine-tuning locally and then uploads the parameters or gradient summaries. The cloud performs aggregation and verification before distributing a stable version. Figure 6 The continuous learning and model version closed-loop update mechanism of this invention is explained in detail. Preferably, the cloud verification stage includes at least accuracy regression verification, edge resource consumption regression verification, and key scenario sample replay verification. The new version is marked as a releasable version only when all three meet the threshold.

[0091] Example 2: Smart water conservancy pump and gate monitoring scenario.

[0092] In the smart water conservancy pumping station monitoring scenario, the end-side sensing nodes include upstream and downstream cameras of the gate, cameras inside the pumping station, water level gauges, flow meters, and equipment status acquisition terminals. Data from different perspectives may be affected by rain, fog, nighttime supplemental lighting, reflective water surfaces, and partial obstruction. With this invention, the edge computing gateway first synchronizes the images and sensor data from each perspective, and then DMVAF-Net calculates the perspective reliability weights. When the upstream camera is affected by fog, the system reduces the weight of that perspective and makes greater use of downstream camera, water level gauge, and equipment status data to complete the gate status fusion judgment. Furthermore, the gate status can be subdivided into multiple categories such as "open, closed, stuck, abnormal vibration, and foreign object blockage," and the water level change rate, flow rate fluctuation amplitude, and equipment current fluctuation are used as auxiliary temporal perspectives to jointly input into the fusion model, thereby enhancing the ability to identify operating conditions that are difficult to distinguish when relying solely on images.

[0093] like Figure 9As shown, the process of the intelligent water conservancy pumping station monitoring embodiment can be divided into six stages: "multi-source sensing access—edge synchronous preprocessing—quality assessment and weighted fusion—gate status identification—abnormal alarm and task transmission—cloud review and incremental update". Specifically, upstream and downstream cameras of the gate, cameras inside the pumping station, water level gauges, flow meters, and equipment status acquisition terminals first complete synchronous acquisition at the end side; the edge computing gateway then performs time alignment, noise reduction, scale normalization, and view grouping; when the upstream camera is affected by fog, Figure 9 The quality assessment branch in the process will reduce the weight of this perspective and increase the contribution of downstream camera, water level gauge and equipment status data to the fusion result; if the confidence of the edge side identification result is high and it belongs to normal operating conditions, a local operation record will be generated directly; if the identification result shows that the gate is suspected of being stuck, the flow is abnormal or the equipment current is suddenly increased, the edge side will immediately trigger a local alarm and upload key frames, structured features and corresponding time windows, which will be reviewed by the cloud in combination with historical operating conditions and samples from nearby sites.

[0094] Figure 9 This also illustrates the closed-loop operation mode of the present invention in a pumping station scenario: the edge side is responsible for rapid status judgment and local response, while the cloud side is responsible for high-precision verification by combining historical operating conditions, cross-site samples, and teacher models, and then transmitting the new strategy and new model version back to the edge gateway. As can be seen from this embodiment, the present invention is not only applicable to static image recognition, but also to complex water conservancy equipment monitoring scenarios that include video perspectives and time-series sensor perspectives. Preferably, when low confidence or multi-view conflicts are detected within multiple consecutive time windows, the system automatically adds the sample to the difficult example pool for priority training by the cloud in subsequent incremental distillation stages.

[0095] Example 3: Industrial Internet of Things (IoT) production line inspection scenario.

[0096] In industrial IoT production line inspection scenarios, multiple cameras capture the status of workpieces or equipment from the front, sides, and top. Due to the fast production line cycle time, uploading all raw data to the cloud would cause significant bandwidth pressure and response latency. This invention deploys MobileNetV3-Small and DMVAF-Net student models in an edge computing gateway. Early dismissal inference is performed on normal workpieces, while more channels are retained for suspected abnormal workpieces, and keyframes are uploaded to the cloud for verification, thus balancing real-time performance and recognition accuracy. Furthermore, the system can further categorize workpiece defects into types such as missing parts, incorrect parts, surface scratches, color deviation, size deviation, and positional offset. For workstations with particularly fast cycle times, a dual-threshold strategy can be set at the edge: high-confidence normal parts are released early, medium-confidence samples enter full edge inference, and low-confidence or viewpoint conflict samples are then sent to the cloud for verification.

[0097] like Figure 10As shown, the process of an industrial IoT production line inspection embodiment includes six stages: "multi-camera acquisition—lightweight edge feature extraction—normal part early termination judgment—retaining more channels for suspected abnormal parts—uploading key frames to the cloud for verification—results being written back to the production line system." After multiple cameras simultaneously acquire the status of the workpiece or equipment from the front, side, and top, the edge computing gateway first performs synchronization and standardization, and then calls the student model for rapid inference; if the confidence level of the intermediate layer indicates that the workpiece is a normal sample, then according to... Figure 10 The early exit path directly outputs the detection results to reduce latency and computing power consumption; if the detection confidence is insufficient or multiple perspectives conflict, it enters the full inference path that retains more channels, and uploads the keyframes, defect candidate boxes, perspective contribution and decision logs to the cloud for review; after the cloud completes the review, the final results, defect categories and confidence are written back to the MES system, PLC controller or production line traceability platform.

[0098] Figure 10 This further illustrates the practical value of the invention in high-cycle production lines: by rapidly processing most normal samples at the edge, and only sending a small number of problematic samples to the cloud for review, bandwidth pressure is alleviated while ensuring the accuracy of anomaly identification. For issues such as missing parts, incorrect parts, surface defects, and positional offsets in the production line, the system can make an initial judgment at the edge, and generate high-confidence review and continuous model optimization results in the cloud. Preferably, if the number of a certain type of defect samples increases significantly within a shift, the cloud can also perform correlation analysis based on shift labels, workstation labels, and equipment logs to determine whether it is due to model deviation, equipment drift, or process anomaly, and then send the correction strategy back to the corresponding edge gateway.

[0099] Example 4: Continuous operation under weak network conditions.

[0100] When communication between the edge computing gateway and the cloud is interrupted, the system continues to perform local inference and local alarms at the edge, caching key samples, structured features, viewpoint weights, model versions, and inference results locally. After communication is restored, the system resumes interrupted transmission, the cloud performs supplementary diagnosis and incremental training on the cached samples, and associates the supplementary diagnosis results with the real-time results to the same business object's timeline. Preferably, the system prioritizes cached content according to business value: high-priority cache includes alarm event keyframes, abnormal time window features, and model judgment criteria; medium-priority cache includes structured logs and low-resolution thumbnails; and low-priority content is retained locally for a short period before being cyclically overwritten.

[0101] like Figure 11As shown, the process of continuous operation under weak network conditions begins with network status monitoring. The system first determines whether cloud-edge communication is normal; if normal, it maintains the real-time upload and cloud verification path; if abnormal, it immediately switches to edge local inference, local alarm, and key sample caching mode. The cached content preferably includes key samples, structured features, viewpoint weights, model version numbers, and local inference results, thereby ensuring business continuity and the integrity of the evidence chain during communication interruption. Furthermore, the local cache directory can be organized according to "object number - time slice - event level - model version" so that re-upload can be performed in priority order after communication is restored, and the confusion of evidence chains of different business objects can be avoided.

[0102] Figure 11 The document also details the retransmission and re-diagnosis mechanism after communication recovery: When the system detects communication recovery, it first performs breakpoint resumption on the local cached content, then the cloud performs re-diagnosis and incremental training on the cached samples, and associates the re-diagnosis results with the real-time edge results during the network outage to the same business object timeline. If the cloud re-diagnosis finds misjudgments or omissions during the network outage, it can correct the final business conclusion and update the model parameters synchronously. This embodiment illustrates that the present invention has a stable switching capability between the three states of weak network, network outage, and network recovery. Preferably, after receiving the retransmitted data, the cloud first verifies the timestamp integrity, cache digest hash, and model version consistency before deciding whether to include the batch of data in incremental training to avoid training pollution caused by incomplete breakpoint resumption.

[0103] In summary, this invention achieves real-time, accurate, and sustainable fusion of multi-perspective data in complex engineering scenarios through three-level collaboration between edge, cloud, and endpoint, a lightweight multi-view fusion model, dynamic pruning, task scheduling, and incremental collaborative update mechanisms. For those skilled in the art, any substitutions of data types, edge device models, teacher models, or student models without departing from the spirit and substance of this invention should fall within the scope of protection of this invention.

Claims

1. An edge-cloud collaborative lightweight multi-view data adaptive fusion system, characterized in that, This includes edge sensing nodes, edge computing gateways, and cloud management and control platforms; The edge sensing node collects data from different observation angles. The collected data includes multi-view images, video frames, time-series sensor data, and service tags of the same target object or monitoring area. The collected data is also appended with timestamps, device numbers, and viewpoint identifiers. The edge computing gateway performs time synchronization, viewpoint calibration, noise suppression, missing viewpoint labeling, and scale normalization on the collected data to form a multi-view sample group. MobileNetV3-Small is used to extract lightweight features from each viewpoint data. Viewpoint reliability weights are generated based on occlusion ratio, blur level, signal-to-noise ratio, illumination stability, sensor status, and historical reliability. A fusion representation is generated through the cross-viewpoint attention fusion module and dynamic gating fusion module of DMVAF-Net. Furthermore, channel retention rate, viewpoint branch mask, and early warning are generated based on remaining computing power, network bandwidth, service latency requirements, and on-site interference intensity. A threshold is set to perform dynamic channel pruning, low-quality viewpoint branch closure, and early exit of intermediate inference heads. The edge computing gateway executes real-time inference, local alarms, and emergency response tasks on the edge side, and schedules global analysis, cross-regional sample mining, and model optimization tasks to the cloud management platform. The cloud management platform uses SwinTransformer-Base and Cross-ViewTransformer to form a cloud teacher model, performs cross-viewpoint modeling and anomaly sample verification on all samples, and performs knowledge distillation on the edge student model through soft labels, attention distribution, and intermediate features. The cloud-based management platform receives local incremental sample summaries, hard example sample indexes, gradient summaries, or parameter summaries uploaded from the edge side, and completes aggregation, verification, and model version distribution. The edge computing gateway outputs target identification results, state classification results, anomaly level, fusion confidence, view contribution, scheduling decision and model version information.

2. The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system according to claim 1, wherein, The edge sensing nodes include one or more of the following: fixed cameras, mobile inspection cameras, drone cameras, infrared or multispectral sensing devices, online sensors, and edge acquisition controllers. Adjacent viewpoints have overlapping observation areas or temporal correlations for the same business object. 3.The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system of claim 1, wherein, The edge computing gateway uses a sliding time window to match timestamps from different perspectives. When the difference between timestamps from multiple perspectives is less than the synchronization threshold, the corresponding data is assigned to the same candidate sample group. When there is slight asynchrony, alignment is achieved through linear interpolation, nearest neighbor frame interpolation, or short-term buffering. When a perspective is severely lagging, a delay flag is written to that perspective and its initial fusion weight is reduced.

4. The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system according to claim 1, wherein, The DMVAF-Net includes a multi-view lightweight feature extraction branch, a view quality evaluation branch, a cross-view attention fusion module, a dynamic gating fusion module, a dynamic channel pruning module, a complete inference output head, and at least one early termination inference head; the multi-view lightweight feature extraction branch shares or partially shares the parameters of the MobileNetV3-Small backbone network.

5. The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system according to claim 4, characterized in that, The view quality assessment branch aggregates the sharpness index, occlusion and disconnection degree, signal-to-noise ratio index, equipment health status and historical stability into a view quality score, and obtains the reliability weight of each view through softmax normalization. The cross-perspective attention fusion module calculates the correlation between features from different perspectives based on query, key and value mapping, and then uses the perspective reliability weight to further weight the attention results.

6. The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system according to claim 4, characterized in that, The dynamic channel pruning module determines the channel retention rate based on the remaining computing power on the edge side, available network bandwidth, on-site interference intensity, and service latency constraint pressure, and determines the view branch mask based on the view reliability weight and view reliability threshold. When the confidence level of the intermediate layer reaches the early termination threshold, the corresponding early termination inference head outputs the result; otherwise, it enters the complete inference output head.

7. The edge-cloud collaborative lightweight multi-view data adaptive fusion system according to claim 1, wherein, The cloud-based management platform performs classification distillation, feature distillation, and attention distillation on the cloud-based teacher model, and performs weighted aggregation of parameters or parameter summaries uploaded from multiple edge computing gateways according to the corresponding effective sample size. After the new model version is verified by accuracy regression, edge resource occupation regression, and key scenario sample replay, it is distributed in a grayscale. If the accuracy decreases or the latency exceeds the limit, the version is rolled back. 8.The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system of claim 1, wherein, When communication between the edge computing gateway and the cloud management platform is abnormal, the edge computing gateway continues to perform local inference and local alarms, and caches key samples, structured features, perspective weights, model versions and inference results according to business value levels. After communication is restored, the breakpoint resumes transmission, the cloud management platform performs supplementary diagnosis on the retransmitted data, and associates the supplementary diagnosis results with the real-time results on the edge side during the network outage to the same business object timeline. 9.The end-edge-cloud collaborative lightweight multi-view data adaptive fusion system of claim 1, wherein, The edge-cloud task scheduling uses a set of candidate scheduling schemes for selection. The candidate scheduling schemes include at least edge independent execution, edge initial screening and cloud verification, cloud main judgment and edge supplementation, and cached delayed upload. The execution position, data type and rollback strategy of the current task are determined by comprehensively considering end-to-end latency, edge energy consumption, expected accuracy and network transmission overhead.

10. A lightweight multi-view adaptive data fusion method utilizing the edge-cloud collaborative lightweight multi-view data fusion system according to any one of claims 1 to 9, characterized in that, include: Simultaneously collect multi-view data of the same target object or monitoring area from the device side, and record the timestamp, device number and view identifier; At the edge, the multi-view data is time-synchronized, viewpoint calibrated, noise-suppressed, missing viewpoints are marked, and scale is normalized to form a multi-view sample group; MobileNetV3-Small is used to extract lightweight features of each viewpoint, and viewpoint reliability weights are calculated based on image quality, sensor status, occlusion degree, signal-to-noise ratio, and historical reliability. The fusion representation is generated using the cross-view attention fusion module and dynamic gating fusion module of DMVAF-Net. The channel retention rate, view branch mask, and early termination threshold are adjusted based on edge computing power, network bandwidth, service latency requirements, and on-site interference intensity. When the reliability weight of a view is lower than the threshold or there is occlusion, disconnection, or strong noise, the fusion contribution of that view is reduced or the view branch is closed. Real-time inference, local alarm, and emergency response tasks are retained on the edge side, while global analysis, long-term trend judgment, and model optimization tasks are scheduled to the cloud. The cloud-based teacher model performs cross-view modeling on all samples and sends distillation supervision or parameter updates to the edge-side student model. The edge side performs fine-tuning or hard example screening based on new data, uploads parameter summaries to the cloud for aggregation, and outputs the fusion results, confidence level, view contribution, and model version.

Images