A control method and system for incomplete multi-view data

Abstract

This invention discloses a control method and system for incomplete multi-view data, belonging to the field of data recovery control technology. The invention extracts monitoring data features from each view and evaluates their quality; views below a threshold are marked as incomplete. Initial recovery results are then obtained by migrating samples from similar neighborhoods of other views. All features are mapped to a unified latent space to obtain fused semantic representation features. By comprehensively considering multiple factors such as missing data labels, data quality, spatiotemporal features, and semantic consistency, the fusion impact of each view is quantified. Then, based on the relationship between the quantified value and a set threshold, the fusion weights of each view are adaptively calculated. Finally, a global fusion representation feature is generated based on this weighted average, thereby reconstructing and recovering the incomplete view. Anomaly detection is performed, and alarm information is output, improving the accuracy of incomplete view data recovery results and solving the problem of reduced accuracy in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of data recovery control technology, and in particular to a control method and system for incomplete multi-view data. Background Technology

[0002] With the rapid development of IoT and computer vision technologies, multi-camera surveillance systems are widely used in complex scenarios such as security monitoring, intelligent transportation, and smart cities. In these applications, systems typically collect multi-view data of the same scene or target using cameras deployed in different spatial locations to obtain more comprehensive and three-dimensional feature information, thereby improving the performance of downstream tasks such as target recognition, trajectory analysis, and anomaly detection. However, in real-world open environments, due to numerous uncontrollable factors such as sensor hardware failures, network transmission delays, physical occlusion, or blind spots in the field of view, the collected multi-view data often faces severe view loss problems, resulting in so-called "incomplete multi-view data." How to effectively extract consistent correlations from this highly incomplete data and achieve high-quality data recovery has become a key challenge that urgently needs to be solved in the fields of artificial intelligence and pattern recognition.

[0003] To address the challenges posed by incomplete multi-view data, existing technologies typically employ methods based on cross-view neighborhood transfer and unified latent space fusion for view restoration and feature representation. The core idea of ​​these methods is to map heterogeneous data from different perspectives into a common latent low-dimensional feature space, attempting to capture the underlying consistent structure shared across multiple views while eliminating specific noise from each view. Building upon this, existing technologies often utilize observed available view data, combined with cross-view local neighborhood topology relationships, to reconstruct or complete missing views through information transfer and feature fusion mechanisms. Under ideal conditions, this approach can, to some extent, compensate for the inadequacy of information from a single view, providing fundamental data support for subsequent monitoring and analysis tasks.

[0004] While the aforementioned existing technologies have achieved certain results in specific scenarios, they still reveal significant limitations in complex multi-camera surveillance environments. Specifically, due to the often extremely complex view loss patterns and significant semantic differences between cross-viewpoints, coupled with the inherent continuity requirements of temporal monitoring data and accumulated errors in computation, existing methods based on cross-view neighborhood migration and latent space fusion are prone to bottlenecks in practical applications due to "coupling conflicts in multi-factor fusion control." Particularly during the fusion control process, conflicts frequently occur between multiple control factors (e.g., some migrated features may have high local quality but be discontinuous temporally, or they may be spatially similar but semantically unrelated). The lack of a unified, global decision-making coordination mechanism leads to highly unstable weight learning when the system processes these conflicting factors. This multi-factor conflict and decision imbalance ultimately severely restricts the accuracy of incomplete view data recovery results and reduces the reliability of the monitoring system in anomaly detection. Summary of the Invention

[0005] To address the technical problem of reduced accuracy in the recovery results of incomplete view data in existing technologies, embodiments of the present invention provide a control method and system for incomplete multi-view data. The technical solution is as follows:

[0006] On the one hand, a control method for incomplete multi-view data is provided, the method including:

[0007] The process involves acquiring monitoring sample data for each view of the target sample, extracting monitoring data features for each view, evaluating the data quality of each view's monitoring data features to obtain a data quality score, and determining whether the data quality score exceeds a preset quality threshold. If not, the view is marked as incomplete; otherwise, the data quality score of the next view is determined. Neighborhood samples similar to the missing target in the incomplete view are searched for in the monitoring data features of the remaining views, and these samples are transferred to the incomplete view to obtain an initial recovery result. This initial recovery result is then mapped to a unified latent space along with the monitoring data features of each view to obtain the fused semantic representation features of each view. Within the unified latent space, a dataset of feature fusion influencing factors for each view is acquired, and the degree of cross-view semantic representation fusion influence is quantified using preset initial fusion weights to obtain the quantified value of the fusion influence of each view. The dataset includes missing data markers, data quality scores, semantic consistency scores, view spatial topology evaluation values, and temporal continuity scores. It determines whether the fusion impact quantification value of each view is greater than a set fusion quality threshold. If it is, the corresponding fusion weight is obtained by mapping based on the fusion impact quantification value of that view. Otherwise, it determines whether the fusion impact quantification value is less than the minimum fusion quality limit. If it is, the fusion semantic representation features of the corresponding view are removed, and the initial fusion weights of the remaining views are adjusted to obtain the fusion weights. If not, the initial fusion weights of the view are adjusted based on the fusion impact quantification value and its historical fusion impact quantification values ​​to obtain the fusion weights. The fusion semantic representation features of each view are fused according to their fusion weights to obtain global fusion representation features. Based on the global fusion representation features, the original features are reconstructed to restore the incomplete view, and anomaly detection is performed to output corresponding alarm information.

[0008] On the other hand, a control system for incomplete multi-view data is provided, the system comprising:

[0009] The module consists of several modules: a view sample data acquisition module, a monitoring data processing module, and a view feature processing module. The former acquires monitoring sample data for each view and extracts monitoring data features for each view. The latter performs data quality assessment on the monitoring data features of each view to obtain a data quality score. It determines whether the data quality score is greater than a preset quality threshold. If not, the view is marked as incomplete; otherwise, the data quality score of the next view is determined. The module searches for neighboring samples similar to the missing target in the incomplete view from the monitoring data features of the remaining views and migrates these samples to the incomplete view to obtain an initial recovery result. This initial recovery result is then mapped to a unified latent space to obtain the fusion semantic representation features of each view. The latter module analyzes the view feature fusion impact, acquiring a dataset of feature fusion influencing factors for each view in the unified latent space. It then quantifies the degree of fusion impact across view semantic representations using preset initial fusion weights to obtain a quantified value for the degree of fusion impact of each view. The feature fusion influencing factor dataset includes missing labels, data quality scores, semantic consistency scores, view spatial topology evaluation values, and temporal continuity scores. The dynamic adjustment module for fusion weights determines whether the quantified value of the fusion influence of each view is greater than a set fusion quality threshold. If it is, the corresponding fusion weight is obtained by mapping based on the quantified value of the fusion influence of that view. Otherwise, it determines whether the quantified value of the fusion influence is less than the minimum fusion quality limit. If it is, the fusion semantic representation features of the corresponding view are removed, and the initial fusion weights of the remaining views are adjusted to obtain the fusion weights. If not, the initial fusion weight of the view is adjusted based on the quantified value of the fusion influence and its historical quantified values ​​to obtain the fusion weights. The view feature recovery and reconstruction module fuses the fusion semantic representation features of each view according to their fusion weights to obtain global fusion representation features. Based on the global fusion representation features, the original features are reconstructed to restore the incomplete view, and anomaly detection is performed to output corresponding alarm information.

[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0011] 1. This invention provides a control method and system for incomplete multi-view data. By evaluating the quality of each view data of the sample to be monitored, it can effectively identify views with low data quality or missing data and mark them as missing. Through a data quality score evaluation mechanism, it automatically determines whether the view data meets a preset quality threshold. In cases of missing or insufficient data, it can automatically mark the view as incomplete. This quality evaluation not only saves time and cost from manual inspection but also provides a clear basis for subsequent recovery and fusion. Especially in complex multi-view data monitoring systems, this method can promptly identify and isolate problematic views, preventing them from affecting subsequent fusion and recovery processes, ensuring the stability and efficiency of the entire system. Through this mechanism, the system can autonomously perform data quality evaluation and missing data marking, thus providing a foundation for subsequent sample recovery and data fusion. This automated process greatly reduces reliance on manual intervention, improves processing speed, and enhances the accuracy of data processing. It ensures that the monitoring system can respond promptly to data quality fluctuations, improves the overall intelligence level of data processing, and helps solve the problem of reduced accuracy in incomplete view data recovery results in existing technologies.

[0012] 2. This invention, after marking missing data in incomplete views, allows the system to migrate and recover data based on the monitoring data characteristics of similar neighboring samples in other views. This mechanism can automatically migrate neighboring samples according to the data characteristics of different views and semantically fuse the monitoring data characteristics of each view in a unified latent space to obtain the fused semantic representation features of each view. This cross-view data recovery method not only fills in missing data but also effectively avoids the performance degradation of the control system caused by the lack of data in a single view. The construction of the unified latent space enables seamless connection and fusion of data between different views, thereby further improving the integrity and consistency of monitoring data. This fusion mechanism also forms a complete feature fusion influencing factor dataset by combining factors such as multidimensional data quality scores and semantic consistency, making the fusion results of different views more accurate.

[0013] 3. By intelligently adjusting the fusion weights of each view, the method ensures that the final fusion result remains highly accurate and stable even when the data is incomplete or of inconsistent quality. By calculating the quantified value of the fusion influence of each view and adjusting the corresponding fusion weights based on this value, the method can dynamically optimize the weight allocation of different views in global fusion. In particular, for low-quality or incomplete views, the system can adjust the fusion weights or remove unreliable semantic representation features to ensure that the final global fusion representation features have high quality. This intelligent adjustment mechanism not only improves the system's adaptability to incomplete data conditions but also optimizes the performance of the entire control system. Ultimately, the reconstruction of the original features based on the global fusion representation features can effectively recover incomplete views, providing a more efficient solution for various complex multi-view monitoring systems. Especially in dynamically changing data environments, it can achieve automated and efficient anomaly warning and response. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of a control method for incomplete multi-view data provided in this application embodiment;

[0016] Figure 2 This is a schematic diagram of the structure of a control system for incomplete multi-view data provided in an embodiment of this application. Detailed Implementation

[0017] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0018] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0019] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0020] like Figure 1 The diagram shown is a flowchart of a control method for incomplete multi-view data provided in an embodiment of this application. The method includes the following steps:

[0021] Step 1: Obtain the monitoring sample data for each view of the sample to be monitored, and extract the monitoring data features for each view.

[0022] Specifically, the sample to be monitored can be a target object or target event in the monitoring scene. Multiple views correspond to the observation results of multiple cameras on the sample to be monitored at the same time or in adjacent time periods. Different cameras, due to differences in installation location, shooting angle, and acquisition conditions, form different perspectives of data description for the same sample to be monitored.

[0023] During the data acquisition phase, raw monitoring data is obtained from each camera. This raw monitoring data may include video frame image data, target detection result data, and timestamp information. To ensure the correspondence between multiple views, the data from each camera is time-aligned, so that data from different views form a set of corresponding samples, i.e., monitoring sample data, within the same time index or preset time window.

[0024] During the data extraction stage, feature extraction is performed on the raw monitoring data of each view to obtain the monitoring data features of each view. Feature extraction may include, but is not limited to, the following methods: performing convolutional neural network encoding on video frame images to extract visual semantic feature vectors; performing regional feature extraction on detected target areas to obtain target-level feature representations; performing differential or optical flow calculations on continuous frames to extract time change features; and encoding camera identification and time information to obtain auxiliary context features.

[0025] After the above processing, the monitoring data of the c-th view at time t can be represented as a feature vector:

[0026] ;

[0027] in, This represents the monitoring data of the c-th view at time t. This indicates the feature dimension of the corresponding view. The feature dimensions of different views can be the same or different.

[0028] Furthermore, to adapt to subsequent multi-view fusion processing, the features of each view can be normalized or standardized to reduce the distribution differences between different views and improve the consistency of features.

[0029] Ultimately, a multi-view data set consisting of multiple view features is obtained:

[0030] ;

[0031] in, This represents the monitoring data of the first view at time t. This represents the monitoring data of the second view at time t. Represents a multi-view data collection. Indicates the number of cameras.

[0032] Step 2: Perform data quality assessment on the monitoring data characteristics of each view to obtain the data quality score of each view. Determine whether the data quality score is greater than the preset quality threshold. If not, mark the view as missing and record it as an incomplete view. Otherwise, determine the data quality score of the next view. The quality threshold is a preset value, which is set in advance by preset staff based on historical data and experience rules and entered into the preset database for later use.

[0033] It is important to understand that the data quality score for each view is obtained by evaluating the data quality characteristics of each view. The specific method for obtaining this score is as follows:

[0034] First, acquire data quality assessment data for the monitoring data characteristics of each view. The data quality assessment data includes, but is not limited to, sharpness indicators, brightness indicators, frame change indicators, target presence indicators, and signal integrity indicators.

[0035] The sharpness index is obtained by calculating the Laplace variance. Brightness index , This represents the total number of pixels in the image that are involved in the calculation. Indicates the first The brightness value of each pixel, frame change index , This represents the monitoring data of the c-th view at time t-1, and the target existence index. Signal integrity index .

[0036] Next, the data quality impact weights corresponding to each data quality assessment data are obtained from the preset database. The data quality impact weights include, but are not limited to, the impact weights of sharpness, brightness, frame change, target presence, and signal integrity.

[0037] It should be noted that the weights of the data quality impact are preset by staff based on historical data within a historical period and empirical rules, and the sum of the weights of each data quality impact is 1.

[0038] Finally, the data quality assessment data and the data quality impact weights are weighted and fused to obtain the data quality score of the corresponding view.

[0039] Specifically, data quality score The specific expression is as follows:

[0040] ;

[0041] In the formula, This indicates that clarity affects the weight. This indicates that brightness affects the weight. This indicates that frame changes affect the weights. This indicates the weight of the impact of the existence of the target. This indicates the weights that affect signal integrity.

[0042] Step 3: Find neighborhood samples similar to the missing target in the incomplete view from the monitoring data features of the remaining views, and transfer the obtained neighborhood samples of each view to the incomplete view to obtain the initial recovery result. Map the results to the unified latent space with the monitoring data features of each view to obtain the fused semantic representation features of each view.

[0043] Specifically, at time There exists the first If the monitoring data features of a view are missing or unavailable, that view is recorded as an incomplete view, and its corresponding features are recorded as missing. For the set of other available views at the same time... , denoted as:

[0044] ;

[0045] in, Indicates the first Each view at time Availability markers.

[0046] In the monitoring data features of the remaining views, for the target sample to be recovered, its neighborhood sample set is found based on feature similarity metrics. Specifically, Euclidean distance or cosine similarity can be used as the similarity metric for any available view. Calculate the features of the current sample The distance between the feature and the historical or candidate sample features in the view, and the selection of the top ones. The most similar samples form a neighborhood set :

[0047] ;

[0048] The neighborhood samples obtained from each available view are mapped to the feature space corresponding to the target missing view. Specifically, for the neighborhood sample set, its representation in the target view is extracted. The corresponding feature representations (if historical mapping relationships or cross-view correspondences exist) are aggregated to obtain the initial recovery features of the missing views. :

[0049] ;

[0050] in, Indicates the neighborhood sample index number. Indicates the first Neighboring samples in the target view The corresponding features in.

[0051] The above results As the initial recovery result of the missing view at the current moment, it is used to replace the missing features, thereby obtaining the completed multi-view feature set. :

[0052] ;

[0053] in, Indicates the first Each view at time The completed features, No. Each view at time The characteristics of the original monitoring data Indicates the first Each view at time To eliminate feature heterogeneity between different views, the missing state labels are mapped to a unified latent space using corresponding encoding functions. For the first... The mapping process for each view is as follows:

[0054] ;

[0055] in, Indicates the first Feature encoding function for each view This represents the view in the unified latent space.

[0056] The representations of each view in the unified latent space are fused to obtain the fused semantic representation features. :

[0057] ;

[0058] in, Indicates the first Each view at time The fusion weight.

[0059] Step 4: In the unified latent space, obtain the feature fusion influencing factor dataset for each view, and combine it with the preset initial fusion weight to quantify the degree of fusion influence of cross-view semantic representation fusion to obtain the quantified value of the fusion influence of each view. The feature fusion influencing factor dataset includes missing labels, data quality scores, semantic consistency scores, view spatial topology evaluation values, and temporal continuity scores.

[0060] The preset initial fusion weight is a weight value initially set by the staff based on the contribution of each data in the dataset of various feature fusion influencing factors to the incomplete view restoration. The higher the contribution, the greater the corresponding initial fusion weight.

[0061] Furthermore, the method for obtaining the quantitative value of the degree of fusion impact is as follows:

[0062] The first step is to assign a missing score to each view by marking it as missing: if a view has a missing score, the missing score of that view is assigned a first preset value, such as 0; otherwise, it is assigned a second preset value, such as 1. The first preset value is less than the second preset value.

[0063] The second step involves processing the acquired semantic consistency assessment data to obtain a semantic consistency score. The semantic consistency assessment data includes view feature similarity, latent space distance, and cross-view alignment values.

[0064] Specifically, for the same moment Multi-view monitoring data feature set First, the features of each view are uniformly represented or standardized to reduce the distribution differences between different views. Then, a pre-defined encoding function maps the features of each view to the latent space, resulting in the latent space representation. :

[0065] ;

[0066] in, Indicates the first Encoding functions for each view, This represents the corresponding latent space representation.

[0067] Based on this, semantic consistency evaluation data is constructed, specifically including:

[0068] (1) View feature similarity: For any two views and The feature similarity is calculated to measure the consistency of different views in the original feature space. View feature similarity. Cosine similarity can be used to represent it: .

[0069] (2) Latent space distance: In a unified latent space, the distance between different view representations is calculated to measure the degree of semantic difference. This can be represented using Euclidean distance: , Specifically, the first The view and the first Each view at time The potential space distance, Indicates the first Each view at time The latent space representation features, Indicates the first Each view at time The latent space representation features; to facilitate subsequent fusion processing, the latent space distance can be normalized or converted into a similarity metric, for example: .

[0070] (3) Cross-view alignment score: This measures the overall alignment of different views in the latent space. Specifically, it can be calculated based on a preset alignment model or a contrastive learning mechanism to determine the consistency score between different view representations. ,For example: ,in, The alignment calculation function can be a similarity function or a correlation function obtained from contrastive learning training.

[0071] The semantic consistency evaluation data mentioned above are then fused together to obtain the semantic consistency score. Specifically, a weighted combination method can be used:

[0072] ;

[0073] in, , , Let the weight parameters satisfy: .

[0074] Furthermore, the results of all view pairs can be aggregated to obtain a global semantic consistency score. :

[0075] ;

[0076] in, Represents the total number of views and the semantic consistency score. It is used to reflect the overall semantic consistency of the current multi-view data.

[0077] The third step is to process the acquired view topology data to obtain the view space topology evaluation value. The view topology data includes view space distance and view overlap value.

[0078] Specifically, for multiple views (corresponding to multiple cameras) within the same monitoring area, the spatial location information and imaging parameter information of each camera are pre-acquired or estimated online, and view topology data is constructed based on the information. Let the first... The view and the first The camera spatial positions corresponding to each view are respectively Their orientation vectors are respectively .

[0079] Based on this, the following two types of evaluation indicators are constructed:

[0080] (1) View space distance This is used to characterize the spatial proximity between different cameras. Specifically, Euclidean distance can be used for calculation. ;

[0081] To facilitate integration with other indicators, spatial distance can be normalized or converted into a similarity measure. ,For example: ;in, This is a distance scale parameter used to adjust the distance decay rate.

[0082] (2) Value of visual field overlap This measures the degree of overlap between the observation areas of two views in space. Specifically, the overlap ratio can be calculated based on the camera's field-of-view model. Let the first... The field of view of each view is , No. The field of view of each view is Then, their degree of overlap can be expressed as:

[0083] ;in, Indicates the area or volume of a region.

[0084] In practice, the field of view can be estimated using the camera's intrinsic and extrinsic parameters and imaging model, or it can be obtained through statistical analysis of historical observation data.

[0085] Based on the aforementioned spatial distance similarity and view overlap, different view pairs are comprehensively evaluated to obtain a view spatial topology evaluation value. Specifically, a weighted fusion method can be used:

[0086] ;in, , Let the weight parameters satisfy: .

[0087] Furthermore, the topology evaluation values ​​for all view pairs can be normalized or a topology relation matrix can be constructed: The view space topology evaluation value is used to reflect the degree of correlation between different views in terms of spatial structure.

[0088] The fourth step is to process the time continuity assessment data to obtain the time continuity score. The time continuity assessment data includes the change amplitude between consecutive frames, the trajectory continuity quantization value, and the historical stability quantization value.

[0089] Specifically, for the same sample to be monitored in a continuous time series The multi-view feature representation within the data constructs time continuity assessment data and quantifies it to measure the consistency and stability between current data and historical data.

[0090] Let the first Each view at time The features are represented as Its representation in the latent space is as follows .

[0091] Based on the above, we can calculate:

[0092] (1) Variation between consecutive frames This characterizes the degree of change between the current frame and the previous frame. Specifically, the feature differences between adjacent time steps can be calculated in the latent space. ;

[0093] To facilitate standardized measurement, the magnitude of change between consecutive frames can be converted into a continuity score. ,For example:

[0094] ;in, To adjust the parameters.

[0095] (2) Trajectory continuity quantification value: used to measure whether the target's trajectory is continuous in a time series. Let the target's spatial position at consecutive moments be . Then the continuity of the trajectory can be measured by the smoothness of the positional changes, for example:

[0096] ;in, Indicates the first Each view at time The trajectory continuity quantization value, Indicates the first Each view at time The target position vector, Indicates the first The view in the previous moment The target position vector, Indicates the first The view in the first two moments The target position vector, This represents the trajectory change scale adjustment parameter, used to adjust the sensitivity to trajectory fluctuations. The second-order difference term reflects the smoothness of trajectory changes; the smoother the change, the higher the continuity.

[0097] (3) Historical stability quantification: This measures the consistency between the current feature and the feature distribution over a historical period. Specifically, it calculates the deviation between the current feature and the mean of the feature within the historical window.

[0098] ;

[0099] ;

[0100] in, Indicates the length of the time window. Indicates the first Each view at time Corresponding historical average latent space characteristics, This represents a historical time index variable used to iterate through historical time windows. Indicates the first Each view at a historical moment The latent space representation features, Indicates the first Each view at time The historical stability quantification value represents the first... Each view at the current moment The latent space representation features, This represents the historical average potential space characteristics. This represents the historical stability scaling parameter, used to control the sensitivity to historical offsets.

[0101] By integrating the above three types of indicators, a time continuity score is obtained. Specifically, a weighted approach can be used:

[0102] ;

[0103] in, , , Let the weight parameters satisfy: .

[0104] Furthermore, the temporal continuity scores of all views can be aggregated to obtain the overall temporal continuity assessment result.

[0105] The fifth step is to obtain the fusion factor analysis factors that represent the contribution of each fusion influencing factor to the fusion weight analysis in the feature fusion influencing factor dataset. The fusion factor analysis factors include missing label analysis factors, data quality analysis factors, semantic consistency analysis factors, view space topology analysis factors, and temporal continuity analysis factors.

[0106] It should be added that the various fusion factor analysis factors are preset values ​​set by the preset staff, which are generally preset based on historical data and empirical rules within a historical period.

[0107] The sixth step is to perform weighted fusion processing on each data point in the feature fusion influencing factor dataset with the corresponding fusion factor analysis factor to obtain the quantitative value of the degree of fusion influence.

[0108] In this embodiment, by comprehensively evaluating multiple factors, the influence of each view in the data fusion process is quantified more accurately, thereby effectively optimizing the quality and stability of data fusion. First, the missing data labeling and scoring mechanism automatically identifies and distinguishes views with missing data, assigning them lower missing data scores to ensure that views with higher data integrity are prioritized during fusion. Second, the processing of evaluation data such as semantic consistency, view spatial topology, and temporal continuity helps the system quantify the similarity and consistency between views, avoiding interference from inconsistent or distorted data on the fusion results. Through weighted fusion of these factors, the quantification value of the fusion influence of each view can be calculated more accurately, providing a scientific basis for subsequent fusion weight adjustments. This multi-dimensional data evaluation mechanism can more comprehensively consider the interrelationships between views, data quality, and temporal changes, ensuring high-quality feature fusion can still be achieved in incomplete or unstable data environments. By dynamically adjusting the fusion weights, the system can automatically optimize the contribution of views, resulting in a more accurate and stable global fusion representation. This intelligent fusion process not only improves the system's adaptability but also enhances the accuracy of data recovery and anomaly detection.

[0109] Step 5: Determine whether the fusion impact quantification value of each view is greater than the set fusion quality threshold. If it is greater, then map the corresponding fusion weight based on the fusion impact quantification value of the view. Otherwise, determine whether the fusion impact quantification value is less than the minimum fusion quality limit. If it is, then remove the fusion semantic representation features of the corresponding view and adjust the initial fusion weight of the remaining views to obtain the fusion weight. If not, then adjust the initial fusion weight of the view based on the fusion impact quantification value and its historical fusion impact quantification value to obtain the fusion weight.

[0110] It is important to understand that the fusion quality threshold and the minimum fusion quality limit are adaptively determined based on the distribution of the quantized values ​​of the degree of fusion influence of each view at the current moment. The fusion quality threshold is calculated based on the mean and standard deviation of the quantized values ​​and is used to filter high-quality views. The minimum fusion quality limit is determined based on the quantiles or mean offset of the quantized values ​​and is used to eliminate low-quality views, thereby achieving dynamic control over the multi-view fusion process.

[0111] The fusion weight is obtained by removing the fusion semantic representation features of the corresponding view and adjusting the initial fusion weights of the remaining views. The specific process is as follows:

[0112] The number of views with removed fusion weights and the sum of the fusion weights generated by the remaining views are counted. The difference between the preset sum of fusion weights and the sum of the remaining fusion weights is calculated to obtain the fusion weight compensation value.

[0113] The fusion weights of the remaining views are sorted by data, and the compensation analysis ratio is obtained based on the data sorting results; wherein, the data sorting is sequential.

[0114] Specifically, the values ​​corresponding to the sorting results are summed to obtain the total compensation analysis. Then, the sorting values ​​corresponding to each view are compared with the total compensation analysis to obtain the corresponding compensation analysis ratio. For example, if the fusion weight sorting results of the other views are 1, 2, 3, 4, and 5, and the fusion weight sorting result of a certain view is 3, then the compensation analysis ratio corresponding to that view is 3 / 1+2+3+4+5, which is 1 / 5.

[0115] The compensation weight for each view is obtained by multiplying the compensation analysis ratio and the fusion weight compensation value of each view.

[0116] The weights to be compensated for in each view are added to the corresponding initial fusion weights to obtain the fusion weights, and the values ​​of the initial fusion weights for each view are updated to the values ​​of the corresponding fusion weights.

[0117] Preferably, the initial fusion weight of the view is adjusted by changing the degree of influence based on the quantified value of the fusion influence and its historical quantified values. The specific method is as follows:

[0118] The first trend value is obtained by calculating the difference between the fusion quality threshold and the quantified value of fusion impact, and the second trend value is obtained by calculating the difference between the quantified value of fusion impact and the minimum limit value of fusion quality.

[0119] The numerical comparison between the first trend degree value and the second trend degree value includes the following three cases:

[0120] In the first case, if the first trend degree value is greater than the second trend degree value, the initial fusion weight of the view is reduced based on the second trend degree value.

[0121] The initial fusion weight of the view is adjusted downward based on the second trend degree value. The specific process is as follows:

[0122] Input the second trend degree value into the pre-trained descent control data table and output the descent control ratio.

[0123] Specifically, by inputting the second trend degree value into the downregulation data table, the corresponding downregulation ratio can be obtained. This data sequence is used to fit the mapping relationship between the second trend degree value and the downregulation ratio. The construction method is as follows: in the initial data table built based on the gradient boosting regression algorithm, the second trend degree value collected in the historical time period and the downregulation ratio set according to the empirical rules are selected as training samples. The model is trained based on the XGBoost framework with the least square error as the objective function, and finally the trained downregulation data table is obtained.

[0124] The initial fusion weight of the view is multiplied by the descent control ratio to obtain the descent weight value.

[0125] The initial fusion weight and the decreasing weight are used to calculate the difference to obtain the fusion weight of the view, and then the view is updated.

[0126] In the second scenario, if the second trend degree value is greater than the first trend degree value, the initial fusion weight of the view is adjusted upward based on the first trend degree value.

[0127] Similarly, the initial fusion weight of the view is adjusted upward based on the first trend degree value. The specific process is as follows:

[0128] Input the first trend degree value into the pre-trained upward regulation data table, and output the upward regulation ratio.

[0129] Specifically, by inputting the first trend degree value into the upward regulation data table, the corresponding upward regulation ratio can be obtained. This data sequence is used to fit the mapping relationship between the first trend degree value and the upward regulation ratio. The construction method is as follows: in the initial data table constructed based on the gradient boosting regression algorithm, the first trend degree value collected in the historical time period and the upward regulation ratio set according to the empirical rules are selected as training samples. The model is trained based on the XGBoost framework with the least square error as the objective function, and finally the trained upward regulation data table is obtained.

[0130] The initial fusion weight of the view is multiplied by the rise control ratio to obtain the rise weight value.

[0131] The initial blending weight and the rising weight are summed to obtain the blending weight of the view, and then updated.

[0132] In the third scenario, if the first trend degree value equals the second trend degree value, the view is adjusted based on the historical fusion influence degree value.

[0133] It should also be noted that the specific steps for adjusting this view based on the historical fusion impact value are as follows:

[0134] Obtain the historical fusion impact value of the corresponding view and perform smoothing to obtain the representative value of the fusion impact of the view; the smoothing can be performed by averaging.

[0135] If the fusion impact value is higher than the set fusion judgment value, the initial fusion weight of the view is increased based on the first trend degree value; otherwise, the initial fusion weight of the view is decreased based on the second trend degree value.

[0136] In addition, adjusting the degree of influence of the initial fusion weights of this view also includes:

[0137] The difference between the first trend degree value and the second trend degree value is calculated, and the absolute value is taken to obtain the control average value.

[0138] If the average value of the adjustment is greater than the set difference amount, the initial fusion weight of the view will continue to be adjusted based on the first trend degree value and the second trend degree value. The set difference amount is the data set by the preset staff.

[0139] If the average value of the adjustment is not greater than the set difference, the initial fusion weight of the view is adjusted based on the median between the fusion quality threshold and the minimum fusion quality limit.

[0140] Step 6: Based on the fusion weight of each view, fuse the fusion semantic representation features of each view to obtain the global fusion representation features. Based on the global fusion representation features, reconstruct the original features to restore the incomplete view, and perform anomaly detection to output the corresponding alarm information.

[0141] Specifically, set at time , No. The fusion semantic representation features of each view in the unified latent space are: The corresponding fusion weight is The global fusion representation features are obtained through weighted fusion:

[0142] ,in, This represents a global fusion representation in a unified latent space, used to comprehensively reflect the semantic information of multiple views.

[0143] Based on global fusion representation features The incomplete view is restored by mapping it back to the original feature space of each view through a decoding function. For the first... The reconstruction process of a view can be represented as follows:

[0144] ;

[0145] in, Indicates the first Decoding functions corresponding to each view This indicates the features after reconstruction.

[0146] Furthermore, for incomplete views (i.e., views with missing parts), the reconstructed features are used as the recovery result; for complete views, residual correction methods can be used for optimization.

[0147] ;

[0148] in, The first is represented by the globally fused representation. Each view at time Correction items, Indicates the first Each view at time The characteristics of reconstruction and restoration Indicates the first Each view at time The original characteristics.

[0149] Anomaly detection is performed on the current data based on the difference between the reconstructed results and the original features. Specifically, the reconstruction error can be calculated:

[0150] ;in, Indicates the first Each view at time Reconstruction error.

[0151] Furthermore, the error is compared with a preset anomaly threshold. Comparison:

[0152] ;

[0153] in, Indicates the first Each view at time Abnormal detection results This indicates that an anomaly has been detected. In some implementations, the anomaly determination result can be corrected by combining semantic consistency score or temporal continuity score to reduce the false alarm rate.

[0154] It should also be noted that when an anomaly is detected, a corresponding alarm message is generated. The alarm message may include, but is not limited to, the time of the anomaly occurrence, the anomaly view number, the anomaly type identifier, and the anomaly severity score. The alarm message is then sent to the monitoring system or user terminal to achieve real-time early warning.

[0155] In this embodiment, by dynamically adjusting the quantified value of the fusion impact and its historical values, the initial fusion weights of each view can be adaptively adjusted based on data quality and semantic consistency. By calculating the first and second trend values, the system can more clearly determine the performance trend of a view in the current fusion, thereby deciding whether to adjust upward or downward. Compared with the static fusion weight allocation method, this method can optimize the contribution of each view in real time when the multi-view data is incomplete or fluctuates in quality, ensuring the accuracy and stability of the global fusion representation features, while reducing the adverse effects of low-quality views on the final fusion result, and improving the robustness of the system in complex environments.

[0156] In the specific processes of upward and downward control, this method uses a pre-trained control data table to map the trend degree values, thereby calculating the control ratio and updating the fusion weights. Downward control obtains the downward weight value through multiplication, ensuring a reasonable reduction in the weight of low-quality views. Upward control obtains the upward weight value through summation, allowing high-quality views to fully play their role. This refined control mechanism can more accurately adjust the fusion weights under different trend conditions, balancing stability and flexibility. Simultaneously, smoothing by incorporating historical fusion impact values ​​further reduces the impact of sudden data anomalies on the fusion weights, enabling the control system to maintain continuity and reliability even when facing data gaps or fluctuations.

[0157] Furthermore, this method introduces a control mean value judgment mechanism. By comparing the difference between the first trend degree value and the second trend degree value, it determines whether to continue control based on the trend value or to use the median between the fusion quality threshold and the minimum limit value for weight adjustment. This mechanism can achieve balanced control when there are large differences in data from different views, preventing abnormal data from a single view from excessively affecting the global fusion effect. Overall, this method, through dynamic, refined, and balanced fusion weight control, makes the fusion of multi-view data in incomplete situations more intelligent and efficient. It not only improves the accuracy and stability of global feature fusion but also provides a solid data foundation for subsequent feature recovery and anomaly detection, improving the adaptability and reliability of the multi-view monitoring system in practical applications.

[0158] like Figure 2 The diagram shown is a structural schematic of a control system for incomplete multi-view data provided in an embodiment of this application, including:

[0159] The view sample data acquisition module is used to acquire monitoring sample data of each view of the sample to be monitored, and extract the monitoring data characteristics of each view.

[0160] The monitoring data feature processing module is used to evaluate the data quality of the monitoring data features of each view to obtain the data quality score of each view. It determines whether the data quality score is greater than the preset quality threshold. If not, the view is marked as missing and recorded as an incomplete view. Otherwise, the data quality score of the next view is determined. The module searches for neighborhood samples similar to the missing target in the incomplete view in the monitoring data features of the remaining views, and migrates the obtained neighborhood samples of each view to the incomplete view to obtain the initial recovery result. The module maps the results to the monitoring data features of each view to a unified latent space to obtain the fused semantic representation features of each view.

[0161] The view feature fusion impact analysis module is used to obtain the feature fusion impact factor dataset of each view in a unified latent space, and combine it with the preset initial fusion weight to quantify the degree of fusion impact of cross-view semantic representation fusion to obtain the quantified value of the fusion impact of each view. The feature fusion impact factor dataset includes missing labels, data quality scores, semantic consistency scores, view spatial topology evaluation values, and temporal continuity scores.

[0162] The dynamic adjustment module for fusion weight is used to determine whether the quantified value of the fusion influence of each view is greater than the set fusion quality threshold. If it is greater, the corresponding fusion weight is obtained by mapping based on the quantified value of the fusion influence of that view. Otherwise, it determines whether the quantified value of the fusion influence is less than the minimum limit value of fusion quality. If it is, the fusion semantic representation features of the corresponding view are removed and the initial fusion weight of the remaining views is adjusted to obtain the fusion weight. If not, the initial fusion weight of the view is adjusted based on the quantified value of the fusion influence and its historical quantified value of the fusion influence to obtain the fusion weight.

[0163] The view feature recovery and reconstruction module is used to fuse the fusion semantic representation features of each view according to the fusion weight of each view to obtain the global fusion representation features, reconstruct the original features based on the global fusion representation features to restore the incomplete view, and perform anomaly detection to output the corresponding alarm information.

[0164] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0165] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0166] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units, located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the solution, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method for incomplete multi-view data, characterized in that, Includes the following steps: Acquire monitoring sample data for each view of the sample to be monitored, and extract the monitoring data features for each view. The monitoring data characteristics of each view are evaluated to obtain the data quality score of each view. It is determined whether the data quality score is greater than the preset quality threshold. If not, the view is marked as missing and recorded as an incomplete view. Otherwise, the data quality score of the next view is determined. Find neighborhood samples similar to the missing targets in the incomplete view from the monitoring data features of the remaining views, and transfer the obtained neighborhood samples of each view to the incomplete view to obtain the initial recovery result. Map the results to the unified latent space with the monitoring data features of each view to obtain the fused semantic representation features of each view. In the unified latent space, the feature fusion influencing factor dataset of each view is obtained, and the cross-view semantic representation fusion influence degree is quantified by combining the preset initial fusion weight to obtain the fusion influence degree quantification value of each view. The feature fusion influencing factor dataset includes missing labels, data quality score, semantic consistency score, view space topology evaluation value and temporal continuity score. If the fusion impact quantification value of each view is greater than the set fusion quality threshold, the corresponding fusion weight is obtained by mapping based on the fusion impact quantification value of that view. Otherwise, if the fusion impact quantification value is less than the minimum fusion quality limit, the fusion semantic representation features of the corresponding view are removed and the initial fusion weight of the remaining views is adjusted to obtain the fusion weight. If not, the initial fusion weight of the view is adjusted based on the fusion impact quantification value and its historical fusion impact quantification value to obtain the fusion weight. The fusion semantic representation features of each view are fused according to the fusion weight of each view to obtain the global fusion representation features. The incomplete view is restored by reconstructing the original features based on the global fusion representation features, and anomaly detection is performed on it to output the corresponding alarm information.

2. The control method for incomplete multi-view data as described in claim 1, characterized in that: The data quality score for each view is obtained by evaluating the data characteristics of each view. The specific method for obtaining the score is as follows: Data quality assessment data of monitoring data characteristics of each view is obtained, including sharpness index, brightness index, frame change index, target presence index and signal integrity index; The data quality impact weights corresponding to each data quality assessment data are obtained from a preset database. The data quality impact weights include sharpness impact weight, brightness impact weight, frame change impact weight, target presence impact weight, and signal integrity impact weight. The data quality score of the corresponding view is obtained by weighting and fusing the data quality assessment data and the data quality impact weights.

3. The control method for incomplete multi-view data as described in claim 1, characterized in that: The method for obtaining the quantitative value of the degree of fusion impact is as follows: Each view is assigned a missing score by marking it as missing: if a view has a missing score, the missing score of that view is assigned a first preset value; otherwise, it is assigned a second preset value. The first preset value is less than the second preset value. The semantic consistency score is obtained by processing the acquired semantic consistency evaluation data, which includes view feature similarity, latent space distance, and cross-view alignment value. The view topology data is processed to obtain the view space topology evaluation value, which includes the view space distance and the degree of view overlap. A time continuity score is obtained by processing the time continuity assessment data, which includes the change amplitude between consecutive frames, the trajectory continuity quantization value, and the historical stability quantization value. The feature fusion influencing factors dataset is obtained to represent the contribution of each fusion influencing factor to the fusion weight analysis. The fusion factor analysis factors include missing label analysis factors, data quality analysis factors, semantic consistency analysis factors, view space topology analysis factors, and temporal continuity analysis factors. The data in the feature fusion influencing factor dataset are weighted and fused with the corresponding fusion factor analysis factors to obtain the quantitative value of the degree of fusion influence.

4. The control method for incomplete multi-view data as described in claim 1, characterized in that: The process of removing the fusion semantic representation features of the corresponding view and adjusting the initial fusion weights of the remaining views to obtain the fusion weights is as follows: The number of views with removed fusion weights and the sum of the fusion weights generated by the remaining views are counted. The difference between the preset sum of fusion weights and the sum of the fusion weights is calculated to obtain the fusion weight compensation value. The fusion weights of the remaining views are sorted according to the data, and the compensation analysis ratio is obtained based on the data sorting results. The compensation weight to be compensated for each view is obtained by multiplying the compensation analysis ratio and the fusion weight compensation value of each view. The weights to be compensated for in each view are added to the corresponding initial fusion weights to obtain the fusion weights, and the values ​​of the initial fusion weights for each view are updated to the values ​​of the corresponding fusion weights.

5. The control method for incomplete multi-view data as described in claim 1, characterized in that: The fusion weight is obtained by adjusting the initial fusion weight of the view based on the quantified value of the fusion impact degree and its historical quantified value of the fusion impact degree. The specific method is as follows: The first trend value is obtained by calculating the difference between the fusion quality threshold and the quantified value of fusion impact, and the second trend value is obtained by calculating the difference between the quantified value of fusion impact and the minimum limit value of fusion quality. If the first trend degree value is greater than the second trend degree value, the initial fusion weight of the view is reduced based on the second trend degree value. If the second trend degree value is greater than the first trend degree value, the initial fusion weight of the view is adjusted upward based on the first trend degree value; If the first trend degree value equals the second trend degree value, then the view is adjusted according to the historical fusion influence degree value.

6. The control method for incomplete multi-view data as described in claim 5, characterized in that: The process of adjusting the initial fusion weight of the view based on the second trend degree value is as follows: Input the second trend degree value into the pre-trained descent control data table and output the descent control ratio; The initial fusion weight of the view is multiplied by the descent control ratio to obtain the descent weight value; The initial fusion weight and the decreasing weight are used to calculate the difference to obtain the fusion weight of the view, and then the view is updated.

7. The control method for incomplete multi-view data as described in claim 5, characterized in that: The initial fusion weight of the view is adjusted upward based on the first trend degree value, and the specific process is as follows: Input the first trend degree value into the pre-trained upward regulation data table, and output the upward regulation ratio; The initial fusion weight of the view is multiplied by the rise control ratio to obtain the rise weight value. The initial blending weight and the rising weight are summed to obtain the blending weight of the view, and then updated.

8. The control method for incomplete multi-view data as described in claim 5, characterized in that: The specific steps for adjusting the view based on the historical fusion impact value are as follows: Obtain the historical fusion impact value of the corresponding view and perform smoothing to obtain the representative value of the fusion impact of the view; If the fusion impact value is higher than the set fusion judgment value, the initial fusion weight of the view is increased based on the first trend degree value; otherwise, the initial fusion weight of the view is decreased based on the second trend degree value.

9. The control method for incomplete multi-view data as described in claim 5, characterized in that: The adjustment of the influence degree of the initial fusion weight of the view also includes: The difference between the first trend degree value and the second trend degree value is calculated, and the absolute value is taken to obtain the control average value; If the average adjustment value is greater than the set difference amount, the initial fusion weight of the view will continue to be adjusted based on the first trend degree value and the second trend degree value. If the average value of the adjustment is not greater than the set difference, the initial fusion weight of the view is adjusted based on the median between the fusion quality threshold and the minimum fusion quality limit.

10. A system applying the control method for incomplete multi-view data as described in any one of claims 1-9, characterized in that, include: The view sample data acquisition module is used to acquire monitoring sample data of each view of the sample to be monitored, and extract the monitoring data characteristics of each view from the data. The monitoring data feature processing module is used to evaluate the data quality of the monitoring data features of each view to obtain the data quality score of each view, and to determine whether the data quality score is greater than the preset quality threshold. If not, the view is marked as missing and recorded as an incomplete view; otherwise, the data quality score of the next view is determined. Find neighborhood samples similar to the missing targets in the incomplete view from the monitoring data features of the remaining views, and transfer the obtained neighborhood samples of each view to the incomplete view to obtain the initial recovery result. Map the results to the unified latent space with the monitoring data features of each view to obtain the fused semantic representation features of each view. The view feature fusion impact analysis module is used to obtain the feature fusion impact factor dataset of each view in a unified latent space, and combine it with the preset initial fusion weight to quantify the degree of cross-view semantic representation fusion impact to obtain the fusion impact quantification value of each view. The feature fusion impact factor dataset includes missing labels, data quality scores, semantic consistency scores, view space topology evaluation values, and temporal continuity scores. The dynamic adjustment module for fusion weight is used to determine whether the fusion influence degree quantification value of each view is greater than the set fusion quality threshold. If it is greater, the corresponding fusion weight is obtained by mapping based on the fusion influence degree quantification value of the view. Otherwise, it determines whether the fusion influence degree quantification value is less than the minimum limit value of fusion quality. If it is, the fusion semantic representation feature of the corresponding view is removed and the initial fusion weight of the remaining views is adjusted to obtain the fusion weight. If not, the initial fusion weight of the view is adjusted based on the fusion influence degree quantification value and its historical fusion influence degree quantification value to obtain the fusion weight. The view feature recovery and reconstruction module is used to fuse the fusion semantic representation features of each view according to the fusion weight of each view to obtain the global fusion representation features, reconstruct the incomplete view based on the global fusion representation features, and perform anomaly detection to output the corresponding alarm information.

Images