A multi-modal data acquisition and calibration method and system

Abstract

The application discloses a kind of multimodal data acquisition and calibration method and system, belong to data processing field, method includes: through sensor network, multimodal data acquisition is carried out to smart factory, obtains multiple original data streams, and original data stream is carried out multi-granularity desensitization processing, generates desensitization acquisition feature;Desensitization acquisition feature is carried out observability and privacy exposure degree analysis, establishes calibration anchor point layer, carries out space-time correlation matching detection to multiple desensitization acquisition features, establishes acquisition calibration relationship diagram;Multiple desensitization acquisition features are carried out cross-modal calibration labeling, and cross-modal calibration labeling space is established;Cross-modal calibration labeling space is carried out closed loop transformation residual error detection compensation, obtains calibration labeling enhancement space, and multiple desensitization acquisition features are carried out automatic calibration.The present application solves the technical problem that existing multimodal data calibration scheme cannot consider data privacy security and calibration accuracy, and is easy to cause privacy information leakage risk.

Description

Technical Field

[0001] This invention relates to the field of data processing, and specifically to a multimodal data acquisition and calibration method and system. Background Technology

[0002] Smart factories have widely deployed various heterogeneous sensing devices, including video surveillance, environmental parameter acquisition, personnel access control, and equipment operation status monitoring, forming a multimodal data acquisition network with full coverage. However, in actual deployment and operation, sensing devices of different modes and from different manufacturers use independent sampling clocks and spatial coordinate systems. Moreover, the entire data acquisition link is simultaneously affected by multi-layered coupling interference from device hardware parameter drift, external environmental physical condition disturbances, and data transmission link fluctuations. This results in insufficient spatiotemporal alignment and large numerical deviations in multimodal data, making it difficult to directly use for cross-modal fusion analysis. At the same time, the raw data contains a large amount of sensitive content such as facial information, personnel identification, and core equipment operation and maintenance parameters. Existing multimodal data calibration solutions generally cannot balance data privacy and security with calibration accuracy. If the raw data is anonymized to meet compliance requirements, it is easy to cause information loss of calibration-related features, posing a compliance risk of privacy information leakage and making it impossible to maintain stable and reliable calibration results in the long term. Summary of the Invention

[0003] This application provides a multimodal data acquisition and calibration method and system, which aims to solve the technical problem that existing multimodal data calibration schemes cannot simultaneously ensure data privacy and security and calibration accuracy, and are prone to privacy information leakage risks.

[0004] In view of the above problems, this application provides a multimodal data acquisition and calibration method and system.

[0005] The first aspect disclosed in this application provides a method for multimodal data acquisition and calibration, the method comprising: Multimodal data acquisition of a smart factory is performed using a sensor network to obtain multiple raw data streams. These raw data streams undergo multi-granularity de-identification processing to generate multiple de-identified acquisition features. Observability and privacy exposure analysis is performed on these de-identified acquisition features to establish a calibration anchor layer. Spatiotemporal correlation matching detection is then performed on the multiple de-identified acquisition features through this calibration anchor layer to establish an acquisition calibration relationship graph. Cross-modal calibration annotation is performed on the multiple de-identified acquisition features based on the acquisition calibration relationship graph to establish a cross-modal calibration annotation space. Closed-loop transformation residual detection and compensation are then performed on the cross-modal calibration annotation space to obtain a calibration annotation enhancement space. Finally, the multiple de-identified acquisition features are automatically calibrated based on this calibration annotation enhancement space.

[0006] Another aspect of this application discloses a multimodal data acquisition and calibration system, the system comprising: The data acquisition module is used to acquire multimodal data from the smart factory through a sensor network, obtain multiple raw data streams, and perform multi-granularity desensitization processing on the multiple raw data streams to generate multiple desensitized acquisition features. The analysis module is used to perform observability and privacy exposure analysis on the multiple desensitized acquisition features, establish a calibration anchor layer, and perform spatiotemporal correlation matching detection on the multiple desensitized acquisition features through the calibration anchor layer to establish an acquisition calibration relationship graph. The annotation module is used to perform cross-modal calibration annotation on the multiple desensitized acquisition features according to the acquisition calibration relationship graph to establish a cross-modal calibration annotation space. The calibration module is used to perform closed-loop transformation residual detection and compensation on the cross-modal calibration annotation space to obtain a calibration annotation enhancement space, and automatically calibrate the multiple desensitized acquisition features according to the calibration annotation enhancement space.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: By performing multi-granular hierarchical desensitization processing on the raw data stream, the effective information of calibration-related features is fully preserved while ensuring data privacy compliance. Furthermore, a calibration anchor layer is constructed based on the bidirectional screening results of observability and privacy exposure, and a data acquisition calibration relationship graph is established. This enables cross-modal unified calibration labeling of coupling interference in the three layers of equipment, environment, and transmission. Finally, the calibration labeling accuracy is dynamically optimized through closed-loop transform residual detection and feedback compensation mechanisms, completing the automatic calibration of all desensitized acquisition features. This effectively balances data privacy security and calibration accuracy requirements, and improves the spatiotemporal alignment effect and anti-interference calibration capability of multi-source heterogeneous sensing data.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 This application provides a flowchart illustrating a multimodal data acquisition and calibration method. Figure 2 This application provides a schematic diagram of the structure of a multimodal data acquisition and calibration system.

[0010] Figure labeling: Acquisition module 11, Analysis module 12, Labeling module 13, Calibration module 14. Detailed Implementation

[0011] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0012] The overall concept of the technical solution provided in this application is as follows: This application provides a multimodal data acquisition and calibration method and system. By employing multi-granularity hierarchical desensitization processing, core calibration features are preserved while ensuring data privacy and compliance. Combined with a calibration anchor layer and an acquisition-calibration relationship graph, spatiotemporal correlation matching of multi-source data is achieved. Furthermore, cross-modal unified calibration labeling of coupling interference across the device, environment, and transmission layers is completed. Finally, calibration accuracy is dynamically optimized through closed-loop transform residual detection and feedback compensation mechanisms, achieving automatic calibration of all desensitized acquisition features and improving the spatiotemporal consistency and numerical accuracy of multi-source heterogeneous sensing data.

[0013] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0014] Example 1, as Figure 1 As shown in the figure, this application provides a multimodal data acquisition and calibration method, which includes: S100: The smart factory is subjected to multimodal data acquisition through a sensor network, multiple raw data streams are obtained, and the multiple raw data streams are subjected to multi-granularity desensitization processing to generate multiple desensitized acquisition features.

[0015] Specifically, firstly, multimodal data acquisition is carried out through a sensor network deployed throughout the smart factory. This sensor network refers to a distributed sensing system formed by various heterogeneous sensing devices, such as video surveillance cameras, environmental temperature and humidity sensors, personnel access control card readers, equipment operation status acquisition terminals, and acoustic acquisition devices, distributed across different scenarios including public areas, building interiors, and equipment rooms. This system is networked via wired or wireless communication links, enabling continuous sensing coverage of the entire factory space. Multimodal data acquisition involves the simultaneous collection of sensing data with different data formats and semantic dimensions, specifically encompassing both structured and unstructured data such as video images. The system collects data in various modalities, including environmental physical quantity data, personnel access and identification data, equipment operating condition data, and audio data. Different modalities of data reflect different aspects of the same event or operating state in the factory from their respective dimensions. During the data collection process, each sensor continuously outputs data according to its own sampling frequency and original data format, ultimately resulting in multiple independent raw data streams with different timestamps and data formats. The raw data streams are the original sensing data sequences directly output by the sensors without any processing, which fully preserve the original sampling accuracy and full information dimensions of the data. However, they are also mixed with inherent noise from the equipment, transmission interference, and unsensitized privacy information.

[0016] After acquiring multiple raw data streams, multi-granularity desensitization processing is performed to generate multiple desensitized acquisition features. This multi-granularity desensitization processing refers to a hierarchical desensitization mechanism that employs differentiated desensitization intensity and processing granularity for data of different types and sensitivity levels. The process first performs two types of feature recognition operations on the multiple raw data streams. The first type is calibration-related feature recognition, which extracts feature elements directly related to subsequent spatiotemporal calibration and cross-modal matching from the raw data streams, such as the sampling timestamps corresponding to the data, equipment spatial location identifiers, event trigger feature values, and physical quantity change trend features. These features are the core numerical basis for subsequent calibration calculations, and the resulting feature set constitutes the first acquisition feature set. The second type is privacy-sensitive feature recognition, which identifies sensitive information in the raw data streams that can be directly or indirectly associated with the identity of a specific natural person, corporate trade secrets, or core factory operation and maintenance parameters. Examples include facial information and license plate information in video streams, personnel names and employee numbers in access control data, and core process parameters in equipment operation data. If these features directly participate in subsequent calculations, there is a high risk of privacy leakage; the resulting feature set constitutes the second acquisition feature set.

[0017] After dividing the feature sets into two categories, the first collection feature set is irreversibly encoded according to multiple preset desensitization granularities. The preset desensitization granularity refers to the pre-set encoding strength level corresponding to the calibration importance level of different features. For features with higher calibration correlation, a coarser-grained encoding is used to retain their calibration value to the greatest extent. For features with lower calibration correlation, a finer-grained encoding is used to strengthen information protection. Irreversible encoding refers to the use of encoding methods such as hash mapping and feature generalization that cannot reverse the original specific values. The original calibration-related features are converted into encoded values ​​that can be used for calibration calculations but cannot be used to deduce the original specific information. The recognizability of the original information is eliminated while preserving the relative numerical relationship between features and the usability of calibration.

[0018] Simultaneously, the second collection feature set is desensitized based on a privacy-sensitive processing mechanism. This mechanism includes three specific implementation methods: privacy-sensitive feature deletion, privacy-sensitive feature masking, and privacy-sensitive feature isolation. Privacy-sensitive feature deletion refers to directly removing highly sensitive features that do not need to participate in subsequent calibration calculations. Privacy-sensitive feature masking refers to masking and replacing visually sensitive information such as faces and license plates using methods such as pixelation and semantic replacement, eliminating identity recognition while preserving scene information. Privacy-sensitive feature isolation refers to separating and storing sensitive features from calibration-usable features, and controlling their access only at necessary stages of the calibration process. Through differentiated processing methods, multiple data desensitization results corresponding to different original data streams are obtained.

[0019] Finally, the anonymized data processing results are associated and bound with the corresponding device identifier, original timestamp, and anonymization processing parameters. The device identifier is the unique identification code of each sensor, used to accurately distinguish the source device and spatial location attributes of the data. The original timestamp is the native marker of the data sampling time, used to preserve the time dimension benchmark of the data to support subsequent spatiotemporal calibration. The anonymization processing parameters are the granularity level, encoding algorithm, and processing rules used in this anonymization process, used to verify the traceability of features during subsequent calibration. In the end, multiple anonymized collection features are formed that simultaneously possess calibration usability and privacy compliance.

[0020] S200: Perform observability and privacy exposure analysis on the multiple desensitized acquisition features, establish a calibration anchor layer, and perform spatiotemporal correlation matching detection on the multiple desensitized acquisition features through the calibration anchor layer to establish an acquisition calibration relationship graph.

[0021] Specifically, firstly, observability analysis and privacy exposure analysis are conducted sequentially on all de-identified collected features. Observability analysis quantifies the ability of each de-identified feature to represent the real physical events and operational status of the factory from multiple dimensions, such as signal-to-noise ratio, temporal sampling resolution, spatial coverage effectiveness, and numerical fluctuation stability. The result is multiple observability scores for each feature; higher scores indicate a clearer reflection of the real state and greater suitability as a calibration reference. Privacy exposure analysis quantifies the residual privacy leakage risk of each de-identified feature from dimensions such as the number of associative dimensions, the degree of coupling with the original sensitive information, and the feasibility of reversibly restoring sensitive information through association. The result is multiple privacy exposure scores for each feature; higher scores indicate a higher risk of information leakage even after de-identification.

[0022] After completing the two quantitative evaluations, pre-set screening criteria are obtained. These criteria consist of an observability score threshold and a privacy exposure score threshold. The observability score threshold is used to define the minimum representational capability standard for calibrated features. Features below this threshold cannot support accurate calibration due to insufficient information quality. The privacy exposure score threshold is used to define the maximum risk limit for privacy and security. Features above this threshold are not allowed to participate in calibration calculations due to excessive security risks. Subsequently, based on all observability and privacy exposure scores, all de-identified collected features are bidirectionally screened according to the pre-set screening criteria. After removing features that do not meet the admission criteria, a collection feature screening domain with both qualified calibration representational capability and compliant privacy and security level is obtained, which is the candidate feature set that meets the dual-dimensional admission requirements.

[0023] Based on this, event attribution determination is performed on all features within the feature selection domain. This involves classifying and aggregating features according to their corresponding physical event type, spatial region, and time segment. The determination identifies which selected features commonly point to the same real event within the factory, such as personnel passage events, equipment status anomaly events, or environmental parameter fluctuation events. Features with clear spatiotemporal references and the strongest signal stability under the same event are established as calibration anchors. Calibration anchors are high-reliability reference feature points used to unify spatiotemporal references during cross-modal calibration. The reference system formed by organizing all calibration anchors hierarchically according to event attribution is the calibration anchor layer. This layer provides a unified spatiotemporal alignment reference for subsequent full-process calibration operations.

[0024] After the calibration anchor layer is constructed, it serves as the spatiotemporal reference benchmark. Spatiotemporal correlation matching detection is then performed on all multiple desensitized acquisition features. This detection is divided into two parallel stages: time dimension matching and spatial dimension matching. Time dimension matching uses the unified time benchmark of the calibration anchor point as a reference to align the timestamp offsets of each desensitized acquisition feature, correct sampling frequency differences, and match the time windows of event occurrences to determine the correspondence between different features in the time series. Spatial dimension matching uses the spatial location of the calibration anchor point as a reference to match the installation location of each feature acquisition device, the degree of overlap of the sensing coverage area, and the spatial coordinate mapping relationship of event occurrences to determine the degree of correlation between different features in the spatial dimension. Through bidirectional matching detection, it can be determined whether desensitized acquisition features from different devices and different modalities correspond to the same physical event and whether they possess calibrable spatiotemporal correlation.

[0025] Finally, based on the results of spatiotemporal correlation matching detection, an acquisition calibration relationship graph is constructed. This graph is a directed graph structure with each acquisition device as a node and the spatiotemporal correlation strength between the corresponding desensitized acquisition features of the devices as edge weights. Each node carries the desensitized acquisition features and attribute information of the corresponding device. The edges between nodes represent that the features of the two devices have verifiable spatiotemporal calibration correlations, and the weight values ​​of the edges represent the credibility and matching accuracy of the correlation. The topology network formed by all nodes and associated edges is the acquisition calibration relationship graph, which fully describes the calibration correlation logic between all acquisition features.

[0026] S300: Perform cross-modal calibration annotation on the multiple desensitized acquisition features according to the acquisition calibration relationship diagram, and establish a cross-modal calibration annotation space.

[0027] Specifically, cross-modal calibration annotation refers to the standardized calibration attribute labeling of desensitized acquisition features of different modalities such as video, environmental physical quantities, and equipment operating conditions, from the perspective of interference sources. It clarifies the type, degree, and source tracing basis of interference of each feature by various factors. First, it traces the device coupling interference of multiple desensitized acquisition features. Device coupling interference refers to the coupled impact of the hardware state abnormality, performance degradation, parameter drift, and other intrinsic factors of the acquisition sensor itself on the accuracy of the acquired data. Device coupling interference tracing is to trace the causal relationship between the numerical deviation of each desensitized acquisition feature and the abnormal state of the acquisition device itself. Specifically, it first collects the sensor state parameters corresponding to each desensitized acquisition feature to obtain the device state vector. The device state vector is a vectorized representation formed by splicing the device's own state parameters such as the sensor's operating voltage, sampling frequency offset, hardware operating temperature, cumulative running time, and historical calibration deviation according to a unified dimension. It is used to quantitatively describe the current working health status of the sensor.

[0028] Subsequently, anomaly detection is performed on the state vectors of each device to obtain the anomaly detection results. Anomaly detection involves identifying abnormal items in the device state vectors that deviate from the normal working range using a preset normal state threshold range or anomaly detection algorithm. This determines whether the device has abnormal states such as hardware drift, performance degradation, or precursors to failure. The detection results for the anomaly type and degree of each device are output. Then, attention is allocated to the state vectors of each device based on the anomaly detection results to obtain the feature vectors of each device. Attention allocation involves assigning differentiated weights to different dimensions of the state vector according to the degree of anomaly. State dimensions with higher anomaly degrees are assigned higher attention weights to highlight the core device factors causing the acquisition deviation. The vector formed after weighted adjustment is the device feature vector. Finally, interference identification and tracing are performed on multiple desensitized acquisition features based on the feature vectors of each device. This involves associating and mapping the attention weights of the devices with the numerical deviations of the corresponding desensitized acquisition features to identify which feature deviations are caused by the device's own anomalies and the contribution ratio of device factors to the deviations. Finally, the interference association results of all device dimensions are organized hierarchically to generate an acquisition device coupling interference layer. This layer fully records the type, degree, and source tracing basis of interference of each acquisition feature by the device's own factors.

[0029] After completing interference tracing at the equipment level, environmental coupling interference tracing is performed on multiple desensitized acquisition features. Environmental coupling interference refers to the coupling effect of changes in environmental physical conditions within the smart factory, including external environmental factors such as temperature fluctuations, humidity changes, light intensity changes, electromagnetic interference, and rain / snow weather, on the acquisition accuracy of different sensors. Environmental coupling interference tracing involves reversely deducing the correlation between the deviation of desensitized acquisition features and the environmental conditions at the corresponding acquisition time. Specifically, the factory-wide environmental monitoring data corresponding to the sampling time of each desensitized acquisition feature is matched, and the corresponding trend of the change amplitude of environmental parameters and the deviation of feature values ​​is analyzed. At the same time, the sensitivity differences of different modal sensors to different environmental factors are distinguished. For example, optical sensors are sensitive to light and dust, while electrical sensors are sensitive to temperature, humidity, and electromagnetic interference. This allows for the accurate identification of the degree and type of interference of environmental factors on each acquisition feature. Finally, the interference analysis results of all environmental dimensions are integrated to form an acquisition environment coupling interference layer, which carries the annotation information of the influence of the external environment on each acquisition feature.

[0030] Next, transmission coupling interference is traced for multiple desensitized acquisition features. Transmission coupling interference refers to the timestamp offset, missing values, or distortion of acquisition features caused by transmission problems such as communication link delay fluctuations, data packet loss, signal attenuation, and network congestion during the data transmission from the sensor terminal to the data processing node. Transmission coupling interference tracing involves tracking the correspondence between the data transmission link status and the abnormal acquisition features. During execution, the transmission log of the original data stream corresponding to each desensitized acquisition feature is retrieved to record transmission parameters such as transmission delay, packet loss rate, and link signal-to-noise ratio. The abnormal transmission parameters are correlated and matched with the time offset and value jump of the acquisition features to determine the degree of interference of transmission factors on the accuracy of the features. Finally, the interference labeling results of all transmission dimensions are summarized to form the acquisition transmission coupling interference layer. This layer fully labels the impact of transmission link abnormalities on each acquisition feature.

[0031] After constructing the three interference layers, cross-modal calibration association is performed on the acquisition device coupling interference layer, acquisition environment coupling interference layer, and acquisition transmission coupling interference layer based on the node topology and association weights of the acquisition calibration relationship graph. Cross-modal calibration association refers to mapping and matching the interference annotation information of three different dimensions to each desensitized acquisition feature node based on the spatiotemporal association relationship of each node in the acquisition calibration relationship graph. This unifies the dimension, format, and spatiotemporal reference benchmark of calibration annotation for features of different modalities and sources, eliminates the differences in interference annotation rules of different modal data, and allows the annotation information that originally belonged to different interference dimensions and different data modalities to be aligned and compatible under the same framework. The resulting cross-modal calibration annotation space is a multi-dimensional annotation space with the acquisition calibration relationship graph as the topological skeleton, all desensitized acquisition features as data nodes, and each node carrying the three layers of interference calibration annotation information of device, environment, and transmission. All features in the space have a unified calibration annotation specification and spatiotemporal reference system, which can be directly used for subsequent residual detection and calibration compensation calculation.

[0032] S400: Perform closed-loop transformation residual detection compensation on the cross-modal calibration annotation space to obtain the calibration annotation enhancement space, and automatically calibrate the multiple desensitized acquisition features according to the calibration annotation enhancement space.

[0033] Specifically, the closed-loop transformation residual detection and compensation refers to performing spatiotemporal coordinate system transformation verification on each calibration labeling result between different associated device nodes based on the node topology association of the collected calibration relationship diagram. The deviation residual is calculated by the difference between the transformation result and the unified benchmark, and the labeling parameters are then corrected in reverse according to the magnitude of the residual. Specifically, the Sth calibration labeling result is extracted from the cross-modal calibration labeling space, where S is a positive integer. The Sth calibration labeling result refers to the complete labeling information corresponding to the Sth feature extracted sequentially from all feature nodes included in the cross-modal calibration labeling space according to event attribution or node sorting. It covers the three-layer interference labeling data of the device, environment, and transmission attached to the feature, the unified spatiotemporal benchmark information assigned by the calibration anchor point, and the corresponding original desensitized feature value. It is an independent processing unit for this closed-loop verification.

[0034] After extracting a single annotation result, the Sth calibration annotation result is subjected to closed-loop transformation residual detection based on the acquisition calibration relationship diagram to obtain the Sth closed-loop verification residual. This detection process is divided into transformation verification in two dimensions: time and space. In the time dimension, according to the acquisition calibration relationship diagram, the Sth calibration annotation result is subjected to time transformation between different device nodes to obtain the Sth time transformation result. The time transformation refers to transforming the time reference of the current calibration annotation result to the time coordinate system of its associated other device nodes based on the time mapping relationship between associated nodes in the acquisition calibration relationship diagram. This simulates the time annotation result obtained from observing the same event from the perspective of other associated acquisition devices, thereby cross-verifying the cross-node consistency of the time annotation. The corresponding time annotation value output after coordinate system transformation is the Sth time transformation result.

[0035] Subsequently, based on the difference between the unified event time in the Sth time transformation result and the Sth calibration annotation result, the Sth time transformation residual is obtained. The unified event time is the standard time reference for the corresponding event established by the previous calibration anchor layer. The time transformation residual is the numerical difference between the annotation value obtained after time transformation and the standard reference time, which directly reflects the degree of deviation of the current annotation result in the time dimension.

[0036] In the spatial dimension, based on the acquisition and calibration relationship diagram, the Sth calibration annotation result is spatially transformed between different device nodes to obtain the Sth spatial transformation result. The spatial transformation refers to mapping and transforming the spatial position reference of the current calibration annotation result to the spatial coordinate system of other associated device nodes based on the spatial position coordinates of the nodes in the acquisition and calibration relationship diagram and the perception coverage overlap relationship. This simulates the spatial position annotation result obtained from observing the same event from the perspective of other associated acquisition devices, thereby cross-verifying the cross-node consistency of spatial annotation. The corresponding position annotation value output by the coordinate system mapping is the Sth spatial transformation result.

[0037] Next, based on the difference between the unified event position in the Sth spatial transformation result and the Sth calibration annotation result, the Sth spatial transformation residual is determined. The unified event position is the standard spatial position benchmark of the corresponding event established by the previous calibration anchor layer. The spatial transformation residual is the deviation between the annotation value obtained after spatial transformation and the standard benchmark position, which directly reflects the degree of deviation of the current annotation result in the spatial dimension.

[0038] After completing the two-dimensional residual calculation, the S-th time transformation residual and the S-th spatial transformation residual are weighted and fused to generate the S-th closed-loop verification residual. The weighted fusion refers to assigning corresponding weight coefficients to the time residual and the spatial residual according to the modal attributes of the corresponding features and the accuracy requirements of the calibration scenario, and then summing them to obtain the total residual value that can comprehensively reflect the overall spatiotemporal deviation of the annotation result. The larger the residual value, the greater the deviation between the current calibration annotation result and the cross-node benchmark, and the lower the annotation accuracy.

[0039] After obtaining the Sth closed-loop verification residual, it is determined whether the residual is greater than or equal to the predetermined closed-loop verification residual. The predetermined closed-loop verification residual is a pre-set calibration labeling accuracy threshold, representing the maximum acceptable labeling deviation upper limit for the calibration scenario. If the residual is lower than the threshold, it means that the labeling accuracy meets the subsequent calibration requirements; if it is higher, it means that the labeling deviation is too large and needs to be corrected and optimized. If the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual, the Sth calibration labeling result is compensated based on the Sth closed-loop verification residual to obtain the Sth calibration labeling correction result. The feedback compensation refers to adjusting the weight assignment of the time labeling parameters, spatial labeling parameters, and three interference layers in the corresponding calibration labeling result in reverse, based on the magnitude and direction of the closed-loop verification residual, to accurately offset the deviation amount corresponding to the residual. The labeling result after deviation correction is the Sth calibration labeling correction result.

[0040] Subsequently, the correction result of the Sth calibration annotation is added to the calibration annotation enhancement space. If the residual of the Sth closed-loop verification is less than the predetermined residual of the closed-loop verification, it means that the accuracy of the annotation result has met the standard and no additional correction is required. The Sth calibration annotation result is then directly added to the calibration annotation enhancement space. By performing the above closed-loop verification, judgment and correction process on all calibration annotation results in the cross-modal calibration annotation space one by one, the final set of calibration annotations that meet the accuracy requirements is obtained, namely the calibration annotation enhancement space. Compared with the original cross-modal calibration annotation space, all annotation results in this space have undergone cross-node cross-verification and deviation optimization, and the spatiotemporal consistency and annotation accuracy are significantly improved. It is the high-precision benchmark system for the final data calibration.

[0041] After completing the construction of the calibration annotation enhancement space, multiple desensitized acquisition features are automatically calibrated based on this space. Specifically, the three-layer interference annotation parameters and spatiotemporal alignment parameters in the calibration annotation enhancement space are used as a unified basis to automatically correct the timestamp offset, spatial position deviation, and numerical interference distortion of all original desensitized acquisition features. This accurately offsets various acquisition errors caused by three types of factors: equipment drift, environmental interference, and transmission anomalies. Finally, the calibrated multimodal feature data with accurate spatiotemporal alignment and reliable numerical accuracy is output.

[0042] Furthermore, in the method provided in this application embodiment, multi-granularity de-identification processing is performed on the multiple raw data streams to generate multiple de-identified acquisition features, including: performing calibration association feature identification on the multiple raw data streams to obtain a first acquisition feature set; performing privacy-sensitive feature identification on the multiple raw data streams to obtain a second acquisition feature set; performing irreversible encoding on the first acquisition feature set according to multiple preset de-identification granularities, and performing de-identification processing on the second acquisition feature set based on a privacy-sensitive processing mechanism to obtain multiple data de-identification processing results; associating each data de-identification processing result with the corresponding device identifier, original timestamp, and de-identification processing parameters to obtain the multiple de-identified acquisition features.

[0043] Specifically, multi-granularity desensitization processing is performed on the raw data stream to generate multiple desensitized collection features. This multi-granularity desensitization process employs a tiered desensitization mechanism that differentiates the desensitization intensity and processing method based on the calibration value and privacy risk level of different features, rather than applying a uniform standard desensitization operation to all data. This ensures maximum retention of the effective information required for calibration while protecting privacy and security. Specifically, calibration-related feature identification is performed on multiple raw data streams to obtain the first collection feature set. This identification involves selecting and extracting feature elements directly related to subsequent core calibration processes such as spatiotemporal alignment, cross-modal matching, and residual verification from the full data dimension of the raw data streams through methods such as feature importance assessment and calibration contribution matching. These features include data sampling timestamp features, device spatial location identifiers, physical quantity change trend features, event trigger threshold features, and device operating baseline state parameters. These features are the core numerical basis supporting subsequent calibration calculations and generally possess high calibration value and weak direct sensitivity. The feature set composed of all calibration-related features constitutes the first collection feature set.

[0044] Simultaneously, multiple raw data streams are subjected to privacy-sensitive feature identification in parallel to obtain a second collection feature set. Privacy-sensitive feature identification refers to identifying sensitive data dimensions in the raw data streams that can be directly or indirectly associated with the identity of specific natural persons, factory trade secrets, or core operation and maintenance parameters through technologies such as entity extraction, semantic matching, and sensitive rule matching. Examples include facial features and license plate information in video data streams, personnel identification in access control data streams, and core process operation parameters in equipment data streams. If these features are directly involved in subsequent calculations, there is a high risk of information leakage. Overall, they have the characteristics of high privacy risk and low direct contribution to calibration. The feature set composed of all privacy-sensitive features is the second collection feature set.

[0045] After dividing the feature sets into two categories, the first collection feature set is irreversibly encoded according to multiple preset desensitization granularities. The preset desensitization granularity refers to multiple levels of desensitization intensity based on the calibration importance of the features. Features with higher calibration correlation correspond to coarser desensitization granularity to preserve the numerical relationships and feature attributes required for calibration as much as possible. Features with lower calibration correlation correspond to finer desensitization granularity to strengthen information protection. Irreversible encoding refers to the use of encoding methods such as hash mapping, numerical binning generalization, and feature embedding transformation, which cannot restore the original specific values ​​through reverse operation, to convert the original feature values ​​in the first collection feature set into encoded feature expressions. The conversion process completely preserves the relative numerical relationships, temporal change patterns, and spatial correlation attributes between features, ensuring the logical validity of subsequent calibration operations, while completely eliminating the identifiability of the original values, making it impossible to deduce the specific content of the original data from the encoding results.

[0046] While encoding the first set of collected features, the second set of collected features is desensitized based on a privacy-sensitive processing mechanism. This mechanism includes three differentiated processing paths: privacy-sensitive feature deletion, privacy-sensitive feature masking, and privacy-sensitive feature isolation. Privacy-sensitive feature deletion refers to directly removing purely sensitive features that do not participate in any calibration calculations from the feature set, eliminating the risk of information leakage at the source. Privacy-sensitive feature masking refers to eliminating the identifiability of sensitive features that need to retain scene structure information but do not require identity recognition attributes, such as faces and license plates in video footage, using pixelation masking and semantic replacement, while retaining calibration-usable information such as motion trajectories and regional locations in the image. Privacy-sensitive feature isolation refers to separating and storing sensitive features that need to be accessed under controlled conditions in certain special scenarios from regular calibration features. The main calibration process does not directly access these features, but only calls them as needed in dedicated controlled stages.

[0047] After the above two types of differentiated processing, each original data stream corresponds to a set of de-identified feature data, which together form multiple data de-identification results. Finally, each data de-identification result is associated and bound with the corresponding device identifier, original timestamp, and de-identification parameters. The device identifier is the unique identification code of the corresponding acquisition sensor, carrying attribute information such as the device's installation location, device type, and historical calibration baseline. It is used to trace the data source and match spatial association in the subsequent calibration process. The original timestamp is the native time mark at the moment of data sampling, which completely preserves the original sampling time reference and is used for time alignment and residual calculation in subsequent spatiotemporal calibration. The de-identification parameters record the granularity level, encoding algorithm, and specific processing rules used in this de-identification. They are used to verify the de-identification level of the features and trace the processing process in the subsequent verification, supporting the quantitative assessment of observability and privacy exposure. After binding and encapsulating the de-identified features with the three types of metadata one by one, multiple de-identified acquisition features with complete attributes are finally obtained, which serve as the standard input for all subsequent calibration steps.

[0048] Furthermore, in the method provided in this application embodiment, the observability and privacy exposure analysis of the plurality of de-identified collection features is performed to establish a calibration anchor layer, including: evaluating the observability of the plurality of de-identified collection features to obtain a plurality of observability scores; evaluating the privacy exposure of the plurality of de-identified collection features to obtain a plurality of privacy exposure scores; obtaining predetermined screening conditions, the predetermined screening conditions including an observability score threshold and a privacy exposure score threshold; based on the plurality of observability scores and the plurality of privacy exposure scores, screening the plurality of de-identified collection features according to the predetermined screening conditions to obtain a collection feature screening domain; and determining event attribution for the collection feature screening domain to generate the calibration anchor layer.

[0049] Specifically, the observability of multiple desensitized acquisition features is first evaluated to obtain multiple observability scores. The observability evaluation refers to the quantitative assessment of each desensitized acquisition feature from multiple dimensions, such as signal-to-noise ratio, temporal sampling resolution, effective spatial coverage, numerical fluctuation stability, and event response sensitivity. This comprehensively measures the feature's ability to accurately represent real physical events and operating status in the factory. During the evaluation process, the scores for each dimension are assigned and weighted based on the corresponding sensor's equipment type, historical calibration baseline, and scenario adaptability. Finally, the observability score for each desensitized acquisition feature is output. The higher the score, the better the perception quality of the feature, the higher the degree of restoration of the real state, and the more suitable it is as a reference benchmark for cross-modal calibration.

[0050] After completing the observability evaluation, the privacy exposure of multiple de-identified collection features is evaluated simultaneously, resulting in multiple privacy exposure scores. The privacy exposure evaluation refers to a quantitative risk assessment for each de-identified collection feature, considering factors such as the degree of coupling between the de-identified feature and the original sensitive information, the feasibility of reversibly restoring sensitive information through multi-feature association, and the number of identifiable identity dimensions carried by the feature. This comprehensively measures the potential privacy leakage risk that remains even after de-identification. During the evaluation process, the de-identification granularity level, the proportion of residual sensitive information, and the difficulty of association derivation are combined to assign a comprehensive value. Finally, a privacy exposure score is output for each de-identified collection feature. The higher the score, the higher the risk of information leakage for that feature, and the less suitable it is for deep participation in calibration calculations.

[0051] After completing the two quantitative evaluations, predetermined screening criteria are obtained. These criteria include an observability score threshold and a privacy exposure score threshold. The observability score threshold is a pre-set minimum quality threshold for calibrated features. Features below this threshold cannot provide reliable numerical data for calibration due to insufficient perception accuracy and high information distortion. The privacy exposure score threshold is a pre-set maximum risk limit acceptable to the calibration process. Features above this threshold are not allowed to participate in subsequent core calibration steps due to excessive privacy leakage risks.

[0052] Subsequently, based on all observability scores and all privacy exposure scores, a two-way screening is performed on all de-identified features according to predetermined screening conditions. Features with unacceptable observability or excessive privacy risk are eliminated, while features that simultaneously meet the dual requirements of observability score ≥ threshold and privacy exposure score ≤ threshold are retained. The resulting set of qualified features is the feature screening domain. This screening domain eliminates low-quality and high-risk invalid features and is a high-quality candidate pool for constructing calibration anchors.

[0053] After obtaining the feature selection domain, event attribution is determined for all features within it, generating a calibration anchor point layer. Event attribution determination refers to clustering and matching based on the physical event type, spatial region, and time window corresponding to the feature to identify which selected features correspond to the same real event within the factory, such as personnel intrusion, equipment temperature rise, or abnormal fluctuation of environmental parameters. Within the feature cluster corresponding to each type of event, the feature with the highest observability score, clearest spatiotemporal reference, and strongest long-term signal stability is selected as the calibration anchor point for that calibration scenario. The calibration anchor point is a highly reliable reference feature point used to unify spatiotemporal reference and verify deviations of other features during cross-modal calibration. It possesses the core attributes of high reference accuracy, low risk level, and strong stability. After organizing the calibration anchor points corresponding to all events hierarchically according to event level and spatial region, a systematic calibration anchor point layer is finally formed.

[0054] Furthermore, in the method provided in this application embodiment, cross-modal calibration annotation is performed on the multiple desensitized acquisition features according to the acquisition calibration relationship diagram to establish a cross-modal calibration annotation space, including: tracing device coupling interference of the multiple desensitized acquisition features to obtain an acquisition device coupling interference layer; tracing environmental coupling interference of the multiple desensitized acquisition features to obtain an acquisition environment coupling interference layer; tracing transmission coupling interference of the multiple desensitized acquisition features to obtain an acquisition transmission coupling interference layer; and performing cross-modal calibration association on the acquisition device coupling interference layer, the acquisition environment coupling interference layer, and the acquisition transmission coupling interference layer according to the acquisition calibration relationship diagram to generate the cross-modal calibration annotation space.

[0055] Specifically, cross-modal calibration annotation refers to labeling the type, degree, source basis, and correction direction of each feature affected by various interference factors according to a unified calibration dimension specification for desensitized collection features of different modalities such as video, environmental physical quantities, equipment operating conditions, and acoustic signals, so that features of heterogeneous modalities have a unified calibration annotation system.

[0056] Specifically, the process begins by tracing device coupling interference for multiple desensitized acquisition features to obtain the device coupling interference layer. Device coupling interference refers to the source of endogenous deviation coupled with the accuracy of the acquired data, caused by abnormal hardware attributes and operating states of the acquisition sensors themselves. This includes parameter drift due to aging of sensor components, numerical offset due to power supply voltage fluctuations, timestamp misalignment caused by sampling clock deviation, and device calibration baseline offset. This type of interference originates from the device itself and will continuously affect all acquired data output by the device. Tracing device coupling interference involves reverse deducing the causal relationship between the numerical deviation of each desensitized acquisition feature and the corresponding state of the acquisition device, accurately locating the interference source and its impact at the device level. Specifically, the sensor state parameters corresponding to each desensitized acquisition feature are collected to obtain the device state vector. These sensor state parameters include endogenous parameters reflecting the health status of the device, such as real-time operating voltage, internal operating temperature, clock offset, cumulative running time, historical calibration deviation records, and component attenuation coefficients. The device state vector is a vectorized representation formed by splicing the above multi-dimensional state parameters according to a unified standard, used to quantitatively characterize the current working state and performance level of a single sensor.

[0057] Subsequently, anomaly detection is performed on the state vectors of each device to obtain the anomaly detection results for each device. Anomaly detection refers to identifying abnormal dimensions in the state vector that deviate from the normal working range by using a preset normal state threshold range for the device or an anomaly detection algorithm trained based on historical normal data. This determines whether the device has abnormal states such as hardware drift, performance degradation, or latent fault precursors. The detection conclusions of the anomaly type, degree, and dimension for each device are output. Then, attention allocation is performed on the state vectors of each device based on the anomaly detection results to obtain the feature vectors of each device. Attention allocation refers to assigning differentiated weight coefficients to different dimensions of the state vector according to the degree of anomaly. The state dimension with a higher degree of anomaly is assigned a higher weight ratio, thereby amplifying the core device factors that cause acquisition deviation and weakening the interference of normal state dimensions. The vector formed after weighted adjustment is the device feature vector, which can intuitively reflect the core factors affecting acquisition accuracy at the device level.

[0058] Finally, based on the feature vectors of each device, interference identification and tracing are performed on multiple desensitized acquisition features. This involves associating the attention weight of the device with the magnitude and direction of the numerical deviation of the corresponding desensitized acquisition feature, quantifying the contribution of each device anomaly to the numerical deviation of the corresponding feature, identifying the part of the deviation caused by the device itself, and finally organizing the interference labeling results of all device dimensions into a structured hierarchical system according to the correspondence between device level and feature, generating the acquisition device coupling interference layer. This layer fully records the type, degree, and source tracing basis of interference of each desensitized acquisition feature by the device itself.

[0059] After completing interference tracing at the device level, environmental coupling interference tracing is simultaneously performed on multiple desensitized acquisition features to obtain the acquisition environment coupling interference layer. Environmental coupling interference refers to the coupled impact of changes in external environmental physical conditions within the smart factory on the acquisition accuracy of different types of sensors. Specifically, this includes the impact of environmental temperature and humidity fluctuations on the parameters of electrical sensors, the impact of changes in light intensity and dust obstruction on the perception of optical sensors, interference from strong electromagnetic environments on signal sensors, and numerical disturbances of outdoor sensors caused by weather conditions such as rain, snow, and frost. This type of interference is an external factor, and different modal sensors exhibit significant differences in sensitivity to different environmental factors. Environmental coupling interference tracing involves matching the full-domain environmental monitoring data at the sampling time of each desensitized acquisition feature. By reverse deducing the correlation between the numerical deviation of features and the corresponding changes in environmental conditions, the source and degree of interference at the environmental level are located. Specifically, synchronous data from environmental monitoring nodes in the corresponding spatial area are retrieved to analyze the correlation between the magnitude and trend of changes in environmental parameters and the numerical deviation of collected features. At the same time, combined with the environmental sensitivity curves of different modal sensors, the influence weight of different environmental factors on different features is distinguished, and the degree of interference of each type of environmental factor on the corresponding collected feature is accurately quantified. Finally, the interference labeling results of all environmental dimensions are organized into layers according to the type of environmental factor and spatial region to form a collection environment coupling interference layer. This layer carries the complete labeling information of the influence of external environment on each collected feature, clarifying the action path and influence level of environmental interference.

[0060] Simultaneously, transmission coupling interference is traced in parallel for multiple desensitized acquisition features to obtain the acquisition transmission coupling interference layer. Transmission coupling interference refers to the distortion of acquisition features caused by communication link anomalies during the transmission of raw data from the sensor terminal to the data processing node. Specifically, this includes timestamp offsets caused by network latency fluctuations, numerical missingness or interpolation deviations caused by data packet loss, numerical distortion due to signal attenuation, and timing misalignments caused by network congestion. This type of interference originates in the data transmission link and directly damages the timing accuracy and numerical integrity of the acquisition features. Tracing transmission coupling interference involves retrieving the transmission link logs corresponding to each raw data stream to trace... The study traces the correspondence between abnormal transmission status and deviations in the acquired features, identifies interference sources and their impact at the transmission layer, and matches transmission parameters such as transmission delay, packet loss rate, link signal-to-noise ratio, and retransmission count for each de-identified acquired feature during execution. It then correlates and matches abnormal transmission parameter periods with time offsets, numerical jumps, and missing data periods of the acquired features to determine the degree of interference of transmission factors on the feature's time precision and numerical accuracy. Finally, it integrates the interference labeling results of all transmission dimensions according to time sequence and link level to form an acquisition and transmission coupling interference layer. This layer fully labels the impact and time sequence distribution of transmission link anomalies on each acquired feature.

[0061] After independently constructing the three interference layers (equipment, environment, and transmission), cross-modal calibration associations are performed on the acquisition equipment coupling interference layer, acquisition environment coupling interference layer, and acquisition transmission coupling interference layer based on the node topology and edge weight information of the acquisition calibration relationship graph. This generates a cross-modal calibration annotation space. Cross-modal calibration association refers to mapping and matching the interference annotation information of the three independent dimensions to each desensitized acquisition feature node based on the spatiotemporal relationships and event attribution of each node in the acquisition calibration relationship graph. This provides a unified dimensional definition, numerical specification, and spatiotemporal reference base for calibration annotations of features from different modalities and sources. Accuracy eliminates the heterogeneous differences in interference annotation rules and benchmark systems of different modal data, enabling annotation information that originally belonged to different interference dimensions and different data modalities to be aligned and compatible under the same spatiotemporal framework. The resulting cross-modal calibration annotation space is a multi-dimensional annotation system with the acquisition and calibration relationship diagram as the topological skeleton, all desensitized acquisition features as data nodes, and each node carrying complete interference calibration annotation information of three layers: equipment, environment, and transmission. All features in the space have unified calibration annotation specifications and spatiotemporal reference standards, which can be directly used as standardized inputs for subsequent closed-loop residual detection and calibration compensation calculations.

[0062] Furthermore, in the method provided in this application embodiment, the device coupling interference tracing of the plurality of desensitized acquisition features to obtain the acquisition device coupling interference layer includes: acquiring sensor state parameters corresponding to each desensitized acquisition feature to obtain each device state vector; performing anomaly detection on each device state vector to obtain each device anomaly detection result; allocating attention to each device state vector according to the anomaly detection result to obtain each device feature vector; and performing interference identification and tracing of the plurality of desensitized acquisition features based on the device feature vector to generate the acquisition device coupling interference layer.

[0063] Specifically, the device coupling interference tracing refers to the reverse derivation of the causal relationship between the numerical and temporal deviations of each desensitized acquisition feature and the abnormal operating status of the corresponding acquisition device, so as to accurately locate the interference source, interference degree and action path at the device level. The acquisition device coupling interference layer is a structured hierarchical system that carries the interference annotation information of all acquisition feature device dimensions, and is one of the components of the cross-modal calibration annotation space.

[0064] Specifically, the sensor status parameters corresponding to each desensitized acquisition feature are first collected to obtain the status vector of each device. The sensor status parameters refer to the intrinsic parameters that can reflect the sensor's own operating health status and performance level. These parameters include multi-dimensional indicators such as the device's real-time operating voltage, sampling clock offset, internal operating temperature, cumulative running time, historical calibration baseline deviation, core component attenuation coefficient, and sampling frequency fluctuation amplitude. These parameters are independent of the collected business data and directly reflect the device's own working status. The device status vector is a vectorized representation formed by splicing the above multi-dimensional status parameters according to a unified dimension sorting and normalization standard. Each acquisition sensor corresponds to a unique device status vector, which can quantitatively characterize the current comprehensive performance status of a single device and realize the standardized and calculable expression of the device status.

[0065] After constructing the device state vectors, anomaly detection is performed on each device's state vector to obtain the anomaly detection results. Anomaly detection refers to comparing all dimensions of each device's state vector with a benchmark model established using the sensor's factory-calibrated normal operating parameter range and historical normal operating data. This identifies parameter dimensions that deviate from the normal operating range and determines the specific type and degree of the anomaly. Anomaly types include parameter drift caused by component aging, timestamp misalignment caused by clock crystal oscillator deviation, numerical offset caused by unstable power supply, and decreased sensitivity caused by device performance degradation. The degree of anomaly is quantified by the deviation of the actual value from the normal benchmark value. The final output of the anomaly detection results for each device will fully record the anomaly dimensions, anomaly types, deviation magnitudes, and risk levels of each device, clearly identifying all anomalies that may affect the acquisition accuracy.

[0066] After obtaining the equipment anomaly detection results, attention is allocated to the state vectors of each equipment based on the anomaly detection results to obtain the feature vectors of each equipment. The attention allocation refers to assigning differentiated weight coefficients to different dimensions of the equipment state vector according to the degree of anomaly of the corresponding dimension. Dimensions with higher anomaly degrees are assigned higher weight proportions, while dimensions in the normal range are assigned lower weights. By adjusting the weights differently, the core equipment factors that cause the acquisition deviation are amplified, while the ineffective interference of normal state dimensions on interference tracing is weakened. This avoids averaging processing from masking the core anomaly causes. The vector obtained after weight adjustment is the equipment feature vector. This vector can intuitively highlight the core equipment anomalies that affect the acquisition accuracy, filter out irrelevant normal state information, and allow subsequent interference tracing to focus on the core causes.

[0067] Finally, based on the feature vectors of each device, interference identification and tracing are performed on multiple desensitized acquisition features to generate a coupled interference layer for acquisition devices. Interference identification and tracing refers to associating and mapping high-weight anomalies in the device feature vectors with the numerical and temporal deviations of the corresponding desensitized acquisition features output by the sensors, and quantifying the relationship between each device anomaly and the acquisition feature deviation. The contribution ratio of device anomalies to feature deviations is quantified, and the specific magnitude and direction of deviations caused by device factors are clarified. For example, sampling clock offset anomalies are correlated with feature timestamp deviations, and component attenuation anomalies are correlated with the overall numerical offset of features. After completing the device interference tracing for all features, all interference labeling results are structured according to device level and feature affiliation to form the coupled interference layer for acquisition devices. In this layer, each desensitized acquisition feature is labeled with the specific type of interference, degree of interference, tracing basis, and corresponding anomaly, forming a complete device-dimensional interference labeling system.

[0068] Furthermore, in the method provided in this application embodiment, the closed-loop transformation residual detection compensation is performed on the cross-modal calibration annotation space to obtain the calibration annotation enhancement space, including: extracting the Sth calibration annotation result according to the cross-modal calibration annotation space, where S is a positive integer; performing closed-loop transformation residual detection on the Sth calibration annotation result according to the acquired calibration relationship diagram to obtain the Sth closed-loop verification residual; determining whether the Sth closed-loop verification residual is greater than or equal to a predetermined closed-loop verification residual; if the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual, performing feedback compensation on the Sth calibration annotation result according to the Sth closed-loop verification residual to obtain the Sth calibration annotation correction result, and adding the Sth calibration annotation correction result to the calibration annotation enhancement space.

[0069] Specifically, the closed-loop transformation residual detection and compensation refers to mapping and transforming a single calibration annotation result to the local coordinate system of the associated device, supported by the node association topology of the acquisition calibration relationship graph. The verification residual is calculated by the deviation between the transformation result and the unified benchmark, and then the annotation parameters are corrected in reverse based on the residual. Specifically, the S-th calibration annotation result is first extracted from the cross-modal calibration annotation space, where S is a positive integer. The cross-modal calibration annotation space is a multi-dimensional annotation set with the acquisition calibration relationship graph as the topological skeleton, which carries all the three layers of interference annotations corresponding to the desensitized acquisition features, the environment, and the transmission, as well as the unified spatiotemporal benchmark. The S-th calibration annotation result is a complete annotation unit corresponding to a single feature extracted from the space according to node sorting or event affiliation, containing the unified event time of that feature. Unified event location The three-layer interference quantification annotation value, the list of associated device nodes, and other full annotation information constitute the smallest independent processing unit for closed-loop verification. Successive extraction ensures that all annotation results in the space undergo complete accuracy verification and optimization.

[0070] After extracting a single annotation result, the S-th calibration annotation result is subjected to closed-loop transformation residual detection based on the acquisition and calibration relationship diagram to obtain the S-th closed-loop verification residual. The closed-loop transformation residual detection relies on the clock offset mapping and spatial coordinate transformation relationships between nodes recorded in the acquisition and calibration relationship diagram. It transforms the unified benchmark of the current annotation to the local coordinate system of each associated device node, quantifying the annotation deviation by comparing the difference between the transformation result and the benchmark value. The detection is performed in two parallel dimensions: time and space, and finally, a weighted fusion is used to obtain the comprehensive residual. In the time dimension, based on the acquisition and calibration relationship diagram, the S-th calibration annotation result undergoes time transformation between different device nodes to obtain the S-th time transformation result. The time transformation pointer points to the i-th device node associated with the S-th calibration annotation result and calls the clock offset parameter of that node pre-stored in the acquisition and calibration relationship diagram. Time mapping coefficient The uniform event time will be marked Transform to the local time coordinate system of this node, simulate the time stamp values ​​obtained from observing the same event from the perspective of this device, and the first... The formula for calculating the time transformation result of each node is: ; The time transformation results of all associated nodes together form the Sth time transformation result set; Then, the mean of the absolute values ​​of the differences between the time transformation results of each node and the unified event time is calculated to obtain the S-th time transformation residual. The calculation formula can be expressed as: ; Where n is the total number of associated nodes. It directly quantifies the cross-node consistency deviation of the current annotation results in the time dimension; the larger the residual value, the more significant the deviation in time annotation. In the spatial dimension, it synchronously performs spatial transformation on the S-th calibration annotation result across different device nodes according to the acquisition and calibration relationship graph, and obtains the S-th spatial transformation result. The spatial transformation pointer calls the pre-stored spatial coordinate offset vector of the node associated with the S-th calibration annotation result to the i-th device node in the acquisition and calibration relationship graph. With spatial mapping matrix The uniform event location will be marked. Transform to the local spatial coordinate system of this node, simulate the location label value obtained from observing the same event from the perspective of this device, and calculate the spatial transformation result of the i-th node using the following formula: ; The spatial transformation results of all associated nodes together form the S-th spatial transformation result set. Then, the mean Euclidean distance between the spatial transformation result of each node and the unified event location is calculated to obtain the S-th spatial transformation residual. The calculation formula can be expressed as: ; in Represents Euclidean distance operations in two-dimensional or three-dimensional space. It directly quantifies the cross-node consistency deviation of the current annotation results in the spatial dimension; After completing the two-dimensional residual calculation, the S-th time transformation residual and the S-th spatial transformation residual are weighted and fused to generate the S-th closed-loop verification residual. The weighted fusion method assigns weights to the temporal residuals based on the modal attributes of the current features and the accuracy requirements of the calibration scenario. Assign weights to spatial residuals And it satisfies the weight normalization condition. For example, the temporal weighting is increased for time-sensitive acoustic features, and the spatial weighting is increased for spatially sensitive video features. The fusion formula is as follows: ; The residual value comprehensively reflects the overall spatiotemporal deviation level of the S-th calibration annotation result. The larger the residual, the worse the cross-node consistency of the annotation result and the lower the annotation accuracy.

[0071] After obtaining the S-th closed-loop verification residual, determine whether the residual is greater than or equal to the predetermined closed-loop verification residual. Among them, the predetermined closed-loop verification residual It is a qualified threshold set in advance based on the business accuracy requirements of multimodal data of smart factories. It represents the maximum acceptable deviation of calibration and is the criterion for determining whether the calibration result needs to be corrected.

[0072] If the S-th closed-loop verification residual Greater than or equal to the predetermined closed-loop verification residual This indicates that the current annotation result has excessive deviation and the accuracy does not meet the calibration requirements. Feedback compensation is performed on the Sth calibration annotation result based on the Sth closed-loop verification residual to obtain the Sth calibration annotation correction result. Feedback compensation refers to adjusting the time reference parameters and spatial location parameters in the annotation result in reverse, based on the magnitude and direction of the closed-loop verification residual. Simultaneously, it corrects the weight assignments of the interference annotations at the equipment, environment, and transmission layers, accurately offsetting the deviation corresponding to the residual. For example, it corrects the offset of the unified event time for the time residual and the offset of the position coordinates for the spatial residual. The correction process can be achieved through reverse superposition of deviations. ; in , For the revised unified spacetime reference, The sign of the average direction of the time deviation. Let S be the average direction vector of the spatial deviation. The complete annotation result after deviation correction is the Sth calibration annotation correction result. Then, the Sth calibration annotation correction result is added to the calibration annotation enhancement space.

[0073] If the residual of the Sth closed-loop verification is less than the predetermined residual of the closed-loop verification, it means that the cross-node consistency and accuracy of the annotation result have met the requirements and no additional correction is needed. The calibration annotation result is directly added to the calibration annotation enhancement space. After all the calibration annotation results in the cross-modal calibration annotation space have completed the above closed-loop verification, judgment and optimization process, the final set of full compliance annotations is the calibration annotation enhancement space. All the annotation results in this space have undergone cross-node cross-verification and deviation optimization. The spatiotemporal consistency and annotation accuracy are significantly improved compared with the original cross-modal calibration annotation space.

[0074] Furthermore, in the method provided in this application embodiment, the method of performing closed-loop transformation residual detection on the Sth calibration labeling result according to the acquisition calibration relationship diagram to obtain the Sth closed-loop verification residual includes: performing time transformation on the Sth calibration labeling result between different device nodes according to the acquisition calibration relationship diagram to obtain the Sth time transformation result; obtaining the Sth time transformation residual based on the difference between the Sth time transformation result and the unified event time in the Sth calibration labeling result; performing spatial transformation on the Sth calibration labeling result between different device nodes according to the acquisition calibration relationship diagram to obtain the Sth spatial transformation result; determining the Sth spatial transformation residual based on the difference between the Sth spatial transformation result and the unified event position in the Sth calibration labeling result; and weightedly fusing the Sth time transformation residual and the Sth spatial transformation residual to generate the Sth closed-loop verification residual.

[0075] Specifically, through cross-node coordinate system transformation and deviation calculation in both spatiotemporal dimensions, the spatiotemporal consistency deviation of the calibration annotation is transformed into a quantifiable numerical indicator. The closed-loop transformation residual detection refers to transforming the global unified reference of the S-th calibration annotation result to the local coordinate system of each associated device node based on the spatiotemporal mapping parameters pre-stored in the acquisition calibration relationship diagram. By comparing the difference between the transformed result and the original reference, the cross-node consistency deviation of the annotation result is quantified. Specifically, based on the acquisition calibration relationship diagram, a time transformation is performed on the S-th calibration annotation result across different device nodes to obtain the S-th time transformation result. This time transformation refers to mapping the global unified time reference in the annotation result to the local time coordinate system of a single device based on the clock parameters of each associated device node stored in the acquisition calibration relationship diagram, simulating the time annotation value output by the device when independently observing the same event. Specifically, for the i-th device node associated with the S-th calibration annotation result, the time scale coefficient of that node pre-stored in the acquisition calibration relationship diagram is called. Characterizing the rate difference between the local clock and the standard clock, and the clock offset. This characterizes the fixed phase difference between the local clock and the standard clock, combined with the uniform event time in the S-th calibration annotation result. The calculation is performed, where the unified event time is the global standard time reference for the corresponding event, established in the early stage through the calibration of the anchor layer. It is the core reference for time alignment of all modal data. The calculation formula for the single-node time transformation result is as follows: ; In the formula This is the time transformation output value of the i-th device node. All transformation values ​​obtained by traversing all associated nodes together form the S-th time transformation result set, i.e., the S-th time transformation result.

[0076] After completing the time transformation, the S-th time transformation residual is obtained based on the difference between the S-th time transformation result and the unified event time in the S-th calibration annotation result. The time transformation residual is a quantitative indicator measuring the consistency of time annotation across nodes, reflecting the overall deviation of the current annotation's time base from different device perspectives. The calculation is based on the average of the absolute deviations between the time transformation results of all associated nodes and the unified event time. The calculation formula is as follows: ; In the formula, n is the total number of associated device nodes. This is the S-th time transformation residual. The larger the residual value, the worse the cross-node alignment accuracy of the time label.

[0077] After completing the time-dimensional detection calculations, the spatial transformation of the S-th calibration annotation result is performed synchronously between different device nodes based on the acquisition and calibration relationship diagram to obtain the S-th spatial transformation result. This spatial transformation refers to mapping the globally unified position reference in the annotation result to the local observation coordinate system of a single device, based on the spatial coordinate mapping parameters of each associated device node stored in the acquisition and calibration relationship diagram. This simulates the position annotation value output by the device when independently observing the same event. Specifically, for the i-th device node, the spatial mapping matrix of that node pre-stored in the acquisition and calibration relationship diagram is called. It includes transformation parameters for coordinate system rotation and scaling, and spatial offset vector. This characterizes the translation between the two coordinate systems, combined with the unified event location in the S-th calibration annotation result. The calculation involves using the unified event location, which is the global standard spatial coordinate of the corresponding event established in the early stage through the calibration of the anchor point layer. This coordinate serves as the core reference for spatial alignment of all modal data. The calculation formula for the single-node spatial transformation result is as follows: ; In the formula The spatial transformation output vector of the i-th device node is used. All transformation vectors obtained by traversing all associated nodes together form the S-th spatial transformation result set, i.e., the S-th spatial transformation result. Then, based on the difference between the S-th spatial transformation result and the unified event position in the S-th calibration annotation result, the S-th spatial transformation residual is determined. The spatial transformation residual is a quantitative indicator measuring the consistency of spatial annotations across nodes, reflecting the overall deviation of the current annotation's positional reference from different device perspectives. The calculation is based on the average Euclidean distance between the spatial transformation results of all associated nodes and the unified event position. The calculation formula is: ; In the formula Euclidean norm operations represent two-dimensional or three-dimensional space, used to quantify the straight-line distance between two spatial coordinate points. This is the S-th spatial transformation residual. The larger the residual value, the worse the cross-node alignment accuracy of the spatial annotation.

[0078] Finally, the S-th time transformation residual and the S-th spatial transformation residual are weighted and fused to generate the S-th closed-loop verification residual. The weighted fusion refers to assigning differentiated weight coefficients to the time and spatial residuals based on the modal attributes of the current labeled features and the accuracy requirements of the calibration scenario. This adapts to the calibration focus of different modal data; for example, increasing the time weight for time-sensitive acoustic and vibration features, and increasing the spatial weight for spatially sensitive video and infrared features. Let the weight of the time residual be... Spatial residual weights are And it satisfies the weight normalization constraint. Then the formula for calculating the final S-th closed-loop verification residual is: ; In the formula This is the total verification residual that integrates the spatiotemporal deviations, and it is the core numerical basis for determining whether feedback compensation needs to be performed on the annotation results.

[0079] Furthermore, in the method provided in this application embodiment, the privacy-sensitive processing mechanism includes privacy-sensitive feature deletion, privacy-sensitive feature masking, and privacy-sensitive feature isolation.

[0080] Specifically, a tiered desensitization mechanism consisting of three differentiated paths is adopted for privacy-sensitive processing. This mechanism matches different intensities and methods of desensitization processing based on the sensitivity level of features, data modal attributes, and value contribution to subsequent calibration processes. Among them, privacy-sensitive feature deletion is a processing method for sensitive features with the highest risk and lowest calibration value. It refers to the complete removal of highly sensitive features that do not participate in subsequent calibration calculations from the feature dataset, eliminating the risk of leakage of such information from the source of the data. These features typically include pure identity fields such as personnel names and employee numbers in access control data streams, core commercial confidential parameters in equipment operation and maintenance data, and personal attribute information that has no relation to spatiotemporal calibration. Specifically, according to the preset highly sensitive deletion list, the feature data of the corresponding dimension is completely removed from the second collection feature set without retaining any reversible original fields. At the same time, the deleted feature type and processing log are recorded to support subsequent privacy compliance audits.

[0081] Privacy-sensitive feature masking is a processing method for features that have calibration reuse value but pose a risk of identity or precise attribute leakage. It refers to the elimination of individual identifiability and precise sensitive attributes of features through techniques such as pixel replacement, semantic generalization, and numerical coarsening, while retaining core calibration information such as spatiotemporal location, motion trajectory, and numerical change trends. These features are mainly visual and audio-visual modal data, such as faces and license plate information in video images, voiceprint features in audio data, and precise address information at the point level. During the process, for visual data, pixelation masking and general contour replacement are used to remove individual identity identifiers, while fully retaining the target's spatial coordinates, appearance time, motion trajectory, and other calibration-necessary information. For numerical data, interval generalization and encoding masking are used to replace precise sensitive values, while retaining the relative size and temporal relationship between data. The processed features cannot be reversed to restore the original sensitive information, but can participate normally in subsequent spatiotemporal correlation matching and calibration calculations.

[0082] Privacy-sensitive feature isolation is a method for handling highly sensitive features that need to be accessed in controlled ways in special scenarios but are not suitable for inclusion in the regular calibration process. It refers to the process of separating and storing such sensitive features from regular calibration features through logical or physical data splitting and access control. These features typically include original identity data and core calibration baseline parameters of the equipment, which are not needed for routine calibration but need to be retrieved as needed in special accuracy verification scenarios. During execution, these features are separated from the main data stream and stored in an independent high-privilege privacy data domain, which is logically isolated from the main calibration operation link. This data cannot be accessed throughout the entire process, including regular observability analysis, cross-modal calibration, and residual detection. It is only accessed in controlled ways by a dedicated module in special verification scenarios with strict access control and full traceability, thus preventing the spread of sensitive information in the calibration link.

[0083] Furthermore, in the method provided in this application embodiment, determining whether the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual includes: if the Sth closed-loop verification residual is less than the predetermined closed-loop verification residual, adding the Sth calibration labeling result to the calibration labeling enhancement space.

[0084] Specifically, the S-th closed-loop verification residual is a comprehensive deviation value obtained by weighted fusion of deviations in both time and space dimensions. It fully quantifies the degree of spatiotemporal consistency deviation of the current S-th calibration annotation result across multiple device nodes. The larger the value, the more significant the deviation of the annotation benchmark and the lower the annotation accuracy. The predetermined closed-loop verification residual is a qualified threshold pre-formed by combining the accuracy requirements of different business scenarios in the smart factory and the benchmark accuracy level calibration of the calibration anchor point. It serves as the judgment boundary for whether the annotation result meets the subsequent automatic calibration requirements. Its value will be adjusted according to the scenario. For example, a smaller threshold will be set for high-precision scenarios such as security, while the threshold range can be appropriately widened for environmental monitoring scenarios. Specifically, the calculated S-th closed-loop verification residual is precisely compared with the pre-stored predetermined closed-loop verification residual threshold to complete the logical judgment of the size relationship. If the determination shows that the S-th closed-loop verification residual is less than the predetermined closed-loop verification residual threshold, the value will be determined. The residual verification indicates that after cross-validation of the current S-th calibration annotation result through cross-node spatiotemporal transformation, the overall deviation between the unified event time and the unified event location is within an acceptable accuracy range. The cross-modal alignment accuracy and interference annotation accuracy of the annotation result meet the benchmark requirements for subsequent automatic calibration. There are no systematic deviations that need to be corrected, and no additional feedback compensation calculation is required. At this time, all data such as the device, environment, transmission three-layer interference annotation information, unified spatiotemporal benchmark parameters, and associated node mapping relationship attached to the annotation result will be completely retained. The original S-th calibration annotation result will be directly added to the calibration annotation enhancement space. At the same time, the annotation result will be marked with a verification compliance status indicator, and the corresponding residual value and verification timestamp will be recorded synchronously to form a traceable accuracy certificate. The corresponding residual exceeding the standard will enter the feedback compensation branch. The two together constitute a complete closed-loop judgment logic.

[0085] In summary, the multimodal data acquisition and calibration method provided in this application has the following technical effects: By performing multi-granular hierarchical desensitization processing on the raw data stream, the effective information of calibration-related features is fully preserved while ensuring data privacy compliance. Furthermore, a calibration anchor layer is constructed based on the bidirectional screening results of observability and privacy exposure, and a data acquisition calibration relationship graph is established. This enables cross-modal unified calibration labeling of coupling interference in the three layers of equipment, environment, and transmission. Finally, the calibration labeling accuracy is dynamically optimized through closed-loop transform residual detection and feedback compensation mechanisms, completing the automatic calibration of all desensitized acquisition features. This effectively balances data privacy security and calibration accuracy requirements, and improves the spatiotemporal alignment effect and anti-interference calibration capability of multi-source heterogeneous sensing data.

[0086] Example 2, based on the same inventive concept as the multimodal data acquisition and calibration method in the foregoing examples, such as... Figure 2 As shown in the figure, this application provides a multimodal data acquisition and calibration system, which includes: The acquisition module 11 is used to acquire multimodal data from the smart factory through a sensor network, obtain multiple raw data streams, and perform multi-granularity desensitization processing on the multiple raw data streams to generate multiple desensitized acquisition features; the analysis module 12 is used to perform observability and privacy exposure analysis on the multiple desensitized acquisition features, establish a calibration anchor layer, and perform spatiotemporal correlation matching detection on the multiple desensitized acquisition features through the calibration anchor layer to establish an acquisition calibration relationship graph; the annotation module 13 is used to perform cross-modal calibration annotation on the multiple desensitized acquisition features according to the acquisition calibration relationship graph to establish a cross-modal calibration annotation space; the calibration module 14 is used to perform closed-loop transformation residual detection compensation on the cross-modal calibration annotation space to obtain a calibration annotation enhancement space, and automatically calibrate the multiple desensitized acquisition features according to the calibration annotation enhancement space.

[0087] Furthermore, the acquisition module 11 is also used to perform the following steps: perform calibration association feature recognition on the plurality of raw data streams to obtain a first acquisition feature set; perform privacy-sensitive feature recognition on the plurality of raw data streams to obtain a second acquisition feature set; perform irreversible encoding on the first acquisition feature set according to a plurality of preset desensitization granularities, and perform desensitization processing on the second acquisition feature set based on a privacy-sensitive processing mechanism to obtain a plurality of data desensitization processing results; associate each data desensitization processing result with the corresponding device identifier, original timestamp, and desensitization processing parameters to obtain the plurality of desensitized acquisition features.

[0088] Furthermore, the analysis module 12 is also used to perform the following steps: evaluate the observability of the plurality of de-identified collection features and obtain a plurality of observability scores; evaluate the privacy exposure of the plurality of de-identified collection features and obtain a plurality of privacy exposure scores; obtain predetermined screening conditions, the predetermined screening conditions including observability score thresholds and privacy exposure score thresholds; based on the plurality of observability scores and the plurality of privacy exposure scores, screen the plurality of de-identified collection features according to the predetermined screening conditions to obtain a collection feature screening domain; determine the event attribution of the collection feature screening domain to generate the calibration anchor layer.

[0089] Furthermore, the annotation module 13 is also used to perform the following steps: trace device coupling interference of the plurality of desensitized acquisition features to obtain the acquisition device coupling interference layer; trace environmental coupling interference of the plurality of desensitized acquisition features to obtain the acquisition environment coupling interference layer; trace transmission coupling interference of the plurality of desensitized acquisition features to obtain the acquisition transmission coupling interference layer; and perform cross-modal calibration association of the acquisition device coupling interference layer, the acquisition environment coupling interference layer and the acquisition transmission coupling interference layer according to the acquisition calibration relationship diagram to generate the cross-modal calibration annotation space.

[0090] Furthermore, the annotation module 13 is also used to perform the following steps: collecting sensor state parameters corresponding to each desensitized acquisition feature to obtain the state vector of each device; performing anomaly detection on the state vector of each device to obtain the anomaly detection result of each device; performing attention allocation on the state vector of each device according to the anomaly detection result of each device to obtain the feature vector of each device; and performing interference identification and tracing on the multiple desensitized acquisition features based on the feature vector of each device to generate the coupling interference layer of the acquisition device.

[0091] Furthermore, the calibration module 14 is also used to perform the following steps: extracting the Sth calibration label result according to the cross-modal calibration label space, where S is a positive integer; performing closed-loop transformation residual detection on the Sth calibration label result according to the acquired calibration relationship diagram to obtain the Sth closed-loop verification residual; determining whether the Sth closed-loop verification residual is greater than or equal to a predetermined closed-loop verification residual; if the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual, performing feedback compensation on the Sth calibration label result according to the Sth closed-loop verification residual to obtain the Sth calibration label correction result, and adding the Sth calibration label correction result to the calibration label enhancement space.

[0092] Furthermore, the calibration module 14 is also used to perform the following steps: according to the acquisition calibration relationship diagram, perform time transformation on the Sth calibration annotation result between different device nodes to obtain the Sth time transformation result; according to the difference between the Sth time transformation result and the uniform event time in the Sth calibration annotation result, obtain the Sth time transformation residual; according to the acquisition calibration relationship diagram, perform spatial transformation on the Sth calibration annotation result between different device nodes to obtain the Sth spatial transformation result; according to the difference between the Sth spatial transformation result and the uniform event position in the Sth calibration annotation result, determine the Sth spatial transformation residual, and perform weighted fusion on the Sth time transformation residual and the Sth spatial transformation residual to generate the Sth closed-loop verification residual.

[0093] Furthermore, the acquisition module 11 is also used to perform the following steps: the privacy-sensitive processing mechanism includes privacy-sensitive feature deletion, privacy-sensitive feature masking, and privacy-sensitive feature isolation.

[0094] Furthermore, the calibration module 14 is also used to perform the following steps: if the Sth closed-loop verification residual is less than the predetermined closed-loop verification residual, the Sth calibration label result is added to the calibration label enhancement space.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for multimodal data acquisition and calibration, characterized in that, The method includes: The smart factory is equipped with a sensor network to collect multimodal data, acquire multiple raw data streams, and perform multi-granular desensitization processing on the multiple raw data streams to generate multiple desensitized acquisition features. Observability and privacy exposure analysis are performed on the multiple desensitized collection features, a calibration anchor layer is established, and spatiotemporal correlation matching detection is performed on the multiple desensitized collection features through the calibration anchor layer to establish a collection calibration relationship graph; Based on the acquisition calibration relationship diagram, cross-modal calibration annotation is performed on the multiple desensitized acquisition features to establish a cross-modal calibration annotation space; Closed-loop transformation residual detection compensation is performed on the cross-modal calibration annotation space to obtain the calibration annotation enhancement space, and the multiple desensitized acquisition features are automatically calibrated according to the calibration annotation enhancement space; Specifically, observability and privacy exposure analysis are performed on the multiple desensitized collection features, and a calibration anchor layer is established, including: The observability of the multiple desensitized collection features is evaluated to obtain multiple observability scores; The privacy exposure of the multiple de-identified collection features is evaluated to obtain multiple privacy exposure scores; Obtain predetermined filtering conditions, including observability score threshold and privacy exposure score threshold; Based on the multiple observability scores and the multiple privacy exposure scores, the multiple de-identified collection features are filtered according to the predetermined filtering conditions to obtain the collection feature filtering domain; The event attribution of the collected feature filtering domain is determined to generate the calibration anchor point layer.

2. The multimodal data acquisition and calibration method as described in claim 1, characterized in that, The multiple raw data streams are subjected to multi-granularity de-identification processing to generate multiple de-identified acquisition features, including: The multiple raw data streams are subjected to calibration and correlation feature recognition to obtain a first collection feature set; Privacy-sensitive feature identification is performed on the multiple raw data streams to obtain a second collection feature set; The first collection feature set is irreversibly encoded according to multiple preset desensitization granularities, and the second collection feature set is desensitized based on a privacy-sensitive processing mechanism to obtain multiple data desensitization processing results; The data anonymization results are associated with the corresponding device identifier, original timestamp, and anonymization parameters to obtain the multiple anonymized collection features.

3. The multimodal data acquisition and calibration method as described in claim 1, characterized in that, Based on the acquisition calibration relationship diagram, cross-modal calibration annotation is performed on the multiple desensitized acquisition features to establish a cross-modal calibration annotation space, including: Device coupling interference tracing is performed on the multiple desensitized acquisition features to obtain the acquisition device coupling interference layer; Environmental coupling interference tracing is performed on the multiple desensitized acquisition features to obtain the acquisition environment coupling interference layer; The transmission coupling interference is traced for the multiple desensitized acquisition features to obtain the acquisition transmission coupling interference layer; Based on the acquisition calibration relationship diagram, cross-modal calibration association is performed on the acquisition device coupling interference layer, the acquisition environment coupling interference layer, and the acquisition transmission coupling interference layer to generate the cross-modal calibration annotation space.

4. The multimodal data acquisition and calibration method as described in claim 3, characterized in that, Device coupling interference tracing is performed on the multiple desensitized acquisition features to obtain the acquisition device coupling interference layer, including: Collect the sensor state parameters corresponding to each desensitized collection feature to obtain the state vector of each device; Anomaly detection is performed on the state vectors of each device to obtain the anomaly detection results for each device. Based on the anomaly detection results of each device, attention is allocated to the state vector of each device to obtain the feature vector of each device. Based on the feature vectors of each device, interference identification and tracing are performed on the multiple desensitized acquisition features to generate the coupling interference layer of the acquisition device.

5. The multimodal data acquisition and calibration method as described in claim 1, characterized in that, Perform closed-loop transform residual detection compensation on the cross-modal calibration annotation space to obtain the calibration annotation enhancement space, including: The Sth calibration annotation result is extracted from the cross-modal calibration annotation space, where S is a positive integer; Based on the acquired calibration relationship diagram, the Sth calibration labeling result is subjected to closed-loop transformation residual detection to obtain the Sth closed-loop verification residual. Determine whether the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual; If the Sth closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual, the Sth calibration label result is compensated based on the Sth closed-loop verification residual to obtain the Sth calibration label correction result, and the Sth calibration label correction result is added to the calibration label enhancement space.

6. The multimodal data acquisition and calibration method as described in claim 5, characterized in that, Based on the aforementioned acquisition calibration relationship diagram, the Sth calibration annotation result is subjected to closed-loop transformation residual detection to obtain the Sth closed-loop verification residual, including: Based on the acquisition and calibration relationship diagram, time transformation is performed on the Sth calibration labeling result across different device nodes to obtain the Sth time transformation result; Based on the difference between the Sth time transformation result and the uniform event time in the Sth calibration labeling result, obtain the Sth time transformation residual; Based on the acquisition and calibration relationship diagram, perform spatial transformation on the Sth calibration annotation result between different device nodes to obtain the Sth spatial transformation result; Based on the difference between the unified event positions in the Sth spatial transformation result and the Sth calibration annotation result, the Sth spatial transformation residual is determined, and the Sth time transformation residual and the Sth spatial transformation residual are weighted and fused to generate the Sth closed-loop verification residual.

7. The multimodal data acquisition and calibration method as described in claim 2, characterized in that, The privacy-sensitive processing mechanism includes privacy-sensitive feature deletion, privacy-sensitive feature masking, and privacy-sensitive feature isolation.

8. The multimodal data acquisition and calibration method as described in claim 5, characterized in that, Determining whether the S-th closed-loop verification residual is greater than or equal to the predetermined closed-loop verification residual includes: If the Sth closed-loop verification residual is less than the predetermined closed-loop verification residual, the Sth calibration label result is added to the calibration label enhancement space.

9. A multimodal data acquisition and calibration system, characterized in that, The system is used to implement the multimodal data acquisition and calibration method according to any one of claims 1 to 8, wherein the system comprises: The acquisition module is used to acquire multimodal data from the smart factory through a sensor network, obtain multiple raw data streams, and perform multi-granular desensitization processing on the multiple raw data streams to generate multiple desensitized acquisition features. The analysis module is used to perform observability and privacy exposure analysis on the multiple desensitized collection features, establish a calibration anchor layer, and perform spatiotemporal correlation matching detection on the multiple desensitized collection features through the calibration anchor layer to establish a collection calibration relationship graph; The annotation module is used to perform cross-modal calibration annotation on the multiple desensitized acquisition features according to the acquisition calibration relationship diagram, and to establish a cross-modal calibration annotation space; The calibration module is used to perform closed-loop transformation residual detection compensation on the cross-modal calibration annotation space, obtain the calibration annotation enhancement space, and automatically calibrate the multiple desensitized acquisition features according to the calibration annotation enhancement space.

Images