A processing method for multi-modal data acquisition of an intelligent manufacturing system

Abstract

This invention discloses a method for processing multimodal data acquisition in intelligent manufacturing systems, relating to the field of industrial data governance technology. The method includes: acquiring multimodal data streams; segmenting raw data blocks and standardizing and encapsulating them to generate metadata records; establishing time indexes, batch indexes, and process indexes; hashing the metadata records to generate content addresses and writing them to object storage; recalculating the hash and comparing the content addresses to complete consistency verification and writing the results to a commit log table; generating operator records for successfully committed data blocks and hashing them to obtain operator addresses; registering lineage edges to form directed acyclic lineage relationships; generating leaf lists according to a preset range and aggregating them to obtain root digests; verifying content addresses object by object based on the leaf lists and root digests; replaying and recalculating to generate replay-derived data blocks; comparing them to generate audit records and storing them. This invention ensures continuous monitoring of data quality during big data acquisition through the replay and recalculation processing of directed acyclic lineage relationships.

Description

Technical Field

[0001] This invention relates to the field of industrial data governance technology, and in particular to a method for processing multimodal data acquisition in intelligent manufacturing systems. Background Technology

[0002] In current production processes, intelligent manufacturing systems involve the real-time acquisition and processing of data from various sensors and devices. This includes not only traditional mechanical operation data but also multimodal data such as temperature, pressure, vibration, and displacement. Multimodal data acquisition provides more comprehensive and detailed monitoring of the production process through different data types. The acquisition system needs to perform time synchronization and data integration to ensure that the acquired data accurately reflects each stage of the production process. Existing multimodal data acquisition methods mainly rely on timestamp calibration, data segmentation, and standardized encapsulation techniques to maintain data integrity and auditability.

[0003] Traditional methods often rely on static block rules and have relatively simple time-correction processes, making them unable to be flexibly adjusted in complex production environments. In particular, under variable data acquisition conditions, they suffer from error accumulation. Furthermore, existing methods struggle to achieve finer-grained tracking and verification between data sources and processing operations during data block generation and processing. This is especially true in intelligent manufacturing systems with complex data flows and multiple processing stages, where the ability to trace data sources and processes is insufficient, failing to effectively guarantee data integrity and consistency. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a processing method for multimodal data acquisition in intelligent manufacturing systems to solve the problems of error accumulation caused by static block rules and insufficient tracking capabilities of data sources and processing operations in existing technologies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for processing multimodal data acquisition in an intelligent manufacturing system, comprising: Collect multimodal data streams, segment the multimodal data into raw data blocks, standardize and encapsulate the raw data blocks, generate metadata records, and establish time, batch, and process indexes for the metadata records; Hash the metadata records to obtain the content address and write it to the object storage. Recalculate the hash of the data block corresponding to the content address in the object storage and perform consistency verification by comparing the content address to generate a commit log table. When processing actions are performed on successfully submitted data blocks in the commit log table, corresponding operator records are generated, hash calculations are performed on the operator records to obtain the operator addresses, and lineage edges are registered with content addresses as nodes and operator addresses as association identifiers to form directed acyclic lineage relationships. The content addresses corresponding to successfully submitted data blocks are extracted from the commit log table to generate a leaf list, and the root digest is generated by aggregation of the leaf list and appended to the registration. Based on the leaf list and root summary, the system performs retrieval, recalculation and verification, and object-by-object content address verification. According to the directed acyclic lineage, under the constraints of operator records, the system performs replay recalculation to generate replay derived data blocks. The replay derived data blocks are then compared for consistency, and audit records are generated and stored.

[0007] As a preferred embodiment of the multimodal data acquisition and processing method for the intelligent manufacturing system described in this invention, the steps of acquiring the multimodal data stream and dividing the multimodal data into raw data blocks are as follows: By acquiring multimodal data streams and adding acquisition timestamps, reading reference time from time sources, combining reference time and acquisition timestamps, calculating offset observations and offset predictions, generating time correction information records, and combining multimodal data streams to correct acquisition timestamps, the corrected multimodal data is obtained. The corrected multimodal data is segmented to generate original data blocks.

[0008] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps of standardizing and encapsulating the original data blocks to generate metadata records and establishing time, batch, and process indexes for the metadata records are as follows. The original data block is normalized and encapsulated using a fixed set of fields and field order; Based on the header information encapsulated in the original data block, time index, batch index, and process index are established in the metadata storage location by writing the metadata record field set.

[0009] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps of performing hash calculations on metadata records, obtaining content addresses, and writing them to object storage are as follows: Locate the data block corresponding to the metadata record based on the metadata record, use a cryptographic hash function to obtain the content address, write the data block corresponding to the metadata record into the object storage, and generate the object's physical location; Write the content address and the object's physical location into the mapping table to form a mapping relationship between the content address and the object's physical location, and then fill the content address back into the corresponding metadata record.

[0010] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps of performing recalculation hashing on the data block corresponding to the content address in the object storage, and generating a commit log table by comparing the content addresses, are as follows: The physical location of the object is queried by mapping the relationship table, and the data block corresponding to the content address is read from the object storage. The read data block is obtained and a cryptographic hash function is used to generate the recalculated content address. The recalculated content address and the content address are compared byte by byte to obtain the output consistency check status flag, which is then combined with the content address to generate the commit log table.

[0011] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps are as follows: when processing actions are performed on successfully submitted data blocks in the submission log table, a corresponding operator record is generated, and a hash calculation is performed on the operator record to obtain the operator address. Filter the list of successfully submitted content addresses and the set of successfully submitted data blocks from the commit log table; Execute processing actions on the set of successfully submitted data blocks, obtain the output data of the processing actions, and combine it with the list of successfully submitted content addresses to write it into the operator record according to a fixed set of fields and field order. Operator addresses are generated based on operator records using cryptographic hash functions.

[0012] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the step of forming a directed acyclic lineage relationship by registering lineage edges with content addresses as nodes and operator addresses as association identifiers is as follows: The output data of the processing action is normalized and encapsulated to generate a derived normalized encapsulated data block, and a cryptographic hash function is used to generate the address of the derived content. By writing the derived normalized encapsulated data block and the derived content address into the object storage, the physical location of the derived object is obtained, and the derived content address and the physical location of the derived object are written into the mapping table. Register lineage edges using content addresses as nodes and operator addresses as association identifiers, obtain lineage edge records, and confirm the directed acyclic lineage relationship status of lineage edge records by performing ancestor content address set queries and inclusion relationship checks on the lineage edge records.

[0013] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps of extracting the content addresses corresponding to successfully submitted data blocks from the submission log table, generating a leaf list, aggregating the leaf list to generate a root digest, and appending the records are as follows: Filter the commit log table for records whose consistency check status is marked as successful during consistency check, sort them, generate an ordered content address sequence and encode it, and generate a leaf list. Based on the leaf list, perform hash aggregation on the ordered content address sequence to obtain the root digest, determine the root digest record writing position by the timestamp, and append the root digest and leaf list to the root digest record writing position.

[0014] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps of performing retrieval, recalculation verification, and object-by-object content address verification based on the leaf list and root summary are as follows: Based on the root digest record write location and root digest, locate the leaf list location identifier associated with the root digest, and extract the leaf list content address sequence from the leaf list; Perform hash aggregation on the address sequence of the leaf list content layer by layer to obtain the recalculated root digest and compare it byte by byte with the root digest to generate a recalculation verification status flag; Based on the leaf list content address sequence, the physical location of the object is located through the mapping relationship table for each content address, and the data block corresponding to the content address is read from the object storage to generate a read data block. The read data block is then subjected to recalculation hashing to obtain the recalculated content address and is compared byte by byte with the content address to generate a set of object-by-object verification records.

[0015] As a preferred embodiment of the multimodal data acquisition processing method for the intelligent manufacturing system described in this invention, the steps are as follows: Based on the directed acyclic lineage relationship, under operator record constraints, playback recalculation processing is performed to generate playback derived data blocks; consistency comparison is performed on the playback derived data blocks; audit records are generated and stored. Combining directed acyclic lineage and content address, query the parent content address list and operator address, perform recalculation hashing on operator record, generate operator record verification status flag, execute processing actions according to operator record constraints, generate replay derived data block and generate replay derived content address through hash calculation; Perform a byte-by-byte comparison of the content addresses of the replay derived content address and the leaf list content address sequence to generate a replay consistency comparison status flag. Combine the replay consistency comparison status flag, the replay derived content address, the operator address, and the parent content address list to generate an audit record and write it to the audit record storage location.

[0016] The beneficial effects of this invention are as follows: by retrieving and recalculating based on leaf lists and root digests, the consistency of data blocks is accurately located, and the integrity and consistency of data are achieved during large-scale data collection; by replaying and recalculating the directed acyclic lineage relationship, the traceability of the system and the transparency of data processing are enhanced, and continuous monitoring of data quality is achieved during big data collection. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a multimodal data acquisition and processing method for intelligent manufacturing systems.

[0019] Figure 2 This is a flowchart of the time-sharing and segmentation of the data stream.

[0020] Figure 3 A flowchart for hash calculation and consistency verification of metadata records.

[0021] Figure 4 This is a flowchart for blood relations and replay recalculation. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for processing multimodal data acquisition in an intelligent manufacturing system, comprising the following steps: S1. Collect multimodal data streams, divide the multimodal data into raw data blocks, standardize and encapsulate the raw data blocks, generate metadata records, and establish time, batch, and process indexes for the metadata records.

[0026] By acquiring multimodal data streams and adding acquisition timestamps, the reference time is read from the time source. By combining the reference time and the acquisition timestamps, the offset observation value and the offset prediction value are calculated, and time correction information records are generated. By combining the multimodal data streams, the acquisition timestamps are corrected to obtain the corrected multimodal data.

[0027] Furthermore, acquisition channel registration information is established for each channel in the multimodal data stream. Data is read from each channel through acquisition configuration reference. Acquisition timestamps are added to the data frames and messages returned for each data read. This is achieved by reading the current count value of the local timer and writing the acquisition timestamp into the header field of the data frame. The multimodal data stream maintains a continuous sequence structure with acquisition timestamps at the channel level. The acquisition timestamps and channel identifiers are written into the acquisition process record. Time source docking processing is performed before acquiring the multimodal data stream. A session is established with the time source through the time synchronization interface. When the time source reads the reference time, the reference time is received periodically according to the message format defined by the time synchronization interface. The reference time is written into the reference time buffer, which records the reference time and the corresponding reception time marker.

[0028] The data acquisition channel registration information includes the channel identifier and data acquisition configuration reference; the time synchronization interface is the network time synchronization interface.

[0029] It should be noted that data reading is accomplished using both the driver-level acquisition interface and the bus acquisition interface.

[0030] The driver layer acquisition interface is used to return data frames from sensors or acquisition cards, while the bus acquisition interface is used to return messages and event records from the control bus.

[0031] It should be noted that the output of the time synchronization interface is a reference time.

[0032] It should be noted that the receiving timestamp is generated by the local timer at the acquisition end and is used to establish the correspondence between the reference time and the acquisition timestamp.

[0033] It should be noted that the multimodal data stream includes visual data, acoustic data, vibration data, electrical parameter data, torque data, temperature data, displacement data, and control logs.

[0034] Among them, the data read and return data frames corresponding to visual data, acoustic data, vibration data, electrical parameter data, torque data, temperature data, and displacement data, and the data read and return messages and event records corresponding to control logs.

[0035] Furthermore, based on the reference time and the acquisition timestamp, sample alignment processing is performed to obtain an aligned sample set. When calculating the offset observations, each aligned sample set is processed item by item to calculate the difference between the reference time and the acquisition timestamp. The difference between the reference time and the acquisition timestamp is expressed in a unified time unit and written into the offset observation sequence. Outlier removal processing is performed during the formation of the offset observation sequence. Based on the offset observation sequence, the offset prediction value is calculated using the sliding window truncated mean method. The offset observation sequence and the offset prediction value are organized according to a fixed set of fields and the field order to generate a time correction information record.

[0036] It should be noted that the sample alignment process specifically involves selecting a reference time record in the reference time buffer that is adjacent to the receiving time marker of the acquisition timestamp, and pairing the receiving time marker of the reference time record with the acquisition timestamp to obtain an aligned sample.

[0037] It should be noted that when there are several channels in a multimodal data stream, the sample alignment process uses the channel identifier as the grouping key to form an aligned sample set.

[0038] It should be noted that the sliding window truncated mean method specifically involves: based on the offset observation sequence, determining the window capacity and the number of offset observations to be removed at both ends according to the acquisition configuration reference; maintaining the window offset observation set according to the generation order of the offset observation sequence during the formation of the offset observation sequence; ensuring that the window offset observation set contains only the offset observations with the closest window capacity at any given time; adding new offset observations to the offset observation sequence, adding the new offset observations to the window offset observation set, and removing the earliest added offset observation from the window offset observation set to maintain a constant window capacity; sorting the window offset observation set to obtain a sorted window offset observation set; removing the smallest number of offset observations to be removed at both ends and the largest number of offset observations to be removed at both ends from the sorted window offset observation set to obtain the truncated window offset observation set; and calculating the mean of the truncated window offset observation set to generate the offset prediction value.

[0039] The content carriers of the time correction information record include channel identifier, reference time summary, acquisition timestamp summary, offset observation summary, and offset prediction value; the reference time summary and acquisition timestamp summary are used to identify the time range covered by the aligned sample set, and the offset observation summary is used to identify the statistical characteristics of the offset observation sequence.

[0040] It should be noted that the window size and the number of rejections at both ends remain unchanged during a single data collection task.

[0041] Furthermore, based on the multimodal data stream and time correction information records, the acquisition timestamp correction process is performed on the multimodal data stream to obtain the corrected multimodal data.

[0042] It should be noted that the acquisition timestamp correction process specifically involves performing a correction operation on the acquisition timestamp of each data frame and message in the multimodal data stream, using the offset prediction value as the correction amount. The acquisition timestamp is shifted according to the offset prediction value to form a corrected timestamp field. The corrected timestamp field is written into the data frame header field and the message extension field, while the original acquisition timestamp value is retained in the history field to generate corrected multimodal data.

[0043] The corrected multimodal data retains the channel identifier and the data frame or message content unchanged, and only the acquisition timestamp field is corrected, replaced, and appended.

[0044] The corrected multimodal data is segmented to generate original data blocks.

[0045] Furthermore, based on the acquisition configuration reference, the preset segmentation rules are determined, and the corrected multimodal data is sequentially scanned using the channel identifier as the grouping key. The segmentation boundaries are identified according to the monotonically increasing order of the corrected timestamp field, and the original data blocks are output.

[0046] It should be noted that the preset segmentation rules are determined based on the acquisition configuration reference. Specifically, the segmentation rule field set is read from the acquisition configuration reference. After grouping the corrected multimodal data by channel identifier, the starting position of segmentation processing is determined for each group based on the segmentation trigger condition field. Starting from the starting position of segmentation processing, the corrected timestamp field is sequentially traversed. Based on the segmentation boundary judgment field, candidate segmentation points are registered for the corrected timestamp field that meets the interval judgment rule and the cumulative coverage judgment rule, forming a candidate segmentation point set. Segmentation boundary alignment processing is performed on the candidate segmentation point set to obtain the segmentation boundary. The segmentation boundary alignment processing is determined based on the segmentation boundary alignment field, which is used to limit the alignment method and alignment priority of the boundary and the corrected timestamp field. After the segmentation boundary alignment processing is completed, the corrected multimodal data is segmented according to the segmentation boundary and the original data blocks are output.

[0047] The segmentation rule field set includes a segmentation trigger condition field, a segmentation boundary determination field, and a segmentation boundary alignment field. The segmentation trigger condition field is used to limit the starting conditions for the corrected multimodal data to enter the segmentation process. The segmentation boundary determination field is used to limit the judgment criteria for the segmentation boundary. The segmentation boundary alignment field is used to limit the alignment criteria for the segmentation boundary. The interval determination rule is used to register candidate segmentation points when the interval between adjacent corrected timestamp fields meets the interval condition. The interval condition is based on the time difference between the corrected timestamp fields and is determined by the data acquisition frequency. For example, when one data point is collected every second, the preset interval condition is defined as a time difference of 1 second. The cumulative coverage determination rule is used to register candidate segmentation points when the cumulative coverage of the corrected timestamp fields meets the preset coverage condition. The segmentation boundary alignment field limits the alignment method between the segmentation boundary and the corrected timestamp field, including forward alignment and backward alignment. The segmentation boundary alignment field limits the alignment priority between the segmentation boundary and the corrected timestamp field and is used to select the final segmentation boundary when there are several candidate segmentation points.

[0048] The original data block is normalized and encapsulated with a fixed set of fields and field order.

[0049] Furthermore, after generating the original data blocks, the original data blocks are standardized and encapsulated with a fixed set of fields and field order to obtain the data blocks corresponding to the metadata records.

[0050] The fixed field set includes the encapsulation header information field set and the encapsulation body information field set. The encapsulation header information field set contains the channel identifier, the original data block start correction timestamp, and the original data block end correction timestamp. The encapsulation body information field set contains the continuous data frames and messages carried by the original data block.

[0051] It should be noted that the original data block is standardized and encapsulated with a fixed set of fields and field order. Specifically, the fixed set of fields and field order are read according to the acquisition configuration reference, and field writing processing is performed on the original data block. When writing the encapsulation header information field set, the channel identifier, data block start correction timestamp, and data block end correction timestamp are written in sequence, and the field integrity is verified on the writing result of the encapsulation header information field set. When writing the encapsulation body information field set, the continuous data frames and messages carried by the original data block are written into the encapsulation body information field set in the field order, and the boundary consistency is verified on the writing result of the encapsulation body information field set to generate the data block corresponding to the metadata record.

[0052] Among them, the field integrity check is used to determine that the encapsulation header information field set contains all the fields defined by the fixed field set and that the field order of the written result is consistent with the field order; the boundary consistency check is used to determine that the time range of continuous data frames and messages in the encapsulation body information field set is consistent with the data block start correction timestamp and data block end correction timestamp.

[0053] It should be noted that the field writing process includes writing the encapsulation header information field set in the field order and writing the encapsulation body information field set in the field order.

[0054] Based on the header information encapsulated in the original data block, time index, batch index, and process index are established in the metadata storage location by writing the metadata record field set.

[0055] Furthermore, based on the encapsulated header information in the data block corresponding to the metadata record, metadata record generation processing is performed. The encapsulated header information field set is parsed from the data block corresponding to the metadata record, and the channel identifier, data block start correction timestamp, and data block end correction timestamp are read from the encapsulated header information field set. According to the acquisition configuration reference, the metadata record field set is read, and the channel identifier, data block start correction timestamp, and data block end correction timestamp are written into the metadata record field set according to the field order of the metadata record field set. The batch identifier and process identifier are also written into the metadata record field set to obtain the metadata record. The metadata record is written to the metadata storage location. A time index is created in the metadata storage location using the data block start correction timestamp and data block end correction timestamp as keys, a batch index is created in the metadata storage location using the batch identifier as a key, and a process index is created in the metadata storage location using the process identifier as a key.

[0056] It should be noted that the metadata record field set includes channel identifier, data block start correction timestamp, data block end correction timestamp, time correction information record location identifier, calibration version information location identifier, batch identifier, and process identifier.

[0057] Among them, the batch identifier and process identifier are read and written to the metadata record field set by referencing the acquisition configuration.

[0058] S2. Perform hash calculation on the metadata records, obtain the content address and write it to the object storage, perform re-hash processing on the data block corresponding to the content address in the object storage, and perform consistency verification by comparing the content address to generate a commit log table.

[0059] Locate the data block corresponding to the metadata record based on the metadata record, use a cryptographic hash function to obtain the content address, write the data block corresponding to the metadata record into object storage, and generate the object's physical location.

[0060] Furthermore, metadata records are read from the metadata storage location, and the channel identifier, data block start correction timestamp, and data block end correction timestamp are parsed from the metadata records. Based on the channel identifier, data block start correction timestamp, and data block end correction timestamp, matching and locating are performed in the write queue of the data block corresponding to the metadata record to obtain the data block corresponding to the target metadata record. Cryptographic hashing is performed on the data block corresponding to the target metadata record, and a hash digest is generated based on the normalized encapsulated byte sequence of the data block corresponding to the target metadata record. The hash digest is used as the content address. After the content address is generated, the data block corresponding to the target metadata record is written to the object storage. The object storage allocates an object physical location for the data block corresponding to the target metadata record during the write process.

[0061] Specifically, the matching and positioning process in the write queue of the data block corresponding to the metadata record is used to retrieve the data block corresponding to the metadata record in the write queue that has the same channel identifier in the encapsulation header information field set and the same data block start correction timestamp and data block end correction timestamp in the encapsulation header information field set; the physical location of the object includes the object storage bucket identifier and the object key value, which are used to accurately locate the stored data block in the object storage.

[0062] It should be noted that the cryptographic hashing process performed on the data block corresponding to the target metadata record involves: extracting the normalized encapsulated byte sequence of the data block corresponding to the target metadata record as the byte sequence to be hashed; determining the type of cryptographic hash function based on the collection configuration reference, such as SHA-256, SHA-1, and MD5; performing hash calculation by calling the cryptographic hash function on the byte sequence to be hashed to generate a hash digest; and performing encoding processing on the hash digest according to encoding rules, such as hexadecimal encoding rules, to obtain the content address.

[0063] Write the content address and the object's physical location into the mapping table to form a mapping relationship between the content address and the object's physical location, and then fill the content address back into the corresponding metadata record.

[0064] Furthermore, by obtaining the content address and object physical location from the target metadata record, the content address and object physical location are written as key-value pairs into the mapping table, forming a mapping relationship between the content address and the object physical location; by obtaining the channel identifier, data block start correction timestamp, and data block end correction timestamp from the target metadata record, the content address and the field set of the metadata record are connected, and the content address is written into the corresponding metadata record field.

[0065] The mapping table includes mapping data between content addresses and object physical locations; this mapping data represents the relationship between storage locations, ensuring that each content address corresponds to a unique object physical location.

[0066] The physical location of the object is queried by mapping the relationship table, and the data block corresponding to the content address is read from the object storage. The read data block is obtained and a cryptographic hash function is used to generate the recalculated content address.

[0067] Furthermore, the physical location of the corresponding object is found in the mapping table based on the content address, and the data block location corresponding to the physical location of the object is obtained; the data block storage location is located through the physical location of the object, the data block content is read and the read data block is obtained; a cryptographic hash function is used to perform hash calculation on the byte sequence of the read data block to generate a recalculated content address.

[0068] It should be noted that the cryptographic hash function is determined based on the collection configuration reference, and the purpose of hash calculation is to ensure the integrity and consistency of data blocks during the storage process.

[0069] The recalculated content address and the content address are compared byte by byte to obtain the output consistency check status flag, which is then combined with the content address to generate the commit log table.

[0070] Furthermore, the recalculated content address and the stored content address are compared byte by byte. When the bytes meet the requirement of a complete match, the comparison is consistent, and a status flag indicating that the consistency check has passed is generated. When the bytes do not meet the requirement of a complete match, the comparison is inconsistent, and a status flag indicating that the consistency check has failed is generated. The status flags indicating that the consistency check has passed and the status flags indicating that the consistency check has failed are counted to generate a consistency comparison result. Based on the consistency comparison result, combined with the original content address and the recalculated content address, a consistency check result is generated. The consistency check result and the relevant content address information are written to the commit log table.

[0071] The consistency verification result includes the verification status and verification information. The verification status is a status mark indicating whether the consistency verification has passed or failed. The verification information includes the original content address, the recalculated content address, the verification timestamp, and related identification information. When the comparison is inconsistent, the verification information also includes the position of the inconsistent bytes. The related identification information of the verification information is used for log tracking.

[0072] S3. When processing actions are performed on data blocks that have been successfully submitted in the submission log table, corresponding operator records are generated, hash calculations are performed on the operator records to obtain the operator addresses, and lineage edges are registered with content addresses as nodes and operator addresses as association identifiers to form directed acyclic lineage relationships. The content addresses corresponding to the successfully submitted data blocks are extracted from the submission log table, a leaf list is generated, and the root digest is generated by aggregation of the leaf list and appended to the registration.

[0073] Filter the list of successfully submitted content addresses and the set of successfully submitted data blocks from the commit log table.

[0074] Furthermore, retrieve all successfully committed data block records from the commit log table; filter the successfully committed data block records by checking the consistency check status to include records containing the content address of each data block and the metadata of the related data blocks; extract the content address list from the filtered metadata records and arrange them in order of content address; extract the corresponding set of successfully committed data blocks based on the content address.

[0075] It should be noted that when performing the filtering operation, the filtering conditions are limited according to the preset range; the extracted data block set includes all field information of each data block; by accessing the content address field in the submission log table, the corresponding content address list is generated, forming a set of successfully submitted data blocks.

[0076] Each data block contains information in all fields, including timestamps, correction information, and the actual content of the data block.

[0077] It should be noted that the filtering criteria based on the preset range include filtering based on time range, data type, and verification status. Specifically, it limits the time range of successfully submitted data block records to ensure that only successfully submitted data block records are filtered from the specified time period. For example, the time range can be limited by the correction timestamp and the submission timestamp. It filters data block records that meet specific types based on the data block type and identifier. For example, it selects records of specific sensors, measurement types, and data sources. It extracts only records that have passed the consistency verification to ensure that the filtered data blocks are successfully verified and have their integrity verified.

[0078] Execute processing actions on the set of successfully submitted data blocks, obtain the output data of the processing actions, and combine it with the list of successfully submitted content addresses to write it into the operator record according to a fixed set of fields and field order.

[0079] Furthermore, each data block is extracted one by one from the set of successfully submitted data blocks, and the corresponding content address is obtained by matching the content address from the list of successfully submitted content addresses based on the content address of each data block; for each data block, the corresponding processing action is performed according to the specific content to obtain the processed output data; the processed output data is combined with the corresponding content address in the list of successfully submitted content addresses, and written into each field of the operator record in the order of the fixed field set and field order to obtain the operator record.

[0080] It should be noted that the operator record includes the operator name, operator implementation identifier, parameter set, input content address list, input time range, output structure summary, and runtime environment fingerprint; the operator name identifies the name of the processing action, such as decoding; the operator implementation identifier is a unique identifier that identifies the operator implementation and is used to track the operator version and source; the parameter set is the set of parameters required for operator processing, such as sampling rate and filter parameters; the input content address list is the content address of the operator input data block and is used to track the source of the input data; the input time range indicates the time interval of the input data block and is used for time series data processing; the output structure summary records the summary information of the operator output results, such as statistical characteristics and structured data; the runtime environment fingerprint identifies the environment information when the operator is executed, such as hardware and software versions, and is used for environment traceability.

[0081] It should be noted that the processing actions include, but are not limited to, decoding, resampling, filtering, denoising, alignment, window truncation, compression, statistical summary generation, feature generation, and sample construction.

[0082] It should be noted that the corresponding processing actions are performed according to the specific content. For example, when the data source is sensor data, the processing actions are denoising and resampling; when the data source is image data, the processing actions are featureization and compression operations.

[0083] It should be noted that when performing data writing, the content address of the operator record is filled with the content address of the corresponding data block, ensuring a unique correspondence between the output of each processing action and the data block.

[0084] Operator addresses are generated based on operator records using cryptographic hash functions.

[0085] Furthermore, all relevant fields are extracted from the operator record, arranged word by word in a fixed order, and the value of each field is converted into byte form to form a sequence of bytes to be hashed. A cryptographic hash function, such as SHA-256, is then used to perform hash calculations on the sequence of bytes to be hashed, generating a fixed-length hash digest. This fixed-length hash digest is then used as the operator address. The operator address is a unique identifier for the current operator, used to identify the content address of the operator record. All processing operations related to the operator are tracked and associated using the current operator address.

[0086] The output data of the processing action is normalized and encapsulated to generate a derived normalized encapsulated data block, and a cryptographic hash function is used to generate the address of the derived content.

[0087] Furthermore, the processed output data is extracted from the operator record. Based on a fixed set of fields and field order, the output data is normalized and encapsulated into a unified byte stream format to generate a derived normalized encapsulated data block. The generated derived normalized encapsulated data block is converted into a byte sequence. The cryptographic hash function type is determined based on the collection configuration reference, such as SHA-256. The cryptographic hash function is used to perform hash calculation on the byte sequence to generate a hash digest. The hash digest is used as the address of the derived content.

[0088] The generated derived normalized encapsulated data block includes the processed output data and related metadata; the derived content address is the unique identifier of the current derived data block.

[0089] It should be noted that the derived content address corresponds to the original content address and is used to identify new data blocks generated during processing.

[0090] By writing the derived normalized encapsulated data block and the address of the derived content into the object storage, the physical location of the derived object is obtained, and the address of the derived content and the physical location of the derived object are written into the mapping table.

[0091] Furthermore, the processed output data and metadata are extracted from the derived normalized encapsulated data blocks. Based on a fixed set of fields and field order, the derived normalized encapsulated data blocks are written to object storage. The physical location of the derived object is allocated to each derived data block through object storage, and the content of the derived data block and the address of the derived content are written to object storage together. The address of the derived content and the corresponding physical location of the derived object are written to a mapping table.

[0092] It should be noted that the physical location of a derived object includes the object bucket identifier and the object key value; the mapping table records the mapping relationship between the address of the derived content and the physical location of the derived object, ensuring the accurate location and access of the derived data.

[0093] Register lineage edges using content addresses as nodes and operator addresses as association identifiers, obtain lineage edge records, and confirm the directed acyclic lineage relationship status of lineage edge records by performing ancestor content address set queries and inclusion relationship checks on the lineage edge records.

[0094] Furthermore, based on the content address and operator address, the lineage relationship registration table is used with the content address as the node and the operator address as the association identifier to perform the lineage edge registration operation and generate lineage edge records. When registering lineage edges, the records are written to the lineage relationship database to save the mapping relationship between the content address and the operator address. By performing an ancestor content address set query on the lineage edge records, all ancestor content addresses of the current content address are identified from the registered lineage edges. An inclusion relationship check is performed on all ancestor content addresses. When the inclusion relationship check passes, the directed acyclic lineage relationship status of the lineage edge record is confirmed to be valid.

[0095] It should be noted that the ancestor content address refers to the hierarchical relationship of content addresses in the lineage registry, which traces the origin of a data block. Specifically, the ancestor content address represents the content address of the upstream data block of the current data block. The upstream data block has undergone several processing steps before generating the current data block. Each data block is associated with a content address and a specific operator address. The ancestor content address is the content address of the data block at the next higher level that directly and indirectly generated the current data block from the parent data block of the current data block.

[0096] It should be noted that the registration operation of bloodline edges is specifically as follows: in the bloodline relationship registration table, for each data block, a new record is generated with the content address as the node and the operator address as the association identifier, which saves the dependency relationship between the content address and the operator address and clearly identifies the dependency between the data block and the operator.

[0097] It should be noted that performing an ancestor content address set query on the lineage edge record specifically involves retrieving all ancestor content addresses related to the current content address from the registered lineage edge records, tracing the source of the data block through the ancestor content address set, and confirming the generation path of the data block.

[0098] It should be noted that the inclusion relationship check is performed on the ancestor content address. Specifically, if the current content address is in the set of ancestor content addresses, the inclusion relationship check is confirmed to fail; if the current content address is not in the set of ancestor content addresses, the inclusion relationship check is confirmed to pass.

[0099] It should be noted that the ancestor content address set is obtained by statistically analyzing all ancestor content addresses.

[0100] The commit log table is filtered to select records whose consistency verification status is marked as successful, sorted, and an ordered content address sequence is generated and encoded to generate a leaf list.

[0101] Furthermore, from the commit log table, all records marked as "consistency verification passed" are filtered out according to a preset time range, and sorted according to the lexicographical order of the content addresses to form an ordered content address sequence; the content addresses in the ordered content address sequence are encoded using Base64 encoding; and a leaf list is generated by assembling the encoded content address sequences into a new list.

[0102] It should be noted that the root summary record writing position is determined by timestamps. Specifically, the timestamps of successfully committed data blocks are extracted from the commit log table, and the writing position of the root summary record is determined according to the start and end range of the timestamps, ensuring that each generation and update of the root summary can accurately match the content address of the relevant data block and the leaf list record.

[0103] The start range of the timestamp indicates the starting time of data collection and recording, while the end range of the timestamp indicates the ending time of data collection and recording.

[0104] Based on the leaf list, perform hash aggregation on the ordered content address sequence to obtain the root digest, determine the root digest record writing position by the timestamp, and append the root digest and leaf list to the root digest record writing position.

[0105] Furthermore, records that have passed consistency verification are selected from the commit log table and sorted lexicographically by content address to generate an ordered content address sequence. Based on the ordered content address sequence, a hash algorithm, such as SHA-256, is applied to each content address to generate a hash value. The hash values ​​are then aggregated using the XOR aggregation method to merge all hash values ​​and generate a root digest. The root digest is written to a specified position in the root digest record. According to a preset time range, the root digest, leaf list, and preset time range are appended to the root digest record.

[0106] It should be noted that after the root digest is generated, the root digest serves as a unique identifier representing the overall information of the ordered content address sequence.

[0107] S4. Based on the leaf list and root summary, perform retrieval, recalculation verification, and object-by-object content address verification. According to the directed acyclic lineage, perform replay recalculation under operator record constraints to generate replay derived data blocks. Perform consistency comparison on the replay derived data blocks, generate audit records, and store and register them.

[0108] Based on the root digest record write location and root digest, locate the leaf list location identifier associated with the root digest, and extract the leaf list content address sequence from the leaf list.

[0109] Furthermore, starting from the root digest record write position, the leaf list identifier associated with the current root digest is located based on the root digest information; the corresponding leaf list record is located by querying the metadata management structure in the database; the content addresses of all data blocks that have passed the consistency check in the leaf list are extracted, organized in lexicographical order, and the content address sequence is extracted from the leaf list.

[0110] It should be noted that the root summary record is written to the timestamp of the successfully committed data block extracted from the commit log table, ensuring that each generation and update of the root summary matches the content address of the relevant data block and the leaf list record.

[0111] Perform hash aggregation layer by layer on the address sequence of the leaf list content, obtain the recalculated root digest, compare it byte by byte with the root digest, and generate a recalculation verification status flag.

[0112] Furthermore, an ordered sequence of content addresses is extracted from the leaf list, and a hash aggregation operation is performed on each content address in sequence. Based on the content addresses in the leaf list, each content address is hashed using a specified hash algorithm, such as SHA-256, to generate a hash value. All generated hash values ​​are aggregated to form a recalculated hash result, which is used as the recalculated root digest. The recalculated root digest and the original root digest are compared byte by byte. When the bytes of the recalculated root digest and the original root digest are completely identical, the comparison is considered successful, and a consistency check status is generated and marked as successful. When the bytes of the recalculated root digest and the original root digest are inconsistent, a consistency check status is generated and marked as failed.

[0113] It should be noted that the original root digest refers to the root digest generated during the root digest generation process based on the same sequence of leaf list content addresses.

[0114] It should be noted that the hash algorithm and aggregation method used in the comparison process are consistent with those used in the generation of the root digest.

[0115] Based on the leaf list content address sequence, the physical location of the object is located through the mapping relationship table for each content address, and the data block corresponding to the content address is read from the object storage to generate a read data block. The read data block is then subjected to recalculation hashing to obtain the recalculated content address and is compared byte by byte with the content address to generate a set of object-by-object verification records.

[0116] Furthermore, the content address sequence is extracted from the leaf list, and each content address is processed sequentially. For each content address, the corresponding object's physical location is queried by accessing the mapping table. Based on the object's physical location, the corresponding data block is read from the object storage. A recalculation hash is performed on the read data block, hashing the byte sequence to generate a recalculated content address. The recalculated content address and the original content address are compared byte-by-byte. When the recalculated content address and the original content address are completely identical, the current data block's consistency verification is considered successful, and a consistency verification record status of "consistency successful" is generated. When inconsistent bytes exist, the current data block's consistency verification is considered unsuccessful, and a consistency verification record status of "consistency unsuccessful" is generated. The consistency verification record statuses are statistically analyzed, and the comparison results for each content address are obtained to generate a set of object-by-object verification records.

[0117] It should be noted that the original content address refers to the content address generated by cryptographic hash calculation when the data is initially stored.

[0118] It should be noted that the re-hash processing of the read data blocks uses the same hash algorithm as when the data was initially stored, such as SHA-256.

[0119] Combining directed acyclic lineage and content address, query the parent content address list and operator address, perform recalculation hashing on the operator record, generate operator record verification status flag, execute processing actions according to operator record constraints, generate replay derived data block and generate replay derived content address through hash calculation.

[0120] Furthermore, based on the current content address, the parent content address list and operator address related to the current content address are queried through the mapping information in the lineage table. For each operator record related to the current content address, a recalculation hash is performed on the operator record. During the recalculation hash calculation, an operator record verification status flag is generated. According to the constraints in the operator record, the corresponding processing actions are performed to generate a replay derived data block. The byte sequence of the replay derived data block is hashed to generate the replay derived content address.

[0121] The constraints in the operator record include the operator operation type and input / output requirements.

[0122] It should be noted that performing re-hashing on the operator record specifically involves extracting information from the operator record, hashing the byte sequence of each field in the operator record according to a specified cryptographic hash function, such as SHA-256, generating a hash digest of the operator record, and using the hash digest as the operator address. It should be noted that the parent content address list refers to the content addresses of all parent data blocks related to the current data block. The parent content address list is used to trace the source of data generation.

[0123] It should be noted that the operator record verification status flag is determined by verifying that the hash value of the operator record matches the stored operator hash value. When the hash value calculated by the operator record matches the stored hash value, it is marked as the operator record consistency verification passed. When the hash value calculated by the operator record does not match the stored hash value, it is marked as the operator record consistency verification failed.

[0124] Perform a byte-by-byte comparison of the content addresses of the replay derived content address and the leaf list content address sequence to generate a replay consistency comparison status flag. Combine the replay consistency comparison status flag, the replay derived content address, the operator address, and the parent content address list to generate an audit record and write it to the audit record storage location.

[0125] Furthermore, the replay derived content address and the original content address are compared byte by byte. When the replay derived content address and the original content address are the same, it is marked as replay derived consistency passed. When the replay derived content address and the original content address are different, it is marked as replay derived consistency failed. Based on the replay consistency comparison status flag, combined with the replay derived content address, operator address and parent content address list, an audit record is generated and written to the audit record storage location.

[0126] It should be noted that the audit log includes the replay consistency comparison status flag, the replay derived content address, the operator address, and the list of parent content addresses.

[0127] In summary, this invention accurately locates the consistency of data blocks through retrieval and recalculation verification based on leaf lists and root digests, thus achieving data integrity and consistency during large-scale data collection. Furthermore, by replaying and recalculating directed acyclic lineage relationships, it enhances the traceability of the system and the transparency of data processing, enabling continuous monitoring of data quality during big data collection.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for processing multimodal data acquisition in an intelligent manufacturing system, characterized in that: include, Collect multimodal data streams, segment the multimodal data into raw data blocks, standardize and encapsulate the raw data blocks, generate metadata records, and establish time, batch, and process indexes for the metadata records; Hash the metadata records to obtain the content address and write it to the object storage. Recalculate the hash of the data block corresponding to the content address in the object storage and perform consistency verification by comparing the content address to generate a commit log table. When processing actions are performed on successfully submitted data blocks in the commit log table, corresponding operator records are generated, hash calculations are performed on the operator records to obtain the operator addresses, and lineage edges are registered with content addresses as nodes and operator addresses as association identifiers to form directed acyclic lineage relationships. The content addresses corresponding to successfully submitted data blocks are extracted from the commit log table to generate a leaf list, and the root digest is generated by aggregation of the leaf list and appended to the registration. Based on the leaf list and root summary, the system performs retrieval, recalculation and verification, and object-by-object content address verification. According to the directed acyclic lineage, under the constraints of operator records, the system performs replay recalculation to generate replay derived data blocks. The replay derived data blocks are then compared for consistency, and audit records are generated and stored.

2. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 1, characterized in that: The process of acquiring multimodal data streams and segmenting the multimodal data into raw data blocks is as follows. By acquiring multimodal data streams and adding acquisition timestamps, reading reference time from time sources, combining reference time and acquisition timestamps, calculating offset observations and offset predictions, generating time correction information records, and combining multimodal data streams to correct acquisition timestamps, the corrected multimodal data is obtained. The corrected multimodal data is segmented to generate original data blocks.

3. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 2, characterized in that: The steps for normalizing and encapsulating the original data blocks, generating metadata records, and establishing time, batch, and process indexes for these metadata records are as follows. The original data block is normalized and encapsulated using a fixed set of fields and field order; Based on the header information encapsulated in the original data block, time index, batch index, and process index are established in the metadata storage location by writing the metadata record field set.

4. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 3, characterized in that: The steps for performing hash calculations on metadata records, obtaining content addresses, and writing them to object storage are as follows: Locate the data block corresponding to the metadata record based on the metadata record, use a cryptographic hash function to obtain the content address, write the data block corresponding to the metadata record into the object storage, and generate the object's physical location; Write the content address and the object's physical location into the mapping table to form a mapping relationship between the content address and the object's physical location, and then fill the content address back into the corresponding metadata record.

5. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 4, characterized in that: The steps for performing rehashing on the data blocks corresponding to the content addresses in object storage, verifying consistency by comparing content addresses, and generating a commit log table are as follows. The physical location of the object is queried by mapping the relationship table, and the data block corresponding to the content address is read from the object storage. The read data block is obtained and a cryptographic hash function is used to generate the recalculated content address. The recalculated content address and the content address are compared byte by byte to obtain the output consistency check status flag, which is then combined with the content address to generate the commit log table.

6. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 5, characterized in that: When processing actions are performed on successfully submitted data blocks in the commit log table, corresponding operator records are generated. A hash calculation is then performed on these operator records to obtain the operator addresses. The steps are as follows: Filter the list of successfully submitted content addresses and the set of successfully submitted data blocks from the commit log table; Execute processing actions on the set of successfully submitted data blocks, obtain the output data of the processing actions, and combine it with the list of successfully submitted content addresses to write it into the operator record according to a fixed set of fields and field order. Operator addresses are generated based on operator records using cryptographic hash functions.

7. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 6, characterized in that: The process of registering lineage edges using content addresses as nodes and operator addresses as association identifiers to form directed acyclic lineage relationships is as follows: The output data of the processing action is normalized and encapsulated to generate a derived normalized encapsulated data block, and a cryptographic hash function is used to generate the address of the derived content. By writing the derived normalized encapsulated data block and the derived content address into the object storage, the physical location of the derived object is obtained, and the derived content address and the physical location of the derived object are written into the mapping table. Register lineage edges using content addresses as nodes and operator addresses as association identifiers, obtain lineage edge records, and confirm the directed acyclic lineage relationship status of lineage edge records by performing ancestor content address set queries and inclusion relationship checks on the lineage edge records.

8. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 7, characterized in that: The steps are as follows: extracting the content addresses corresponding to successfully submitted data blocks from the commit log table, generating a leaf list, aggregating the leaf list to generate a root digest, and appending the digests. Filter the commit log table for records whose consistency check status is marked as successful during consistency check, sort them, generate an ordered content address sequence and encode it, and generate a leaf list. Based on the leaf list, perform hash aggregation on the ordered content address sequence to obtain the root digest, determine the root digest record writing position by the timestamp, and append the root digest and leaf list to the root digest record writing position.

9. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 8, characterized in that: The steps for performing retrieval, recalculation verification, and object-by-object content address verification based on the leaf list and root digest are as follows: Based on the root digest record write location and root digest, locate the leaf list location identifier associated with the root digest, and extract the leaf list content address sequence from the leaf list; Perform hash aggregation on the address sequence of the leaf list content layer by layer to obtain the recalculated root digest and compare it byte by byte with the root digest to generate a recalculation verification status flag; Based on the leaf list content address sequence, the physical location of the object is located through the mapping relationship table for each content address, and the data block corresponding to the content address is read from the object storage to generate a read data block. The read data block is then subjected to recalculation hashing to obtain the recalculated content address and is compared byte by byte with the content address to generate a set of object-by-object verification records.

10. The processing method for multimodal data acquisition in an intelligent manufacturing system as described in claim 9, characterized in that: Based on the directed acyclic lineage relationship, under the constraints of operator records, replay recalculation is performed to generate replay derived data blocks. Consistency comparison is then performed on the replay derived data blocks, and audit records are generated and stored. The steps are as follows: Combining directed acyclic lineage and content address, query the parent content address list and operator address, perform recalculation hashing on operator record, generate operator record verification status flag, execute processing actions according to operator record constraints, generate replay derived data block and generate replay derived content address through hash calculation; Perform a byte-by-byte comparison of the content addresses of the replay derived content address and the leaf list content address sequence to generate a replay consistency comparison status flag. Combine the replay consistency comparison status flag, the replay derived content address, the operator address, and the parent content address list to generate an audit record and write it to the audit record storage location.

Images