Intelligent unloading workshop multi-source data storage and tracing method and system

Abstract

The present application belongs to the technical field of industrial information and data processing and memory addressing control, and relates to a kind of intelligent unloading workshop multi-source data storage and traceability method and system, comprising: according to the topological evolution relationship of material identification, logical adjacency matrix is constructed in key value storage to establish logical traceability topological mapping layer;When archiving, extract timestamp and write into logical adjacency matrix after synchronous rectification using clock offset table;The target identifier of analysis search request is searched and time stamp routing vector is generated by topological path recursive addressing;Based on time stamp routing vector, the physical row address offset of main data table is calculated to control the jump of reading pointer to read target data, the present application decouples topological correlation and physical storage, eliminates the cascading update overhead under nonlinear flow, effectively shortens the query response delay, and guarantees the consistency of data traceability.

Description

Technical fields: This invention belongs to the field of industrial information and data processing and memory addressing control technology. It relates to a method and system for multi-source data storage and traceability in intelligent material handling workshops. It is used for integrated industrial information and data processing and full life cycle reverse traceability of discrete multi-source heterogeneous industrial data in intelligent manufacturing material handling workshops. It can decouple topological association and physical storage, eliminate the cascading update overhead under nonlinear flow, and effectively shorten the query response latency. Background technology: Currently, with the deepening development of intelligent manufacturing and industrial digitalization, utilizing digital threads to build integrated management and full lifecycle traceability of multi-source heterogeneous data has become a core technological path. However, existing technologies still have limitations in handling the semantic consistency of massive heterogeneous data, the stability of the underlying hardware time base, and the dynamic adaptability of cross-domain associations. First, semantic mapping integration technology based on digital mainline; for example, Chinese invention patent CN121071045A discloses a multi-source heterogeneous data integration management method based on digital mainline, which uses a deep ontology fusion model and attention mechanism to realize automated semantic mapping between cross-domain models. Its shortcomings are: the scheme focuses on semantic alignment at the model level. Due to the large overhead of deep learning models and attention calculation, it is difficult to balance the contradiction between semantic parsing depth and real-time response speed when facing extremely high frequency industrial real-time data streams, and it lacks the ability to correct time deviations caused by the hardware status of the data source.

[0001] Second, big data processing technology based on relational graphs; for example, Chinese invention patent CN121188497A discloses an industrial big data processing system based on cloud computing, which generates data DNA nodes by slicing data and constructs relational graphs to support data traceability analysis. Its drawback is that the establishment of its association mainly relies on predefined logical models or feature similarity, ignoring the time reference error caused by crystal oscillator drift during long-term operation of sensing hardware. As a result, in high-precision traceability at the microsecond level, the data association chain is prone to logical breakage due to time reference misalignment.

[0002] Third, time synchronization correction technology for sensing terminals; for example, Chinese invention patent CN120378039A discloses a time synchronization drift self-correction method for dedicated transformer acquisition terminals, which uses the zero intersection point of the power grid to back-calculate the reference time for dynamic self-correction. Its drawback is that the scheme is limited to the physical clock correction of a single acquisition terminal and fails to deeply couple the physical time accuracy of the hardware layer with the semantic logic layer of the upper digital mainline, resulting in the lack of a unified time reference with global semantic awareness for data flow across systems and professional fields.

[0003] Therefore, the technical problem to be solved by this invention is how to construct a prediction and integration management framework that has the ability to perceive underlying hardware drift and can be deeply coupled with cross-domain semantic mapping, so that the data management logic converges to the real high-precision physical time and semantic boundary throughout the entire life cycle. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and solve the technical problems of static physical mounting triggering high-concurrency write amplification, cross-table cascading retrieval causing query response delay, and distributed clock drift causing traceability link breakage in the prior art, so as to provide a method and system for multi-source data storage and traceability in intelligent material handling workshops.

[0005] To achieve the above-mentioned objectives, this invention provides a method for multi-source data storage and traceability in an intelligent material handling workshop, comprising the following steps: Step S1, solidify multi-source heterogeneous data and align to construct a global flow timeline: acquire discrete multi-source heterogeneous data records and solidify them to the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. Align the discrete multi-source heterogeneous data records stored in the underlying storage block in sequence on the timeline to construct a global flow timeline. Step S2, construct a topology mapping layer and record the material batch evolution logic graph: establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream, separate the material identifier from the discrete multi-source heterogeneous data record, and construct an independent logical relationship graph in the logical traceability topology mapping layer that records the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. Step S3: Use a Bloom filter to coarsely screen the data and complete the consistency verification: During the coarse screening process, use a Bloom filter to filter the status of non-associated material blocks in the collected material flow chain data, intercept non-target material data from entering the core retrieval chain, and complete the data consistency verification based on the checksum generated by the flow node. Step S4, Calculate the logical path to generate a routing vector to read node data: During the topology path addressing process, the shortest logical path is calculated in the independent logical relationship graph based on the query time interval. The address offset routing vector is generated through the shortest logical path. The address offset routing vector is used to point to the underlying storage block in real time. The solidified material flow node data in the underlying storage block is read directly through the address pointer.

[0006] The step S1 of this invention, which records discrete multi-source heterogeneous data stored in the underlying storage block on the time axis and aligns them sequentially to construct a global flow time axis, includes the following steps: Step S11, obtaining the global clock signal in the distributed network; Step S12, calculating the clock drift between the local clock and the global clock signal of each flow node; Step S13, using the clock drift to calibrate the timestamps of enterprise resource planning instructions, workshop process parameters, and quality inspection logs.

[0007] The step S4 of this invention, which generates an address offset routing vector through the shortest logical path, includes the following steps: Step S41, counting the total number of logical nodes traversed by the shortest logical path as the path length of the shortest logical path; Step S42, reading the fixed base storage block size of the underlying storage block; Step S43, calculating the address offset routing vector by multiplying the path length by the base storage block size; Step S44, superimposing the memory address offset onto the underlying physical starting base address, and directly jumping to locate the data page of the target material block.

[0008] The step S3 of this invention, which uses a Bloom filter to filter the state of non-associated material blocks in the collected material flow chain data, includes the following steps: Step S31, constructing a Bloom filter mapping space containing independent hash function matrices; Step S32, extracting the target material batch number from the material flow chain data as the feature to be retrieved; Step S33, calculating a multidimensional hash index value for the target material batch number using the hash function matrix, and retrieving the position state corresponding to the multidimensional hash index value in the Bloom filter mapping space; Step S34, if there is a 0 in the position state, it is determined that the material data block corresponding to the target material batch number is not mounted in the current underlying storage block, and the target material batch number is blocked from entering the core retrieval link; Step S35, if all position states are 1, the target material batch number is allowed and input into the core retrieval link.

[0009] The step S2 of this invention, which involves constructing an independent logical relationship graph recording the evolution topology of material batch numbers in the logical traceability topology mapping layer, includes the following steps: Step S21, when a parent batch identifier is divided into multiple child batch identifiers, a directed edge is created in the independent logical relationship graph, radiating from a parent node to multiple child nodes, generating material batch splitting topology data; Step S22, when multiple preceding batch identifiers converge to the same aggregation processing node, a directed edge is created in the independent logical relationship graph, converging from multiple preceding nodes to the same aggregation node, generating material batch aggregation topology data; Step S23, the material batch splitting topology data and the material batch aggregation topology data are stored in a memory key-value database in key-value pair form, constructing a logical evolution topology graph decoupled from the underlying physical table storage structure.

[0010] The step S4 of this invention, which calculates the shortest logical path in the independent logical relationship graph based on the query time interval, includes the following steps: Step S45, in response to the input reverse-tracking material batch number and the target query time interval, retrieve the initial topology node corresponding to the reverse-tracking material batch number in the memory key-value database; Step S46, extract the set of local topology edges that match the target query time interval, and truncate the topology branches that exceed the target query time interval in the independent logical relationship graph; Step S47, use the breadth-first search algorithm to calculate the shortest logical edge sequence from the initial topology node to the target source node within the pruned set of local topology edges, and construct a routing trajectory representing the full-link logical evolution of the quality node.

[0011] The data consistency self-check based on the checksum generated by the flow node in step S3 of the present invention includes the following steps: Step S36, collecting the material feature checksum generated by the sensors of each flow node; Step S37, recalculating the real-time checksum of the read material data block during data backtracking; Step S38, comparing the real-time checksum with the material feature checksum, and if the two are consistent, determining that no data tampering or packet loss has occurred in the core retrieval link.

[0012] In the discrete multi-source heterogeneous data records described in this invention, a distributed relational database is used to store enterprise resource planning instructions, a time-series database is used to store workshop process parameters, and a non-relational document database is used to store quality inspection logs.

[0013] The step S4 of this invention, which uses the address offset routing vector to point to the underlying storage block in real time, includes: translating the address offset routing vector into the physical sector offset of the block device of the underlying distributed file system, bypassing the table join mechanism of the upper relational database, directly reading the data page of the disk device in a directed graph roaming mode, and after the data backtracking reading is completed, outputting the traceability visualization topology result to the terminal of the manufacturing execution system, and caching the target material data block of the backtracking location to the high-speed buffer of the main memory.

[0014] This invention also provides a multi-source data storage and traceability system for an intelligent material handling workshop, which is used to implement a method for multi-source data storage and traceability in an intelligent material handling workshop, including: The data acquisition module is used to acquire discrete multi-source heterogeneous data records and solidify them into the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. The discrete multi-source heterogeneous data records stored in the underlying storage block are aligned sequentially on the time axis to construct a global flow time axis. The topology construction module is used to establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream. It separates the material identifier from the discrete multi-source heterogeneous data records and constructs an independent logical relationship graph in the logical traceability topology mapping layer to record the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. The data coarse screening module is used to perform state screening of non-related material blocks in the collected material flow chain data during the data coarse screening process using a Bloom filter, intercepting non-target material data from entering the core retrieval link, and completing data consistency verification based on the checksum generated by the flow node. The path addressing module is used to calculate the shortest logical path in the independent logical relationship graph based on the query time interval during the topology path addressing process. It generates an address offset routing vector through the shortest logical path, uses the address offset routing vector to point to the underlying storage block in real time, and directly reads the solidified material flow node data in the underlying storage block through the address pointer.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: 1. In the multi-source data storage of the intelligent material feeding workshop, a directed graph model parallel to the physical data flow is constructed in a logical topology memory independent of the physical business data table. The derivative dependencies and temporal relationships between the mother furnace batch identifier, the daughter furnace batch identifier, and the ingot section identifier are transformed into a logical adjacency matrix for independent storage. The directed graph model only retains the connection topology between identifiers and does not include process parameters. When the material flow path changes due to branching or merging, the system control unit directly completes the addition of node associations in the logical topology memory, eliminating the need for the physical data table to trigger high-frequency cascading synchronous updates under nonlinear splitting or merging conditions of material batches. This avoids the write amplification bottleneck under high-concurrency write conditions and ensures the stability of the data write state.

[0016] 2. Based on the received retrieval request, the identifier of the dimension to be queried is parsed. The logical topology memory completes recursive addressing according to its internal directed graph model and generates a timestamp routing vector covering the entire link in real time. The system control unit converts the timestamp routing vector into the address offset of the underlying physical storage block and controls the reading device to complete a jump reading of the target physical storage block, locate the key node data on the material flow chain, eliminate the multi-table joint retrieval load generated by scanning the full amount of heterogeneous data in the traditional cross-system association query process, and transform the complex association retrieval into physical addressing based on routing weight, thereby improving the query response speed under high concurrency conditions.

[0017] 3. Utilize the clock offset correction table stored inside the logical topology memory to perform a synchronization correction operation on the physical time axis of the timestamps collected by each heterogeneous node under distributed operating conditions before controlling the reading device to locate the physical storage address; ensure that the data records generated by different production systems are aligned in physical time, avoid the risk of traceability link breakage triggered by global clock asynchrony in a distributed network environment, and ensure the data consistency of reverse traceability results of quality nodes. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the multi-source heterogeneous data storage and topology traceability process in the intelligent material feeding workshop of this invention; Figure 2 This is a schematic diagram of the structure of the intelligent material unloading workshop multi-source data storage and topology traceability system of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings.

[0020] Example 1: This embodiment discloses a method for multi-source data storage and traceability in an intelligent material handling workshop, including the following steps: Step S1, solidify multi-source heterogeneous data and align to construct a global flow timeline: acquire discrete multi-source heterogeneous data records and solidify them to the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. Align the discrete multi-source heterogeneous data records stored in the underlying storage block in sequence on the timeline to construct a global flow timeline. Step S2, construct a topology mapping layer and record the material batch evolution logic graph: establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream, separate the material identifier from the discrete multi-source heterogeneous data record, and construct an independent logical relationship graph in the logical traceability topology mapping layer that records the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. Step S3: Use a Bloom filter to coarsely screen the data and complete the consistency verification: During the coarse screening process, use a Bloom filter to filter the status of non-associated material blocks in the collected material flow chain data, intercept non-target material data from entering the core retrieval chain, and complete the data consistency verification based on the checksum generated by the flow node. Step S4, Calculate the logical path to generate a routing vector to read node data: During the topology path addressing process, the shortest logical path is calculated in the independent logical relationship graph based on the query time interval. The address offset routing vector is generated through the shortest logical path. The address offset routing vector is used to point to the underlying storage block in real time. The solidified material flow node data in the underlying storage block is read directly through the address pointer.

[0021] In this embodiment, step S1, which records the discrete multi-source heterogeneous data stored in the underlying storage block on the time axis and aligns them sequentially to construct a global flow time axis, includes the following steps: Step S11, obtaining the global clock signal in the distributed network; Step S12, calculating the clock drift between the local clock and the global clock signal of each flow node; Step S13, using the clock drift to calibrate the timestamps of enterprise resource planning instructions, workshop process parameters, and quality inspection logs.

[0022] In this embodiment, step S4, generating the address offset routing vector through the shortest logical path, includes the following steps: Step S41, counting the total number of logical nodes traversed by the shortest logical path to use as the path length of the shortest logical path; Step S42, reading the fixed base storage block size of the underlying storage block; Step S43, calculating the address offset routing vector by multiplying the path length by the base storage block size; Step S44, adding the memory address offset to the underlying physical starting base address and directly jumping to the data page of the target material block.

[0023] In this embodiment, step S3, which uses a Bloom filter to filter the state of non-associated material blocks in the collected material flow chain data, includes the following steps: Step S31, constructing a Bloom filter mapping space containing independent hash function matrices; Step S32, extracting the target material batch number from the material flow chain data as the feature to be searched; Step S33, calculating a multidimensional hash index value for the target material batch number using the hash function matrix, and searching for the position state corresponding to the multidimensional hash index value in the Bloom filter mapping space; Step S34, if there is a 0 in the position state, it is determined that the material data block corresponding to the target material batch number is not mounted in the current underlying storage block, and the target material batch number is blocked from entering the core retrieval link; Step S35, if all position states are 1, the target material batch number is allowed and input into the core retrieval link.

[0024] In this embodiment, step S2, which involves constructing an independent logical relationship graph recording the evolution topology of material batch numbers in the logical traceability topology mapping layer, includes the following steps: Step S21, when a parent batch identifier is divided into multiple child batch identifiers, directed edges are created in the independent logical relationship graph, radiating from a parent node to multiple child nodes, generating material batch splitting topology data; Step S22, when multiple preceding batch identifiers converge to the same aggregation processing node, directed edges are created in the independent logical relationship graph, converging from multiple preceding nodes to the same aggregation node, generating material batch aggregation topology data; Step S23, the material batch splitting topology data and the material batch aggregation topology data are stored in a memory key-value database in key-value pair form, constructing a logical evolution topology graph decoupled from the underlying physical table storage structure.

[0025] In this embodiment, step S4, which calculates the shortest logical path in the independent logical relationship graph based on the query time interval, includes the following steps: Step S45, in response to the input reverse-tracking material batch number and the target query time interval, the initial topology node corresponding to the reverse-tracking material batch number is retrieved from the memory key-value database; Step S46, the set of local topology edges that match the target query time interval is extracted, and the topology branches that exceed the target query time interval are truncated in the independent logical relationship graph; Step S47, the shortest logical edge sequence from the initial topology node to the target source node is calculated in the truncated local topology edge set using a breadth-first search algorithm, and a routing trajectory representing the full-link logical evolution of the quality node is constructed.

[0026] In this embodiment, step S3, which involves completing a data consistency self-check based on the checksum generated by the flow nodes, includes the following steps: Step S36, collecting the material feature checksum generated by the sensors of each flow node; Step S37, recalculating the real-time checksum of the read material data block during data backtracking; Step S38, comparing the real-time checksum with the material feature checksum. If the two are consistent, it is determined that no data tampering or packet loss has occurred in the core retrieval link.

[0027] In the discrete, multi-source, heterogeneous data records described in this embodiment, a distributed relational database stores enterprise resource planning instructions, a time-series database stores workshop process parameters, and a non-relational document database stores quality inspection logs.

[0028] In step S4 of this embodiment, using the address offset routing vector to point to the underlying storage block in real time includes: translating the address offset routing vector into the physical sector offset of the block device of the underlying distributed file system, bypassing the table join mechanism of the upper relational database, directly reading the data page of the disk device in a directed graph roaming mode, and after the data backtracking reading is completed, outputting the traceability visualization topology result to the terminal of the manufacturing execution system, and caching the target material data block of the backtracking location to the high-speed buffer of the main memory.

[0029] This embodiment also provides a multi-source data storage and traceability system for an intelligent material handling workshop, including: The data acquisition module is used to acquire discrete multi-source heterogeneous data records and solidify them into the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. The discrete multi-source heterogeneous data records stored in the underlying storage block are aligned sequentially on the time axis to construct a global flow time axis. The topology construction module is used to establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream. It separates the material identifier from the discrete multi-source heterogeneous data records and constructs an independent logical relationship graph in the logical traceability topology mapping layer to record the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. The data coarse screening module is used to perform state screening of non-related material blocks in the collected material flow chain data during the data coarse screening process using a Bloom filter, intercepting non-target material data from entering the core retrieval link, and completing data consistency verification based on the checksum generated by the flow node. The path addressing module is used to calculate the shortest logical path in the independent logical relationship graph based on the query time interval during the topology path addressing process. It generates an address offset routing vector through the shortest logical path, uses the address offset routing vector to point to the underlying storage block in real time, and directly reads the solidified material flow node data in the underlying storage block through the address pointer.

[0030] Example 2: In this embodiment, during the production scheduling and flow of the intelligent material handling workshop, the multi-source data management center collects material requisition instructions from the enterprise resource planning system, real-time operating data from the process parameter monitoring unit, and feedback logs from the quality inspection module in real time. Addressing the issue of traceability data misalignment caused by the splitting of the mother furnace batch into multiple sub-ingot sections during multi-task sequence flow and the slight deviations in the physical clocks of the heterogeneous systems, this invention implements the following data organization and mapping logic: The system constructs a logical adjacency matrix in a non-relational memory key-value storage. This matrix is ​​stored independently of the underlying business physical data table. Its node set includes the mother furnace batch number, sub-furnace batch number, and ingot section number, while the edge set records the evolution path relationship of materials from the mother furnace batch to the ingot section. When material flow data triggers a write operation, the system synchronously extracts the timestamps from each data source, performs correction calculations based on the pre-stored clock offset table, and associates the corrected unified timestamp with the material identifier, updating the logical adjacency matrix.

[0031] When the retrieval logic is executed, the user terminal initiates a query request for the target identifier of a specific ingot. The tracing processing module performs recursive topological path addressing in the logical adjacency matrix based on the target identifier. In the specific implementation, recursive addressing is achieved by maintaining a recursive function call stack in memory with the target identifier as the initial input parameter. Its recursive basis is set as the current probe node having no preceding parent node or the timestamp of the current node exceeding the target query time interval. The recursive iteration method uses matrix multiplication of the adjacency row vector of the current node with the logical adjacency matrix to obtain all direct preceding nodes. At the same time, in order to optimize the stack overhead in a large-scale directed graph, the layer-by-layer expansion and backtracking process of the recursive function is equivalently carried and iteratively addressed by the first-in-first-out queue mechanism of the breadth-first search algorithm at the bottom layer. The complete traversal of the entire link dependency is achieved by the layer-by-layer iteration of the node state in the queue. The system uses the target ingot as the root node and searches backwards for all path branches connected to the mother furnace batch to generate a routing vector containing the nodes of the entire link. This vector is mapped to a set of physical pointers recording block addresses in the underlying distributed file system. To eliminate the differences in underlying physical file organization between distributed relational databases, time-series databases, and non-relational document databases, the underlying distributed file system uses uniformly sized, flattened block files as the container for underlying storage blocks. The storage engines of each heterogeneous database write their data pages sequentially into these pre-allocated physical storage blocks in a uniform binary stream format. The product result contained in the address offset routing vector is essentially a logical block slot index pointing to the internal logical block slot index of this uniformly flattened block file. The storage controller directly converts this slot index into the absolute physical sector number offset of the underlying raw block device through a preset underlying block device driver image table. In the address mapping unit of the main memory, physical addressing of the underlying block device storage space and associated industrial information and data processing are realized, thereby bypassing the schema parsing and complex multi-table join retrieval load of each database layer, realizing a directed graph roaming direct jump read of disk hard sector data. The storage controller directly reads the corresponding discrete storage block based on the physical address offset contained in the routing vector, avoiding the cascading join calculation of the entire business data table.

[0032] When material splitting or merging adjustments occur during production, the system control unit directly adds node relationships to the logical adjacency matrix; the record rows in the physical business data table remain in a static storage state, eliminating the overhead of high-frequency cascading synchronous updates of the data table triggered by nonlinear flow; this architecture decouples the material logical evolution topology from the physical storage of business records, and achieves fast addressing and adaptive clock deviation correction of heterogeneous data in the material unloading workshop while ensuring the consistency of traceability throughout the entire chain.

[0033] Example 3: This embodiment verifies the performance of the logical traceability topology mapping layer in handling high-concurrency writes and non-linear data flow through logically related verification data. The test platform is built on a dual-processor server cluster, configured with a multi-core processor with a main frequency of 3.0GHz and 128GB of memory. The data persistence layer adopts a distributed file system and is configured with a network transmission bandwidth of 1000Mbps. The test design includes a control group and an experimental group. The control group adopts a static physical mounting scheme, that is, embedding the association key in each business data record of the unloading workshop. When the material batch is merged or split, the system performs a full cascading update on the underlying physical business table. The experimental group adopts the logical traceability topology mapping mechanism of the aforementioned embodiment 2, storing the material flow topology relationship independently in the memory key-value storage. During the test, the working condition of generating 10,000 multi-source heterogeneous data records in the unloading workshop in a single hour is simulated, and the system response latency under different working condition complexities is recorded. In order to address the random disturbance of the data flow and sensor measurement errors, Gaussian white noise with a signal-to-noise ratio of 20dB is actively superimposed at the input end of the test environment.

[0034] In the stable production phase with a batch flow complexity of 1:1, the average single-data query latency of the control group was 45.2ms, while the query latency of the experimental group remained stable at 12.8ms. As the production process evolved to the batch splitting phase with a batch complexity of 1:5, the average query latency of the control group increased to 215.6ms due to the triggering of cascading write operations on large-scale business physical tables, accompanied by frequent deadlock waiting. The query latency of the experimental group increased slightly to 14.3ms because it only needed to add nodes to the logical adjacency matrix, demonstrating extremely high stability.

[0035] For the calibration and verification of distributed node physical clock drift, the experiment set up three heterogeneous data acquisition nodes, with initial manual settings for clock offsets of 15ms, -22ms, and 8ms for each node. After clock offset correction, the timestamp error range recorded in the logical traceability topology mapping layer of the experimental group converged to within 0.5ms, ensuring the logical consistency of the traceability link across heterogeneous nodes. Data results show that when the material flow path undergoes nonlinear changes, the technical solution of this invention avoids the write amplification effect of the physical storage layer by decoupling the material batch evolution logic from the underlying physical business table physical storage. When processing a complex topology containing 50,000 material flow records, the reverse traceability success rate of the experimental group reached 100%, and no deadlock or data consistency deviation occurred in the full-link addressing operation. The above comparative data confirms that this invention achieves fast addressing and adaptive clock offset correction for discrete business data through the logical mapping layer, and the architecture has engineering applicability to support real-time storage and traceability of large-scale heterogeneous industrial data.

[0036] Example 4: This embodiment combines Figures 1 to 2This paper describes the multi-source data storage and traceability methods and systems in intelligent material handling workshops, such as... Figure 1 As shown, step S1 is to solidify multi-source heterogeneous data and align and construct a global flow timeline. Then, the process continues to step S2, which is to construct a topology mapping layer and record the material batch evolution logic graph. Step S3 is to use a Bloom filter to coarsely screen the data and complete the consistency verification. Finally, it proceeds to step S4, which is to calculate the logical path to generate a routing vector to read the node data.

[0037] like Figure 2 As shown, the system architecture mainly consists of a clock reference transmitter, heterogeneous data acquisition nodes, a dual-processor server cluster, and a manufacturing execution system terminal. The heterogeneous data acquisition nodes are internally configured with sensors. The clock reference transmitter and the heterogeneous data acquisition nodes are connected by a dotted line. The heterogeneous data acquisition nodes perform unidirectional data input to the dual-processor server cluster. The dual-processor server cluster is logically divided into two parts: a non-relational memory key-value store and an underlying distributed file system. The non-relational memory key-value store contains a logical adjacency matrix, a high-speed buffer, and a clock offset table. The underlying distributed file system contains a distributed relational database and its associated enterprise resource planning instructions, a time-series database and its associated workshop process parameters, a non-relational document database and its associated quality inspection logs, and underlying storage blocks contained in the physical sectors of block devices. The top-level termination of the dual-processor server cluster provides unidirectional output to the manufacturing execution system terminal.

[0038] Example 5: In this embodiment, when the multi-source discrete heterogeneous data acquisition nodes in the intelligent material unloading workshop are in a long-cycle operation state, the distributed clock drift caused by crystal oscillator aging between the heterogeneous data acquisition nodes causes the original timestamps extracted by the data management center when archiving distributed storage blocks to be misaligned on the global flow time axis. In order to eliminate the interference of clock deviation on the stability of the topological association of the material identification directed graph, the clock variation calibration loop in the system acquires the delay messages fed back by each heterogeneous data acquisition node within the sliding window acquisition cycle.

[0039] The clock reference transmitter periodically sends synchronization messages containing the originating timestamp to each heterogeneous data acquisition node. The target data acquisition node receives the synchronization message and records the receive register timestamp. The control unit calculates the clock offset observation value for the current period based on the difference between the receive register timestamp and the originating timestamp, and deducts the preset fixed network transmission delay loss. ,in, For time sample indexing; to suppress random pulse interference caused by network disturbances, the control unit opens a sliding shear window in the cache area to accommodate 10 sampling points, storing the clock offset observations for 10 consecutive cycles. The data is temporarily stored in the sliding shear window. The control unit clears the maximum and minimum values ​​and calculates the arithmetic mean of the remaining 8 observations to generate the node clock drift calibration coefficient for the current time period. Among them, the node clock drift calibration coefficient The unit is milliseconds, and the system control unit sets the node clock drift calibration coefficient. The clock offset table is written to the non-relational memory key-value store, and the timing correction factor of the corresponding heterogeneous data acquisition node is overwritten and updated. Based on the principle of symmetric one-way transmission delay in network clock synchronization, the global transfer timestamp is calculated as follows: , where variables Represents the global flow timestamp, with a value that is a positive rational number; variable The original timestamp is determined by the local clock hardware counter of each flow node; the coefficient... This represents the node clock drift calibration coefficient, which is read from the clock offset table. During the calculation process, the time data is constrained to a uniform flow coordinate axis.

[0040] When the archiving module writes the material identification logical adjacency matrix to the memory, it calls the node clock drift calibration coefficient in the clock offset table. According to mathematical relations The original timestamp is converted into a unified global transfer timestamp, where, This is the converted global flow timestamp. The raw timestamp extracted from the physical storage block. The node clock drift calibration coefficients for the corresponding heterogeneous data acquisition nodes, read from the clock offset table, are preset minimum link thresholds in practical engineering. The integer value is determined based on the total number of core processing steps in the material preparation workshop. In this embodiment, it is fixed at level 4. If the value is lower than this, it is determined that the directed graph evolution path does not cover the complete scheduling process flow. The number of sampling points of the sliding shear window is set to 10. This is based on the experimental determination of the clock jitter characteristics of the heterogeneous acquisition nodes in the workshop under 20dB Gaussian white noise interference. If the number of sampling points is less than 5, it is impossible to effectively suppress the random burst pulses generated by network disturbances by eliminating extreme values. If the number of sampling points is higher than 15, it will cause the calculation of calibration coefficients to produce cumulative update lag, and it will be impossible to correct the dynamic temperature drift of the crystal oscillator in real time during long-term operation. Therefore, 10 sampling points are selected to achieve the best engineering balance between filtering smoothness and response real-time performance. The timing reconstruction loop eliminates the topological association break caused by time axis misalignment. When the system faces a clock bias interference condition of 50ms level, the time deviation of the converted global flow timestamp is reduced to less than 0.5ms, ensuring the orderly alignment of distributed heterogeneous data blocks at the underlying memory level and completing the material full life cycle timing path addressing across heterogeneous systems.

[0041] Example 6: In this embodiment, when the system faces situations such as the addition of a new data acquisition node in the intelligent material handling workshop or changes in network routing topology, the increased latency can cause bias errors in the clock calibration circuit. To establish the initial state of the memory, the control unit triggers a pre-calibration operation before data solidification, controlling the reference transmitter to continuously send 100 ranging empty packet messages to the new data acquisition node. After receiving the messages, the target node returns an acknowledgment signal via a hardware interrupt. The control unit records the entire round-trip time data, removes abnormal data caused by sudden network congestion, and uses half of the average remaining round-trip time as the static reference latency constant for the new channel. The control unit then writes this value into a non-relational key-value store.

[0042] According to the formula The initial clock bias factor is calculated, where, This is the initial clock bias factor. This is the local timestamp when the target node receives the message. This is the initial timestamp. As a static reference delay constant, the storage controller writes the result into the clock offset table as the zero-point reference value for calibration; this pre-processing eliminates the static deviation caused by communication protocol dissipation, controls the initial alignment error of the acquisition network to within 1ms, enables the production records written during high concurrency to complete physical alignment on the global flow time axis, and controls the calculation accuracy of the topology path recursive addressing routing vector.

[0043] Example 7: In this embodiment, when the system faces high-concurrency network congestion in the intelligent material unloading workshop, leading to packet loss of material identification flow messages and causing local path breaks in the topology relationship of material batch splitting recorded in the memory key-value storage, the addressing module recursively addresses the target identifier in the non-relational memory key-value storage; when the addressing module detects that the depth of the currently generated routing vector is less than the preset minimum link threshold... At that time, among them For the preset integer link depth order, the system control unit interrupts the current recursive probe and calls the physical metadata auxiliary alignment loop. It controls the main storage controller to read the header descriptor of the underlying storage block corresponding to the broken path. The header descriptor contains the physical transaction log stream written during solidification. The system control unit parses the physical transaction log stream to reconstruct the missing batch topology dependency nodes and overwrites the repaired topology evolution path to the logical adjacency matrix. In order to accurately locate the target storage block under the condition of directed graph path breakage and logical mapping failure, the main storage controller extracts the material batch number and its actual timestamp of the last valid logical node before the break point and uses it as the time window benchmark for retrieval.

[0044] Because the underlying storage blocks possess strong time-axis alignment during physical writes, the main storage controller directly invokes the file system's time-series metadata index. Within a preset millisecond delay range after the time window baseline, it performs range addressing on adjacent physical storage blocks generated sequentially within the physical channel. This allows for precise identification and reading of the underlying physical storage block header descriptors corresponding to the physical occurrence sequence of the broken path, without relying on upper-layer logical topology paths, through physical time-series proximity. The physical metadata-assisted alignment loop parses the data stream: reading the change data capture log stream embedded in the physical storage block header descriptor; extracting the data manipulation language change messages contained in the log stream; filtering out log frames containing material batch number update events; and reversing the process within the log frames. The previous batch field value and the current batch field value are reassembled to form a directed dependency edge from the parent batch node to the child batch node. The directed dependency edge is used as the repaired topology evolution path to overwrite the logical adjacency matrix, completing the directed graph topology network. The updated logical adjacency matrix forms the directed graph topology network. The addressing module performs a second calculation based on the repaired topology evolution path and outputs an address offset routing vector covering the entire lifecycle. The storage controller receives the address offset routing vector, calculates the physical row address offset of the corresponding master data table, and drives the read pointer to directly jump to read the target discrete process parameter data block. This reduces the reverse tracing interruption rate under abnormal network disturbances in the acquisition layer, and the end-to-end cascade addressing latency is less than or equal to 15.0ms.

Claims

1. A method for multi-source data storage and traceability in an intelligent material handling workshop, characterized in that, Includes the following steps: Step S1: Obtain discrete multi-source heterogeneous data records and solidify them into the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. Align the discrete multi-source heterogeneous data records stored in the underlying storage block in sequence on the time axis to construct a global flow time axis. Step S2: Establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream. Extract the material identifier from the discrete multi-source heterogeneous data record. Construct an independent logical relationship map in the logical traceability topology mapping layer that records the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. Step S3: During the data coarse screening process, a Bloom filter is used to screen the status of non-associated material blocks in the collected material flow chain data, intercepting non-target material data from entering the core retrieval link, and completing data consistency verification based on the checksum generated by the flow node. In step S4, during the topology path addressing process, the shortest logical path is calculated in the independent logical relationship graph based on the query time interval. An address offset routing vector is generated through the shortest logical path. The address offset routing vector is used to point to the underlying storage block in real time. The solidified material flow node data in the underlying storage block is read directly through the address pointer.

2. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S1 involves recording the discrete, multi-source, heterogeneous data stored in the underlying storage block on the timeline and aligning them sequentially to construct a global flow timeline. This includes the following steps: Step S11, obtaining the global clock signal within the distributed network; Step S12, calculating the clock drift between the local clock and the global clock signal of each flow node; Step S13, using the clock drift to calibrate the timestamps of enterprise resource planning instructions, workshop process parameters, and quality inspection logs.

3. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S4, generating the address offset routing vector through the shortest logical path, includes the following steps: Step S41, counting the total number of logical nodes traversed by the shortest logical path to determine the path length of the shortest logical path; Step S42, reading the fixed base storage block size of the underlying storage block; Step S43, calculating the address offset routing vector by multiplying the path length by the base storage block size; Step S44, adding the memory address offset to the underlying physical starting base address and directly jumping to the data page of the target material block.

4. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S3 involves using a Bloom filter to filter the status of non-associated material blocks in the collected material flow chain data, including the following steps: Step S31, constructing a Bloom filter mapping space containing independent hash function matrices; Step S32, extracting the target material batch number from the material flow chain data as the feature to be retrieved; Step S33, calculating a multidimensional hash index value for the target material batch number using the hash function matrix, and retrieving the corresponding position status in the Bloom filter mapping space; Step S34, if a position status contains 0, determining that the material data block corresponding to the target material batch number is not mounted in the current underlying storage block, and intercepting the target material batch number from entering the core retrieval link; Step S35, if all position statuses are 1, allowing the target material batch number to pass and input it into the core retrieval link.

5. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S2 involves constructing an independent logical relationship graph in the logical traceability topology mapping layer to record the evolution topology of material batch numbers, including the following steps: Step S21, when a parent batch identifier is divided into multiple child batch identifiers, a directed edge is created in the independent logical relationship graph, radiating from a parent node to multiple child nodes, generating material batch splitting topology data; Step S22, when multiple preceding batch identifiers converge to the same aggregation processing node, a directed edge is created in the independent logical relationship graph, converging from multiple preceding nodes to the same aggregation node, generating material batch aggregation topology data; Step S23, the material batch splitting topology data and the material batch aggregation topology data are stored in a memory key-value database in key-value pair form, constructing a logical evolution topology graph decoupled from the underlying physical table storage structure.

6. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S4, which calculates the shortest logical path in the independent logical relationship graph based on the query time interval, includes the following steps: Step S45, in response to the input reverse-tracking material batch number and the target query time interval, retrieve the initial topology node corresponding to the reverse-tracking material batch number in the memory key-value database; Step S46, extract the set of local topology edges that match the target query time interval, and truncate the topology branches that exceed the target query time interval in the independent logical relationship graph; Step S47, use the breadth-first search algorithm to calculate the shortest logical edge sequence from the initial topology node to the target source node within the pruned set of local topology edges, and construct the routing trajectory representing the full-link logical evolution of the quality node.

7. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S3, which involves a self-check of data consistency based on the checksums generated by the flow nodes, includes the following steps: Step S36, collecting the material feature checksums generated by the sensors at each flow node; Step S37, recalculating the real-time checksum of the read material data block during data backtracking; Step S38, comparing the real-time checksum with the material feature checksum. If the two are consistent, it is determined that no data tampering or packet loss has occurred in the core retrieval link.

8. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, In discrete, multi-source, heterogeneous data records, distributed relational databases store enterprise resource planning instructions, time-series databases store workshop process parameters, and non-relational document databases store quality inspection logs.

9. The method for multi-source data storage and traceability in an intelligent material handling workshop according to claim 1, characterized in that, Step S4 involves using the address offset routing vector to point to the underlying storage block in real time. This includes translating the address offset routing vector into the physical sector offset of the block device in the underlying distributed file system, bypassing the table join mechanism of the upper relational database, directly reading the data page of the disk device in a directed graph roaming mode, and outputting the traceability visualization topology result to the terminal of the manufacturing execution system after the data backtracking is completed. The target material data block of the backtracking location is cached in the high-speed buffer of the main memory.

10. A multi-source data storage and traceability system for an intelligent material handling workshop, used to implement the multi-source data storage and traceability method for an intelligent material handling workshop as described in claim 1, characterized in that, include: The data acquisition module is used to acquire discrete multi-source heterogeneous data records and solidify them into the underlying storage block. The discrete multi-source heterogeneous data records include enterprise resource planning instructions, workshop process parameters and quality inspection logs. The discrete multi-source heterogeneous data records stored in the underlying storage block are aligned sequentially on the time axis to construct a global flow time axis. The topology construction module is used to establish a logical traceability topology mapping layer that is synchronously mapped with the physical data stream. It separates the material identifier from the discrete multi-source heterogeneous data records and constructs an independent logical relationship graph in the logical traceability topology mapping layer to record the evolution topology relationship of the material batch number. The evolution topology relationship includes the material batch splitting topology relationship and the material batch convergence topology relationship. The data coarse screening module is used to perform state screening of non-related material blocks in the collected material flow chain data during the data coarse screening process using a Bloom filter, intercepting non-target material data from entering the core retrieval link, and completing data consistency verification based on the checksum generated by the flow node. The path addressing module is used to calculate the shortest logical path in the independent logical relationship graph based on the query time interval during the topology path addressing process. It generates an address offset routing vector through the shortest logical path, uses the address offset routing vector to point to the underlying storage block in real time, and directly reads the solidified material flow node data in the underlying storage block through the address pointer.

Images