A heterogeneous data source unified access method

Abstract

This invention relates to the field of digital information transmission technology, specifically a unified access method for heterogeneous data sources. The invention parses and classifies data streams, injecting hardware priority tags into non-real-time data; performs dynamic diversion of non-real-time data and concurrent push of real-time data hardware-level cloning; synchronously acquires the physical layer transmission counter and calculates the remaining transmission time in nanoseconds; calculates a weighted preemption score based on hardware tag weights and remaining time to lock the target network interface, and performs interruption hard overhead suppression judgment; if there is a benefit, it triggers physical layer transmission error interruption long frame transmission, resets the media access control queue, and prioritizes the transmission of emergency data clones; finally, it achieves partial retransmission of preempted data through hardware proxy caching. This invention solves the problem of line-head blocking in the underlying physical link, ensuring hard real-time emergency commands while balancing service integrity and system throughput efficiency.

Description

Technical Field

[0001] This invention relates to the field of digital information transmission technology, specifically a method for unified access to heterogeneous data sources. Background Technology

[0002] In modern industrial settings, a vast and diverse array of sensing nodes and control devices are deployed. To break down "information silos," industrial edge gateways typically need to provide a unified data access and routing mechanism across protocols to simultaneously handle highly differentiated heterogeneous data streams. In typical scenarios, a single physical gateway must handle not only high-bandwidth, non-real-time IT data but also extremely low-latency, hard real-time OT data. To ensure the real-time performance of critical control commands during unified access, existing technologies typically introduce semantic classification engines at the gateway application layer to parse the business semantics of heterogeneous data and combine them with multi-dimensional dynamic queue managers and non-blocking egress schedulers to perform queue isolation and distribution based on strict priorities. These mechanisms attempt to prioritize emergency stop commands over machine vision images during the software queuing phase.

[0003] However, although the aforementioned mechanisms based on unified application semantic awareness and strict priority queuing can optimize the transmission order at the software queue management level, they are still fundamentally limited by the underlying traditional Ethernet MAC layer physical transmission mechanism when facing typical IT / OT hybrid concurrent scenarios. The transmission of traditional standard Ethernet data frames on the physical link is non-preemptive, leading to a fatal "head-of-line blocking" effect on the underlying physical link. In actual heterogeneous network convergence scenarios, high-bandwidth non-real-time data streams typically consist of long data frames at the maximum transmission unit level. When the system's physical port has just begun sending a low-priority long data frame, if a critical microsecond-level emergency control command arrives at the unified access interface and is instantly parsed and placed at the head of the highest priority queue, this emergency command must also wait until the current low-priority long packet has been completely transmitted on the physical link before it can gain transmission rights. This hardware-level queuing blind spot caused by the uninterruptible mechanism of the underlying physical link introduces tens to hundreds of microseconds of nondeterministic transmission jitter on a micro-time scale. Therefore, existing heterogeneous data unified access and routing scheduling technologies, due to their inability to overcome the "head-of-line blocking" effect of physical links, have always struggled to control the end-to-end transmission jitter of high-priority critical instructions within a strictly predictable microsecond-level upper limit in real industrial scenarios with mixed high-frequency congestion, thus failing to achieve hard real-time assurance in an engineering sense.

[0004] In summary, how to solve the problem that critical high-priority instructions cannot be interrupted because the underlying port is sending low-priority long data frames, thus preventing the end-to-end routing scheduling from achieving absolute hard real-time guarantee, is a technical problem that urgently needs to be solved in this field.

[0005] To address this, a unified access method for heterogeneous data sources is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a unified access method for heterogeneous data sources. This invention parses and classifies data streams, injects hardware priority tags into non-real-time data, performs dynamic distribution of non-real-time data and concurrent pushes hardware-level clones of real-time data, synchronously acquires the physical layer transmission counter and calculates the remaining transmission time in nanoseconds, calculates a weighted preemption score based on hardware tag weights and remaining time to lock the target network interface, and performs interruption hard overhead suppression judgment; if there is a benefit, it triggers a physical layer transmission error to interrupt long frame transmission, resets the media access control layer queue, and prioritizes the transmission of emergency data clones; finally, it achieves partial retransmission of preempted data through hardware proxy caching. This invention solves the problem of line-head blocking in the underlying physical link, ensuring hard real-time emergency commands while balancing service integrity and system throughput efficiency.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for unified access to heterogeneous data sources, the method being applied to an industrial edge gateway configured with a first physical network port and a second physical network port, comprising: The application layer protocol of the received multi-source heterogeneous data stream is parsed through a unified access interface, business semantic identifiers are extracted, and heterogeneous data streams are classified into non-real-time data and real-time emergency data. The target distribution port is determined from the two physical network ports, and non-real-time data is dynamically distributed and sent. At the same time, real-time emergency data is mirrored and cloned at the hardware layer, and the two clones are pushed concurrently to the media access control layer of the two physical network ports. When the clone reaches the media access control layer, it synchronously reads the transmission status counters of the underlying physical layer chips of the two physical network ports and calculates the remaining transmission time of the non-real-time data being transmitted by each physical network port. Based on the remaining transmission time, the weighted preemption score of each physical network interface is determined, and the physical network interface with the highest weighted preemption score is selected as the target preemption network interface. A physical layer transmission error interrupt signal is triggered to the underlying physical layer chip of the target preemption network interface, forcibly terminating the non-real-time data currently being transmitted by the target preemption network interface at the physical level, and prioritizing the transmission of the corresponding clone. At the same time, the transmission state of the other physical network interface remains unchanged, and the corresponding clone is transmitted after the current data frame is completed.

[0008] Preferably, parsing the application layer protocol of the received multi-source heterogeneous data streams, extracting service semantic identifiers, and classifying the heterogeneous data streams into non-real-time data and real-time urgent data includes: stripping the network layer packet header and transport layer packet header of the multi-source heterogeneous data streams, reading the application layer payload data; performing string matching comparison between the application layer payload data and a preset semantic feature matrix, extracting the successfully matched feature fields as service semantic identifiers; querying a pre-configured service mapping table, which is divided into a first type of service label set and a second type of service label set; when the service semantic identifier belongs to the first type of service label set, determining that the corresponding data stream is real-time urgent data with absolute time constraints; when the service semantic identifier belongs to the second type of service label set, determining that the corresponding data stream is non-real-time data dependent on throughput.

[0009] Preferably, the target distribution port is determined from two physical network ports, and non-real-time data is dynamically distributed and sent. This includes: extracting the source Internet Protocol address, target Internet Protocol address, source port number, and target port number of the non-real-time data to form a data stream quintuple; querying a preset flow state mapping table at the hardware layer to determine whether the data stream quintuple is bound to a target network port; if it is, the non-real-time data is directly driven into the sending queue of the bound physical network port; if it is not, the current queue backlog byte length of the two physical network ports is extracted, the physical network port with the shorter backlog byte length is selected as the target distribution port, and the binding relationship between the data stream quintuple and the target distribution port is written into the flow state mapping table, and then the non-real-time data is driven into the sending queue of the target distribution port.

[0010] Preferably, real-time emergency data is mirrored and cloned at the hardware layer, and the two clones are concurrently pushed to the media access control layer of the two physical network ports. This includes: intercepting real-time emergency data in a field-programmable gate array and writing its binary payload into a single storage area of ​​a dual-port random access memory; generating two independent transfer descriptors pointing to the starting address of the storage area as clones within one hardware clock cycle through an internal direct memory access controller; and using the two independent read ports of the dual-port random access memory, injecting the two clones directly into the media access control layer reserved interface of the corresponding physical network port through two mutually isolated hardware bus channels, bypassing the operating system kernel.

[0011] Preferably, the remaining transmission time of the non-real-time data being transmitted by each physical network port is calculated, including: reading the total byte length of the non-real-time data packet being transmitted from the descriptor ring of the media access control layer; polling the internal status registers of the underlying physical layer chips of the two physical network ports respectively through the internal management bus; extracting the transmission status counter value representing the number of physical symbols sent from the internal status register; performing a subtraction operation to subtract the transmission status counter value from the total byte length of the packet to obtain the remaining untransmitted byte length; querying and extracting the nominal link transmission rate value of the two physical network ports; and performing a division operation to divide the remaining untransmitted byte length of the two physical network ports by the corresponding nominal link transmission rate value to obtain the remaining transmission time of the non-real-time data being transmitted by the two physical network ports.

[0012] Preferably, the weighted preemption score for each physical network interface is determined based on the remaining transmission time, including a weighted determination step based on hardware tags: while synchronously reading the transmission status counter of the physical layer chip at the media access control layer, the hardware priority tags in the media access control layer packet headers of the non-real-time data being transmitted by the two physical network interfaces are extracted in parallel within one hardware clock cycle using a field-programmable gate array; using hardware combinational logic circuits, the service weight coefficient mapped by the hardware priority tags is weighted and determined by the remaining transmission time to generate their respective weighted preemption scores; the weighted preemption scores of the two physical network interfaces are compared, and the physical network interface with the higher weighted preemption score is selected as the target preemption interface, and preemption avoidance protection is implemented for high-priority non-real-time data.

[0013] Preferably, a physical layer transmission error interrupt signal is triggered to the underlying physical layer chip of the target preempted network port to intervene and terminate the non-real-time data currently being transmitted by the target preempted network port at the physical level. This includes: starting the hardware interrupt generator corresponding to the target preempted network port; sending a trigger level control command to the media independent interface of the target preempted network port; driving the transmission error dedicated pin of the underlying physical layer chip to be continuously in the trigger level state; forcing the underlying physical layer chip to reverse the polarity of the frame check sequence in the current physical symbol stream and inject error coding symbols into it; outputting the damaged physical layer residual frame to the physical cable; completing the intervention and termination action of non-real-time data; clearing the residual non-real-time data terminated by intervention from the media access control layer queue; and pushing the corresponding clone into the physical layer chip for priority transmission.

[0014] Preferably, after terminating the non-real-time data currently being transmitted by the target preempted network port, a hardware proxy partial retransmission step for the non-real-time data is also included: In the static random access memory (SRAM) inside the field-programmable gate array (FPGA), the non-real-time data pushed into the media access control (MAC) layer queue of the target preempted network port is mirrored and buffered; after the physical layer transmission error interrupt signal is triggered and the corresponding clone has been sent first, the new data is blocked from being sent by the operating system kernel; the complete frame of the non-real-time data that was terminated and buffered in the SRAM is re-injected into the MAC layer transmission ring of the target preempted network port, and a transparent physical layer retransmission is performed.

[0015] Preferably, after sending the corresponding clone, the method further includes a spatial deduplication step for the cloned packets: the target receiving device listens for and receives packets arriving from the corresponding links of the two physical network interfaces; the target receiving device extracts the sequence number and timestamp parameters of the first arriving packet on the timeline, records them in the local doubly linked list memory, and submits the first arriving packet to the application layer to perform an emergency shutdown action; the sequence number of subsequent arriving packets is continuously compared with the sequence number recorded in the local doubly linked list memory; when the sequence numbers match, the subsequent arriving packets are determined to be redundant cloned packets; the target receiving device executes a silent discard instruction on the redundant cloned packets at the network card driver level to eliminate the risk of service state machine re-entry anomalies caused by duplicate control packets.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention effectively improves the determinism and response timeliness of critical data transmission. By concurrently pushing real-time emergency data through dual physical network ports at the media access control layer, and combining synchronous sensing of the underlying physical layer chip status and a transmission error interruption intervention mechanism, this method can directly terminate non-real-time data transmission at the physical level. This hardware-level cross-layer intervention mechanism significantly shortens the queuing period of real-time emergency data at the network card level, enabling critical control commands to achieve better transmission timeliness guarantees in complex concurrent network environments.

[0017] 2. This invention achieves balanced allocation and rational reuse of underlying network resources. It employs an asymmetric data routing and distribution architecture, dynamically diverting routine, non-real-time data to different target network interfaces for transmission, while only performing hardware-level mirroring and cloning on real-time emergency data. This scheduling strategy, while ensuring the multi-path redundant transmission characteristics of emergency service data, fully releases the daily communication bandwidth potential of the dual physical network interfaces, avoiding long-term idleness and waste of physical link resources.

[0018] 3. This invention balances the reliability of multi-service concurrency with overall data integrity. Based on a comprehensive preemption decision scoring system, it implements targeted intervention, ensuring not only hard real-time transmission of urgent data but also intelligently identifying and protecting non-real-time data streams (such as critical visual quality inspection frames) that are time-consuming but have higher service value, even when multiple physical network ports are occupied, while preempting lower-weight regular traffic. Combined with the redundancy deduplication mechanism of the target receiving device and the protocol stack timeout retransmission mechanism of the source device, this solution prioritizes the initial transmission of urgent real-time data while effectively reducing the impact on high-value broadband services, achieving a multi-dimensional balance between communication timeliness, service importance, and data integrity. Attached Figure Description

[0019] Figure 1 A flowchart of a method for unified access to heterogeneous data sources provided in an embodiment of the present invention; Figure 2 This is a functional module structure diagram of an industrial edge gateway provided in an embodiment of the present invention; Figure 3 The hardware preemption determination and interrupt suppression logic diagram provided for embodiments of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1 to 3 This invention provides a unified access method for heterogeneous data sources, the technical solution of which is as follows: A unified access method for heterogeneous data sources, applied to an industrial edge gateway configured with a first physical network port and a second physical network port, includes: The application layer protocol of the received multi-source heterogeneous data stream is parsed through a unified access interface, business semantic identifiers are extracted, and heterogeneous data streams are classified into non-real-time data and real-time emergency data. The target distribution port is determined from the two physical network ports, and non-real-time data is dynamically distributed and sent. At the same time, real-time emergency data is mirrored and cloned at the hardware layer, and the two clones are pushed concurrently to the media access control layer of the two physical network ports. When the clone reaches the media access control layer, it synchronously reads the transmission status counters of the underlying physical layer chips of the two physical network ports and calculates the remaining transmission time of the non-real-time data being transmitted by each physical network port. Based on the remaining transmission time, the weighted preemption score of each physical network interface is determined, and the physical network interface with the highest weighted preemption score is selected as the target preemption network interface. A physical layer transmission error interrupt signal is triggered to the underlying physical layer chip of the target preemption network interface, forcibly terminating the non-real-time data currently being transmitted by the target preemption network interface at the physical level, and prioritizing the transmission of the corresponding clone. At the same time, the transmission state of the other physical network interface remains unchanged, and the corresponding clone is transmitted after the current data frame is completed.

[0022] Example 1: This embodiment applies to complex edge computing scenarios in the Industrial Internet of Things (IIoT) where information technology and operational technology are deeply integrated. In a typical smart manufacturing workshop, the unified access interface of the industrial edge gateway needs to simultaneously access and process high-bandwidth image data from a binocular machine vision quality inspection system, as well as hard real-time emergency stop commands from safety light curtains. Existing network scheduling mechanisms, when processing long-byte image data, suffer from severe line-heading congestion due to the non-preemptive nature of the underlying physical links. This forces extremely small emergency stop commands (real-time emergency data) to queue for tens of microseconds, leading to uncontrollable transmission jitter and seriously threatening production safety. This embodiment aims to provide a specific implementation method for data processing and cross-layer routing scheduling.

[0023] As one embodiment of the present invention, refer to Figure 1 A flowchart of a unified access method for heterogeneous data sources, refer to Figure 2 The functional module structure diagram of the industrial edge gateway is shown in the reference diagram. Figure 3 Hardware preemption determination and interrupt suppression logic diagram.

[0024] Before parsing the application layer protocol of the received multi-source heterogeneous data stream through the unified access interface, a predictive transport layer fragmentation shaping step is also included: a timing prediction engine based on the time-sensitive network arrival cycle is run inside the gateway; when it is predicted that the current timeline is entering the high probability arrival window of real-time urgent data, the maximum transmission unit splitting mechanism of the gateway network layer is actively intervened; non-real-time data that is in the network layer and whose total volume reaches the preset long frame threshold is physically cut and encapsulated into multiple micro independent Internet Protocol data packets in advance, and then sent down to the media access control layer for queuing and transmission.

[0025] The time-series prediction engine is constructed using a Long Short-Term Memory (LSTM) network. Its training and prediction process is as follows: During the initial operation phase, the gateway continuously monitors and records the absolute timestamps of periodic real-time emergency data arrivals via the underlying network interface card, building a historical time-series sample library. Based on this, the absolute timestamp sequence of real-time emergency data arriving at the gateway within the past T periods is used as the input feature of the LSM network; the actual arrival time window of the real-time emergency data in the (T+1)th period is used as the target label for supervised learning. In the prediction phase, the most recent T timestamps are input, and the model outputs the predicted arrival time of the emergency data in the next period; this predicted arrival time is pushed forward by a preset tolerance time (e.g., 1 millisecond) as the starting point of the high-probability arrival time window pre-stage.

[0026] Specifically, to reduce the physical waiting blind spots that may be caused by line congestion at the source, a predictive transport layer fragmentation shaping mechanism is deployed in advance at the front end of the unified access interface for receiving multi-source heterogeneous data streams. The gateway integrates a timing prediction engine based on time-sensitive network scheduling, which can learn and predict the microsecond-level time windows for the arrival of periodic emergency control commands. When the system clock approaches the high-probability arrival window of such real-time emergency data (e.g., 1 millisecond before the command is expected to arrive), the prediction engine issues an active intervention command to the operating system's network stack. Upon receiving the intervention command, the network layer temporarily modifies the maximum transmission unit (MTB) splitting strategy for non-real-time broadband streams (such as machine vision images). At this point, the image payload, originally packaged in 1500-byte blocks, is forcibly sliced ​​and encapsulated into multiple miniature independent Internet Protocol (IP) data packets containing only 128 or 256 bytes of payload, before being submitted to the media access control layer for queuing. This proactive physical slicing significantly shortens the exclusive period of a single non-real-time data frame on the underlying physical medium, reserving dense penetration gaps for upcoming emergency data.

[0027] This invention breaks through the limitations of passive preemption by using a timing prediction engine to proactively fragment massive non-real-time network layer packets into compliant micro-datagrams just before critical control commands are expected. This not only reduces the time of a single physical occupancy by several times from the source, but also significantly reduces the probability of triggering destructive underlying physical layer transmission errors and interruptions, thus achieving high-frequency congestion avoidance without changing the underlying physical specifications.

[0028] Furthermore, the application layer protocol of the received multi-source heterogeneous data stream is parsed, service semantic identifiers are extracted, and the heterogeneous data streams are classified into non-real-time data and real-time emergency data. This includes: stripping the network layer packet header and transport layer packet header of the multi-source heterogeneous data stream, reading the application layer payload data; performing string matching and comparison between the application layer payload data and a preset semantic feature matrix, extracting the successfully matched feature fields as service semantic identifiers; and querying a pre-configured service mapping table, which is divided into a first type of service tag set and a second type of service tag set.

[0029] The semantic feature matrix can be stored in the form of a two-dimensional array or a key-value mapping table. Each row (or each key) includes at least: a protocol type field, a target port number field, a feature keyword field, and a corresponding business tag field. The protocol type field identifies the application layer protocol category to which the data stream belongs; the target port number field identifies the destination port number of the data stream; the feature keyword field stores one or more semantic keywords that can be used for string matching; and the business tag field indicates the business semantic category corresponding to the feature, such as "real-time shutdown command," "machine vision image stream," or "ordinary log data." After reading the application layer load data, the parsing engine sequentially matches the protocol type field, the target port number field, and the feature keyword field. When a match is successful, the business tag field of the corresponding row is used as the business semantic identifier. The business mapping table can be implemented in the form of a hash table or an array. For each business semantic identifier, it records the first or second set of business tags to which it belongs, for subsequent division of the data stream into real-time emergency data with absolute time constraints or non-real-time data dependent on throughput. Those skilled in the art can extend or tailor the above fields according to actual application scenarios without departing from the spirit of the invention.

[0030] When the business semantic identifier belongs to the first type of business tag set, the corresponding data stream is determined to be real-time urgent data with absolute time constraints; when the business semantic identifier belongs to the second type of business tag set, the corresponding data stream is determined to be non-real-time data that depends on throughput.

[0031] When parsing the application layer protocol of the received multi-source heterogeneous data stream through the unified access interface and classifying the heterogeneous data stream into non-real-time data and real-time emergency data, the process also includes a pre-hardware tag injection step: further calculating the sub-priority of the non-real-time data based on the service attributes of the non-real-time data, mapping and encoding the sub-priority into a hardware priority tag, and pre-writing it into the media access control layer message header of the non-real-time data. Specifically, at the edge gateway's access processing front end, deep packet inspection technology is used to parse multi-source heterogeneous data streams layer by layer. First, the protocol parsing module, according to the network layer and transport layer protocol specifications, peels off the Internet Protocol header and Transmission Control Protocol or User Datagram Protocol header layer by layer to extract the original application layer payload message. Then, the built-in semantic feature matching engine is invoked to compare the key fields of the application layer payload message with a preset semantic feature matrix. For example, when a target port number of 9100 is matched and the message format conforms to the machine vision image transmission protocol standard, this feature is extracted as the first type of business semantic identifier; when parsing an industrial control protocol message and matching a specific register address or a specific emergency flag, this feature is extracted as the second type of business semantic identifier. The system has a pre-configured business mapping table that records the correspondence between various semantic identifiers and data transmission dimensions. The parsing engine queries the service mapping table and classifies the data streams with the first type of service semantic identifier as non-real-time data that depends on throughput scale. This type of data usually has a message length of up to the maximum transmission unit limit of 1500 bytes. At the same time, the data streams with the second type of service semantic identifier are classified as real-time urgent data with absolute time constraints. This type of data usually has a message length of about 64 bytes.

[0032] This invention overcomes the technical limitations of traditional gateways, which can only perform coarse-grained classification based on network layer addresses, by implementing deep service semantic parsing and classification during the access phase. This step provides accurate data attribute basis for subsequent fine-grained physical resource allocation and targeted scheduling, ensuring that the subsequent processing pipeline can accurately identify urgent services that need to be guaranteed and broadband services that can be delayed, laying the foundation for heterogeneous data classification and scheduling.

[0033] Furthermore, after completing the basic hardware and software service classification, the parsing engine also performs sub-priority evaluation on high-bandwidth packets classified as non-real-time data. For example, high-definition video streams from binocular machine vision are evaluated as high sub-priority, while ordinary device status log files are evaluated as low sub-priority. This sub-priority evaluation result is then mapped and encoded into a short virtual LAN priority code point label. Before the data packet is sent to the lower layer, the network card driver directly writes this hardware priority label into the Media Access Control (MAC) header of these non-real-time data packets. This pre-labeling operation successfully solidifies the complex upper-layer service semantics into binary physical level characteristics that the lower-layer hardware can instantly recognize, preparing data credentials in advance for subsequent nanosecond-level intelligent decision-making.

[0034] Further, the target distribution port is determined from the two physical network ports, and non-real-time data is dynamically distributed and sent. This includes: extracting the source Internet Protocol address, target Internet Protocol address, source port number, and target port number of the non-real-time data to form a data stream quintuple; querying a preset flow state mapping table at the hardware layer to determine whether the data stream quintuple is bound to a target network port; if it is, the non-real-time data is directly driven into the sending queue of the bound physical network port; if it is not, the current queue backlog byte length of the two physical network ports is extracted, the physical network port with the shorter backlog byte length is selected as the target distribution port, and the binding relationship between the data stream quintuple and the target distribution port is written into the flow state mapping table, and then the non-real-time data is driven into the sending queue of the target distribution port.

[0035] Specifically, for high-bandwidth packets classified as non-real-time data, in order to fully utilize the bandwidth of the dual physical network interfaces while strictly avoiding out-of-order transmission control protocol packets and congestion misjudgments caused by dynamic packet-by-packet scheduling at the underlying level, this solution adopts a session-keeping diversion mechanism based on a hardware flow state mapping table. The diversion decision module first extracts the source Internet Protocol address, destination Internet Protocol address, source port number, destination port number, and transport layer protocol type of the non-real-time data packet, forming a standard data flow 5-tuple. Then, at the hardware layer, a high-speed query of the preset flow state mapping table is performed using a hash index to determine whether the data flow 5-tuple exists and is bound to a target physical network interface. If the query is successful (i.e., a binding relationship exists, indicating that the packet belongs to an established active data flow), the diversion decision module directly drives the non-real-time data into the sending queue of the bound target physical network interface. This mechanism overcomes the interference of dynamic queue length and absolutely guarantees that the physical sending order of all subsequent packets within the same TCP data flow is consistent with the sending order at the source. If the query fails to find a match (i.e., the packet belongs to the first packet of a newly created data stream), the status monitoring module reads the current queue backlog byte lengths of the first and second physical network interfaces via the internal management bus. The hardware arbitrator performs a simple comparison operation and directly selects the physical network interface with the shorter current backlog byte length as the target distribution network interface for the newly created data stream. Next, the binding relationship between the data stream's 5-tuple and the selected target distribution network interface is written into the stream status mapping table, and then the first packet is driven into the sending queue of the target distribution network interface. Simultaneously, the stream status mapping table is configured with a hardware timer aging mechanism. When no new packet matches the 5-tuple of a data stream within a preset timeout period, the hardware logic automatically clears the entry to dynamically release table space for subsequent new streams.

[0036] This invention combines five-tuple flow state mapping with dynamic first-packet backlog assessment to achieve "flow-level" intelligent load balancing of non-real-time data across dual network interfaces. This mechanism effectively avoids the accumulation of new flows on a single network interface, fully releasing the potential of dual links. At the same time, it completely eliminates the packet out-of-order problem caused by traditional "packet-by-packet" scheduling from the bottom layer, avoiding fast retransmission and congestion-induced speed reduction at the receiving end, and significantly improving the effective throughput and long-term connection quality of high-bandwidth services under complex concurrent conditions.

[0037] Furthermore, at the hardware layer, real-time emergency data is mirrored and cloned, and the two clones are concurrently pushed to the media access control layer of the two physical network ports. This includes: intercepting real-time emergency data in a field-programmable gate array and writing its binary payload into a single storage area of ​​a dual-port random access memory; within one hardware clock cycle, generating two independent transfer descriptors pointing to the starting address of the storage area as clones through an internal direct memory access controller; and using the two independent read ports of the dual-port random access memory, injecting the two clones directly into the media access control layer reserved interface of the corresponding physical network port, bypassing the operating system kernel, through two mutually isolated hardware bus channels.

[0038] Real-time emergency data is intercepted in the field-programmable gate array (FPGA) and its binary payload is written into a single storage area of ​​the dual-port random access memory (DRAM). Within one hardware clock cycle, two independent transfer descriptors pointing to the starting address of the storage area are generated by the internal direct memory access controller (DMC), serving as two data clones. Using the two independent read ports of the DMC, the two clones are pushed directly into the media access control layer transmission ring of the corresponding physical network port through two isolated hardware bus channels, bypassing the operating system kernel.

[0039] Specifically, for high-priority messages identified as real-time urgent data, a pure hardware-level bypass mirroring mechanism is adopted to completely eliminate the processing latency caused by the CPU's soft interrupt response and the operating system kernel protocol stack. A field-programmable gate array (FPGA) module is deployed in the edge gateway's hardware architecture. When the service pipeline determines that the current message is real-time urgent data, it directly transfers the message's memory descriptor to this FPGA. The FPGA is internally configured with a high-speed dual-port random access memory (DRAM). Upon receiving the urgent message data stream, the FPGA's control logic drives a write enable signal, writing its binary payload into a single physical storage area of ​​the DRAM. Subsequently, within a single hardware clock cycle (e.g., only 8 nanoseconds at a 125 MHz operating frequency), the internal direct memory access controller generates two independent transfer descriptors, each pointing to the starting address of this storage area, serving as two data clones. Following this, two physically isolated high-speed peripheral component interconnect bus channels are established using the two independent read ports of the DRAM. The first clone (i.e., the data stream pulled by the first read port) is directly pushed into the highest priority transmission ring of the Media Access Control (MAC) layer of the first physical network port through the first bus channel. The second clone (i.e., the data stream pulled by the second read port) is synchronously pushed into the MAC layer of the second physical network port through the second bus channel. The entire cloning and pushing process does not go through the operating system's kernel queuing network stack at all.

[0040] This invention, by introducing a hardware pass-through cloning mechanism using a field-programmable gate array (FPGA) and a dual-port memory, completely breaks through the performance bottleneck of traditional gateways performing memory copying and protocol stack polling at the software level. This method significantly reduces the processing and mirroring time of emergency messages from the conventional hundreds of microseconds to the tens of nanoseconds, providing a solid underlying hardware foundation and multi-path physical redundancy guarantee for achieving ultra-low latency transmission of hard real-time data.

[0041] Further, the remaining transmission time of the non-real-time data being transmitted by each physical network port is calculated, including: reading the total byte length of the non-real-time data packet being transmitted from the descriptor ring of the media access control layer; polling the internal status registers of the underlying physical layer chips of the two physical network ports respectively through the internal management bus; extracting the transmission status counter value representing the number of physical symbols sent from the internal status register; performing a subtraction operation to subtract the transmission status counter value from the total byte length of the packet to obtain the remaining untransmitted byte length; querying and extracting the nominal link transmission rate values ​​of the two physical network ports; performing a division operation to divide the remaining untransmitted byte length of the two physical network ports by the corresponding nominal link transmission rate value to obtain the remaining transmission time of the non-real-time data being transmitted by the two physical network ports.

[0042] The system synchronously reads the transmit status counters of the underlying physical layer chips of the two physical network ports and calculates the remaining transmission time of the non-real-time data being transmitted by each physical network port. This includes: constructing a hardware counter inside the field-programmable gate array (FPGA); hard-wired the clock enable pin of the hardware counter to the transmit enable pin of the gigabit independent medium interface between the medium access control layer and the physical layer chip; when the transmit enable pin is in a high-level state, driving the hardware counter to increment by one byte of physical symbol value in each transmit clock cycle; obtaining the total byte length of the currently transmitted non-real-time data; and calculating the remaining transmission time accurate to the nanosecond level by subtracting the real-time value of the hardware counter from the total byte length.

[0043] Specifically, when the two emergency data clones arrive at the two Media Access Control (MAC) layers ready to transmit, they are highly likely to encounter a line-heading blockage where both physical ports are transmitting 1500-byte non-real-time long messages. To implement precise intervention, the slow-speed management data input / output bus polling method, which is prone to latency, is abandoned in favor of pure hardware-level nanosecond-level status monitoring. An independent high-frequency hardware counter is pre-built inside the gateway's field-programmable gate array (FPGA). The clock enable pin of this hardware counter is directly hardwired to the transmit enable pin (TX_EN pin) of the gigabit independent media interface between the MAC layer and the physical layer chip via physical traces on the printed circuit board. When non-real-time data is being transmitted and the TX_EN pin is high, the hardware logic automatically increments the counter by one byte of physical symbol value in each 125 MHz transmit clock cycle. The hardware arbiter first reads the total byte length of the current non-real-time packet from the media access control layer descriptor. Then, it performs a subtraction operation, subtracting the real-time transmitted byte value fed back by the hardware counter from this total byte length to calculate how many bytes of the current packet remain to be sent. Finally, combining this with the nominal transmission rate of the Gigabit Ethernet link, it accurately calculates the remaining transmission time required for the first and second physical network ports to send long packets. In the example hardware configuration using a Gigabit Ethernet GMII interface and a drive clock frequency of 125MHz, the resolution of this time calculation can be controlled within the nanosecond range with a basic step size of 8 nanoseconds.

[0044] This invention overcomes the latency and measurement errors caused by traditional low-speed management bus polling by physically hard-connecting a hardware counter directly to the transmit enable pin of the gigabit media independent interface. This technique enables real-time capture of the number of bytes sent within the clock cycle of the interaction between the media access control layer and the physical layer, providing nanosecond-level precision remaining time for targeted preemption decisions and effectively reducing intervention errors caused by status read lag.

[0045] It should be noted that the nanosecond-level measurement accuracy described in this embodiment is determined based on the standard operating frequency of the Gigabit Ethernet independent interface. Under the 125MHz hardware-driven clock used in this embodiment, the single clock cycle of the internal counter of the field-programmable gate array (FPGA) is 8 nanoseconds. Therefore, the measurement resolution of the remaining transmission time is based on 8 nanoseconds as the basic unit. By real-time accumulation and subtraction of the number of transmitted physical symbols, time determination is achieved on the order of 10 nanoseconds. This measurement method based on a hardware synchronous clock eliminates the microsecond-level uncertainty caused by software polling, ensuring the physical accuracy of the scheduling decision.

[0046] Furthermore, the weighted preemption score for each physical network interface is determined based on the remaining transmission time, including a weighted determination step based on hardware tags: while the transmission status counter of the physical layer chip is synchronously read at the media access control layer, the hardware priority tags in the media access control layer packet headers of the non-real-time data being transmitted by the two physical network interfaces are extracted in parallel within one hardware clock cycle using a field-programmable gate array; using hardware combinational logic circuits, the service weight coefficient mapped by the hardware priority tags is weighted and determined by the remaining transmission time to generate their respective weighted preemption scores; the weighted preemption scores of the two physical network interfaces are compared, and the physical network interface with the higher weighted preemption score is selected as the target preemption interface, and preemption avoidance protection is implemented for high-priority non-real-time data.

[0047] The business weighting coefficients can be pre-defined in the read-only memory or configuration register of a field-programmable gate array (FPGA) in the form of a lookup table. Different hardware priority tags correspond to different weighting coefficient values. In this embodiment, both the weighting coefficients and the time factor are preferably quantized to fixed-point numbers between 0 and 1, so that the hardware combinational logic circuit can perform simple multiplication or shift-addition operations to quickly generate a weighted preemption score. Those skilled in the art can adjust the specific values ​​of the above weight lookup table according to different business scenarios, or use other equivalent linear or piecewise linear function forms to implement weighted determination without affecting the basic principles of the present invention.

[0048] The specific logic of the weighted judgment operation is as follows: a baseline maximum transmission time constant is preset (e.g., 12000 nanoseconds for a 1500-byte message); the remaining transmission time is divided by the baseline maximum transmission time constant to obtain a dimensionless time factor with a value range between 0 and 1; then, the service weight coefficient mapped by the hardware priority label (also limited to a value range of 0 to 1, with lower service value resulting in higher weight) is multiplied with the time factor to obtain the final weighted preemption score.

[0049] Specifically, after determining the remaining transmission time of the two physical network ports to the nanosecond level, to prevent the loss of high-value image data due to blind interruptions, the probes inside the field-programmable gate array (FPGA) intercept the Media Access Control (MAC) headers of the non-real-time data being transmitted by the two physical network ports in parallel within the same transmission clock cycle, instantly extracting the previously written Virtual LAN Priority Code Point (VLAN) hardware tag. Subsequently, pure hardware combinational logic immediately intervenes, performing a weighted product operation on the service importance weight coefficient mapped to this hardware tag and the just-calculated remaining transmission time, generating a first weighted preemption score and a second weighted preemption score. For example, if the first network port has a longer remaining transmission time but is transmitting an extremely important video stream (with a very low weight coefficient), while the second network port has a slightly shorter remaining transmission time but is transmitting ordinary logs (with a very high weight coefficient), after weighted calculation, the second network port's weighted preemption score is actually higher. Finally, the hardware arbitrator selects the physical network port with the larger weighted preemption score as the target preemption port, thus successfully avoiding accidental preemption of high-priority non-real-time data while satisfying the minimum latency insertion requirement for urgent data.

[0050] After selecting a target network port to preempt, the process includes an interruption hardware overhead suppression step: The total overhead of the internal hardware pipeline switching cycle from triggering a physical layer transmission error interruption to clearing the Media Access Control (MAC) layer queue and allowing new data transmission is pre-calculated and fixed; a difference judgment is performed, comparing the calculated remaining transmission time with the total overhead of the internal hardware pipeline switching cycle; if the remaining transmission time is less than the total overhead, the triggering action of the physical layer transmission error interruption signal is bypassed, allowing the non-real-time data currently being transmitted by the target network port to be transmitted smoothly and naturally, and after strictly meeting the Ethernet standard frame gap requirements, the prepared emergency data clone is immediately transmitted with the minimum legal physical interval. This mechanism effectively avoids the reverse latency penalty caused by hardware reset, while ensuring the protocol compliance of the physical layer waveform. If the remaining transmission time is greater than or equal to the total overhead of the internal hardware pipeline switching cycle, a physical layer transmission error interruption signal is triggered to the underlying physical layer chip of the target network port.

[0051] Specifically, after selecting a target to preempt the network port, the hardware arbiter does not blindly issue interrupt commands, but instead enters a stringent interrupt overhead suppression logic. Internally, the system pre-calibrates and solidifies the time required for the field-programmable gate array to execute a series of state machine transitions, i.e., the total overhead of the internal hardware pipeline switching cycle required from triggering the dedicated pin for transmission errors, issuing a queue reset command to clear the buffer, to finally pushing the clone into the physical cable (e.g., solidified to 30 nanoseconds). The hardware arbiter performs high-speed differential discrimination, comparing the remaining transmission time precisely calculated in the previous steps with this 30 nanosecond internal overhead in real time. If it is found that the non-real-time packet currently being transmitted by the target preempting the network port is nearing its end, with only 15 nanoseconds remaining (strictly less than the internal overhead), the hardware arbiter will immediately bypass and block the trigger command sent to the interrupt generator. At this time, the remaining 15 nanoseconds of non-real-time data are allowed to be transmitted smoothly and naturally, and after meeting the standard frame interval requirements, the prepared emergency data clone is immediately sent. This mechanism effectively avoids the reverse latency penalty caused by hardware reset.

[0052] Physical hardware inevitably incurs inherent physical clock cycle overhead when executing actions such as pulling up dedicated pins for transmission errors, issuing queue reset commands, and redirecting memory pointers. When only a few bytes of a long message remain to be sent, blindly executing an interrupt will result in a pipeline recovery time that is longer than the time it would take for the message to be sent naturally. This solution intelligently suppresses "unnecessary interruption overhead" through nanosecond-level difference comparison logic, ensuring that the gateway can select the shorter physical transmission path based on real-time calculations in any micro-edge scenario. Under given network configuration and service load conditions, it compresses the queuing time and jitter of emergency commands to an engineering-acceptable microsecond range, thereby significantly improving hard real-time assurance capabilities.

[0053] Furthermore, a physical layer transmission error interrupt signal is triggered to the underlying physical layer chip of the target preempted network port, intervening at the physical level to terminate the non-real-time data currently being transmitted by the target preempted network port. This includes: activating the hardware interrupt generator corresponding to the target preempted network port; sending a trigger level control command to the media-independent interface of the target preempted network port; driving the transmission error-dedicated pin of the underlying physical layer chip to be continuously in the trigger level state; forcing the underlying physical layer chip to reverse the polarity of the frame check sequence in the current physical symbol stream and inject error-coded symbols into it; outputting damaged physical layer fragments to the physical cable; and completing the intervention termination action for non-real-time data. The remaining non-real-time data terminated by intervention is cleared from the media access control layer queue, and the corresponding clone is pushed into the physical layer chip for priority transmission.

[0054] Triggering a physical layer transmission error interrupt signal to the underlying physical layer chip forcibly terminates the transmission of current data on the network interface at a physical level, including: controlling the duration of the transmission error-specific pin of the physical layer chip to be high at the trigger level through a hardware watchdog mechanism; limiting the duration to a continuous range of four to eight transmission clock cycles, and then restoring the transmission error-specific pin to a low level; injecting error-coded symbols into the currently transmitted non-real-time data during the duration to generate corrupted fragment frames, while maintaining the continuous connection state of the physical layer chip.

[0055] Specifically, if, based on the aforementioned difference determination, the remaining transmission time of the target preempted network port is found to be greater than or equal to the total overhead of the internal hardware pipeline switching cycle (i.e., it is determined to have the benefit of physical interruption), then the hardware interrupt triggering process is formally initiated. Once the target preempted network port is locked based on the aforementioned weighted determination logic, the hardware interrupt generator immediately sends a command to the gigabit media independent interface of that network port, forcibly pulling the level of its dedicated transmission error pin (TX_ER pin) to a high-level trigger state. At this time, the built-in hardware watchdog mechanism is activated to strictly control the triggering behavior, precisely limiting the duration of the TX_ER pin maintaining a high level within the range of 4 to 8 consecutive transmission clock cycles, and then immediately restoring it to the default low-level state. During this brief triggering cycle, the physical layer chip injects specific error-coded symbols into the cable in the currently outputting physical symbol stream, artificially outputting an incomplete and corrupted fragment frame, effectively interrupting the physical transmission process of the long message. At the same time, this limited-cycle assertion avoids automatic negotiation crashes within the triggering chip, maintaining the active state of the physical link connection. Furthermore, within the same microprocessor clock cycle that triggers the physical layer transmission error interrupt signal, the hardware control logic synchronously sends a highest-priority queue reset command to the gateway device's media access control layer driver controller. Upon receiving this reset command, the direct memory access controller immediately and forcibly clears all subsequent descriptors of the non-real-time long packet that have not yet been fully transferred to the physical layer from its internal buffer FIFO, effectively preventing the accumulation of subsequent invalid data. Immediately afterwards, the direct memory access controller's memory start address read pointer is instantly redirected to the real-time emergency data buffer that has already been cloned in the previous steps. This allows the blocked emergency data clones to bypass the remaining queue debris and be seamlessly pushed into the physical cable for rapid transmission once the physical transmission channel is cleared.

[0056] This invention effectively solves the problem of underlying physical link disconnection and renegotiation downtime that may be caused by conventional continuous blocking by introducing a hardware watchdog mechanism to limit the trigger level of the transmission error pin to a specific micro-cycle range. While ensuring successful injection of error symbols and termination of non-real-time long packet transmission, it maintains the stable working state of the physical layer chip, achieving minimally invasive data blocking and improving the overall reliability of the network link. By issuing a queue reset command to the memory direct access controller at the same time as triggering the physical layer error interruption, a full-link clearing channel from the upper-layer memory buffer to the lower-layer physical level transmission is opened, effectively clearing the remaining non-real-time data fragments in the media access control layer queue, so that the transmission pipeline of the target preemptive network port can be quickly zeroed out, ensuring that the subsequent real-time emergency data clones can seamlessly enter the transmission stream, guaranteeing hard real-time preemption scheduling.

[0057] Furthermore, after terminating the non-real-time data currently being transmitted by the target preempted network port, a hardware proxy partial retransmission step for the non-real-time data is also included: In the static random access memory (SRAM) inside the field-programmable gate array (FPGA), the non-real-time data pushed into the media access control (MAC) layer queue of the target preempted network port is mirrored and buffered; after the physical layer transmission error interrupt signal is triggered and the corresponding clone has been sent first, the new data is blocked from being sent by the operating system kernel; the complete frame of the non-real-time data that was terminated and buffered in the SRAM is re-injected into the MAC layer transmission ring of the target preempted network port, and a transparent physical layer retransmission is performed.

[0058] Specifically, for non-real-time data that is physically terminated at the target preemptive network interface, a pure hardware-level low-level proxy caching and transparent retransmission mechanism is adopted. When the non-real-time data is initially pushed into the media access control layer queue of the target preemptive network interface, the direct memory access controller inside the field-programmable gate array (FPGA) will simultaneously mirror its original complete data frame losslessly to the built-in SRAM (static random access memory) for preservation. When the transmission process of this non-real-time data is forcibly interrupted by a physical layer transmission error, and the clone of the real-time urgent data successfully completes priority queue insertion, the gateway's hardware arbitrator will immediately pull up a blocking signal on the internal control bus, temporarily blocking the operating system kernel network stack from sending any new regular data to the media access control layer.

[0059] Utilizing this microsecond-level transmission gap, the Direct Memory Access Controller (DMI) instantly retrieves the complete original frame of the non-real-time data that was interrupted and cached in SRAM, and directly injects it into the Media Access Control (MAC) layer transmission loop of the target's preempted network interface. Since this retransmission is completed entirely within a closed loop at the underlying network card driver and physical hardware level, the target receiving device will ultimately receive a complete and error-free non-real-time data frame. This mechanism achieves seamless retransmission at the lower level, successfully avoiding the hundreds of milliseconds of protocol stack waiting and retransmission overhead at the source device.

[0060] This invention utilizes the high-speed static random access memory within a field-programmable gate array (FPGA) to construct a hardware proxy caching mechanism, forming a low-level local self-healing closed loop that transcends software and hardware boundaries. By transforming the lengthy and uncontrollable upper-layer network packet loss retransmission into microsecond-level low-level hardware in-situ retransmission, it minimizes the recovery cost after non-real-time data is preempted, while eliminating the 'line-head blocking' delay of emergency control commands. This ensures the data integrity of high-bandwidth services and achieves a highly efficient balance between communication timeliness and overall network throughput.

[0061] Furthermore, after prioritizing the sending of the corresponding clones, a spatial deduplication step for the cloned messages is also included: Before pushing the clones, the field-programmable gate array (FPGA) inserts a custom redundant control header containing a timestamp and a unique sequence number before the Ethernet payload of the real-time emergency data; the target receiving device listens for and receives messages arriving from the corresponding links of the two physical network interfaces; the target receiving device extracts the sequence number and timestamp parameters of the first arriving message on the timeline, records them in the local doubly linked list memory, and submits the first arriving message to the application layer to perform an emergency shutdown action; the sequence number of subsequent arriving messages is continuously compared with the sequence number recorded in the local doubly linked list memory; when the sequence numbers match, the subsequent arriving messages are determined to be redundant cloned messages; the target receiving device executes a silent discard instruction on the redundant cloned messages at the network card driver level to eliminate the risk of service state machine re-entry anomalies caused by duplicate control messages.

[0062] Before concurrently pushing clones, the field-programmable gate array (FPGA) uses a hardware protocol encapsulation engine to insert a custom redundant control header before the Ethernet load of real-time emergency data. The redundant control header contains a nanosecond-level timestamp generated by gateway clock synchronization and a unique sequence number generated by a monotonically increasing hardware counter.

[0063] Specifically, because the gateway implements concurrent push of real-time emergency data via dual physical network ports at the hardware layer, the two corresponding physical ports of the target receiving device (such as the servo drive controlling the robotic arm) will receive two identical emergency instruction clone messages sequentially. When the network card driver of the target receiving device captures the first clone message arriving on the timeline (assuming this message was sent by the network port that performed physical preemption and arrived first), it immediately extracts the unique sequence number parameter and the timestamp parameter carried in the application layer message header. The driver quickly records the sequence number and timestamp in a doubly linked list status table maintained in local memory and immediately submits the instruction content of the first message to the application layer control module, driving the servo motor to immediately perform a physical-level emergency stop. After a small time difference (e.g., 12 microseconds later, after the other link that was not physically preempted has finished queuing), the second clone message arrives late at the other network card of the target receiving device. The target network card driver also extracts the sequence number parameter of this subsequent message and uses a hash lookup algorithm to traverse and compare it with the sequence number already recorded in the local doubly linked list status table. When the comparison logic identifies that the sequence number is exactly the same as the sequence number just registered in the table, the subsequent message is determined to be a redundant clone message. The network interface card driver layer of the target device will immediately execute a silent discard command on the redundant message and will no longer submit it to the application layer.

[0064] This invention effectively eliminates redundant control commands caused by the gateway's dual-transmission mechanism by deploying a nanosecond-level comparison and deduplication mechanism based on the initial sequence number at the target receiving device. This method eliminates the risk of service state machine re-entry anomalies caused by repeatedly executing the same urgent commands from a spatial perspective, ensuring that the terminal execution device maintains the absolute uniqueness and security of its control logic while obtaining extremely low-latency commands.

[0065] This embodiment shortens the physical link occupancy period and endows the underlying scheduling service with awareness capabilities by pre-fragmentation shaping and hardware tag injection. Combined with weighted preemption decision-making and interruption hard overhead suppression mechanisms, it accurately identifies and protects high-value data, effectively avoids useless interruption delays, and ensures deterministic transmission of urgent instructions at the microscale. In conjunction with the local self-healing mechanism of hardware proxy caching, it significantly reduces data retransmission costs while ensuring hard real-time penetration, thereby improving the overall communication efficiency and stability in heterogeneous network environments.

[0066] Example 2: This embodiment aims to elaborate on the complete implementation process of a unified access method for heterogeneous data sources, focusing on the technical solution process. It emphasizes how hardware-level scheduling and software business semantics can deeply collaborate in complex industrial scenarios to completely eliminate the "head-of-line blocking" effect of the underlying physical links. This embodiment relies on an industrial edge gateway architecture configured with dual gigabit physical network ports and a high-performance field-programmable gate array (FPGA).

[0067] As one embodiment of the present invention, refer to Figure 1 A flowchart of a unified access method for heterogeneous data sources, refer to Figure 2 The functional module structure diagram of the industrial edge gateway is shown in the reference diagram. Figure 3 Hardware preemption determination and interrupt suppression logic diagram.

[0068] In the initial stage of the data processing flow, the gateway receives heterogeneous data sources from different levels in real time through a unified access interface, such as high-bandwidth machine vision data and extremely low-latency control commands. To achieve a shift from passive preemption to proactive avoidance, this embodiment first runs a predictive transport layer fragmentation shaping mechanism. The system's internal time-sensitive network timing prediction engine accurately predicts the high-probability arrival time window of the next cycle's urgent real-time data through deep learning of periodic real-time tasks. In the pre-prediction stage of the predicted time window (e.g., 1 millisecond in advance), the engine actively intervenes in the MTU (Maximum Transmission Unit) splitting strategy of the operating system's network stack. At this time, the non-real-time long data frames originally encapsulated according to the 1500-byte standard are forcibly cut into multiple micro data packets of less than 256 bytes. This pre-shaping action shortens the maximum time base occupied by a single physical link from the source, reserving dense physical gaps for the penetration of subsequent urgent commands, effectively reducing the probability of triggering hardware interruptions.

[0069] Next, the process moves to deep packet inspection and semantic classification. The protocol parsing module separates the network and transport layer headers of the heterogeneous data streams, extracting the application layer payload. The semantic feature matching engine compares the extracted fields with a preset feature matrix (such as industrial control register addresses or specific protocol flags) to accurately classify the data into real-time emergency data (such as shutdown instructions) and non-real-time data (such as quality inspection messages). In this crucial step, this embodiment of the invention performs a pre-implementation hardware tag injection. Based on the business importance of non-real-time data, its sub-priority is further calculated and mapped to a short VLAN priority code point tag, which is then directly written into the media access control layer header of the packet by the driver. This action transforms complex application layer business logic into physical features that can be instantly extracted by the underlying hardware, laying the data foundation for subsequent weighted decision-making.

[0070] During the routing phase, the gateway extracts the five-tuple of non-real-time data and queries the hardware-level flow state mapping table. For newly established data flows, it dynamically binds them to the less loaded physical egress based on the current backlog byte length of the two physical network interfaces. For existing data flows, it strictly follows the binding relationship in the flow state mapping table for forwarding, thereby achieving load balancing between the two network interfaces and physically preventing out-of-order TCP packets. Meanwhile, real-time urgent data is processed by the FPGA module. The FPGA uses its internal dual-port RAM to generate two absolutely identical clones within a single clock cycle and, through two isolated high-speed hardware buses, bypasses the operating system kernel protocol stack, directly injecting them into the highest priority transmission ring of the MAC (Media Access Control) layer of the first and second physical network interfaces.

[0071] When a clone of real-time emergency data arrives at the MAC layer in the ready-to-send state, if both network ports' physical links are occupied by non-real-time data, nanosecond-level status synchronization reading is immediately initiated. The FPGA's internal hardware counter, hardwired to the physical layer chip's transmit enable pin, captures the number of bytes sent in real time. Based on the total message length and gigabit transmission rate, it precisely calculates how many nanoseconds remain before each of the two network ports' long packets are completed, with the calculation accuracy strictly locked at the nanosecond level.

[0072] Based on the aforementioned calculation data, targeted decision-making based on weighted preemption scoring is executed. The FPGA probe extracts the PCP hardware tags from the headers of the packets being transmitted by the two network interfaces in parallel within the same transmission cycle, and performs a weighted product operation between the mapped service weight coefficient and the calculated remaining transmission time. For example, although network interface A has a longer remaining time, it transmits highly valuable visual keyframes (very low weight), while network interface B has a shorter remaining time but transmits ordinary logs (very high weight). After weighted calculation, network interface B has a higher preemption decision score and is thus selected as the target preemption network interface. This mechanism cleverly avoids accidental blocking of high-value services.

[0073] Before executing the interrupt action, a further interrupt hardware overhead suppression judgment is performed. The hardware logic compares the remaining transmission time of the target preempting the network interface with the preset total pipeline switching overhead (e.g., 30 nanoseconds). If the remaining time is less than 30 nanoseconds, it is determined that interrupting at this time is slower than naturally completing the transmission, and a bypass shield is triggered, allowing the packet to be transmitted naturally and then an emergency clone is appended. Conversely, if the remaining time is greater than 30 nanoseconds, it is determined that there is a benefit to physical interruption, and the hardware interrupt triggering process is formally initiated.

[0074] During the triggering process, the hardware interrupt generator drives the dedicated error pin of the target network port physical layer chip to go high for 4 to 8 consecutive clock cycles, forcibly injecting error-coded symbols into the physical cable, forming a physical fragment frame and terminating the non-real-time data transmission process. Simultaneously, the FPGA issues a queue reset instruction to clear the residual data in the MAC layer and instantly redirects the memory start address pointer to the emergency data buffer, enabling the clone to be transmitted at high speed with seamless transition.

[0075] In the post-healing stage, to further improve system reliability, this embodiment employs a proxy cache local self-healing mechanism. The large-capacity SRAM inside the FPGA has performed a complete mirror backup of the preempted non-real-time frames. Within the microsecond interval after the emergency data transmission is completed, the FPGA blocks the kernel from sending new data and directly re-injects the original frame from the SRAM into the transmission ring, achieving seamless retransmission at the underlying level, replacing the TCP timeout retransmission that lasts for hundreds of milliseconds. At the target receiving device, by extracting the sequence number of the initial packet and recording it in a local doubly linked list, subsequent redundant cloned packets are silently discarded, effectively preventing the risk of re-entry into the control logic.

[0076] This embodiment solves the core problem of "head-of-line blocking" at the physical level through the above-mentioned closed-loop operation of the entire chain from pre-shaping, semantic tagging, cloning redundancy to targeted preemption and local self-healing, and realizes efficient integration and deterministic scheduling of IT data and OT data in industrial edge scenarios.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for unified access to heterogeneous data sources, characterized in that, An industrial edge gateway configured with a first physical network port and a second physical network port, including: The application layer protocol of the received multi-source heterogeneous data stream is parsed through a unified access interface, business semantic identifiers are extracted, and heterogeneous data streams are classified into non-real-time data and real-time emergency data. The target distribution port is determined from the two physical network ports, and non-real-time data is dynamically distributed and sent. At the same time, real-time emergency data is mirrored and cloned at the hardware layer, and the two clones are pushed concurrently to the media access control layer of the two physical network ports. When the clone reaches the media access control layer, it synchronously reads the transmission status counters of the underlying physical layer chips of the two physical network ports and calculates the remaining transmission time of the non-real-time data being transmitted by each physical network port. Based on the remaining transmission time, the weighted preemption score of each physical network interface is determined, and the physical network interface with the highest weighted preemption score is selected as the target preemption network interface. A physical layer transmission error interrupt signal is triggered to the underlying physical layer chip of the target preemption network interface, forcibly terminating the non-real-time data currently being transmitted by the target preemption network interface at the physical level, and prioritizing the transmission of the corresponding clone. At the same time, the transmission state of the other physical network interface remains unchanged, and the corresponding clone is transmitted after the current data frame is completed.

2. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, The process involves parsing the application layer protocol of received multi-source heterogeneous data streams, extracting business semantic identifiers, and classifying the heterogeneous data streams into non-real-time data and real-time emergency data. This includes: stripping the network layer and transport layer headers of the multi-source heterogeneous data streams and reading the application layer payload data; performing string matching and comparison between the application layer payload data and a preset semantic feature matrix, extracting successfully matched feature fields as business semantic identifiers; and querying a pre-configured business mapping table to divide the data streams corresponding to the business semantic identifiers into real-time emergency data and non-real-time data.

3. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, The process involves determining the target distribution port from two physical network interfaces and dynamically distributing non-real-time data. This includes extracting the source Internet Protocol address, target Internet Protocol address, source port number, and target port number of the non-real-time data to form a data stream quintuple; querying a pre-defined flow state mapping table at the hardware layer to determine if the data stream quintuple is bound to a target network interface; if it is, directly driving the non-real-time data into the transmission queue of the bound physical network interface; if it is not, extracting the current queue backlog byte length of the two physical network interfaces, selecting the physical network interface with the shorter backlog byte length as the target distribution port, writing the binding relationship between the data stream quintuple and the target distribution port into the flow state mapping table, and then driving the non-real-time data into the transmission queue of the target distribution port.

4. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, The process involves mirroring and cloning real-time emergency data, and concurrently pushing the two clones to the media access control layer of two physical network ports. This includes: intercepting real-time emergency data in a field-programmable gate array (FPGA) and writing its binary payload into a single storage area of ​​a dual-port random access memory (DRAM); generating two independent transfer descriptors pointing to the starting address of the storage area as clones within one hardware clock cycle using an internal direct memory access controller (DRAM); and using the two independent read ports of the DRAM, injecting the two clones directly into the media access control layer reserved interface of the corresponding physical network port, bypassing the operating system kernel, through two isolated hardware bus channels.

5. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, The remaining transmission time of the non-real-time data being transmitted by each physical network port is calculated, including: reading the total byte length of the non-real-time data packet being transmitted from the descriptor ring of the media access control layer; polling the internal status registers of the underlying physical layer chips of each of the two physical network ports through the internal management bus; extracting the transmission status counter value representing the number of physical symbols sent from the internal status register; performing a subtraction operation to subtract the transmission status counter value from the total byte length of the packet to obtain the remaining untransmitted byte length; querying and extracting the nominal link transmission rate value of the two physical network ports; and performing a division operation to divide the remaining untransmitted byte length of the two physical network ports by the corresponding nominal link transmission rate value to obtain the remaining transmission time of the non-real-time data being transmitted by the two physical network ports.

6. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, The weighted preemption score for each physical network interface is determined based on the remaining transmission time, including a weighted judgment step based on hardware tags: while the transmission status counter of the physical layer chip is read synchronously at the media access control layer, the hardware priority tags in the media access control layer packet headers of the non-real-time data being transmitted by the two physical network interfaces are extracted in parallel within one hardware clock cycle using a field-programmable gate array; the service weight coefficient mapped by the hardware priority tag is weighted and judged by the remaining transmission time using hardware combinational logic circuits to generate their respective weighted preemption scores; the weighted preemption scores of the two physical network interfaces are compared, and the physical network interface with the higher weighted preemption score is selected as the target preemption network interface.

7. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, Triggering a physical layer transmission error interrupt signal, the system intervenes at the physical level to terminate the non-real-time data currently being transmitted by the target preempted network port. This includes: activating the hardware interrupt generator corresponding to the target preempted network port; sending a trigger level control command to the media-independent interface of the target preempted network port; driving the dedicated transmission error pin of the underlying physical layer chip to be continuously in the trigger level state; forcing the underlying physical layer chip to reverse the polarity of the frame check sequence in the current physical symbol stream and inject error-coded symbols; outputting damaged physical layer fragments to the physical cable; and completing the intervention termination action for non-real-time data. The system also clears the residual non-real-time data that has been terminated from the media access control layer queue and pushes the corresponding clone into the physical layer chip for priority transmission.

8. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, After terminating the non-real-time data currently being transmitted by the target preempted network port, the process also includes a hardware proxy partial retransmission step for the non-real-time data: In the static random access memory (SRAM) inside the field-programmable gate array (FPGA), the non-real-time data pushed into the media access control (MAC) layer queue of the target preempted network port is mirrored and buffered; after a physical layer transmission error interruption signal is triggered and the corresponding clone has finished transmitting, the system kernel is blocked from sending new data; the complete frame of the non-real-time data that was terminated and buffered in the SRAM is re-injected into the MAC layer transmission ring of the target preempted network port, and a transparent physical layer retransmission is performed.

9. The method for unified access to heterogeneous data sources according to claim 1, characterized in that, After sending the corresponding clones first, there is also a spatial deduplication step for the cloned messages: the target receiving device listens for and receives messages arriving from the corresponding links of the two physical network ports; The target receiving device extracts the sequence number and timestamp parameters of the first arriving message on the timeline, records them in a local doubly linked list in memory, and submits the first arriving message to the application layer to perform an emergency shutdown action; it continuously compares the sequence number of subsequent arriving messages with the sequence number recorded in the local doubly linked list in memory; when the sequence number matches, it determines that the subsequent arriving message is a redundant clone message; the target receiving device executes a silent discard instruction on the redundant clone message at the network card driver level to eliminate the risk of service state machine re-entry anomaly caused by duplicate control messages.

Images