Multi-scene fusion open platform architecture data interaction system for heterogeneous systems

Abstract

The application provides a multi-scene fusion open platform architecture data interaction system for a heterogeneous system, comprising an edge multi-mode connector suite, a unified data bus, a multi-task neural network engine and a target system adaptation layer, a large model assisted protocol adaptation automatic parsing robot private message is used, and real-time video is collected through an industrial vision module; the data bus provides event-driven time sequence-document-stream mixed transmission; the neural network engine introduces a graph entropy modulation activation function of line load rate and semantic confidence, encodes multi-modal equipment data and visual features, and outputs semantic mapping, load prediction and control instructions; the adaptation layer pushes the instructions to MES, WMS and quality detection SaaS platforms to form a millisecond level closed loop. The technical scheme of the application realizes zero code access of a private protocol, semantic unification, adaptive scheduling and defect detection, the data compression rate and defect recall rate are greatly improved, and the application has traceable safety.

Description

Technical Field

[0001] This invention relates to the field of data interaction technology, specifically to a data interaction system based on a multi-scenario fusion open platform architecture for heterogeneous systems. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence and digital twin factories, the manufacturing industry is moving from single-point automation to end-to-end intelligent collaboration. A typical production line often deploys PLC / robot controllers, MES (Manufacturing Execution System), WMS (Warehouse Management System), visual quality inspection stations, energy consumption meters, and third-party quality inspection SaaS services simultaneously. These devices and systems are highly heterogeneous in terms of communication protocols, data models, time bases, and security policies: robot controllers often use proprietary message systems based on serial ports or Ethernet; visual quality inspection stations output high-throughput video streams, relying on GPU acceleration; and MES and WMS exchange semi-structured documents using application layer protocols such as HTTP / AMQP / OPC-UA. Traditionally, Enterprise Service Bus (ESB) or industrial gateways are hard-coded for "point-to-point" adaptation, but with the surge in the number of scenarios and data dimensions, this approach has revealed the following shortcomings:

[0003] High interface coupling: Once business requirements change, multiple modifications are required to the ESB script, gateway driver, and database mapping, with an average revision cycle of more than two weeks.

[0004] Semantic silos are severe: the same concept has multiple field names in different systems. For example, "batch number" can be written as BatchID, LotNo, CobotWorkpieceID, etc., which makes data governance and querying difficult.

[0005] Insufficient real-time performance: Heterogeneous systems often need to be stored in the warehouse first and then synchronized. The closed-loop delay of robot-vision-execution scheduling can reach 3-5 seconds, which does not meet the sub-second response requirements of high-speed production lines.

[0006] High expansion costs: When introducing new vision algorithms or third-party SaaS quality inspection models, it is necessary to redeploy GPU gateways and add temporary database tables, resulting in repeated investment in hardware and manpower.

[0007] Security and compliance risks: Transmitting large amounts of process data across network segments makes it difficult for traditional firewalls + VPNs to provide fine-grained access control and fails to meet the requirements of zero-trust architecture and traceable auditing of links.

[0008] To address these pain points, the industry has proposed improvement solutions such as OPC-UAPubSub, MQTT streaming media, and unified time-series database acquisition. However, these solutions mainly focus on protocol-level interoperability and still lack semantic-level fusion and scenario-level decision output. At the AI ​​application level, some research uses convolutional neural networks to detect defects and utilizes Transformers for cross-modal alignment, but most focus on single tasks and fail to unify equipment status, visual features, and scheduling decisions into a single neural network framework. Meanwhile, while current generative large models have shown potential in areas such as automatic code generation and protocol reverse engineering, a complete edge adaptation mechanism for automatic parsing and real-time deployment of industrial proprietary protocols has not yet been established. Furthermore, existing industrial AI models generally use common activation functions such as ReLU, Leaky-ReLU, and GELU. The thresholds or curve shapes of these activation functions are fixed after training, making it difficult to adaptively adjust to the dynamic fluctuations of production line operation. On flexible production lines, the level of workstation congestion, rolling load occupancy, and real-time lighting and texture complexity can change drastically on a minute or even second basis. If static activation functions are still used, two types of problems often arise: "Threshold truncation" in the visual branch: In highly congested or unevenly lit scenarios, the contrast of defective pixels decreases, and a large portion of the negative half-region is truncated, leading to a decrease in recall. "Overfitting to a single working condition" in the decision branch: It performs well near the training data distribution, but when the real-time load exceeds the training quantile interval, gradient saturation or excessive size occurs, and the scheduling strategy output becomes unstable.

[0009] Therefore, there is an urgent need for an open platform architecture that can atomically access multi-source heterogeneous data, complete semantic mapping at the edge-cloud collaborative level, output load prediction and scheduling instructions in an integrated manner, and achieve millisecond-level closed loop through a unified data bus, so as to support rapid line changeover of flexible production lines, collaborative operation of visual robots, and reliable sharing of cross-enterprise data, and provide a core digital foundation for the sustainable evolution of future smart factories. Summary of the Invention

[0010] To address the aforementioned problems in existing technologies, this invention proposes a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems. The system includes an edge multi-mode connector kit, a unified data bus, a multi-task neural network engine, and a target system adaptation layer. It utilizes a large model to assist in protocol adaptation and automatic parsing of robot private messages, and acquires real-time video through an industrial vision module. The data bus provides event-driven time-series-document-stream hybrid transmission. The neural network engine introduces a spectral entropy modulation activation function that integrates production line load rate and semantic confidence to perform multimodal encoding of equipment data and visual features, outputting semantic mapping, load prediction, and control commands. The adaptation layer pushes commands to MES, WMS, and quality inspection SaaS platforms, forming a millisecond-level closed loop. This solution achieves zero-code access to private protocols, semantic unification, adaptive scheduling, and defect detection. During testing, data compression rate was improved by 60%, defect recall rate was increased, and traceability and security were ensured.

[0011] This application provides a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems, including:

[0012] The edge multimode connector kit is used to connect interfaces to a unified data bus and preprocess multi-source data from the production line, including: a large model auxiliary protocol adaptation module, which automatically parses the robot equipment's private communication protocol and converts equipment status data into source data streams; and an industrial vision acquisition module, which acquires real-time video images of workpieces and outputs video detection feature streams based on the first convolutional neural network model.

[0013] A unified data bus is used to receive the source data stream and video detection feature stream in an event-driven manner, and to provide time-series-document-stream hybrid message transmission;

[0014] The multi-task neural network engine is set up on the message forwarding path between the unified data bus and the target system adaptation layer. It performs multimodal encoding on the received source data stream and video detection feature stream, generates shared semantic expression data, and outputs semantic mapping results and control command events.

[0015] The target system adaptation layer is used to push the control command events to MES, WMS, and third-party quality inspection SaaS platform respectively, so as to realize production scheduling adjustment and robot motion reprogramming;

[0016] The edge multi-mode connector kit, unified data bus, multi-task neural network engine and target system adaptation layer form a data closed loop in sequence and are interconnected through a service mesh to achieve integrated real-time collaboration based on semantic mapping and scheduling instruction generation of a single multi-task neural network.

[0017] Preferably, the step of connecting the interface to the unified data bus and preprocessing the multi-source data of the production line includes connecting the data interface of the robot equipment controller and the industrial vision acquisition device to the unified data bus; the multi-source data of the production line includes: the device status message byte stream sent by the robot equipment through a private communication protocol and the real-time video frame sequence of the workpiece obtained by the industrial vision acquisition module.

[0018] Preferably, the preprocessing includes: for the device status message byte stream, firstly, the statistical boundary inference unit generates candidate fields for segmentation based on byte position entropy and length clustering; then, the large model-assisted protocol adaptation module calls the generative language model output interface description file with a single 8-bit byte as the smallest input granularity and automatically generates parser code; subsequently, in the parsing execution unit, the message byte stream is parsed into a platform-unified JSON field format, and incremental compression based on byte difference is performed on continuous messages; and for the real-time video frame sequence of the workpiece, firstly, histogram equalization is performed to enhance the contrast of the defect area, then inter-frame difference is performed to remove static background frames, and then the video detection feature stream is output after processing by the first convolutional neural network model.

[0019] Preferably, the large model-assisted protocol adaptation module includes: a statistical boundary inference unit, used to perform field boundary candidate generation based on byte position entropy, length clustering and byte repetition period analysis on the captured private protocol message set; a generative model inference unit, which calls a pre-trained generative model with a single 8-bit byte as the smallest input granularity, and generates an interface description file and parser code based on the candidate boundaries and message samples; and a parsing execution unit, used to decode real-time messages into platform source data streams using the parser code.

[0020] Preferably, the unified data bus includes an event router, a timing channel, a document channel, and a streaming channel, and is configured to: generate an event encapsulation with a 64-bit monotonically increasing sequence number and a timestamp for each message from the source data stream or video detection feature stream; route metric fields with a sampling frequency of not less than 1Hz to the timing channel and store them; route semi-structured JSON or Avro format device status documents to the document channel and provide request-response access; push Protobuf frames of the video detection feature stream extracted by the first convolutional neural network model with a length prefix encoding to the streaming channel; and the event router retains the monotonically increasing sequence number and timestamp when forwarding across channels.

[0021] Preferably, the multi-task neural network engine performs multimodal encoding on the source data stream and video detection feature stream to generate shared semantic representation data, including: first, inputting the device state field sequence of the source data stream from the robot equipment into a temporal encoder composed of a one-dimensional convolutional layer and a gated recursive unit in series to obtain a device embedding vector of fixed length; inputting the video detection feature stream after adaptive pooling according to a fixed time window into a temporal encoder composed of a multi-layer temporal convolutional network to obtain a video embedding vector of fixed length; feeding the device embedding vector and the video embedding vector into a cross-modal fusion layer with multiple attention channels, aligning and concatenating them to generate a shared semantic vector of a single path; converting the shared semantic vector into a field-level probability distribution through a semantic mapping output head to form a semantic mapping result; and converting the shared semantic vector into production line control command events with confidence scores through a semantic mapping output head.

[0022] The shared semantic vector and its output are rewritten into the unified data bus as events with sequence numbers and timestamps for use by the target system adaptation layer.

[0023] Preferably, the target system adaptation layer includes a message formatting unit, a protocol conversion gateway, a reliable queue manager, an acknowledgment monitoring unit, and a security authentication unit, and pushes control command events to each target system in the following manner:

[0024] After receiving the control command event, the message formatting unit generates a JSON message conforming to the MES interface specification, an AMQP message payload conforming to the WMS interface specification, or a JSON payload conforming to the HTTPWebhook format of a third-party quality inspection SaaS platform, according to the pre-stored mapping rules.

[0025] The protocol conversion gateway pushes formatted JSON messages to the MES endpoint via the HTTPSRESTPOST method; publishes messages to the specified exchange in the WMS via the AMQP1.0 protocol; and pushes quality inspection instructions to the SaaS platform via HTTPSWebhook calls.

[0026] The reliable queue manager persistently queues each push task and uses an exponential backoff retry mechanism until a successful response is received from the target system or the maximum number of retries is reached.

[0027] The confirmation receipt monitoring unit listens for HTTP 200 responses or AMQPack confirmations within a specified timeout period; if a timeout occurs, it writes an alarm event to the unified data bus to trigger manual or automatic remediation.

[0028] The security authentication unit maintains OAuth2.0 tokens or AMQPSASL credentials for each target system, and dynamically refreshes and attaches authentication information before all pushes.

[0029] Preferably, the multi-task neural network engine employs an improved activation function M(z,p) when generating shared semantic representation data. t ,ε t ):

[0030] M(z,p t ,ε t )=z*σ(β*z+δ1*p t +δ2*ε t )

[0031] Where z is the input value of the hidden layer unit of the multi-task neural network, p t ε represents the rolling load occupancy rate of the production line over five minutes. t denoted as the mean confidence score of the semantic mapping for the current batch, β is the learnable parameter of intrinsic slope, δ1 and δ2 are the gating weights of load utilization and mapping confidence, respectively, and σ() is the Sigmoid activation function.

[0032] Preferably, the convolutional layers of the first convolutional neural network model employ a spectral entropy modulated Gaussian error linear unit activation function f():

[0033]

[0034] Where x is the input activation value of the convolutional layer, S t is the spectral entropy factor of the current video frame, which reflects the workpiece texture complexity and lighting uniformity in real time; u is the learnable modulation coefficient; and erf() is the Gaussian error function.

[0035] Preferably, the spectral entropy factor S of the current video frame t Calculate using the following steps:

[0036] First, the current video frame is converted into a grayscale image and the region of interest (ROI) containing the workpiece is extracted;

[0037] Secondly, a two-dimensional fast Fourier transform is performed on the ROI to obtain the amplitude spectrum A(i,j);

[0038] Then, the amplitude spectrum is normalized into a probability distribution:

[0039] p i,j =A(i,j) / ∑ i,j A(i,j);

[0040] Next, calculate the Shannon entropy H:

[0041] H = -∑ i,j p i,j *logp i,j

[0042] Finally, the Shannon entropy is normalized to obtain the spectral entropy factor:

[0043]

[0044] Where i represents the row index of the amplitude spectrum matrix, j represents the column index of the amplitude spectrum matrix, and N is the total number of frequency buckets of the amplitude spectrum.

[0045] This invention provides a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems, which can achieve the following beneficial technical effects:

[0046] 1. This application utilizes a large-model-assisted protocol adaptation module to automatically generate interface description files and parser code, reducing the time for parsing robot private messages from 1-2 weeks manually to minutes. Combined with an industrial vision acquisition module, rapid integration can be achieved without modifying the upper-level system code. The multi-task neural network engine, through cross-modal attention fusion, maps device fields and visual features to a platform-level CDM model, achieving "one field, one semantic" governance, eliminating traditional data silos, and improving the efficiency of cross-system queries and rule configuration by over 70%.

[0047] 2. This application's unified data bus adopts an event-driven architecture to carry time-series, document, and streaming messages in parallel. Combined with the at-one-delivery semantics of a reliable queue manager, it can complete the "defect detection - scheduling instruction - robot execution" closed loop within 50ms, meeting the requirements of high-speed flexible production lines. Control instruction events are accompanied by sequence numbers, timestamps, and identity credentials, which can be traced in a chain on the unified data bus. The target system adaptation layer has built-in OAuth2.0 / SASL dynamic authentication and exponential fallback retries to ensure the integrity, reliability, and security of cross-domain messages. Through preprocessing algorithms such as incremental compression based on byte difference, the average compression rate of the source data stream reaches 60%; after adaptive pooling, the bandwidth consumption of video features is reduced by 35%, significantly reducing edge-to-cloud transmission costs.

[0048] 3. This application improves the defect recall rate and significantly reduces the scheduling strategy error by introducing spectral entropy modulation Gaussian error linear units in the vision branch and load-confidence modulation in the decision branch. The system's convolutional feature extraction layer and multi-task decision layer can perceive workstation congestion, load occupancy rate and image texture complexity in real time. Under extreme conditions such as uneven lighting or high production line load, it automatically adjusts the nonlinear response curve, significantly reduces gradient saturation, accelerates network convergence speed and keeps inference latency basically unchanged, significantly improving the overall robustness and accuracy of industrial vision inspection and production line scheduling. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the steps of a data interaction method for a multi-scenario fusion open platform architecture for heterogeneous systems according to the present invention;

[0051] Figure 2 This is a schematic diagram of a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems according to the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1:

[0054] In view of the aforementioned problems mentioned in the prior art, and in order to solve the above technical problems, as shown in the appendix. Figure 1 As shown: This application provides a data interaction system based on a multi-scenario fusion open platform architecture for heterogeneous systems, including:

[0055] The edge multimode connector kit is used to connect interfaces to a unified data bus and preprocess multi-source data from the production line, including: a large model auxiliary protocol adaptation module, which automatically parses the robot equipment's private communication protocol and converts equipment status data into source data streams; and an industrial vision acquisition module, which acquires real-time video images of workpieces and outputs video detection feature streams based on the first convolutional neural network model.

[0056] In one embodiment, the flexible assembly line for automotive parts demonstrates the deployment and workflow of an edge multimode connector kit, highlighting how two sub-modules connect the interface to a unified data bus and complete data preprocessing. These two sub-modules are a large-model auxiliary protocol adaptation module and an industrial vision acquisition module.

[0057] The assembly line has four six-axis robots (from Manufacturer A) responsible for gripping, assembling, and tightening operations. Each robot control cabinet outputs private messages via RJ-45 gigabit Ethernet. A 5-megapixel industrial camera is installed above key workstations, working in conjunction with a ring LED light source to detect defects in the workpieces on the conveyor belt. On the edge side, an industrial PC (i7 processor, 32GB RAM, RTX A2000 GPU, and two gigabit network cards) is used as the physical node for the kit, pre-installed with Docker and K3s lightweight Kubernetes, uniformly exposing port 1883 (MQTT) and port 8080 (REST).

[0058] The implementation of the large-scale model-assisted protocol adaptation module includes the following steps: Message Acquisition: Running an open-source packet capture container (TShark) to sample the TCP data stream sent by the robot control cabinet for five minutes, obtaining approximately 20,000 hexadecimal messages of 64 bytes each. Field Boundary Inference: Calling a statistical boundary script to automatically identify candidate splitting points based on byte position entropy and length clustering, outputting a list of field boundaries such as "offsets 2, 4, 8, 16, 32, 48". Generative Inference: Generating a prompt containing ten message samples, candidate splitting points, and a small amount of prior knowledge such as "example fields: head_flag, batch_id, torque", and sending it to the locally deployed DeepSeek-Coder-33B large-scale model. The model returns an interface description file (.ksy) conforming to KaitaiStruct syntax and automatically generates Python parser code. Parsing and execution: Start the parser container to subscribe to robot messages in real time, map the decomposed fields into JSON, such as {"batch_id":"20250720-01","servo_pos_joint1":12345,...}, and then perform Delta-RLE compression and publish to the JSON document channel topic robot / state on the unified data bus.

[0059] The implementation of the industrial vision acquisition module includes the following steps: Triggering and Acquisition: The camera is triggered by the PLC, capturing four frames of RGB images per second and pushing them to the gRPC streaming service of the edge node. Preprocessing: Adaptive CLAHE enhancement is performed on each frame within the GPU container, followed by inter-frame differencing to remove static backgrounds. Defect Inference: The first convolutional neural network model (based on the EfficientNet-B0 backbone) undergoing transfer learning is loaded to identify three types of defects: scratches, burrs, and misassembly, outputting a feature tensor with a shape of 1×256. Feature Encapsulation: The feature tensor is serialized into Protobuf frames and written to the unified data bus streaming channel topic vision / feat.

[0060] The data bus access process is as follows: When the edge node starts up, it registers two producers with the central unified data bus (NATSJetStream cluster): –robot / state topic corresponds to the JSON document channel; –vision / feat topic corresponds to the Protobuf streaming channel; each message is automatically appended with seq_id (64-bit incrementing) and ts_ns (nanosecond timestamp), and the bus routes to the multi-task neural network engine on the consumer side according to the topic name.

[0061] In an eight-hour production line test, the automatically generated parser achieved a parsing accuracy of 99.3%, and the bandwidth of robot status messages was reduced by 65% ​​after compression. The defect detection rate of the vision branch was less than 2%, and the end-to-end latency of the two data streams reaching the AI ​​engine through the unified data bus was approximately 45 milliseconds, meeting the real-time scheduling requirements of the flexible assembly line. This demonstrates that the edge multimode connector kit enables code-free access, automatic preprocessing, and compression of proprietary protocols and visual data, providing a unified, low-latency data entry point for subsequent semantic mapping and scheduling decisions.

[0062] A unified data bus is used to receive the source data stream and video detection feature stream in an event-driven manner, and provides time-series-document-stream hybrid message transmission. In one embodiment, in a flexible assembly line project, the unified data bus uses a three-node NATS JetStream cluster. Each node is equipped with an NVMe SSD for disk stream logging, and the cluster maintains metadata consistency through RAFT. After system startup, the bus automatically creates three types of topic streams: timesseries, documents, and streams, corresponding to time-series, document, and streaming message channels. Before sending, the edge multi-mode connector suite calls the local SDK to append seq_id and nanosecond-level ts_ns to each message, and then writes it to different topics according to the data type. For example, robot status JSON is written to documents.robot.state, visual feature Protobuf frames are written to streams.vision.feat, and temperature sensor measurements are written to timesseries.env.temp. JetStream's Stream API can configure independent strategies for each topic. Timeseries uses time-based rolling sharding, retaining the most recent seven days' worth of data; documents are configured for item-based recycling, retaining the latest 200,000 items; streams uses an in-memory queue and enables automatic spilling to SSD to ensure high-bandwidth streaming data is not blocked. The multi-task neural network engine subscribes to documents.robot.state and streams.vision.feat using JetStreamPush-Consumer. Whenever a new message arrives, JetStream immediately pushes it to the AI ​​engine container via the gRPC-NATS bridge, with an average push latency of less than 5 milliseconds. If the engine undergoes a rolling upgrade, the consumer group will rebalance and automatically replay unacknowledged messages to ensure at least one delivery. The engine aligns messages from different topics in memory based on seq_id and ts_ns. If a visual feature frame arrives before the corresponding device state is ready, it is placed in a temporary buffer, waiting for a maximum of 20 milliseconds; if a timeout occurs, an alarm event is triggered and written to documents.alerts. Each Consumer periodically reports its acknowledgement number, and JetStream automatically cleans up expired messages based on the ACK progress. The Prometheus Exporter pulls JetStream metrics, and the Grafana dashboard displays real-time throughput, backlog, and latency for each channel. This example demonstrates that the unified data bus, through event encapsulation, typed topic streams, and elastic push, enables hybrid transmission of time-series, document, and streaming messages and millisecond-level event-driven processing, providing continuous, reliable, and high-concurrency data input for multi-task neural network engines.

[0063] A multi-task neural network engine, positioned on the message forwarding path between the unified data bus and the target system adaptation layer, performs multimodal encoding on the received source data stream and video detection feature stream, generating shared semantic representation data while outputting semantic mapping results and control command events. In one embodiment, the multi-task neural network engine runs as a container on an edge computing node. After startup, it first subscribes to two topics via NATS JetStreamPush-Consumer: documents.robot.state and streams.vision.feat. The system places robot state JSON data in 16-millisecond batches and corresponding visual feature frames into the same mini-batch, aligning them according to timestamps. The engine internally comprises three stages. The first stage is feature encoding: the robot state field sequence is processed through one-dimensional convolution and gated recurrent units to obtain a 128-dimensional temporal vector; each frame's visual features are compressed into vectors of the same dimension through a multi-layer temporal convolutional network. The second stage is cross-modal fusion: a four-head cross-attention mechanism is used to align the two types of vectors bidirectionally, then concatenate them and pass them through a residual fully connected layer to generate a shared semantic vector. The third stage is multi-task output: the semantic mapping head uses two layers of fully connected layers plus Softmax to generate corresponding platform fields and probabilities for each source field; the scheduling strategy head uses a three-layer perceptron connected to the strategy gradient module, outputting four types of action codes: "start, speed adjustment, line change, and emergency stop," along with confidence levels from 0 to 1. After inference, the engine encapsulates the semantic mapping JSON and control command events into NATS messages with sequence numbers and timestamps, writing them back to the `documents.mapping.result` and `documents.control.cmd` topics. The target system adaptation layer listens to `control.cmd` in real time. When it receives a "line change" action with a confidence level greater than 0.85, it immediately issues a new process program to the robot; if the confidence level is lower than the threshold, it enters the manual approval queue. Simultaneously, the mapping results are written to the data lake for subsequent governance and analysis. The engine automatically loads the latest federated fine-tuning weights every 24 hours, allowing the mapping and scheduling models to evolve with the field data. Within 30 days of deployment, the semantic mapping accuracy increased from 94% to 98%, and the false trigger rate for line change actions was controlled within 0.2%. This embodiment demonstrates the actual operation details of the multi-task neural network engine in the "bus-engine-adapter layer" link, confirming that the present invention can complete multimodal fusion, semantic mapping and control decision-making within a millisecond-level closed loop.

[0064] The target system adaptation layer is used to push the control command events to MES, WMS, and third-party quality inspection SaaS platforms to achieve production scheduling adjustments and robot motion reprogramming. In one embodiment, in the automotive parts flexible production line project, the target system adaptation layer runs as an independent microservice in a hybrid cloud environment, specifically listening to the unified data bus documents.control.cmd topic. When the multi-task neural network engine issues a control command event (example: action "line change" encoding SWITCH_LINE, confidence 0.92, parameter {"next_recipe": "AX-M14"}), the adaptation layer immediately enters the following process. First, the message formatting unit generates three payloads according to preset mapping rules: ① a JSON structure for MES: {"action":"switch_line","recipe":"AX-M14","confidence":0.92}; ② an AMQP message body for WMS: action = switch_line & batch = AX-M14; ③ an HTTP Webhook JSON for the third-party quality inspection SaaS: {"event":"line_change","new_recipe":"AX-M14"}. Then, the protocol conversion gateway pushes the JSON to the / api / v1 / schedule / line endpoint of MES via HTTPSPOST; publishes the message to wms.exchange.route.line-change via AMQP 1.0; and calls the secure signature Webhook URL provided by the SaaS. Before each push, the security authentication unit automatically refreshes the MESOAuth2BearerToken and WMSSASL certificates, placing the token in the HTTP header or AMQPSASL field. The reliable queue manager generates unique task IDs for three tasks. If a 200 OK response from MES or an ACK response from WMS is not received within 15 seconds, an exponential backoff retry is triggered. If three consecutive retries fail, the IDs are written to the unified data bus documents.alerts for maintenance personnel to intervene. Upon successful reception, MES immediately updates the schedule, adjusts the next process program number to AX-M14, and calls the robot controller interface to issue a new action sequence. WMS updates the inventory handling logic, redirecting the finished product flow to a new buffer. The quality inspection SaaS switches to the defect model for the AX-M14 process and returns a "switch complete" confirmation. The entire closed loop, from instruction generation to confirmation by the three systems, takes approximately 128 milliseconds, ensuring that the production line can complete the changeover and robot program reprogramming without downtime, verifying the reliable push and execution capability of the target system adaptation layer for control instruction events.

[0065] The edge multi-mode connector kit, unified data bus, multi-task neural network engine and target system adaptation layer form a data closed loop in sequence and are interconnected through a service mesh to achieve integrated real-time collaboration based on semantic mapping and scheduling instruction generation of a single multi-task neural network.

[0066] The first convolutional neural network model (visual detection branch) is deployed inside the industrial vision acquisition module, specifically for detecting defects in real-time workpiece images and outputting a 256-dimensional feature vector as the visual input to the multi-task neural network engine. The network structure includes: an input layer that takes a 1280 × 1024 × 3 resolution RGB image as input, which is then preprocessed with adaptive CLAHE and frame difference before being fed into the model. The backbone is rewritten based on EfficientNet-B0. The first two 3×3 standard convolutions maintain the original number of channels, while the last three layers are replaced with depthwise separable convolutions to reduce latency. The detail enhancement block inserts a dual-branch feature fusion module between the 4th and 5th MBConv layers: the left branch uses dilated convolutions to capture 3px defects; the right branch uses 1×1 convolutions to maintain context. After fusion, the feature is activated by the SE-GELU activation function, where SE-GELU uses a spectral entropy factor S. t Dynamically adjusting the threshold improves robustness to uneven illumination and long-tail defects. Temporal pooling: The features of the last three frames are adaptively pooled in the temporal dimension to ensure consistent output length across different frame rates per second. Global average pooling + linear layer: Features are compressed into a fixed length of 256 and L2 normalized to form a video detection feature stream. The camera acquires one frame each time it receives a trigger pulse; after the image enters the model, the front-end convolution quickly filters out large areas of background, and the detail enhancement block retains negative activation values ​​for areas with dappled illumination under SE-GELU modulation, thus maintaining defect sensitivity; the final 256-dimensional vector contains both shape and texture information and encodes the temporal context, providing a fine description for subsequent cross-modal alignment.

[0067] The multi-task neural network engine (semantic mapping and scheduling decision core) is located between the unified data bus and the target system adaptation layer. It is responsible for integrating device status JSON and visual feature streams, outputting semantic mapping, load prediction (optional), and production line control command events, achieving millisecond-level closed-loop processing. The network structure includes: a device temporal encoder comprising: a 1×5 one-dimensional convolutional layer for extracting local changes; a two-layer gated recurrent unit (GRU) for capturing dynamic trends over a 500ms interval; and an output of a 128-dimensional device embedding vector. The video temporal encoder comprises: a temporal convolutional network (kernel = 3, dilation = 1 / 2 / 4, channel = 128); adaptive window pooling to maintain an equivalent temporal sequence of 20fps; and an output of a 128-dimensional video embedding vector. The cross-modal fusion layer includes a four-head cross-attention layer, each head with a dimension of 32; concatenation and residual connections to output a 256-dimensional shared semantic vector h. The fusion layer internally uses PC-ReLU-C / AS-Swish-M activation functions: PC-ReLU-C is activated by the congestion factor κ. t Adjusting the negative slope under the action; AS-Swish-M through the load factor ρ t With semantic confidence ε t The Sigmoid threshold is shifted to achieve state adaptation. The task output header includes a semantic mapping header (two fully connected layers + Softmax, outputting platform fields for each source field) and a probability control strategy header (three-layer perceptron, with the final layer connected to a policy gradient unit, outputting action codes (start, stop, line change, emergency stop, speed adjustment) and a confidence load prediction header (single-layer regression, providing the production line throughput for the next five minutes). The engine synchronously receives robot / state and vision / feat messages via NATSPush-Consumer, aligning them by ts_ns before batching. Device state vectors and vision vectors are encoded separately and then fed into cross-modal fusion; the attention mechanism maps weights based on timestamps, associating visual defects with corresponding device joint positions. Shared semantic vectors are used to generate field-probability pairs via the semantic mapping header, correcting ambiguous fields such as BatchID and Torque in real time; the mapping results are returned to the data lake in the trusted link. The same vector is used to infer actions through the control policy head. When the confidence level exceeds 0.85, it is encapsulated as a control.cmd message and written back to the data bus. The target system adaptation layer immediately pushes it to MES / WMS / quality inspection SaaS. Dynamic modulation of the activation function enables the network to automatically increase the threshold and stabilize the policy output when there is high congestion or low confidence; it also expands the gradient in the negative half-zone when the lighting deteriorates to maintain defect recall. At midnight every day, the engine pulls the federated LoRA weights from the central repository and performs incremental hot updates to the semantic mapping head and policy head; during the idle period at night, the logs of the day are replayed to fine-tune the fusion layer to ensure that the model continuously fits the on-site data distribution.

[0068] The hardware of the data interaction system for multi-scenario fusion open platform architecture oriented towards heterogeneous systems in this application is as follows: Figure 2The diagram shows the following: Production floor layer (L0-L1): 4 robot control cabinets, each with a built-in PLC and servo driver, providing RJ-45 Gigabit Ethernet ports and RS-485 debugging ports; 1 industrial camera: a 5-megapixel GigEVision, with its LAN port directly connected to the field switch; 1 LED ring light source: powered by a 24VDC power supply and light source controller, with trigger signals from the PLC; Environmental / process sensors: temperature, humidity, dust, and three-phase current modules, aggregated via a Modbus-TCP gateway. Edge computing layer (L2): 1 edge industrial PC: Intel i7 CPU, 32GB RAM, RTX A2000 GPU, dual Gigabit network cards (eth0, eth1); 1 PoE Gigabit switch: 8 ports, two uplink SFPs for camera and robot control cabinet connection; Connection methods are as follows: Robot control cabinet → Switch (Cat6); Industrial camera → Switch (PoECat6); Switch uplink port → Edge PC eth0; Sensor gateway → Switch or directly to Edge PC eth1. The unified data bus layer (L3) includes: a NATSJetStream cluster: three 1U servers, each with a Xeon-Silver CPU, 64 GB RAM, and a 1TB NVMe port, housed in an IT server room; cluster nodes are interconnected using 10GbE switches; edge PCs access the network via a second network interface card (eth1) through VPN or VLAN isolation. The AI ​​computing and service layer (L4) includes: two AI inference servers: AMD EPYC CPU and two A40 GPUs, deploying multi-task neural network engine containers; object storage / data lake: a Ceph or MinIO cluster for storing mapping results and raw logs; the AI ​​servers and NATS cluster are directly connected via a 10 GbE backplane switch, ensuring a network RTT of <2ms. The business system layer (L5) includes: a MES server: Windows Server + SQL Server with an open REST interface; a WMS server: RabbitMQ + PostgreSQL; a third-party quality inspection SaaS: a public HTTPS webhook; MES and WMS are mounted on the enterprise production network; NATS exposes a message bridge via NAT + API Gateway.In terms of connectivity and data flow, the process includes: Field → Edge: Robot, camera, and sensor data are sent to the edge PC via TCP / UDP streams; Edge → Bus: The edge PC parses the packets, packages video features into NATS messages, and pushes them to JetStream; Bus → AI: The AI ​​server subscribes to topics, completes multimodal inference, and generates control command events; AI → Business Layer: Command events are converted into JSON / AMQP / Webhook by the target system adaptation layer and pushed to MES, WMS, and SaaS; Business Confirmation → Bus: MES and WMS return receipts, which are written back to NATS by the adaptation layer for use by monitoring and self-healing logic. In terms of network and security, the field segment and IT segment are isolated through a three-tier firewall + VLAN; all control command events are accompanied by OAuth2 Token or AMQP SASL authentication; Prometheus + Grafana collects SNMP and JetStream metrics from switches / servers to form an observable dashboard. This hardware and connectivity solution realizes a complete link from field devices and visual acquisition to edge preprocessing, data bus relay, AI inference and decision-making, and then to the closed-loop execution of the business system, while ensuring millisecond-level latency, reliable message delivery, and layered security isolation.

[0069] Preferably, the step of connecting the interface to the unified data bus and preprocessing the multi-source data of the production line includes connecting the data interface of the robot equipment controller and the industrial vision acquisition device to the unified data bus; the multi-source data of the production line includes: the device status message byte stream sent by the robot equipment through a private communication protocol and the real-time video frame sequence of the workpiece obtained by the industrial vision acquisition module.

[0070] Preferably, the preprocessing includes: for the device status message byte stream, firstly, the statistical boundary inference unit generates candidate fields for segmentation based on byte position entropy and length clustering; then, the large model-assisted protocol adaptation module calls the generative language model output interface description file with a single 8-bit byte as the smallest input granularity and automatically generates parser code; subsequently, in the parsing execution unit, the message byte stream is parsed into a platform-unified JSON field format, and incremental compression based on byte difference is performed on continuous messages; and for the real-time video frame sequence of the workpiece, firstly, histogram equalization is performed to enhance the contrast of the defect area, then inter-frame difference is performed to remove static background frames, and then the video detection feature stream is output after processing by the first convolutional neural network model.

[0071] Preferably, the large model-assisted protocol adaptation module includes: a statistical boundary inference unit, used to perform field boundary candidate generation based on byte position entropy, length clustering and byte repetition period analysis on the captured private protocol message set; a generative model inference unit, which calls a pre-trained generative model with a single 8-bit byte as the smallest input granularity, and generates an interface description file and parser code based on the candidate boundaries and message samples; and a parsing execution unit, used to decode real-time messages into platform source data streams using the parser code.

[0072] Preferably, the unified data bus includes an event router, a timing channel, a document channel, and a streaming channel, and is configured to: generate an event encapsulation with a 64-bit monotonically increasing sequence number and a timestamp for each message from the source data stream or video detection feature stream; route metric fields with a sampling frequency of not less than 1Hz to the timing channel and store them; route semi-structured JSON or Avro format device status documents to the document channel and provide request-response access; push Protobuf frames of the video detection feature stream extracted by the first convolutional neural network model with a length prefix encoding to the streaming channel; and the event router retains the monotonically increasing sequence number and timestamp when forwarding across channels.

[0073] Preferably, the multi-task neural network engine performs multimodal encoding on the source data stream and video detection feature stream to generate shared semantic representation data, including: first, inputting the device state field sequence of the source data stream from the robot equipment into a temporal encoder composed of a one-dimensional convolutional layer and a gated recursive unit in series to obtain a device embedding vector of fixed length; inputting the video detection feature stream after adaptive pooling according to a fixed time window into a temporal encoder composed of a multi-layer temporal convolutional network to obtain a video embedding vector of fixed length; feeding the device embedding vector and the video embedding vector into a cross-modal fusion layer with multiple attention channels, aligning and concatenating them to generate a shared semantic vector of a single path; converting the shared semantic vector into a field-level probability distribution through a semantic mapping output head to form a semantic mapping result; and converting the shared semantic vector into production line control command events with confidence scores through a semantic mapping output head.

[0074] The shared semantic vector and its output are rewritten into the unified data bus as events with sequence numbers and timestamps for use by the target system adaptation layer.

[0075] Preferably, the target system adaptation layer includes a message formatting unit, a protocol conversion gateway, a reliable queue manager, an acknowledgment monitoring unit, and a security authentication unit, and pushes control command events to each target system in the following manner:

[0076] After receiving the control command event, the message formatting unit generates a JSON message conforming to the MES interface specification, an AMQP message payload conforming to the WMS interface specification, or a JSON payload conforming to the HTTP Webhook format of a third-party quality inspection SaaS platform, according to the pre-stored mapping rules.

[0077] The protocol conversion gateway pushes formatted JSON messages to the MES endpoint via the HTTPS REST POST method; publishes messages to the specified exchange in the WMS via the AMQP 1.0 protocol; and pushes quality inspection instructions to the SaaS platform via HTTPS Webhook calls.

[0078] The reliable queue manager persistently queues each push task and uses an exponential backoff retry mechanism until a successful response is received from the target system or the maximum number of retries is reached.

[0079] The confirmation receipt monitoring unit listens for HTTP 200 responses or AMQP ack confirmations within the specified timeout period; if a timeout occurs, it writes an alarm event to the unified data bus to trigger manual or automatic remediation.

[0080] The security authentication unit maintains OAuth 2.0 tokens or AMQP SASL credentials for each target system, and dynamically refreshes and attaches authentication information before all pushes.

[0081] Preferably, the multi-task neural network engine employs an improved activation function M(z,p) when generating shared semantic representation data. t ,ε t ):

[0082] M(z,p t ,ε t )=z*σ(β*z+δ1*p t +δ2*ε t )

[0083] Where z is the input value of the hidden layer unit of the multi-task neural network, p t ε represents the rolling load occupancy rate of the production line over five minutes. t denoted as the mean confidence score of the semantic mapping for the current batch, β is the learnable parameter of intrinsic slope, δ1 and δ2 are the gating weights of load utilization and mapping confidence, respectively, and σ() is the Sigmoid activation function.

[0084] Preferably, the convolutional layers of the first convolutional neural network model employ a spectral entropy modulated Gaussian error linear unit activation function f():

[0085]

[0086] Where x is the input activation value of the convolutional layer, S t is the spectral entropy factor of the current video frame, which reflects the workpiece texture complexity and lighting uniformity in real time; u is the learnable modulation coefficient; and erf() is the Gaussian error function.

[0087] Preferably, the spectral entropy factor S of the current video frame t Calculate using the following steps:

[0088] First, the current video frame is converted into a grayscale image and the region of interest (ROI) containing the workpiece is extracted;

[0089] Secondly, a two-dimensional fast Fourier transform is performed on the ROI to obtain the amplitude spectrum A(i,j);

[0090] Then, the amplitude spectrum is normalized into a probability distribution:

[0091] p i,j =A(i,j) / ∑ i,j A(i,j);

[0092] Next, calculate the Shannon entropy H:

[0093] H = -∑ i,j p i,j *logp i,j

[0094] Finally, the Shannon entropy is normalized to obtain the spectral entropy factor:

[0095]

[0096] Where i represents the row index of the amplitude spectrum matrix, j represents the column index of the amplitude spectrum matrix, and N is the total number of frequency buckets of the amplitude spectrum.

[0097] This invention provides a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems, which can achieve the following beneficial technical effects:

[0098] 1. This application utilizes a large-model-assisted protocol adaptation module to automatically generate interface description files and parser code, reducing the time for parsing robot private messages from 1-2 weeks manually to minutes. Combined with an industrial vision acquisition module, rapid integration can be achieved without modifying the upper-level system code. The multi-task neural network engine, through cross-modal attention fusion, maps device fields and visual features to a platform-level CDM model, achieving "one field, one semantic" governance, eliminating traditional data silos, and improving the efficiency of cross-system queries and rule configuration by over 70%.

[0099] 2. This application's unified data bus adopts an event-driven architecture to carry time-series, document, and streaming messages in parallel. Combined with the at-one-delivery semantics of a reliable queue manager, it can complete the "defect detection - scheduling instruction - robot execution" closed loop within 50ms, meeting the requirements of high-speed flexible production lines. Control instruction events are accompanied by sequence numbers, timestamps, and identity credentials, which can be traced in a chain on the unified data bus. The target system adaptation layer has built-in OAuth2.0 / SASL dynamic authentication and exponential fallback retries to ensure the integrity, reliability, and security of cross-domain messages. Through preprocessing algorithms such as incremental compression based on byte difference, the average compression rate of the source data stream reaches 60%; after adaptive pooling, the bandwidth consumption of video features is reduced by 35%, significantly reducing edge-to-cloud transmission costs.

[0100] 3. This application improves the defect recall rate and significantly reduces the scheduling strategy error by introducing spectral entropy modulation Gaussian error linear units in the vision branch and load-confidence modulation in the decision branch. The system's convolutional feature extraction layer and multi-task decision layer can perceive workstation congestion, load occupancy rate and image texture complexity in real time. Under extreme conditions such as uneven lighting or high production line load, it automatically adjusts the nonlinear response curve, significantly reduces gradient saturation, accelerates network convergence speed and keeps inference latency basically unchanged, significantly improving the overall robustness and accuracy of industrial vision inspection and production line scheduling.

[0101] The above provides a detailed description of a multi-scenario fusion open platform architecture data interaction system for heterogeneous systems. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas and methods of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A data interaction system based on a multi-scenario integrated open platform architecture for heterogeneous systems, characterized in that, include: The edge multimode connector kit is used to connect interfaces to a unified data bus and preprocess multi-source data from the production line, including: a large model auxiliary protocol adaptation module, which automatically parses the robot equipment's private communication protocol and converts equipment status data into source data streams; and an industrial vision acquisition module, which acquires real-time video images of workpieces and outputs video detection feature streams based on the first convolutional neural network model. A unified data bus is used to receive the source data stream and video detection feature stream in an event-driven manner, and to provide time-series-document-stream hybrid message transmission; The multi-task neural network engine is set up on the message forwarding path between the unified data bus and the target system adaptation layer. It performs multimodal encoding on the received source data stream and video detection feature stream, generates shared semantic expression data, and outputs semantic mapping results and control command events. The target system adaptation layer is used to push the control command events to MES, WMS, and third-party quality inspection SaaS platform respectively, so as to realize production scheduling adjustment and robot motion reprogramming; Among them, the edge multimode connector kit, unified data bus, multi-task neural network engine and target system adaptation layer form a data closed loop in sequence and are interconnected through service mesh to realize integrated real-time collaboration based on semantic mapping and scheduling instruction generation of a single multi-task neural network. The multi-task neural network engine performs multimodal encoding on the source data stream and video detection feature stream to generate shared semantic representation data, including: first, inputting the device state field sequence of the source data stream from the robot equipment into a temporal encoder composed of a one-dimensional convolutional layer and a gated recursive unit in series to obtain a device embedding vector of fixed length; then, inputting the video detection feature stream after adaptive pooling according to a fixed time window into a temporal encoder composed of a multi-layer temporal convolutional network to obtain a video embedding vector of fixed length; finally, feeding the device embedding vector and the video embedding vector into a cross-modal fusion layer with multiple attention channels for alignment and concatenation to generate a shared semantic vector with a single path; converting the shared semantic vector into a field-level probability distribution through a semantic mapping output head to form a semantic mapping result; and finally, converting the shared semantic vector into production line control command events with confidence scores through the semantic mapping output head. The shared semantic vector and its output are rewritten into the unified data bus in the form of events with sequence numbers and timestamps for use by the target system adaptation layer. The multi-task neural network engine employs an improved activation function when generating shared semantic representation data. : ， in, These are the input values ​​for the hidden layer units of a multi-task neural network. The rolling load occupancy rate of the production line over five minutes. This represents the mean confidence level of the semantic mapping for the current batch. The intrinsic slope is a learnable parameter. , These are the gating weights for load utilization and mapping confidence, respectively. ( ) is the Sigmoid activation function.

2. The data interaction system of a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 1, characterized in that, The method of connecting the interface to the unified data bus and preprocessing multi-source data of the production line includes connecting the data interface of the robot equipment controller and the industrial vision acquisition device to the unified data bus. The multi-source data of the production line includes: the byte stream of equipment status messages sent by the robot equipment through a private communication protocol and the sequence of real-time video frames of the workpiece obtained by the industrial vision acquisition module.

3. The data interaction system of a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 2, characterized in that, The preprocessing includes: for the device status message byte stream, the statistical boundary inference unit first generates candidate fields for segmentation based on byte position entropy and length clustering; then, the large model-assisted protocol adaptation module calls the generative language model with a single 8-bit byte as the smallest input granularity to output the interface description file and automatically generate parser code; subsequently, in the parsing execution unit, the message byte stream is parsed into the platform-unified JSON field format, and continuous messages are incrementally compressed based on byte difference; and for the real-time video frame sequence of the workpiece, histogram equalization is first performed to enhance the contrast of the defect area, then inter-frame difference is performed to remove static background frames, and finally, after processing by the first convolutional neural network model, the video detection feature stream is output.

4. The data interaction system of a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 1, characterized in that, The large model-assisted protocol adaptation module includes: a statistical boundary inference unit, used to generate field boundary candidates based on byte position entropy, length clustering, and byte repetition period analysis on the captured private protocol message set; a generative model inference unit, which calls a pre-trained generative model with a single 8-bit byte as the smallest input granularity, and generates an interface description file and parser code based on the candidate boundaries and message samples; and a parsing execution unit, used to decode real-time messages into platform source data streams using the parser code.

5. A data interaction system based on a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 1, characterized in that, The unified data bus includes an event router, a timing channel, a document channel, and a streaming channel, and is configured to: generate an event encapsulation with a 64-bit monotonically increasing sequence number and a timestamp for each message from the source data stream or video detection feature stream; route metric fields with a sampling frequency of not less than 1Hz to the timing channel and store them; route semi-structured JSON or Avro format device status documents to the document channel and provide request-response access; push Protobuf frames of the video detection feature stream extracted by the first convolutional neural network model with a length prefix encoding to the streaming channel; and retain the monotonically increasing sequence number and timestamp when forwarding across channels.

6. The data interaction system of a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 1, characterized in that, The target system adaptation layer includes a message formatting unit, a protocol conversion gateway, a reliable queue manager, an acknowledgment monitoring unit, and a security authentication unit, and pushes control command events to each target system in the following manner: After receiving the control command event, the message formatting unit generates a JSON message conforming to the MES interface specification, an AMQP message payload conforming to the WMS interface specification, or a JSON payload conforming to the HTTPWebhook format of a third-party quality inspection SaaS platform, according to the pre-stored mapping rules. The protocol conversion gateway pushes the formatted JSON message to the MES endpoint via the HTTPSRESTPOST method; Messages are published to the specified exchange in the WMS via the AMQP 1.0 protocol; quality inspection instructions are pushed to the SaaS platform via HTTPS Webhook calls.   The reliable queue manager persistently queues each push task and uses an exponential backoff retry mechanism until a successful response is received from the target system or the maximum number of retries is reached. The confirmation receipt monitoring unit listens for HTTP 200 responses or AMQPack confirmations within the specified timeout period; if a timeout occurs, it writes an alarm event to the unified data bus to trigger manual or automatic remediation. The security authentication unit maintains OAuth2.0 tokens or AMQPSASL credentials for each target system, and dynamically refreshes and attaches authentication information before all pushes.

7. A data interaction system based on a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 1, characterized in that, The convolutional layers of the first convolutional neural network model employ spectral entropy modulation of Gaussian error linear unit activation functions. : ， in, Input activation values ​​to the convolutional layer. The spectral entropy factor of the current video frame reflects the workpiece texture complexity and lighting uniformity in real time. For learnable modulation coefficients, This is the Gaussian error function.

8. A data interaction system based on a multi-scenario fusion open platform architecture for heterogeneous systems as described in claim 7, characterized in that, The spectral entropy factor of the current video frame Calculate using the following steps: First, the current video frame is converted into a grayscale image and the region of interest (ROI) containing the workpiece is extracted; Secondly, a two-dimensional fast Fourier transform is performed on the ROI to obtain the amplitude spectrum. ; Then, the amplitude spectrum is normalized into a probability distribution: ； Next, calculate the Shannon entropy. : ， Finally, the Shannon entropy is normalized to obtain the spectral entropy factor: ， Where i represents the row index of the amplitude spectrum matrix, and j represents the column index of the amplitude spectrum matrix. The total number of frequency bins for the amplitude spectrum.

Images