A multi-source data intelligent connection management method

Abstract

The application discloses a kind of multi-source data intelligent connection management methods, belong to data processing and information fusion technical field, this method includes: deployment intelligent agent program listens to window message stream and restores operation path;Synchronous capture and parse network request to obtain structured business data;Mapping relationship of operation and data is established by multi-modal fusion analysis;Filter target data based on no-code rule base;Format standardization is carried out using conversion engine;RPA instruction is generated by large model auxiliary;In full-link embedded digital certificate mechanism guarantees data integrity;And by extension point injection dynamic enhancement business logic.The application aims to solve the problem of data collaboration difficulty, high adaptation cost and insufficient security caused by lack of standard interface in heterogeneous business systems, realize high-precision, low-latency, auditable data automatic acquisition, conversion and input across closed systems, significantly shorten the supplier access cycle, improve the efficiency of supply chain four-flow collaboration and system scalability.

Description

Technical Field

[0001] This invention belongs to the field of data processing and information fusion technology, and specifically relates to a method for intelligent connection management of multi-source data. Background Technology

[0002] As enterprises deepen their digital transformation, supply chain collaborative management has become a core element in improving operational efficiency and risk control capabilities. In today's complex business ecosystem, large enterprises often connect with hundreds or even thousands of suppliers, who vary significantly in their IT infrastructure, system architecture, and data standards. This heterogeneity makes it difficult to effectively integrate data across key business processes such as procurement, logistics, and settlement, creating information silos and severely restricting the overall responsiveness and collaborative accuracy of the supply chain. Especially under the goal of achieving refined management that integrates "logistics, contract flow, data flow, and cash flow," the problem of data fragmentation across enterprises is becoming increasingly prominent, urgently requiring a technological approach that can be compatible with diverse information systems and achieve efficient and controllable data interaction.

[0003] Multi-source data intelligent connectivity, as a key technology for breaking down information barriers between upstream and downstream of the supply chain, aims to automatically collect, transform, and transmit business data through non-intrusive means, thereby achieving seamless process integration without modifying existing systems. The basic principle of this technology lies in identifying operational behaviors and communication patterns in different business systems, extracting structured or semi-structured data, and completing semantic mapping and format adaptation according to preset rules. Finally, the processed data is securely injected into the target business system. Compared to traditional docking methods that rely on customized API development, intelligent connectivity technology emphasizes low-code or even zero-code deployment capabilities to reduce the requirements for supplier technical capabilities, shorten implementation cycles, and support dynamic adjustments to adapt to rapidly changing business scenarios.

[0004] However, existing technologies still face multiple bottlenecks in achieving cross-enterprise, multi-source data connectivity. First, mainstream integration solutions heavily rely on standardized API interfaces, but many small and medium-sized suppliers only use localized ERP systems, Excel spreadsheets, or outdated business systems, lacking open interface capabilities, making traditional integration methods difficult to implement. Second, even if usable interfaces exist, their field definitions, data formats, and calling logic vary, lacking a unified data abstraction layer. This necessitates significant manpower for customized development for each new supplier, resulting in high costs and long development cycles. Third, while existing automation tools such as Robotic Process Automation (RPA) can simulate user operations to capture data, their stability and generalization capabilities are insufficient when facing complex interface logic, dynamically loaded content, or multi-step interactive processes, making them prone to failure due to system interface changes. Furthermore, current solutions generally lack inherent security mechanisms, making it difficult to guarantee the authenticity, integrity, and legal validity of digital credentials throughout the entire data collection, transmission, and entry process, failing to meet the needs of highly compliant scenarios such as supply chain finance and electronic contracts. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-source data intelligent connection management method to solve the problems of data collaboration difficulties, high adaptation costs and insufficient security caused by the lack of standard interfaces in existing heterogeneous business systems.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A method for intelligent connection management of multi-source data includes the following specific steps:

[0008] Step 1: Deploy the intelligent agent program in the source business system's operating environment. The intelligent agent program has operating system-level event listening permissions, monitors and captures the window message stream generated during the interaction between the user and the business system in real time. The window message stream includes control creation, focus switching, mouse click and keyboard input events. By performing time sequence analysis and behavior pattern recognition on the event sequence, the complete business operation path is restored, and key data is tagged with time sequence and entered into nodes.

[0009] Step 2: Synchronously start the network communication layer packet sniffing module to intercept and parse all outbound network requests initiated by the source business system, extract the request header, request body and response content under the HTTP or HTTPS protocol, identify the structured business data packets carried therein, and when encrypted transmission is detected, call the local certificate store to complete TLS decryption and parse the plaintext payload to ensure the integrity of the original business data is obtained.

[0010] Step 3: Input the data fragments from the window message stream and the network request stream into the multimodal fusion parsing unit, use the pre-trained deep neural network model to perform spatiotemporal alignment and semantic association between the two, establish the mapping relationship between operation behavior and data change, and generate an operation-data joint record set with timestamp labels. The joint record set serves as the original basis for subsequent data extraction.

[0011] Step 4: Perform data filtering according to the preset data collection rule base. The data collection rules consist of field name matching strategies, data type constraints, and trigger timing logical expressions. The rule base supports a graphical configuration interface to achieve code-free definition. The system extracts target business entities from the operation-data joint record set according to the rules to form a preliminary structured dataset.

[0012] Step 5: Start the data transformation engine, load the pre-configured data transformation rule set, which includes a field mapping table, a format standardization template, a unit conversion coefficient matrix, and a verification logic function. Perform field renaming, numerical normalization, encoding unification, and null value filling on the preliminary structured dataset, and output an intermediate data object that conforms to the target business system interface specification.

[0013] Step 6: Invoke the service orchestration inference module to generate an executable service orchestration script. The script is encapsulated as a microservice component and deployed using a containerized approach. The service orchestration inference module implements this based on the interface capabilities and computing resource configuration of the target business system, using any of the following methods:

[0014] Method 1: For scenarios where the target business system has a standard RESTful interface and sufficient computing resources, a large language model with fine-tuned instructions is adopted. The input context includes API document fragments of the target business system, a library of historical successful call cases, and a description of the data structure to be sent. The output is a JSON Schema definition file conforming to the OpenAPI 3.0 specification and call example code, which is converted into a schedulable task after being verified by a syntax validator.

[0015] Method 2: For edge deployment scenarios where the target business system interfaces are known and computing resources are limited, a rule-driven orchestration engine is adopted. It does not rely on large models and generates executable scripts based on a predefined service template library. The service template library contains the interface call logic, parameter mapping rules and error handling processes of ERP system, database system and enterprise management system.

[0016] Method 3: For business scenarios with extremely high script generation reliability requirements and the need to prevent the illusion of a large model, a structured Prompt is adopted. A Prompt containing role settings, input context, task description and output format constraints is sent to the large model. After receiving the JSON output, a four-layer verification pipeline is executed. The four layers of verification include syntax and schema verification, semantic and reference verification, type and constraint verification, and idempotency and conflict detection. Only when all four layers of verification pass can the generated script be packaged into a microservice component.

[0017] Step 7: Set up a scheduled task to activate the microservice component according to a preset frequency or event triggering conditions. Push the processed data to the target business system through a RESTful interface or message queue protocol. If the target business system does not have an open interface, enable RPA simulation. The RPA simulation module uses computer vision and UI element recognition technology, combined with OCR text recognition and template matching, to generate a GUI operation instruction sequence and complete the human-computer interactive data entry in a controlled environment.

[0018] Step 8: Embed a digital credential generation and verification mechanism in the entire data flow chain. Each piece of business data is bound to a unique identifier when it is first captured, and a data fingerprint is generated through a hash algorithm. The fingerprint is digitally signed using an asymmetric encryption algorithm to form an immutable electronic credential. This credential is transmitted throughout the data flow and is verified for consistency at each key node.

[0019] Step 9: Establish an extension point injection mechanism, reserve plug-in modules in key links of the data processing flow, and allow external programs to insert custom processing logic by registering callback functions. The plug-in modules support three execution modes: pre-verification, post-notification, and exception interception, so as to realize the dynamic enhancement of business rules and the flexible integration of audit tracing functions.

[0020] In step 1, window message listening uses Windows API hook technology or Linux X11 event listening mechanism. The corresponding underlying event capture scheme is selected according to different operating system platforms. The message sampling frequency is set to 60 times per second to ensure that fast and continuous operation actions are captured. At the same time, a sliding window denoising algorithm is introduced to filter high-frequency jitter signals and retain effective user intent expression.

[0021] In step 2, the network communication layer packet sniffing module is configured with TLS decryption proxy function. By installing a trusted root certificate on the host where the source system is located, the proxy can decrypt HTTPS traffic in the middle and extract the business message structure without decrypting sensitive fields. The decrypted data is only temporarily stored in memory and processed immediately, and disk storage is prohibited to ensure privacy and security.

[0022] In step 3, the multimodal fusion parsing unit adopts a dual-channel Transformer architecture. The visual channel receives window screenshots and control tree structure information, while the text channel receives network packets and log entries. The two channels extract features respectively and then fuse them through a cross-attention mechanism to output the fused feature vector. The classification head then determines the operation intent category and the corresponding data field affiliation.

[0023] In step 4, the data collection rule base supports Boolean logic combination conditions, allowing the setting of "triggering data capture when the order status changes to 'shipped' and the carrier field is not empty". Before the rule takes effect, its syntax correctness and conflict are checked by the formal verification module to avoid invalid or contradictory rules from going online.

[0024] The data conversion rule set in step 5 includes an industry standard comparison table, covering common processing logic such as the 1000-fold conversion relationship between tons and kilograms and bank account verification algorithms. The conversion process supports batch processing mode, with a maximum of 100,000 records processed in a single batch and an average processing time of less than 800 milliseconds.

[0025] In step 6, the large model-assisted service orchestration and inference module adopts a language model with hundreds of billions of parameters after instruction fine-tuning. The input context includes API document fragments of the target business system, a library of historical successful call cases, and a description of the data structure to be sent. The output is a JSON Schema definition file and call example code that conforms to the OpenAPI 3.0 specification. After being verified by the syntax validator, it is converted into a schedulable task.

[0026] In step 7, the timed task scheduler is implemented using the distributed Quartz framework, which supports a hybrid mode of Cron expression and time-triggered execution. The task execution status is reported to the central monitoring station in real time, and failed tasks are automatically put into the retry queue, with a maximum of 3 consecutive retries and an exponential backoff interval.

[0027] In step 8, the security strength of the digital credential meets the commercial cryptographic algorithm SM2 / SM3 / SM4 standards. The data fingerprint calculation adopts the SM3 hash algorithm, the signing process uses the SM2 elliptic curve public key system, the private key is stored in the hardware security module HSM, and the signing operation is completed inside the HSM to eliminate the risk of private key export. Alternatively, for ordinary computing environments that do not have a dedicated hardware security module, the hardware security module can be replaced with a software simulator based on a trusted platform module.

[0028] In step 9, the extended point injection mechanism provides a standardized SDK development package, allowing third-party developers to write processing plugins in Python or Java. After the plugin is uploaded, it can be enabled only after static scanning and dynamic behavior monitoring in the sandbox environment confirms that there is no malicious operation. The upper limit of plugin runtime resource usage is set to 20% CPU usage and 300MB memory.

[0029] The multi-source data intelligent connection management method also includes establishing a globally unique business data traceability map, recording each data capture, transformation, and transmission event as a node in the map, with nodes connected by data lineage edges, supporting penetrating queries by order number, contract number, or multi-level supplier path, and fully displaying the data lifecycle trajectory.

[0030] The multi-source data intelligent connection management method also includes configuring an adaptive learning module to regularly collect data processing failure cases and manual correction results, and feed them back to the large model training pipeline for incremental fine-tuning, so that the system can continuously optimize in terms of field mapping recommendation and anomaly recovery strategy generation, with a monthly accuracy improvement of no less than 2 percentage points.

[0031] The multi-source data intelligent connection management method also includes setting up multi-level access control policies, allocating data access scope according to organizational roles, allowing ordinary operators to only view the data flow logs of their own departments, allowing administrators to access the full-process audit records, requiring two-factor authentication authorization for all sensitive operations and leaving a trace, and retaining log records for no less than 5 years.

[0032] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0033] This invention overcomes the technical limitations of traditional interface-dependent integration models by constructing a non-intrusive data perception system that integrates window message listening, network request capture, and multimodal fusion parsing. This enables high-fidelity business data extraction even in closed systems without API open capabilities, significantly expanding the applicable boundaries of data connectivity.

[0034] This invention introduces natural language understanding capabilities into the API call logic generation process through a large model-driven service orchestration mechanism, which greatly reduces the amount of customized development work required for system integration and shortens the new supplier access cycle from an average of 2 weeks in the traditional way to less than 3 days.

[0035] This invention safeguards data authenticity and integrity through a digital credential mechanism based on a national-level encryption algorithm embedded throughout the entire data chain. The theoretical probability of a single data entry being tampered with is lower than [a certain value]. It meets financial-grade compliance requirements.

[0036] This invention, through extended point injection and dynamic rule configuration design, enables the system to possess high flexibility and evolvability. The response time for business logic adjustments is controlled within one week, and the system performance degradation does not exceed 10% even when the data volume increases by 5 times, demonstrating excellent scalability and stability. The overall solution realizes automatic data synchronization in the core business processes of the supply chain, including contracts, orders, shipments, receipts, and reconciliation, with the data deviation rate controlled within 5%. More than 95% of business processes can be completed end-to-end within 3 seconds, and more than 300 data records can be processed per second. It fully achieves the goal of refined management of the integration of the four flows (information, capital, logistics, and information), providing core technical support for building a reliable, efficient, and intelligent modern supply chain collaborative ecosystem. Attached Figure Description

[0037] Figure 1 This is a flowchart of the overall technical solution of a multi-source data intelligent connection management method proposed in this invention;

[0038] Figure 2 This is a schematic diagram of the core principle framework of the composite perception mechanism that integrates window message listening, network request capture, and multimodal parsing in this invention. Detailed Implementation

[0039] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0040] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0041] In the embodiments of the present invention, the same reference numerals denote the same components, and for the sake of brevity, detailed descriptions of the same components are omitted in different embodiments. It should be understood that the thickness, length, width, and other dimensions of various components in the embodiments of the present invention shown in the accompanying drawings, as well as the overall thickness, length, width, and other dimensions of the integrated device, are merely illustrative and should not constitute any limitation on the present invention; the term "multiple" in the present invention refers to two or more (including two).

[0042] Example 1: Basic Implementation Scheme—Intelligent Data Connection Method Based on Multimodal Perception and Large Model-Driven Approach

[0043] This embodiment is the basic implementation scheme of the present invention, and fully describes the specific implementation of steps 1-9. Data from the source system is acquired through dual-channel acquisition via window message listening and network request capture. A mapping relationship between operations and data is established through multimodal fusion parsing. After filtering target data based on a rule base, the conversion engine standardizes the format. A large model is called to automatically generate service orchestration scripts and encapsulate them as microservice components. Data is pushed to the target system via a timer. Digital credentials are embedded throughout the entire chain to ensure data integrity. An extension point injection mechanism is provided to support custom plugins. This embodiment also supplements enhanced functions such as a business data traceability graph, an adaptive learning module, and multi-level access control.

[0044] Currently, in the process of deepening enterprise digital transformation, supply chain collaborative management has become a core link in improving group operational efficiency and risk control capabilities. Large enterprises often connect with hundreds or even thousands of suppliers, and these suppliers have significant differences in their level of information technology construction, system architecture, and data standards, making it difficult to effectively connect data in key business processes such as procurement, logistics, and settlement, forming information silos. Especially under the goal of promoting the integration of "logistics, contract flow, data flow, and cash flow" into a refined management system, the problem of data fragmentation between enterprises is becoming increasingly prominent. To address the above technical problems, this invention proposes to construct a composite perception mechanism that integrates window message listening, network request capture, and multimodal parsing, combined with an automated service orchestration engine based on a large model and a data transmission system with an embedded cryptographic security framework, to achieve a zero-code multi-source data intelligent connection solution without modifying the source system. This enables high-precision, low-latency, and auditable automatic data collection, conversion, and entry across heterogeneous business systems, and is applied to a multi-source data intelligent connection management method.

[0045] refer to Figure 1The diagram illustrates the overall technical solution flowchart of a multi-source data intelligent connection management method. The system of this invention is deployed on the terminal device or virtual machine environment where the source business system resides. Its core components include an intelligent agent program, a network communication layer packet sniffing module, a multimodal fusion parsing unit, a data acquisition rule base, a data transformation engine, a large-model-assisted service orchestration and inference module, a scheduled task scheduler, an RPA simulation module, a digital credential generation and verification mechanism, and an extension point injection mechanism. These components work collaboratively to form a closed-loop data intelligent connection management process. (Reference) Figure 2 The diagram shows the core principle framework of the composite perception mechanism that integrates window message listening, network request capture, and multimodal parsing. This mechanism acquires window message streams and network request streams in parallel through dual channels, and performs spatiotemporal alignment and semantic association in the multimodal fusion parsing unit, thereby establishing a precise mapping relationship between operational behavior and data changes.

[0046] The above-mentioned multi-source data intelligent connection management method includes:

[0047] Step 1: Deploy the intelligent agent program in the source business system's runtime environment. The intelligent agent program has operating system-level event listening permissions, monitoring and capturing window message streams generated during user interactions with the business system in real time. These message streams include control creation, focus switching, mouse clicks, and keyboard input events. By performing time-series analysis and behavioral pattern recognition on the event sequences, the complete business operation path is reconstructed, and key data entry nodes are marked. Specifically, the intelligent agent program dynamically selects the underlying event capture scheme based on the operating system platform of the source business system: if running on Windows, it uses Windows API hook technology, registering global hooks of type WH_CALLWNDPROC or WH_GETMESSAGE through the SetWindowsHookEx function to intercept the message queues of all threads; if running on Linux, it uses the X11 event listening mechanism, subscribing to specific window event types such as ButtonPress, KeyPress, and FocusIn through the XSelectInput function. The message sampling frequency is set to 60 times per second to ensure that rapid, continuous user actions can be captured, such as quickly switching between multiple input boxes and filling in content in an order management system. To eliminate high-frequency noise signals caused by hardware jitter or system interference, the system introduces a sliding window denoising algorithm. This algorithm maintains a time window with a length of 5 sampling periods and performs cluster analysis on similar events within the window, such as consecutive mouse movement events. Only when an event occurs more than 3 times within the window is it considered a valid user intent expression, and its timestamp, window handle, control ID, and event parameters are recorded. The captured raw event stream is organized into structured log entries. Each log entry contains fields such as: event type code, nanosecond-level precision occurrence time, source window title, target control name, control coordinates, and input characters. These log entries are stored in chronological order in a circular memory buffer with a maximum capacity of 10,000 entries to support subsequent backtracking analysis.

[0048] Step 2: Simultaneously launch the network communication layer packet sniffing module to intercept and parse all outbound network requests initiated by the source business system. Extract the request headers, request bodies, and response content under the HTTP or HTTPS protocol, identify the structured business data packets carried within, and when encrypted transmission is detected, call the local certificate store to complete TLS decryption and then parse the plaintext payload, ensuring the integrity of the original business data. Specifically, the network communication layer packet sniffing module operates in transparent proxy mode. By modifying the system host's routing table or configuring WinHTTP / WinINET proxy settings, it redirects all TCP connections initiated by the source business system process to the local proxy port. For HTTP plaintext traffic, the module directly parses the application layer data; for HTTPS encrypted traffic, the module first checks whether a trusted root certificate issued by the system itself has been installed on the host where the source system resides. If it has, the module acts as a man-in-the-middle, dynamically generating a temporary certificate matching the target server's domain name and using the private key to complete the TLS handshake, thereby decrypting the communication content between the client and the server. The decryption process is strictly limited to execution within memory space, and the decrypted plaintext payload is immediately sent to the parsing pipeline, prohibiting any form of disk storage to ensure user privacy and security. The module has a built-in protocol identifier that can accurately extract complete HTTP transactions from TCP streams, including the complete request line, request header set, request body content, and corresponding response status code, response header, and response body. For response bodies with chunked transfer encoding or Gzip compression, the module automatically reassembles and decompresses them to restore the original business data packets. All parsed data packets are appended with metadata, including source IP address, destination IP address, destination port number, session ID, and transaction start and end timestamps, to associate them with the window message stream in step 1.

[0049] Step 3: Data fragments from the window message stream and network request stream are input into the multimodal fusion parsing unit. A pre-trained deep neural network model is used to perform spatiotemporal alignment and semantic association between the two, establishing a mapping relationship between operation behavior and data changes, and generating a joint operation-data record set with timestamp labels. This joint record set serves as the original basis for subsequent data extraction. Specifically, the multimodal fusion parsing unit adopts a dual-channel Transformer architecture. The visual channel receives information derived from the window event stream captured in Step 1, including window screenshots taken at key event trigger moments and control tree structure information obtained in real time through the operating system API. The window screenshots are in PNG format with a resolution of 1920x1080, and the control tree structure information is represented in JSON format, including attributes such as control type, name, coordinates, and status. The text channel receives the network message content parsed in Step 2 and the log entries generated in Step 1, including the request URL, request method, key-value pairs in the request body, and structured data in the response body. The two channels use independent encoders to process the input of their respective modalities: the visual channel encoder, composed of a ResNet-50 backbone network, is used to extract image features; the text channel encoder, composed of a BERT-based model, is used to extract text semantic features. Subsequently, the feature vectors from both channels are fed into a cross-attention fusion layer, which allows visual features to query text features and vice versa, thereby establishing a deep semantic association between control operations and network data fields in a unified representation space. The fused feature vectors are then input into a fully connected classification head, which outputs two results: the operation intent category and the mapping relationship between data fields in each network message and UI controls. For example, the "order_id" field in the response body corresponds to the text box with the ID "txtOrderNo" on the interface. Finally, the system aggregates all relevant events and data fragments within the same business transaction to generate an operation-data joint record. This record contains the operation intent, timestamp range, a list of involved controls, a list of associated network messages, and a field mapping table, serving as the sole reliable source for subsequent data extraction.

[0050] Step 4: Perform data filtering based on the preset data collection rule base. Data collection rules consist of field name matching strategies, data type constraints, and trigger timing logic expressions. The rule base supports a graphical configuration interface for code-free definition. The system extracts target business entities from the operation-data joint record set according to the rules, forming a preliminary structured dataset. Specifically, each rule in the data collection rule base is stored in JSON format, and its core structure includes three parts: matching conditions, triggering logic, and output template. Matching conditions define the names of the fields to be monitored and their expected data types, such as requiring the "Carrier" field to be a non-empty string and the "Shipping Date" field to be in valid ISO 8601 date format. The triggering logic is a Boolean expression that supports AND, OR, and NOT logical operators, used to define the precise timing of data capture, such as "trigger data capture when the order status changes to 'shipped' and the carrier field is not empty." The system provides a web-based graphical configuration interface, allowing business personnel to intuitively construct the above rules by dragging and dropping components and selecting drop-down menu options, without writing any code. Before a rule takes effect, the system calls the formal verification module to perform static analysis, checking the syntax correctness of the expression, the validity of variable references, and whether there are any logical conflicts with other activated rules. For example, two rules might trigger under the same condition but have contradictory output templates. Only rules that pass verification are loaded into the runtime engine. When a new operation-data union record is generated, the rule engine iterates through all activated rules, evaluating their triggering logic one by one. Once a rule is satisfied, the engine extracts the specified field values ​​from the union record according to its output template and populates them into a predefined business entity object, thus forming a preliminary structured dataset. This dataset is stored in row-based format, with each row representing a captured business event instance.

[0051] Step 5: Start the data transformation engine and load the pre-configured data transformation rule set. The rule set includes a field mapping table, a format standardization template, a unit conversion coefficient matrix, and validation logic functions. It performs field renaming, numerical normalization, encoding standardization, and null value filling on the initial structured dataset, outputting intermediate data objects that conform to the target business system's interface specifications. Specifically, the data transformation rule set is organized as pluggable plugins, with each plugin corresponding to a specific transformation logic. The field mapping table defines the mapping relationship between source field names and target field names; for example, mapping "cust_name" in the source system to "customer_full_name" required by the target business system. The format standardization template specifies the output format for fields such as date, time, currency, and phone number; for example, it forces all dates to be converted to the "YYYY-MM-DD" format. The unit conversion coefficient matrix contains industry-standard conversion relationships, such as the 1000-fold conversion between tons and kilograms in weight units, and the 1000-fold conversion between cubic meters and liters in volume units. The validation logic functions are used to perform data quality checks, such as bank account verification algorithms or VAT number verification rules. The data transformation engine supports batch processing mode, capable of loading up to 100,000 records at a time for parallel processing. Internally, the engine employs a pipeline architecture, with each record flowing sequentially through various transformation plugins. For null fields, the system provides multiple filling strategies, such as using default values, inheriting from the previous record, or querying external services for completion. After the complete transformation process, the initial structured dataset is reshaped into one or more intermediate data objects conforming to the target business system interface specifications, ready for the service orchestration phase.

[0052] Step 6: Call the large model-assisted service orchestration inference module, input the intermediate data object and the API document summary of the target business system, and the large model will automatically generate API call sequences, parameter binding schemes and exception handling strategies based on natural language understanding capabilities, generate an executable service orchestration script, and encapsulate the script as a microservice component and deploy it in a containerized manner.

[0053] Specifically, the core of the large-model-assisted service orchestration inference module is a language model with billions of parameters, finely tuned to meet specific instructions. The model's input context consists of three parts: first, a summary of the target business system's API documentation, which extracts key interface paths, methods, request parameters, response structures, and authentication methods from the complete OpenAPI / Swagger documentation using information extraction techniques; second, a library of historical successful call examples, containing numerous verified and valid call examples and their corresponding input data; and third, a structural description of the intermediate data object to be sent. The model is specially trained to understand this contextual information and outputs a structured service orchestration scheme. This scheme details which API to call, in what order, how to bind the fields of the intermediate data object to the API's request parameters, and the retry or degradation strategies to be taken when encountering specific HTTP status codes (such as 429 rate limiting or 500 server error). The output is formatted as a JSON Schema definition file conforming to the OpenAPI 3.0 specification, along with accompanying Python call example code. This code is then fed into a syntax validator for static checking to ensure its executability. The verified code is automatically packaged into a Docker image, encapsulated as an independent microservice component, and pushed to the enterprise's container image repository, awaiting activation by the scheduler.

[0054] Step 7: Configure the scheduled task scheduler to activate microservice components according to a preset frequency or event triggering conditions. Processed data is pushed to the target business system via a RESTful interface or message queue protocol. If the target business system does not have an open interface, the RPA simulation module is used to generate a GUI operation command sequence, enabling interactive data entry in a controlled environment. Specifically, the scheduled task scheduler is implemented using the distributed Quartz framework, capable of coordinating task execution across multiple computing nodes. Two triggering modes are supported: Cron expressions for periodic tasks, such as synchronizing inventory daily at 2 AM; and time-series triggering for event-driven tasks, such as processing new data objects immediately upon generation. Each microservice component is registered as an independent Job, and its execution status is reported in real-time to the central monitoring console for operation and maintenance personnel to view. The execution status is indicated as success, failure, or running. For failed tasks, the scheduler automatically adds them to the retry queue, retrying a maximum of three times consecutively. The retry interval increases exponentially, for example, 1 minute for the first retry, 2 minutes for the second, and 4 minutes for the third. When the target business system has a standard RESTful interface or supports message queue protocols such as AMQP and Kafka, microservice components directly push data through these protocols. If the target business system is a closed, legacy system without any open interfaces, the system will activate the RPA simulation module. This module runs in an isolated virtual desktop environment. After receiving intermediate data objects, it uses computer vision and UI element recognition technology to generate a series of precise GUI operation command sequences, such as "click coordinates (120, 200)," simulating manual operation to enter data into the target business system. The entire process is completed in an unattended, controlled environment.

[0055] Step 8: A digital credential generation and verification mechanism is embedded throughout the entire data flow chain. Each piece of business data is bound with a unique identifier upon its initial capture, and a data fingerprint is generated using a hash algorithm. This fingerprint is then digitally signed using an asymmetric encryption algorithm, forming an immutable electronic credential. This credential is transmitted throughout the data flow and undergoes consistency verification at each key node. Specifically, the security strength of the digital credential meets the commercial cryptographic algorithm standards SM2 / SM3 / SM4. When a piece of business data is confirmed in Step 3 and an operation-data joint record is generated, the system immediately assigns it a globally unique 128-bit UUID as an identifier. Subsequently, the system calls the SM3 hash algorithm to calculate the core content of the joint record, generating a 256-bit data fingerprint. This fingerprint is sent to the hardware security module, where it is digitally signed by the SM2 elliptic curve public-key public-key system private key within the hardware security module. The signing operation is entirely completed within the HSM of the hardware security module, and the private key is never exported. The generated digital signature, along with the original data fingerprint and UUID, constitutes a complete electronic credential. This credential is attached to every data object in the subsequent flow. At each critical juncture in the data processing flow, the system recalculates the SM3 fingerprint of the current data object and verifies the accompanying digital signature using the corresponding public key. If verification fails, the process is immediately interrupted and an alarm is triggered, ensuring that any data tampering can be detected instantly.

[0056] Step 9 establishes an extension point injection mechanism, reserving plugin modules at key stages of the data processing flow. This allows external programs to insert custom processing logic through registered callback functions. Plugin modules support three execution modes: pre-validation, post-notification, and exception interception, enabling dynamic enhancement of business rules and flexible integration of audit trail functionality. Specifically, the extension point injection mechanism pre-defines standardized plugin modules before and after core steps such as data acquisition, data transformation, and service invocation. The system provides SDK development packages in both Python and Java, allowing third-party developers to write processing plugins. Plugins must implement the specified interface methods and declare their expected execution modes: the pre-validation mode allows the plugin to check the legality of input data before the main logic executes and may terminate the process; the post-notification mode allows the plugin to perform additional operations, such as sending notifications or updating audit logs, after the main logic successfully completes; and the exception interception mode allows the plugin to capture and handle exceptions thrown by the main logic. All plugins must undergo rigorous sandbox environment testing before activation, including static code scanning to check for dangerous system calls and dynamic behavior monitoring to monitor their CPU, memory, network I / O, and other resource consumption. Only plugins that pass the tests can be activated. To ensure system stability, the resources available for each plugin are strictly limited, with a maximum CPU usage of 20% and a maximum memory usage of 300MB.

[0057] To further enhance the system's traceability and intelligence, this embodiment also includes establishing a globally unique business data traceability graph. This graph uses a graph database as its storage backend, recording each data capture, transformation, and transmission event as a node. Node attributes include event type, timestamp, operator, and involved data objects. Nodes are connected by "data lineage" edges, clearly showing a complete data flow path from source to final destination. Users can perform penetrating queries using order numbers, contract numbers, or multi-level supplier paths to fully trace the lifecycle of any business data. Furthermore, the system is equipped with an adaptive learning module. This module periodically collects data processing failure cases and manual correction results, forming a high-quality labeled dataset, which is then fed back to the large model training pipeline for incremental fine-tuning. This continuous learning mechanism enables the system to steadily improve its accuracy in areas such as field mapping recommendation and anomaly recovery strategy generation, with a monthly accuracy improvement of no less than 2 percentage points. Meanwhile, the system is equipped with a multi-level access control policy, which allocates data access scope according to organizational roles. Ordinary operators can only view the data flow logs of their own departments, while administrators can access the full-process audit records. All sensitive operations require two-factor authentication authorization and are logged. All log records are kept for no less than 5 years to meet compliance audit requirements.

[0058] Example 2: Alternative Implementation Scheme – Lightweight Rule-Driven Orchestration and RPA Enhancement Method

[0059] This embodiment is an alternative to Embodiment 1. For resource-constrained scenarios, it replaces the large model orchestration with a rule-driven orchestration engine, generating calling scripts based on a predefined service template library, thus reducing computational resource requirements. Simultaneously, the RPA module integrates OCR and template matching technologies to improve integration stability when the interface changes. The digital certificate uses national cryptographic algorithms, and the hardware security module is replaced with a TPM software simulator, suitable for ordinary PC environments.

[0060] To address the diverse needs of enterprises of different sizes and suppliers of different technology stacks, this invention also provides another implementation method, which optimizes the service orchestration and execution mode while maintaining consistency with the core awareness and security mechanisms.

[0061] In this embodiment, the large model-assisted service orchestration inference module is replaced by a lightweight rule-driven orchestration engine. This engine does not rely on a large language model but is based on a predefined, highly structured service template library. Each template corresponds to a common target business system type, such as an ERP system, database system, or enterprise management system. The template hard-codes the calling logic, parameter mapping rules, and error handling procedures for the system's standard interfaces. When the target business system is identified as a known type in the template library, the system directly loads the corresponding template, fills the intermediate data objects into preset placeholders, and generates an executable script. While this approach sacrifices some generalization ability, it significantly reduces the demand for computing resources, allowing the invention to be deployed on resource-constrained edge devices and making it suitable for small and medium-sized enterprise scenarios. Simultaneously, the RPA simulation module is enhanced, integrating a more robust UI element recognition algorithm. This algorithm combines OCR text recognition and template matching technology, enabling accurate positioning of operation controls even with slight changes to the target business system interface, such as button position shifts or color changes, thus improving the stability of interface-less system integration. In this embodiment, the digital credential mechanism also adopts the national cryptographic standard SM2 / SM3, but the hardware security module is replaced by a software simulator based on a trusted platform module chip to adapt to ordinary PC environments that do not have dedicated hardware security modules, ensuring that basic-level data tamper protection can still be provided in cost-sensitive scenarios.

[0062] Example 3: Enhanced Implementation Plan—A Large Model Orchestration Method Based on Structured Prompt and Validator

[0063] This embodiment is an enhancement of step 6 in embodiment 1. It guides the large model to generate JSON output through a structured Prompt (role setting, input context, task description, and output format constraints) and executes a four-layer validation pipeline (syntax validation, semantic validation, type validation, and idempotency detection). The script is only deployed if all validations pass; otherwise, generation is rejected, effectively reducing the risk of large model illusion.

[0064] To further improve the reliability and determinism of the large model generation service orchestration script and reduce the risks caused by large model hallucination, this embodiment provides a specific prompt word engineering scheme and automated verification logic for the output results in step 6.

[0065] The Prompt sent by the system to the large model uses a structured template and includes the following four core parts: role setting, input context, task requirements, and output format constraints. A specific example is shown below:

[0066] {

[0067] "system_prompt": "You are a senior API orchestration engineer specializing in mapping heterogeneous business data to RESTful interfaces of target business systems. Your task is to generate JSON schemas that strictly conform to the OpenAPI 3.0 specification, without including any additional interpretations or comments."

[0068] "user_prompt": {

[0069] "Target Business System API Summary": {

[0070] "API path": "POST / api / v2 / sales / order",

[0071] Authentication method: "Bearer Token"

[0072] "Request body structure": {

[0073] "orderNo": "string (required, maximum 32 characters)",

[0074] "amount": "number (required, unit: yuan)",

[0075] "weightKg": "number (optional, unit: kilograms)",

[0076] "carrierCode": "string (required, enumeration values: C01, C02, C03)"

[0077] }

[0078] },

[0079] "Current data object structure to be sent": {

[0080] "source_order_id": "PO-20251223-001",

[0081] "total_amount": 12500.50,

[0082] "weight_ton": 2.5,

[0083] "shipping_company": "First carrier"

[0084] },

[0085] "Historical Successful Call Case Library": [

[0086] {

[0087] Input: {"source_order_id": "PO-001", "total_amount": 100, "weight_ton": 0.5, "shipping_company": "Secondary carrier"},

[0088] "Output Mapping": {"orderNo": "PO-001", "amount": 100, "weightKg": 500, "carrierCode": "C02"}

[0089] }

[0090] ],

[0091] Task Description: "Please generate a JSON Schema mapping based on the API specifications and data objects described above. Note: Weight units must be converted from tons to kilograms (1 ton = 1000 kilograms). Carrier names must be mapped to enumeration codes: 'First Carrier' → 'C01', 'Second Carrier' → 'C02', 'Third Carrier' → 'C03'. If mapping fails, use an empty string."

[0092] Output format constraints: Output only one JSON object containing the fields 'api_path', 'method', 'auth_type', and 'param_binding', where 'param_binding' is a key-value pair, with the key being the target business system field name and the value being the source field name or a constant expression. Outputting any Markdown tags or natural language is prohibited.

[0093] }

[0094] }

[0095] Expected large model output example, outputting only plain JSON:

[0096] {

[0097] "api_path": " / api / v2 / sales / order",

[0098] "method": "POST",

[0099] "auth_type": "Bearer Token",

[0100] "param_binding": {

[0101] "orderNo": "source_order_id",

[0102] "amount": "total_amount",

[0103] "weightKg": "weight_ton 1000",

[0104] "carrierCode": "case when shipping_company='First Carrier' then 'C01' when shipping_company='Second Carrier' then 'C02' when shipping_company='Third Carrier' then 'C03' else '' end"

[0105] }

[0106] }

[0107] Output validation logic

[0108] The JSON output generated by the large model cannot be executed directly; it must undergo multiple layers of validation. This embodiment designs a four-layer validation pipeline:

[0109] First layer: Syntax and Schema validation

[0110] Use a JSON Schema validator to check if the output conforms to the predefined format (it must contain fields such as api_path, method, and param_binding).

[0111] Check if the method value is an allowed HTTP verb (GET, POST, PUT, DELETE, etc.).

[0112] Check if api_path exactly matches the path in the API summary provided by the target business system.

[0113] Second layer: Semantics and reference verification

[0114] Iterate through all values ​​in param_binding, extract variable names (such as source_order_id), and check if the variable exists in the "current data object structure to be sent".

[0115] If the variable does not exist, a fallback mechanism is triggered: either the lightweight rules engine (the replacement solution in step 6) is invoked or the Prompt is regenerated and the missing field is marked.

[0116] Detect expression safety (e.g., weight_ton) (1000), prohibiting calls to dangerous functions such as eval, exec, and __import__.

[0117] Third layer: Type and constraint verification

[0118] Simulate the execution of the param_binding expression, using sample data from the "current data to be sent object" for calculation.

[0119] Verify whether the calculation results meet the requirements of the target business system API:

[0120] Type checking: For example, orderNo should be a string, and amount should be a number.

[0121] Length / range check: For example, if the string length is ≤32, the value must be non-negative.

[0122] Enumeration check: For example, the result of the carrierCode calculation must be within [C01,C02,C03].

[0123] If the verification fails, the system will automatically attempt to correct it: for example, by truncating the floating-point number to an integer, truncating an excessively long string, or filling in the default enumeration value.

[0124] Fourth layer: Idempotency and conflict detection

[0125] Check if there are multiple assignments to the same target field in the same param_binding.

[0126] Check for circular dependencies (e.g., field A depends on field B, and field B depends on field A).

[0127] Once the validation is successful, the generated JSON Schema is bound to a unique validation_id and stored in the validated script library.

[0128] Final activation logic:

[0129] Only when all four layers of verification pass will the generated orchestration script be packaged into a microservice component and pushed to the container image repository.

[0130] If any verification fails, the system will refuse to generate the data and record the reason for the failure, while simultaneously triggering an alarm to notify the operations and maintenance personnel. The system will not automatically retry more than 3 times to avoid unnecessary resource consumption.

[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent connection management of multi-source data, characterized in that: The specific steps include the following: Step 1: Deploy the intelligent agent program in the source business system's operating environment to monitor and capture the window message flow generated during the interaction between the user and the business system in real time. By performing time sequence analysis and behavior pattern recognition on the window message flow, the complete business operation path is restored, and key data is tagged with time sequence and entered into the nodes. Step 2: Synchronously start the network communication layer packet sniffing module to intercept and parse all outbound network requests initiated by the source business system. When encrypted transmission is detected, call the local certificate store to complete TLS decryption and then parse the plaintext payload. Step 3: Input the data fragments from the window message stream and the network request stream into the multimodal fusion parsing unit. The multimodal fusion parsing unit adopts a dual-channel Transformer architecture, namely a visual channel and a text channel. The two channels extract features respectively and then fuse them through a cross-attention mechanism to output the fused feature vector. Then, the classification head determines the operation intention category and the corresponding data field belonging, establishes the mapping relationship between operation behavior and data change, and generates an operation-data joint record set. Step 4: Perform data filtering according to the preset data collection rule base. The system extracts target business entities from the operation-data joint record set according to the rules to form a preliminary structured dataset. Step 5: Start the data transformation engine, load the pre-configured data transformation rule set, process the initial structured dataset, and output intermediate data objects that conform to the target business system interface specifications; Step 6: Call the large model-assisted service orchestration inference module, and use the preset API semantic mapping template, combined with the parameter semantic features extracted by the large model, to automatically generate an executable service orchestration script containing API call sequences, parameter binding schemes and exception handling strategies. The script is encapsulated as a microservice component and deployed using a containerized approach; Step 7: Configure a scheduled task to activate microservice components according to a preset frequency or event triggering conditions, and complete the interaction between data and the target business system; Step 8: Integrate digital credential generation and verification mechanisms throughout the entire data flow chain; Step 9: Establish an extension point injection mechanism, and reserve plugin interfaces in key stages of the data processing flow to allow external programs to insert custom processing logic by registering callback functions.

2. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 1, window message listening uses Windows API hook technology or Linux X11 event listening mechanism, with the message sampling frequency set to 60 times per second. A sliding window denoising algorithm is introduced to filter high-frequency jitter signals and retain effective user intent expressions. The window message stream includes control creation, focus switching, mouse click, and keyboard input events.

3. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 2, the network communication layer packet sniffing module is configured with TLS decryption proxy function. By installing a trusted root certificate on the host where the source system is located, HTTPS traffic can be decrypted by a man-in-the-middle. The decrypted data is only temporarily stored in memory and processed immediately, and is prohibited from being stored on disk. The interception and parsing refers to extracting the request header, request body and response content under the HTTP or HTTPS protocol, and identifying the structured business data packets carried therein.

4. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 3, the visual channel receives window screenshots and control tree structure information, and the text channel receives network packets and log entries; the mapping relationship is achieved by using a pre-trained deep neural network model to perform spatiotemporal alignment and semantic association on data fragments of the window message stream and network request stream input into a multimodal fusion parsing unit; the operation-data joint record set refers to an operation-data joint record set with timestamps.

5. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 4, the data collection rule base supports Boolean logic combination conditions. Before a rule takes effect, its syntax correctness and conflict resistance are checked by a formal verification module. The data collection rule consists of field name matching strategy, data type constraints, and triggering timing logical expression.

6. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 5, the data conversion rule set includes a field mapping table, a format standardization template, a unit conversion coefficient matrix, and a verification logic function; the data conversion rule set includes an industry standard comparison table, covering the 1000-fold conversion relationship between tons and kilograms, and a bank account verification algorithm. The conversion process supports batch processing mode, with a maximum of 100,000 records processed per batch and an average processing time of less than 800 milliseconds; the processing of the preliminary structured dataset includes field renaming, numerical normalization, encoding unification, and null value filling.

7. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 6, the service orchestration inference module generates an executable service orchestration script based on the interface capabilities and computing resource configuration of the target business system, using any of the following methods: For scenarios where the target business system has a standard RESTful interface and sufficient computing resources: a large language model with fine-tuned instructions is adopted. The input context includes API document fragments of the target business system, a library of historical successful call cases, and a description of the data structure to be sent. The output is a JSON Schema definition file conforming to the OpenAPI 3.0 specification and call example code. After being verified by a syntax validator, it is converted into a schedulable task. For edge deployment scenarios where the target business system interfaces are known and computing resources are limited: a rule-driven orchestration engine is adopted, which does not rely on large models and generates executable scripts based on a predefined service template library. The service template library contains the interface call logic, parameter mapping rules and error handling processes of ERP system, database system and enterprise management system. For business scenarios with extremely high script generation reliability requirements and the need to prevent large model illusions: a structured prompt engineering is adopted to send a structured prompt containing role settings, input context, task description and output format constraints to the large model. After receiving the JSON format output, a four-layer verification pipeline is executed. The four layers of verification include syntax and schema verification, semantic and reference verification, type and constraint verification and idempotency and conflict detection. Only when all four layers of verification pass can the generated script be packaged into a microservice component.

8. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 7, the interaction process refers to pushing the processed data to the target business system through a RESTful interface or message queue protocol. If the target business system does not have an open interface, the RPA simulation module is activated. The RPA simulation module uses computer vision and UI element recognition technology, combined with OCR text recognition and template matching, to generate a GUI operation instruction sequence and complete the human-computer interactive data entry in a controlled environment. In step 8, the security strength of the digital certificate meets the commercial cryptographic algorithm SM2 / SM3 / SM4 standards. The data fingerprint calculation adopts the SM3 hash algorithm, the signing process uses the SM2 elliptic curve public key system, the private key is stored in the hardware security module HSM, and the signing operation is completed inside the HSM. Alternatively, for ordinary computing environments without a dedicated hardware security module, the hardware security module is replaced with a software simulator based on a trusted platform module. The digital certificate generation and verification mechanism refers to binding a unique identifier to each piece of business data when it is first captured, generating a data fingerprint through a hash algorithm, and digitally signing the fingerprint using an asymmetric encryption algorithm to form an immutable electronic certificate. This certificate is transmitted throughout the data flow and undergoes consistency verification at each key node.

9. The multi-source data intelligent connection management method according to claim 1, characterized in that: In step 9, the extension point injection mechanism provides a standardized SDK development package, allowing third-party developers to write processing plugins in Python or Java. After the plugin is uploaded, it can be enabled only after static scanning and dynamic behavior monitoring in the sandbox environment confirms that there is no malicious operation. The upper limit of plugin runtime resource usage is set to 20% CPU usage and 300MB memory. The plugin interface supports three execution modes: pre-verification, post-notification, and exception interception.

10. The multi-source data intelligent connection management method according to claim 1, characterized in that: Also includes: Establish a globally unique business data traceability graph, recording each data capture, transformation, and transmission event as a node in the graph. Nodes are connected by data lineage edges, supporting penetrating queries by order number, contract number, or multi-level supplier paths. Configure an adaptive learning module to regularly collect data processing failure cases and manual correction results, and feed them back to the large model training pipeline for incremental fine-tuning; Set up multi-level access control policies, allocate data access scope according to organizational roles, require two-factor authentication authorization for all sensitive operations and keep log records, and retain log records for no less than 5 years.

Images