Industrial internet of things heterogeneous device log processing method and device, equipment and medium

Abstract

The application discloses a heterogeneous device log processing method and device of an industrial Internet of Things, an apparatus and a medium, relates to the technical field of Internet of Things logs, and comprises the following steps: creating a runtime instance and initializing a log module; collecting log data of an application processor and a communication processor to form a unified log stream; selecting a log analysis strategy according to a source identifier to generate structured log analysis data; and receiving configuration update data to perform dynamic hot updating. The heterogeneous log processing is decoupled from the AP firmware, all logs are isolated and processed in a WASM sandbox, the stability of a host program and an AP system is not affected by an exception, the log module and the strategy configuration can be dynamically and hotly updated, log levels and filtering rules can be adjusted in real time in response to requirements, atomic switching and automatic fallback ensure business continuity, log collection and processing are not interrupted, differential configuration updating effectively reduces network transmission overhead, and the application is particularly suitable for an industrial Internet of Things edge computing scene in which network bandwidth and computing resources are limited.

Description

Technical Field

[0001] This invention relates to the field of IoT log technology, and in particular to a method, apparatus, device, and medium for processing logs of heterogeneous devices in the industrial IoT. Background Technology

[0002] In modern IoT applications such as industrial automation and smart security, the hardware architecture of devices commonly adopts a heterogeneous combination of application processors (APs) and communication processors (CPs). APs typically run general-purpose operating systems such as Linux or Android, handling business logic processing; while CPs (communication modules / modems) run real-time operating systems, specifically handling wireless communication protocols such as 5G / 4G. To achieve comprehensive monitoring and fault diagnosis of device operation, the logging system needs to simultaneously collect and process log information from both the AP and CP processors. In traditional solutions, the log service running on the AP requires writing dedicated native code to parse the CP's log format and process it uniformly with the AP's own logs.

[0003] However, the native libraries used for parsing CP logs are tightly coupled to the firmware version of specific CP models, and are also bound to the CPU architecture and operating system of the application processor (AP). The module that parses CP logs handles untrusted external input and lacks security isolation mechanisms. In particular, when hardware is replaced or firmware is upgraded, it requires re-porting, recompiling, and testing, resulting in high R&D and maintenance costs. The dominance of C / C++ in embedded development languages ​​limits the choice of languages ​​and prevents the utilization of the memory safety and concurrency advantages of modern languages. Furthermore, log filtering rules and output formats are hardcoded into the code, leading to rigid operation and maintenance. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for processing logs of heterogeneous devices in the Industrial Internet of Things (IIoT), in order to solve the technical problems of high R&D and maintenance costs and insufficient security and flexibility caused by rigid logic in heterogeneous log processing in the IoT.

[0005] In a first aspect, embodiments of the present invention provide a method for processing logs of heterogeneous devices in the industrial Internet of Things, including:

[0006] S101: Create a WebAssembly runtime instance and set resource limit parameters. Then, load the WebAssembly logging module while checking its version compatibility and initializing the WebAssembly logging module.

[0007] S102, the first log data is collected through the host program of the AP application processor, the second log data is collected from the CP communication processor through the communication interface, and corresponding source identifiers are added to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module;

[0008] S103, the WebAssembly log module receives the unified log stream, selects the corresponding preset log parsing strategy according to the source identifier, performs log parsing, and generates structured log parsing data;

[0009] S104 receives configuration update data from the cloud and performs incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

[0010] Secondly, embodiments of the present invention provide a heterogeneous device log processing apparatus for the Industrial Internet of Things, comprising:

[0011] The initialization unit is used to create a WebAssembly runtime instance and set resource limit parameters. Then, it loads the WebAssembly logging module, checks its version compatibility, and initializes the WebAssembly logging module.

[0012] The log collection unit is used to collect first log data through the host program of the AP application processor, collect second log data from the CP communication processor through the communication interface, and add corresponding source identifiers to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module.

[0013] The log parsing unit is used by the WebAssembly log module to receive the unified log stream, select the corresponding preset log parsing strategy according to the source identifier, perform log parsing, and generate structured log parsing data.

[0014] The update unit is used to receive configuration update data from the cloud and perform incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

[0015] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for processing logs of heterogeneous devices in the Industrial Internet of Things.

[0019] Fourthly, embodiments of the present invention provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the above-described heterogeneous device log processing method for the Industrial Internet of Things.

[0020] This invention provides a method, apparatus, device, and medium for processing logs of heterogeneous devices in the Industrial Internet of Things (IIoT). The method involves creating a WebAssembly runtime instance and setting resource limitation parameters, loading a WASM log module, and performing a version compatibility check for initialization. It collects first log data from the AP application processor and second log data from the CP communication processor, adds source identifiers to each, forms a unified log stream, and transmits it to the WASM log module. The WASM log module selects a corresponding preset log parsing strategy based on the source identifier, performs log parsing, and generates structured log parsing data. It also receives configuration update data from the cloud and performs differential updates to the WASM log module. It decouples multi-source heterogeneous log processing logic from AP firmware. Even if the log format changes due to CP firmware upgrades, only the WASM log module needs to be updated, without modifying or recompiling the AP firmware, reducing operation and maintenance costs. By dynamically replacing the WASM log module and policy configuration during device operation, hot updates of log parsing capabilities are achieved. Log levels and filtering rules can be adjusted in real time according to actual needs without restarting the device or upgrading the firmware, improving response efficiency. All log processing logic is executed in a WebAssembly sandbox and is subject to strict resource limit parameters. Even if anomalies occur during log module processing, it will not affect the stable operation of the host program and AP system, providing secure isolation at the architectural level. The WebAssembly logging module is compiled into platform-independent standard bytecode, allowing deployment on AP platforms with different CPU architectures without modification. This enables write-once, cross-platform deployment, eliminating repetitive porting and adaptation work. Development teams can choose to develop the logging module in languages ​​such as Rust, C++, and Go, fully leveraging the memory safety and concurrency advantages of each language, breaking the limitation of embedded development to C / C++. Atomic switching and automatic rollback mechanisms ensure business continuity in the event of module update failures, ensuring uninterrupted log collection and processing. Differential configuration updates effectively reduce network transmission overhead, making it particularly suitable for industrial IoT edge computing scenarios with limited network bandwidth and computing resources. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0022] Figure 1This is a flowchart of a heterogeneous device log processing method for industrial Internet of Things as described in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of the hot update process of the WASM log module according to Embodiment 1 of the present invention;

[0024] Figure 3 This is a flowchart of a heterogeneous device log processing method for industrial Internet of Things as described in Embodiment 2 of the present invention;

[0025] Figure 4 This is a flowchart of the batch processing mode switching and back pressure threshold judgment described in Embodiment 2 of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a heterogeneous device log processing device for industrial Internet of Things as described in Embodiment 3 of the present invention;

[0027] Figure 6 This is an architecture diagram of an optional implementation of a heterogeneous device log processing device for industrial Internet of Things as described in Embodiment 3 of the present invention;

[0028] Figure 7 This is a structural diagram of the electronic device described in Embodiment 4 of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0030] Example 1

[0031] Figure 1 The flowchart below illustrates a method for processing heterogeneous device logs in the Industrial Internet of Things (IIoT) according to Embodiment 1 of the present invention. By decoupling the log processing logic from the underlying firmware and executing it in a secure sandbox, and supporting dynamic updates, the method processes heterogeneous IoT logs. Specifically, it includes the following steps:

[0032] S101: Create a WebAssembly runtime instance and set resource limit parameters. Then, load the WebAssembly logging module while checking its version compatibility and initializing the WebAssembly logging module.

[0033] This 5G-based IoT industrial edge gateway uses an ARM-based application processor (AP) running a Linux operating system. The communication processor (CP) uses a third-party 5G communication module and connects to the AP via a serial port. This type of edge gateway is widely used in IoT. The WebAssembly (WASM) logging module is developed in languages ​​such as Rust, C++, or Go and compiled into standard WASM bytecode, independent of CPU architecture. The same bytecode can run directly on APs with different architectures. The host program is a log service process running in the AP's operating system, responsible for log collection, WebAssembly runtime management, log processing scheduling, queue management, and output. Upon gateway startup, initialization is performed by calling the WebAssembly (WASM) runtime library's API to create a new runtime instance. During creation, the host program sets resource limits for the WebAssembly (WASM) runtime instance through the runtime's configuration interface, such as a memory limit of 4MB and a timeout limit of 5 milliseconds for single function calls. This ensures that even if the WebAssembly (WASM) logging module encounters an exception, it will not exhaust the AP's system resources, guaranteeing the stability of both the host program and the AP system.

[0034] After the WASM runtime instance is created, the host program reads the pre-deployed WASM logging module file with the .wasm extension and its associated JSON-formatted policy configuration file from the file system, loads them into the created runtime instance, and performs a version compatibility check. If the check passes, it calls the WASM logging module's init function, passing the configuration string read from the policy configuration file as a parameter. The WASM logging module's internal initialization logic parses this string and caches the parsed configuration object in the WASM logging module's linear memory for direct use during subsequent log processing, avoiding repeated parsing.

[0035] S102, the first log data is collected through the host program of the AP application processor, the second log data is collected from the CP communication processor through the communication interface, and corresponding source identifiers are added to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module.

[0036] For the local logs of the AP application processor, the host program can obtain and form the first log data by listening to the system log interface. For the communication logs of the CP communication processor, the host program can asynchronously read and form the second log data through the serial port device file. The raw logs output by the CP module are usually in AT command response format or a text format defined by the module manufacturer. After collecting the logs, a corresponding source identifier is added to each collected log. The source identifier includes two levels: processor type identifier and subsystem identifier. The processor type identifier is used to distinguish whether the log comes from the AP application processor or the CP communication processor, and the subsystem identifier is used to identify the specific subsystem to which the log belongs. For example, for the first log data from the AP, the identifier AP and the subsystem name such as kernel or network can be added; for the second log data from the CP, the identifier CP and the module model identifier can be added, so that subsequent processing can distinguish the source of the log based on the source identifier and call the corresponding parsing logic. After adding the source identifier, the host program encapsulates the first and second log data into a unified internal data structure, forming a unified log stream containing fields such as timestamp, source identifier, and original log content, and transmits it to the process_log function of the WASM log module through the calling interface provided by the WASM runtime.

[0037] S103, the WebAssembly logging module receives the unified log stream, selects the corresponding preset log parsing strategy based on the source identifier, performs log parsing, and generates structured log parsing data.

[0038] After receiving the unified log stream, the WebAssembly (WASM) logging module first reads the source identifier from its data structure and then looks up the corresponding parsing strategy from the cached configuration object based on the source identifier. The parsing strategy defines the field mapping rules, regular expression templates, and a list of masked fields for that source log. The WASM logging module performs log parsing according to the parsing strategy. For AP logs, it uses regular expression-based parsing to extract fields such as timestamp, process ID, log level, and message content; for CP logs, it performs structured parsing according to a specific format (e.g., [time] + [level] + [module] + [message content]). After parsing, the extracted fields are combined into structured log parsing data in JSON format and returned to the host program. The host program then determines the output method based on its configuration: either writing to local storage or reporting to the cloud management platform.

[0039] S104 receives configuration update data from the cloud and performs incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

[0040] To flexibly adjust log processing strategies during IoT edge gateway operation and improve gateway adaptability for deployment with new devices and environments, configuration updates for the WASM log module can be delivered via cloud-based configuration updates. The update package can be in JSON format, containing a `version` field representing the version number and a `changes` field representing the changes. The `changes` field follows the JSON Patch specification, describing only the fields to be modified and their new values. The need for a configuration update is determined by comparing the currently effective version number with the update package version number; an update is only initiated when the update package version number is higher than the currently effective version number. At this point, the `changes` field in the update package is checked. If the `changes` field is not empty and contains at least one change operation, it is considered an incremental update; if the `changes` field is empty or does not exist, it is considered a full update. Full updates are typically used for initial configuration delivery or configuration reset scenarios.

[0041] During incremental updates, a configuration fragment containing all changed fields is constructed by extracting the change operations from the changed fields. This fragment is then passed as a parameter to the `update_config` function of the WASM logging module. Upon receiving this fragment, the configuration update logic within the WASM logging module iterates through each field, merging the new values ​​into the cached configuration object and overwriting the original values ​​to complete the differential configuration update. Full updates can be performed by directly calling the `init` function, passing in the complete policy configuration file, to overwrite the existing configuration.

[0042] Optionally, receive the new WebAssembly log module update package from the cloud, perform integrity checks based on hash value verification, version compatibility checks based on temporary sandboxes, and functional verification. After successful verification, mark the current WebAssembly log module as a backup and retain it through atomic switching, and switch the new WebAssembly log module as the production execution module.

[0043] To ensure business continuity, the WASM log module can be dynamically hot-updated during device operation without requiring a device reboot or firmware upgrade. For example, such as... Figure 2The diagram illustrates the hot update process for the WASM log module. When the cloud management platform distributes a new version of the WASM log module update package, it first pushes the new module's file and hash value to the IoT edge gateway via the OTA channel. The host program, upon receiving the new module, first saves it to temporary storage and performs hash value verification. If verification fails, the new module is discarded and an error is reported; if verification is successful, the module switching process begins. An independent temporary WASM runtime instance sandbox is created in memory, isolated from the production instance. The newly distributed WASM module is loaded into this temporary sandbox, and a version compatibility check is performed to verify whether the new module exports the necessary init and process_log functions, and whether the function signature matches the host program's expectations. After passing these checks, the host program calls the new WASM log module's init function, passing in a set of test log data containing typical AP and CP log examples to verify whether the new module functions correctly. After successful verification, the host program performs an atomic switch operation, marking the instance of the current production module (old module) in memory as a backup and preserving its state; switching the new WASM log module in the temporary sandbox to the production module and starting to process subsequent logs, while keeping the bytecode file of the old module in the storage area as a rollback backup.

[0044] If the new module fails, it will automatically revert to the backup of the old WebAssembly log module.

[0045] After the switchover is complete, the host application continuously monitors the running status of the new WASM logging module. If, within a preset time window (e.g., 10 seconds), the new WASM logging module experiences three consecutive execution exceptions (such as timeout, memory overflow, or function call error codes), the host application automatically triggers a rollback, stops using the new module, reloads the backed-up old module instance, resumes log processing, and reports the update failure event to the cloud. This atomic switchover and automatic rollback mechanism ensures the continuity of log processing services, preventing interruption of log collection or loss of log data even in the event of a log module update failure.

[0046] This embodiment creates a WebAssembly runtime instance and sets resource limitation parameters, loads the WASM log module, and performs a version compatibility check for initialization. It collects first log data from the AP application processor and second log data from the CP communication processor, adds source identifiers to each, forms a unified log stream, and transmits it to the WASM log module. The WASM log module selects the appropriate preset log parsing strategy based on the source identifier, performs log parsing, and generates structured log parsing data. It also receives configuration update data from the cloud and performs differential updates to the WASM log module. This achieves decoupling of multi-source heterogeneous log processing logic from the AP firmware. Even if the log format changes due to CP firmware upgrades, only the WASM log module needs to be updated, without modifying or recompiling the AP firmware, reducing maintenance costs. By dynamically replacing the WASM log module and policy configuration during device operation, hot updates of log parsing capabilities are achieved. Log levels and filtering rules can be adjusted in real time according to actual needs without restarting the device or upgrading firmware, improving response efficiency. All log processing logic is executed in the WebAssembly sandbox and is subject to strict resource limitation parameters. Even if anomalies occur during log module processing, it will not affect the stable operation of the host program and the AP system, providing security isolation guarantees at the architectural level. The WebAssembly logging module is compiled into platform-independent standard bytecode, which can be deployed to AP platforms with different CPU architectures without modification, achieving cross-platform deployment with a single write and eliminating repetitive porting and adaptation work. The development team can choose to develop the logging module in modern languages ​​such as Rust, C++, and Go according to their needs, giving full play to the memory safety, concurrency and other advantages of each language, breaking the limitation of embedded development to C / C++. The atomic switching and automatic rollback mechanism ensures business continuity in the event of module update failure, ensuring that log collection and processing are uninterrupted. Differential configuration updates can effectively reduce network transmission overhead, which is especially suitable for industrial IoT edge computing scenarios with limited network bandwidth and computing resources.

[0047] Example 2

[0048] Figure 3 This is a flowchart of a heterogeneous device log processing method for industrial IoT according to Embodiment 2 of the present invention. This embodiment is based on the above embodiment and optimized. In this embodiment, S103 is specifically optimized as follows:

[0049] The host program maintains a log buffer queue based on the received unified log stream and determines the log processing mode based on the length of the log buffer queue.

[0050] The log buffer queue length is compared with the preset high watermark threshold. If it exceeds the high watermark threshold, the single processing mode is switched to batch processing mode, and multiple logs in the log buffer queue are packaged into a batch and the batch processing function of the WebAssembly log module is called.

[0051] The log buffer queue length is compared with the preset back pressure threshold. If the back pressure threshold is exceeded, a sampling signal is sent to the host program of the AP application processor to trigger the log sampling down-frequency strategy and suspend the collection of the second log data of the CP communication processor from the communication interface.

[0052] When the WebAssembly logging module fails, the unified log stream can be directly encapsulated and transparently transmitted by switching to degraded mode to bypass the WebAssembly logging module.

[0053] Accordingly, the method for processing logs of heterogeneous devices in the Industrial Internet of Things provided in this embodiment specifically includes:

[0054] S201: Create a WebAssembly runtime instance and set resource limit parameters. Then, load the WebAssembly logging module while checking its version compatibility and initializing the WebAssembly logging module.

[0055] Specifically, it reads the pre-deployed WebAssembly log module and its policy configuration file, creates a WebAssembly runtime instance on the edge gateway, and sets the resource limit threshold and execution time limit threshold.

[0056] When the edge gateway starts, the host program runs automatically as a system service. It first reads the WASM logging module and policy configuration file pre-deployed in the file system. For example, the WASM logging module file is located at ` / usr / lib / wasm / log_processor.wasm`, and the policy configuration file is located at ` / etc / wasm / log_policy.json`. The policy configuration file is in JSON format and includes configuration items such as `global_log_level` (global log level threshold), `subsystem_overrides` (subsystem level override rules), `output_format` (output format), `sensitive_fields` (list of fields to be anonymized), `enable_batch` (whether batch processing mode is enabled), `batch_size` (batch size), `high_watermark` (high watermark threshold), and `backpressure_threshold` (backpressure threshold). Then, it creates a WASM runtime instance on the edge gateway's access point by calling the `wasm_runtime_create` function of the WASM runtime library, such as the WebAssembly Micro Runtime (WAMR) library optimized for embedded devices. During the creation process, resource limit thresholds are set through the configuration interface provided at runtime: the maximum available memory limit is set to 4MB (set via the wasm_runtime_set_memory_limit function); the execution time limit threshold is set to 5 milliseconds for the execution time of a single function call (set via the wasm_runtime_set_timeout function); the limit parameters need to match the AP's hardware resources to ensure that the log module has sufficient resources to complete the log parsing task, and that even if an exception occurs, the AP system resources will not be exhausted.

[0057] Load the WebAssembly logging module into the runtime and perform a version compatibility check.

[0058] An optional implementation of this embodiment is as follows: the version compatibility check includes: enumerating the list of functions exported by the WebAssembly logging module, verifying whether the required init function and process_log function are included, and checking that the function signature is consistent with the host program's expectations. After passing the check, the WebAssembly logging module is initialized.

[0059] The host program obtains a list of exported items of loaded modules through the API enumeration module of the WASM runtime. This list records all functions exposed by the WASM logging module and their type signatures. The program iterates through this list to verify the presence of the required `init` and `process_log` functions. It also checks whether the parameter types and return value types of these two functions match the host program's expected signatures. The `init` function accepts two integer parameters: a memory offset pointing to configuration data and the data length, and returns an integer status code. The `process_log` function should accept log data and a source identifier and return a structured result. If either function is missing, the compatibility check fails, the logging module is refused to load, and an error is logged. After verifying the existence of the required functions, the function signatures are further checked. By obtaining the parameter type list and return value type of each exported function, the expected signature of the `init` function is: accepting two integer parameters (memory offset and data length) and returning an integer status code; the expected signature of the `process_log` function is: accepting three integer parameters (log data offset, data length, and source identifier) ​​and returning an integer (result data offset). If the actual number of parameters, parameter types, or return value types in the signature do not match the expectations, the compatibility check is deemed to have failed and loading is refused.

[0060] Call the initialization function of the WebAssembly logging module and pass in the policy configuration file to cache the configuration object in the global memory of the WebAssembly logging module.

[0061] After all checks pass, the `wasm_runtime_lookup_function` function is called to obtain the handle of the `init` function exported by the module, and then the `wasm_runtime_call` function is invoked, passing the string of the policy configuration file as a parameter. The `init` function parses the passed JSON configuration string and caches the parsed configuration object in the module's linear memory for direct reading by the subsequent `process_log` and `batch_process` functions. The parsed configuration object includes global log level thresholds, subsystem level coverage rules, output format settings, and desensitization switches. Upon initialization, a return status code of 0 indicates success; a non-zero return value indicates initialization failure, an error log is recorded, and a fallback mode is entered, only transmitting the raw logs without WASM module processing. After successful initialization, the system enters a ready state, awaiting log data input. A ready WebAssembly sandbox is used to receive log data and process all log data within the sandbox. Strict resource constraint parameters are applied to the sandbox, ensuring that even if anomalies occur during log module processing, the stable operation of the host program and AP system will not be affected, providing security isolation at the architectural level.

[0062] S202, the first log data is collected through the host program of the AP application processor, the second log data is collected from the CP communication processor through the communication interface, and corresponding source identifiers are added to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module.

[0063] S203, the host program maintains a log buffer queue based on the received unified log stream, and determines the log processing mode based on the length of the log buffer queue.

[0064] To flexibly respond to various log burst scenarios, a first-in-first-out log buffer queue is maintained in memory. The queue length is initially 0. After each raw log is collected and a source identifier is added, the log data is inserted into the tail of the queue, and the queue length is incremented by 1.

[0065] S204 compares the log buffer queue length with the preset high-water mark threshold. If it exceeds the high-water mark threshold, it switches from single-log processing mode to batch processing mode, packages multiple logs in the log buffer queue into a batch, and calls the batch processing function of the WebAssembly log module.

[0066] Simultaneously, a log processing thread can be maintained to continuously retrieve log entries from the head of the log buffer queue for processing. Before processing, the current queue length is checked and compared with several preset thresholds to determine the log processing mode. When the length exceeds the preset high-water mark (e.g., 100 entries), it is determined that a log surge is in effect, and the system will automatically switch from single-entry processing mode to batch processing mode. Up to 50 log entries are retrieved from the queue at a time and packaged into a batch, calling the `batch_process` function of the WASM logging module. This effectively reduces the number of cross-language calls between the host program and the WASM logging module, significantly improving processing throughput. The batch processing function is an optional function exported by the WASM logging module; if this function does not exist, it will fall back to calling each entry individually.

[0067] S205, compare the log buffer queue length with the preset back pressure threshold. If the back pressure threshold is exceeded, send a sampling signal to the host program of the AP application processor to trigger the log sampling down-frequency strategy and suspend the collection of the second log data of the CP communication processor from the communication interface.

[0068] When the log output is compared to a preset backpressure threshold (e.g., 500 logs) and exceeds the threshold, it can be determined that the upstream log generation rate exceeds the processing capacity, at which point the backpressure control mechanism is activated. For example, ... Figure 4The flowchart shown illustrates the batch processing mode switching and backpressure threshold judgment process. For the AP's local logs, the host program sends a sampling signal through the system log interface, triggering a log sampling frequency reduction strategy, lowering the log collection rate from 100% to 10%, and only collecting logs at ERROR level and above. For the CP's communication logs, the host program pauses reading data from the serial port device file and stops receiving new logs from the CP communication module, allowing the serial port buffer to naturally form flow control. When the queue length falls below the corresponding threshold, complete collection of AP logs resumes, reading CP log data from the serial port resumes, and the process switches back to single-log processing mode. Alternatively, batch processing mode can be maintained until the log buffer queue is cleared.

[0069] S206, When the WebAssembly logging module fails, the unified log stream is directly encapsulated and transparently transmitted by switching to degraded mode to bypass the WebAssembly logging module.

[0070] When the WASM logging module malfunctions, such as when the `process_log` function exceeds the 5-millisecond timeout limit, when memory access in the logging module exceeds the bounds causing a runtime exception, or when a function call returns an error code, the WASM runtime instance automatically terminates its current execution and returns an error message to the host program. Upon catching this error, the host program immediately switches to degraded mode. In degraded mode, the raw log data in the log buffer queue can be directly encapsulated into a minimal structured output by bypassing the WASM logging module. For example, the output format is JSON, containing the system's current timestamp (`timestamp`), the source identifier (`source`), and the raw log content (`raw_log`). Simultaneously, the host program starts a background recovery task to attempt to reload the current WASM logging module. If reloading fails after three attempts, it attempts to switch to the backup logging module. If the backup module is also unavailable, the error is reported to the cloud, awaiting a fix or new version.

[0071] An optional implementation of this embodiment is as follows: log parsing includes: first, performing a fast scan, parsing a log header of a preset length, extracting the source identifier and log level information, and comparing the log level information with the level threshold in the policy configuration file.

[0072] The WASM logging module employs a two-stage log parsing strategy to improve log processing efficiency. When the WASM logging module's `process_log` function receives log data, it first performs a fast scan. For example, it reads only the first 64 bytes of the log data because the log level information for AP and CP is usually located in the log header, from which the source identifier and log level can be extracted.

[0073] If the log level is lower than the level threshold, the current log is discarded and an empty result is returned, without further parsing.

[0074] The extracted log level is compared with the level threshold corresponding to the source identifier in the caching policy configuration file. If it is lower than the level threshold, an empty result is returned and no further parsing is performed. If it is higher than the level threshold, further log parsing is performed. This method quickly filters a large number of low-level logs with extremely low computational overhead, saving CPU resources.

[0075] If the log level is higher than the level threshold, the preset log parsing strategy is matched according to the source identifier. The timestamp, event type, and key parameter fields are extracted from the complete unified log stream. The knowledge base built into the WebAssembly log module is queried to expand the device abbreviation into an easy-to-read description. Sensitive fields are anonymized to form structured log parsing data.

[0076] For logs requiring further parsing, a preset parsing strategy is matched based on the source identifier. For example, AP logs can be parsed using regular expressions; CP logs can be extracted using delimiter-based field extraction, extracting fields such as timestamps, event types, and key parameters from the complete log string. During field extraction, a built-in knowledge base can be queried to expand device-specific abbreviations into their full names or easily readable descriptions, reducing the difficulty of understanding the logs. Sensitive fields can be masked or de-identified according to the de-identification switches in the policy configuration file. Finally, all extracted and processed fields are integrated into structured log parsing data in JSON format and returned to the host program.

[0077] S207 receives configuration update data from the cloud and performs incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

[0078] Specifically, it receives configuration update data from the cloud and identifies incremental updates by recognizing the version number and changed fields.

[0079] When log processing strategies need to be adjusted, configuration update data can be sent from the cloud to the IoT edge gateway. Upon receiving the data, the gateway first identifies the version number and the changed content to determine whether an incremental update is necessary. If the received version number is the same as or lower than the current version number, the update is ignored; if the received version number is higher than the current version number and the list of changed fields is not empty, then it can be determined as an incremental update.

[0080] If it is identified as an incremental update, the changed fields are extracted to construct an incremental update fragment, and the configuration update function is called to perform incremental merging and updating of the WebAssembly log module using a differential update mechanism.

[0081] During incremental updates, the host program constructs a configuration fragment containing only the changed fields based on their content, and then calls the `update_config` function of the WASM logging module, passing this configuration fragment as a parameter. The configuration update logic within the WASM logging module receives this fragment, iterates through the changed fields, and merges the new values ​​of each field into the cached configuration object, overwriting the original values. The WASM logging module is compiled into platform-independent standard bytecode, allowing deployment on AP platforms with different CPU architectures without modification. This achieves cross-platform deployment with a single codebase, eliminating redundant porting and adaptation work.

[0082] This embodiment reads the deployed WebAssembly logging module and its policy configuration file during the initialization phase to create a WebAssembly runtime instance on the edge gateway and loads the WASM logging module to perform version compatibility checks. A log buffer queue is maintained based on the unified log stream generated after collection. The single-log processing mode, batch processing mode, and reduced-frequency collection strategy are switched based on the queue length compared with high-water mark and backpressure thresholds. A two-stage parsing strategy is then employed: first, a rapid scan of the log header is used for low-level log filtering; then, the filtered high-level logs undergo complete field extraction, abbreviation expansion, and sensitive field desensitization to generate structured log data. When the WASM logging module fails, a degradation mode is switched to bypass the WASM logging module, allowing the unified log stream to be directly encapsulated and transparently transmitted. Differential configuration updates are used, constructing incremental update fragments by identifying version numbers and changed fields for incremental merging updates. Version compatibility checks ensure consistency between the host program and the WASM module, preventing runtime crashes due to interface incompatibility. High-water mark queue management and backpressure acquisition strategies enable adaptive handling of log surges, effectively preventing resource exhaustion. A two-stage log parsing strategy quickly filters large amounts of low-level logs through lightweight pre-parsing, improving processing efficiency and reducing computational overhead. Built-in knowledge base abbreviation expansion and sensitive field anonymization transform raw logs into easily understandable and securely compliant structured data, providing a data foundation for subsequent analysis and diagnosis. Graceful degradation ensures no log data loss in the event of a WASM module failure. This approach is particularly suitable for industrial IoT edge computing scenarios with limited network bandwidth and computing resources.

[0083] Example 3

[0084] Figure 5 This is a schematic diagram of the structure of a heterogeneous device log processing device for industrial IoT according to Embodiment 3 of the present invention. In this embodiment, the heterogeneous device log processing device for industrial IoT includes:

[0085] Initialization unit 810 is used to create a WebAssembly runtime instance and set resource limit parameters. Then, while loading the WebAssembly logging module, it checks its version compatibility and initializes the WebAssembly logging module.

[0086] The log collection unit 820 is used to collect first log data through the host program of the AP application processor, collect second log data from the CP communication processor through the communication interface, and add corresponding source identifiers to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module.

[0087] The log parsing unit 830 is used by the WebAssembly log module to receive the unified log stream, select the corresponding preset log parsing strategy according to the source identifier, perform log parsing, and generate structured log parsing data.

[0088] The update unit 840 is used to receive configuration update data from the cloud and perform incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

[0089] For example, the device can be implemented using a three-layer decoupling architecture, such as... Figure 6 The diagram shown is an architectural representation of one possible embodiment of the device, comprising:

[0090] The first layer is the data acquisition layer. The host program is responsible for obtaining raw log data streams from the AP locally and from the CP through communication interfaces (such as UART and USB), which serve as a unified data entry point.

[0091] The second layer is the WASM processing layer, which deploys a lightweight WebAssembly runtime on the AP and loads a pre-compiled WASM logging module. All log parsing, filtering, formatting, and de-identification processing logic is executed in this sandbox.

[0092] The third layer is the output layer, where the host program receives structured logs returned by the WASM module and stores them locally or reports them to the cloud.

[0093] This embodiment creates a WebAssembly runtime instance and initializes the WebAssembly logging module through an initialization unit. A log collection unit collects first log data from the AP application processor and second log data from the CP communication processor, forming a unified log stream. A log parsing unit receives this unified log stream, selects a preset log parsing strategy based on the source identifier, and generates structured log parsing data. An update unit receives configuration update data from the cloud and performs incremental or full updates to the WebAssembly logging module configuration. This achieves decoupling of multi-source heterogeneous log processing logic from the AP firmware. Even if the log format changes due to a CP firmware upgrade, only the WASM logging module needs to be updated, without modifying or recompiling the AP firmware, reducing maintenance costs. By dynamically replacing the WASM logging module and policy configuration during device operation, hot updates of log parsing capabilities are achieved. Log levels and filtering rules can be adjusted in real time according to actual needs without restarting the device or upgrading firmware, improving response efficiency. All log processing logic is executed in a WebAssembly sandbox, subject to strict resource constraints. Even if anomalies occur during log module processing, the stable operation of the host program and AP system will not be affected, providing secure isolation at the architectural level. The WebAssembly logging module is compiled into platform-independent standard bytecode, allowing deployment on AP platforms with different CPU architectures without modification. This enables write-once, cross-platform deployment, eliminating repetitive porting and adaptation work. Development teams can choose to develop the logging module in languages ​​such as Rust, C++, and Go, fully leveraging the memory safety and concurrency advantages of each language, breaking the limitation of embedded development to C / C++. Atomic switching and automatic rollback mechanisms ensure business continuity in the event of module update failures, ensuring uninterrupted log collection and processing. Differential configuration updates effectively reduce network transmission overhead, making it particularly suitable for industrial IoT edge computing scenarios with limited network bandwidth and computing resources.

[0094] The heterogeneous device log processing device for industrial IoT provided in this embodiment of the invention can execute the heterogeneous device log processing method for industrial IoT provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0095] Example 4

[0096] Figure 7 This is a structural diagram of an electronic device according to Embodiment 4 of the present invention. Figure 7 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 7 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0097] like Figure 7 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0098] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0099] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0100] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0101] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0102] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the electronic device 12 / server / computer, and / or with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 7 As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 7 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0103] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the heterogeneous device log processing method for the Industrial Internet of Things provided in this embodiment of the invention.

[0104] Example 5

[0105] Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the heterogeneous device log processing method for the Industrial Internet of Things as provided in the above embodiments.

[0106] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0107] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0108] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0109] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for processing logs of heterogeneous devices in the Industrial Internet of Things (IIoT), characterized in that, include: S101: Create a WebAssembly runtime instance and set resource limit parameters. Then, load the WebAssembly logging module while checking its version compatibility and initializing the WebAssembly logging module. S102, the first log data is collected through the host program of the AP application processor, the second log data is collected from the CP communication processor through the communication interface, and corresponding source identifiers are added to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module; S103, the WebAssembly log module receives the unified log stream, selects the corresponding preset log parsing strategy according to the source identifier, performs log parsing, and generates structured log parsing data; S104 receives configuration update data from the cloud and performs incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

2. The method according to claim 1, characterized in that, The method further includes: Receive the new WebAssembly log module update package from the cloud, perform integrity checks based on hash value verification, version compatibility checks based on temporary sandbox, and functional verification. After successful verification, mark the current WebAssembly log module as a backup and retain it through atomic switching, and switch the new WebAssembly log module as the production execution module. If the new module fails, it will automatically revert to the backup of the old WebAssembly log module.

3. The method according to claim 1, characterized in that, S101 includes: Read the pre-deployed WebAssembly log module and its policy configuration file, create a WebAssembly runtime instance on the edge gateway, and set the resource cap threshold and execution time cap threshold; Load the WebAssembly logging module into the runtime and perform a version compatibility check; Call the initialization function of the WebAssembly logging module and pass in the policy configuration file to cache the configuration object in the global memory of the WebAssembly logging module.

4. The method according to claim 3, characterized in that, The version compatibility check includes: Enumerate the list of functions exported by the WebAssembly logging module, verify whether the required init and process_log functions are included, and check that the function signatures are consistent with the host program's expectations. If they are, initialize the WebAssembly logging module.

5. The method according to claim 1, characterized in that, S103 includes: The host program maintains a log buffer queue based on the received unified log stream and determines the log processing mode based on the length of the log buffer queue. The log buffer queue length is compared with the preset high watermark threshold. If it exceeds the high watermark threshold, the single processing mode is switched to batch processing mode, and multiple logs in the log buffer queue are packaged into a batch and the batch processing function of the WebAssembly log module is called. The log buffer queue length is compared with the preset back pressure threshold. If the back pressure threshold is exceeded, a sampling signal is sent to the host program of the AP application processor to trigger the log sampling down-frequency strategy and suspend the collection of the second log data of the CP communication processor from the communication interface. When the WebAssembly logging module fails, the unified log stream can be directly encapsulated and transparently transmitted by switching to degraded mode to bypass the WebAssembly logging module.

6. The method according to claim 5, characterized in that, The log parsing includes: First, a fast scan is performed to parse the log header of the preset length, extract the source identifier and log level information, and compare the log level information with the level threshold in the policy configuration file; If the log level is lower than the level threshold, the current log is discarded and an empty result is returned, requiring no further parsing. If the log level is higher than the level threshold, the preset log parsing strategy is matched according to the source identifier. The timestamp and event type are extracted from the complete unified log stream. The knowledge base built into the WebAssembly log module is queried to expand the device abbreviation into an easy-to-read description. Sensitive fields are anonymized to form structured log parsing data.

7. The method according to claim 1, characterized in that, S104 includes: Receive configuration update data from the cloud and identify incremental updates by recognizing the version number and changed fields; If it is identified as an incremental update, the changed fields are extracted to construct an incremental update fragment, and the configuration update function is called to perform incremental merging and updating of the WebAssembly log module using a differential update mechanism.

8. A heterogeneous device log processing apparatus for the Industrial Internet of Things (IIoT), used to implement the heterogeneous device log processing method for the Industrial Internet of Things as described in any one of claims 1-7, characterized in that, include: The initialization unit is used to create a WebAssembly runtime instance and set resource limit parameters. Then, it loads the WebAssembly logging module, checks its version compatibility, and initializes the WebAssembly logging module. The log collection unit is used to collect first log data through the host program of the AP application processor, collect second log data from the CP communication processor through the communication interface, and add corresponding source identifiers to the first log data and the second log data respectively to form a unified log stream and transmit it to the WebAssembly log module. The log parsing unit is used by the WebAssembly log module to receive the unified log stream, select the corresponding preset log parsing strategy according to the source identifier, perform log parsing, and generate structured log parsing data. The update unit is used to receive configuration update data from the cloud and perform incremental or full updates to the configuration of the WebAssembly log module based on the configuration update data.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the heterogeneous device log processing method for the Industrial Internet of Things as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the heterogeneous device log processing method for the Industrial Internet of Things as described in any one of claims 1-7.

Images