Dynamic data analysis processing method and device for self-adaptive and active adaptive sensing signals
Through the dynamic data analysis method of adaptively and actively adapting the sensor signals, the problem of inconsistent signal formats of multi-source sensors is solved, automatic adaptation and unified analysis of sensor signals are achieved, and data processing efficiency and reliability are improved.
Patent Information
- Application Number
- CN202510817656.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-18
- Publication Date
- 2025-10-17
AI Technical Summary
In the existing technology, sensor equipment has inconsistent signal formats due to differences in manufacturers and models, and lacks the ability to dynamically adapt to multi-source signals, resulting in the inability to centrally process data, low efficiency and prone to errors.
A dynamic data parsing method that adaptively and actively adapts to sensor signals is adopted. Multi-source sensor signals are received through the signal interface module, and feature extraction and protocol identification are performed. The protocol library and machine learning are used to generate parsing rules, which are converted into standardized data formats, and real-time verification and correction are performed to support dynamic updates and scalability.
It achieves automatic adaptation and unified parsing of signals from multiple sensor sources, reduces manual configuration costs, dynamically generates parsing rules, has strong scalability, and improves the accuracy and reliability of processing results.
Smart Images

Figure CN120804561A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data acquisition and processing, and particularly relates to a dynamic data analysis processing method and device for adaptively and actively adapting to sensing signals. BACKGROUND
[0002] In an industrial scene, due to differences in manufacturers and models, signal formats of sensor devices are not unified, for example, protocol types, data encoding, transmission rates and the like are significantly different. In the prior art, a data acquisition device is usually designed for a specific sensor, and lacks dynamic adaptation capability for multi-source signals, so that different sensor data cannot be centrally processed, and manual configuration of analysis rules is required, which is low in efficiency and prone to errors. SUMMARY
[0003] In view of the above problems, the present application provides a dynamic data analysis processing method and device for adaptively and actively adapting to sensing signals, which is suitable for unified processing of signals of multi-source heterogeneous sensors in industrial automation, Internet of Things and intelligent monitoring systems.
[0004] To achieve the above object, the present application provides a dynamic data analysis processing method for adaptively and actively adapting to sensing signals, comprising the following steps:
[0005] S1. receiving original signals from sensors of different manufacturers and models through a signal interface module;
[0006] S2. performing feature extraction on the original signals to identify signal types, protocol formats and data encoding rules;
[0007] S3. dynamically matching a corresponding analysis template from a preset protocol library according to the identification result; if there is no matching template in the protocol library, generating an adaptive analysis rule through a machine learning algorithm;
[0008] S4. converting the original signals into a standardized data format based on the analysis template or the generated analysis rule;
[0009] S5. performing real-time verification on the standardized data to correct data errors and supplement missing fields;
[0010] S6. transmitting the processed data to a centralized processing platform for unified analysis, storage and feedback control.
[0011] Preferably, the feature extraction includes signal waveform analysis, baud rate detection, check bit identification and data frame structure analysis, and the feature extraction adopts the following steps:
[0012] S21. signal preprocessing, inputting an original binary signal stream, denoising the original binary signal stream, and performing sliding window mean filtering, the formula being: Eliminate high-frequency noise; based on level jump detection, divided into independent data frame;
[0013] S22. Baud rate detection, based on joint analysis of time interval statistics and Fourier transform, statistics of adjacent rising edge / falling edge time interval, take the mode as the candidate baud rate, extract the main frequency component of the signal through Fourier transform, verify the candidate baud rate;
[0014] S23. Frame structure analysis, finite state machine combined with dynamic template matching, define states: start bit, data bit, check bit, stop bit, dynamically adjust data bit length, verify frame integrity through cyclic redundancy check;
[0015] S24. Protocol type identification, rule-based feature hashing and similarity matching, extract feature vector frame header flag, check method, data length, calculate cosine similarity with pre-stored protocol template, select the highest matching item.
[0016] Preferably, in step S3, the protocol library supports dynamic updating, and can add or modify protocol templates through a remote server or a local interface. The adaptation analysis rule is as follows:
[0017] S31. Protocol library matching, input feature vector protocol type, frame structure, check method, search matching item in pre-set protocol library, which is based on hash index acceleration, if matching is successful, load corresponding analysis template; otherwise, trigger rule generation process;
[0018] S32. Machine learning driven rule generation, sequence to sequence model based on time sequence attention mechanism, input original signal sequence, output analysis rule including field segmentation position and data type mapping table;
[0019] S33. Training data: labeled original signal and analysis rule pair of historical protocol samples, input unidentified signal, model output analysis rule, rule verification passes through simulated analysis and check bit verification to verify rule effectiveness;
[0020] S34. Dynamic update of protocol library, when adding new protocol template, extract feature vector and store in protocol library, and update hash index.
[0021] Preferably, in step S4, the standardized data format includes unified timestamp, data unit, dimension and data structure, and the standardized data format mapping algorithm is as follows:
[0022] S41. Field mapping table construction, field mapping table construction, its analysis rule field name, data type, unit, according to rule definition field mapping relationship;
[0023] S42. Data conversion, frame-by-frame analysis based on dynamic template, data bit segmentation according to analysis rules, extraction of field values, and execution of unit conversion;
[0024] S43. Timestamp synchronization, adding uniform timestamp precision to milliseconds to each piece of data, and synchronizing system clock based on NTP protocol.
[0025] Preferably, in step S5, the real-time verification includes data range verification, logical correlation check, and redundant data filtering. The real-time verification module detects and corrects data abnormal out-of-range values and logical contradictions through data cleaning and error correction algorithms. The method is as follows:
[0026] S51. Data range verification, preset upper and lower limits of fields. If the limits are exceeded, mark as invalid data or use historical average interpolation;
[0027] S52. Logical correlation check, define the logical relationship between fields, and perform real-time logical verification based on rule engine;
[0028] S53. Redundant data filtering sliding window deduplication. If the difference between consecutive NN frames of data is less than threshold, only the first frame is retained.
[0029] In a second aspect, the application further provides a device for adaptive active adaptive dynamic data analysis processing method of sensing signal, comprising:
[0030] Signal interface module, supporting RS-485, CAN, Modbus and TCP / IP protocol, for receiving multi-source heterogeneous sensor signal;
[0031] Feature recognition module, connected to the signal interface module, realizes signal segmentation, baud rate detection and frame structure analysis through finite state machine and Fourier transform algorithm;
[0032] Protocol adaptation module, including protocol library unit and machine learning unit, the protocol library unit stores the preset analysis template and accelerates the matching through hash index, and the machine learning unit generates the analysis rule of unidentified protocol based on the Transformer model;
[0033] Data conversion module, performing field mapping and unit conversion according to analysis rules, and adding uniform timestamp; verification and correction module, built-in sliding window deduplication algorithm and rule engine, for data cleaning and logical error correction;
[0034] Communication module, using weighted round-robin scheduling strategy to optimize data transmission priority, and supporting breakpoint resume mechanism;
[0035] Self-diagnosis module, real-time monitoring device health status through isolation forest algorithm, triggering alarm signal when abnormal.
[0036] Preferably, the protocol library of the protocol adaptation module supports remote OTA updates, and automatically extracts feature vectors and updates hash indexes when new protocol templates are added.
[0037] Preferably, the health indicators of the self-diagnosis module include CPU usage, protocol matching success rate and communication delay. Anomaly detection calculates the anomaly score of the real-time indicator through the isolation forest algorithm. If the score exceeds the threshold, the faulty module is located.
[0038] Preferably, the breakpoint resume mechanism of the communication module is implemented based on the sliding window of the TCP protocol, recording the transmission progress and continuing the transmission after the network is restored.
[0039] Preferably, the timestamp synchronization accuracy of the data conversion module is at the millisecond level, and is synchronized with the system clock through the NTP protocol.
[0040] The present invention adopts hybrid protocol adaptation: combining rule matching and machine learning to balance efficiency and scalability; dynamic parsing rule generation: based on the Seq2Seq model of Transformer to adapt to new protocols; multi-dimensional data verification: triple protection of scope, logic, and redundancy to improve data reliability; lightweight anomaly detection: isolation forest algorithm to achieve real-time monitoring with low resource consumption.
[0041] Beneficial effects of the present invention:
[0042] 1. Realize automatic adaptation and unified analysis of multi-source sensor signals, reducing manual configuration costs;
[0043] 2. Dynamically generate parsing rules with strong scalability to adapt to the rapid access of new sensors;
[0044] 3. Improve the accuracy and reliability of processing results through data verification and correction. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 Schematic diagram of the process of the present invention;
[0046] Figure 2 Schematic diagram of the structure of the device of the present invention. DETAILED DESCRIPTION
[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.
[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0049] It should be noted that in the description of the present application, the terms indicating the direction or positional relationship of "upper", "lower", "left", "right", "inner", "outer" and the like are based on the direction or positional relationship shown in the drawings, which is only for the convenience of description, and does not indicate or imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.
[0050] In addition, it should also be noted that in the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.
[0051] Embodiment 1, as shown in a dynamic data analysis processing method for adaptive active adaptation of sensing signals, comprising the following steps: Figure 1
[0052] S1. Receiving the original signal from the sensor of different manufacturers and models through the signal interface module;
[0053] S2. Feature extraction is performed on the original signal to identify its signal type, protocol format and data encoding rule. Feature extraction includes signal waveform analysis, baud rate detection, check bit identification and data frame structure analysis. The feature extraction adopts the following steps:
[0054] S21. Signal preprocessing, input original binary signal stream, denoising the original binary signal stream through sliding window mean filtering, the formula is: Eliminate high-frequency noise; based on level jump detection, divide into independent data frames;
[0055] S22. Baud rate detection, based on joint analysis of time interval statistics and Fourier transform, statistics of adjacent rising edge / time interval, take the mode as the candidate baud rate, extract the main frequency component of the signal through Fourier transform, verify the candidate baud rate;
[0056] S23. Frame structure analysis, finite state machine combined with dynamic template matching, define states: start bit, data bit, check bit, stop bit, dynamically adjust the length of data bit, verify the frame integrity through cyclic redundancy check;
[0057] S24. Protocol type identification, rule-based feature hashing and similarity matching, extracting feature vector frame header flag, check mode, data length, calculating the cosine similarity with the pre-stored protocol template, selecting the highest matching item.
[0058] S3. According to the identification result, dynamically match the corresponding analysis template from the pre-stored protocol library; if there is no matching template in the protocol library, generate an adaptive analysis rule through a machine learning algorithm, the protocol library supports dynamic updating, and can add or modify protocol templates through a remote server or a local interface, the adaptive analysis rule is generated by the following method:
[0059] S31. Protocol library matching, input feature vector protocol type, frame structure, check mode, search matching item in pre-stored protocol library, which is based on hash index acceleration, if matching is successful, load corresponding analysis template; otherwise, trigger rule generation process;
[0060] S32. Machine learning driven rule generation, sequence to sequence model based on time sequence attention mechanism, input original signal sequence, output analysis rule including field segmentation position and data type mapping table;
[0061] S33. Training data: labeled original signal and analysis rule pair of historical protocol samples, input unidentified signal, model output analysis rule, rule verification passes through simulated analysis and check bit verification to verify the validity of the rule;
[0062] S34. Dynamic update of protocol library, when adding a new protocol template, extract the feature vector and store it in the protocol library, and update the hash index at the same time.
[0063] S4. Based on the analysis template or generated analysis rule, convert the original signal into a standardized data format, the standardized data format includes unified timestamp, data unit, dimension and data structure, the standardized data format mapping algorithm is as follows:
[0064] S41. Field mapping table construction, field mapping table construction, its analysis rule field name, data type, unit, according to rule definition field mapping relationship;
[0065] S42. Data conversion, frame-by-frame analysis based on dynamic template, split data bits according to analysis rule, extract each field value, and perform unit conversion;
[0066] S43. Time stamp synchronization, add a unified time stamp precision to milliseconds for each data, synchronize system clock based on NTP protocol.
[0067] S5. Real-time checking of standardized data, correction of data errors and supplement of missing fields. The real-time checking includes data range verification, logical correlation check and redundant data filtering. The real-time checking module detects and corrects abnormal out-of-range values and logical contradictions through data cleaning and error correction algorithms. The method is as follows:
[0068] S51. Data range verification. The upper and lower limits of the preset field are checked. If the limits are exceeded, the data is marked as invalid or the historical average value is interpolated.
[0069] S52. Logical correlation check. The logical relationship between fields is defined, and real-time logical verification is performed based on a rule engine.
[0070] S53. Redundant data filtering. If the difference between consecutive NN frames of data is less than a threshold value, only the first frame is retained.
[0071] S6. The processed data is transmitted to a centralized processing platform for unified analysis, storage and feedback control.
[0072] Embodiment 2, as shown in Figure 2 A device for a dynamic data analysis processing method of adaptively actively adapting to a sensing signal, comprising:
[0073] A signal interface module supporting RS-485, CAN, Modbus and TCP / IP protocols for receiving multi-source heterogeneous sensor signals;
[0074] A feature recognition module connected to the signal interface module, which realizes signal segmentation, baud rate detection and frame structure analysis through a finite state machine and a Fourier transform algorithm;
[0075] A protocol adaptation module including a protocol library unit and a machine learning unit. The protocol library unit stores preset analysis templates and accelerates matching through a hash index. The machine learning unit generates analysis rules for unidentified protocols based on a Transformer model. The protocol library of the protocol adaptation module supports remote OTA updates. When a new protocol template is added, the feature vector is automatically extracted and the hash index is updated.
[0076] A data conversion module that performs field mapping and unit conversion according to the analysis rules and adds a unified timestamp; a verification and correction module that includes a sliding window deduplication algorithm and a rule engine for data cleaning and logical error correction. The timestamp of the data conversion module has a millisecond-level synchronization accuracy and is synchronized with the system clock through the NTP protocol;
[0077] A communication module that uses a weighted round-robin scheduling strategy to optimize data transmission priority and supports a breakpoint resume mechanism. The breakpoint resume mechanism of the communication module is based on the sliding window of the TCP protocol, records the transmission progress and continues transmission after network recovery;
[0078] Self-diagnosis module, real-time monitoring of device health status by Isolation Forest algorithm, triggering alarm signal when abnormal, health indicators of self-diagnosis module including CPU usage, protocol matching success rate and communication delay, abnormal detection through Isolation Forest algorithm to calculate real-time index abnormal score, if the score exceeds the threshold, locate the fault module.
[0079] Embodiment 3, the application of the application to the unified processing of multi-source sensors in the industrial automation scene, a stamping workshop of an automobile manufacturing plant deploys pressure sensors from 3 different manufacturers (Modbus protocol sensor from manufacturer A, CAN bus sensor from manufacturer B, and custom TCP / IP protocol sensor from manufacturer C), which are used to monitor the pressure of the hydraulic system in real time. The traditional system needs to configure separate parsing rules for each type of sensor, resulting in high data processing delay and high maintenance cost.
[0080] Implementation steps
[0081] Signal access: sensor signals are accessed to the device through RS-485 (Modbus), CAN bus, and Ethernet (TCP / IP).
[0082] Signal interface module automatically identifies physical interface type and detects pin level.
[0083] Protocol adaptation and data parsing:
[0084] Manufacturer A sensor (Modbus): feature recognition module detects baud rate 9600bps, frame header 0xA5, matches preset Modbus template, and loads parsing rules.
[0085] Manufacturer B sensor (CAN): recognizes standard CAN 2.0B frame format (29-bit identifier), and protocol library matches successfully.
[0086] Manufacturer C sensor (custom TCP / IP): feature extraction discovers unknown protocol, triggers Transformer model to generate parsing rules (field segmentation position: bytes 2-4 are pressure value, unit kPa).
[0087] Data conversion and verification: conversion module converts raw data to JSON format:
[0088] {"timestamp":"2024-10-05T14:23:45.678","pressure":120.5,"unit":"kPa"}
[0089] Verification correction module detects that the data of manufacturer C sensor suddenly changes to -200kPa (out of limit), and replaces it with the previous period average value 118.3kPa.
[0090] Communication and alarm:
[0091] The communication module uploads data in priority of alarm data, and pushes to the workshop control center through the MQTT protocol.
[0092] The self-diagnosis module detects that the CAN bus communication delay is rising (>100ms), triggers an alarm and prompts to check the line interference.
[0093] Example 3 benefit verification
[0094] Protocol compatibility: successfully parse three heterogeneous protocols, no manual configuration, maintenance time reduced by 80%.
[0095] Data processing efficiency: average delay <30ms, meet the real-time control requirements (traditional system >100ms).
[0096] Data reliability: false positive rate reduced from 12% to 1.5% after abnormal data correction.
[0097] Example 4. The application is applied to a smart home system integrating Zigbee temperature and humidity sensor (manufacturer D), Wi-Fi air quality sensor (manufacturer E) and Bluetooth light sensor (manufacturer F). The existing gateway cannot uniformly process multi-protocol data, resulting in the user needing to use multiple APPs to manage the device.
[0098] Implementation steps
[0099] Signal access and feature extraction:
[0100] Zigbee signal is received through the gateway, identified as IEEE 802.15.4 frame structure, baud rate 250kbps.
[0101] Wi-Fi sensor sends JSON format data, but the field naming is not uniform, such as temp vs temperature.
[0102] Bluetooth sensor uses a custom binary protocol, no preset template.
[0103] Dynamic rule generation:
[0104] Wi-Fi sensor: through field similarity matching (cosine similarity >0.9), temp is mapped to standard field temperature.
[0105] Bluetooth sensor: machine learning model parses data stream, identifies bytes 0-1 as light intensity (unit: Lux), and generates rules.
[0106] Data standardization and logic verification: conversion module unified output format:
[0107] {"device_type":"light","value":450,"unit":"Lux","timestamp":"2024-10-05T14:25:30.123"}
[0108] Logical correlation check: If the light intensity > 500 Lux but the curtain status is "closed", mark as abnormal and notify the user.
[0109] Remote control and expansion: User adds LoRa smoke sensor (manufacturer G), protocol library has no matching template. Model automatically parses data frame (byte 3 is smoke concentration), generates rules and stores in protocol library, realizes plug and play.
[0110] Example 4 benefit verification
[0111] Scalability: The access time of new devices is reduced from 2 hours to 5 minutes.
[0112] User experience: Unified display of multi-protocol data, APP operation steps reduced by 70%.
[0113] Energy efficiency optimization: Automatically adjust devices through logical verification, save 15%.
[0114] Example 5, real-time monitoring and fault location of medical gas supply system, application scenario of central oxygen supply system of a certain third-grade hospital deployed 20 oxygen pressure sensors (manufacturer H, RS-485 protocol), need to ensure that the pressure is stable in the range of 300-400kPa, traditional system cannot distinguish between sensor failure and real pressure anomaly.
[0115] Implementation steps
[0116] 1. Signal processing and protocol matching:
[0117] All sensors use Modbus RTU protocol, but manufacturer H defines custom function code (such as function code 0x45 to read pressure value).
[0118] After protocol adaptation module fails to match, model parses data frame structure, confirms that function code 0x45 corresponds to pressure register address 0x0002.
[0119] 2. Double check and fault isolation:
[0120] First judgment: Alarm module detects that sensor S15 pressure value is 420kPa (over threshold), triggers audible and visual alarm.
[0121] Second judgment: Analysis module retrieves S15 historical data, finds that its reading increased from 320kPa to 420kPa within 1 minute, and other sensor readings are stable, determines that S15 is faulty (not real pressure anomaly).
[0122] 3. Fault location and maintenance:
[0123] The warning light of the prompt module is on at S15 installation position, and the maintenance personnel arrives at the scene within 5 minutes to confirm that the sensor circuit board is short-circuited.
[0124] The self-diagnosis module records the fault event (sensor ID, timestamp, abnormal index) and generates a maintenance report.
[0125] 4. Data backup and traceability:
[0126] The storage module retains the last 30 days of data, supports querying historical trend charts by timestamp, and is used for accident analysis.
[0127] Example 5 benefit verification
[0128] Alarm accuracy: false positive rate reduced from 25% to 3%, avoiding oxygen supply interruption caused by false positives.
[0129] Maintenance efficiency: fault location time reduced from 2 hours to 10 minutes.
[0130] System reliability: through historical data analysis, preventive replacement of aging sensors, fault rate reduced by 60%.
[0131] Through the verification of the above examples, the present application has shown significant advantages in industrial, home, medical and other scenarios:
[0132] 1. Adaptive protocol adaptation: covering wired (RS-485, CAN) and wireless (Zigbee, Wi-Fi, Bluetooth) protocols, supporting dynamic rule generation.
[0133] 2. Intelligent data processing: improve alarm accuracy through double verification (threshold + historical trend), combined with machine learning to optimize analysis efficiency.
[0134] 3. Efficient operation and maintenance support: self-diagnosis and prompt module shortens fault response time, and storage module provides complete traceability for data analysis.
[0135] 4. Economic benefits: reduce manual configuration cost by 30%, reduce equipment downtime loss by more than 50%.
[0136] The present application effectively solves the industry problem of unified processing of multi-source heterogeneous sensor signals through innovative algorithm design and modular architecture, and provides reliable technical support for digital transformation in the fields of smart home, smart medical and other fields.
[0137] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.
Claims
1. A dynamic data analysis and processing method for adaptively and actively adapting sensor signals, characterized in that: The following steps are involved: S1. Receive raw signals from sensors of different manufacturers and models through the signal interface module; S2 extracts features from the original signal to identify its signal type, protocol format, and data encoding rules; S3. Based on the recognition results, dynamically match the corresponding parsing template from the preset protocol library; If there is no matching template in the protocol library, an adaptation parsing rule is generated through a machine learning algorithm; S4. Based on the parsing template or the generated parsing rules, the original signal is converted into a standardized data format; S5. Perform real-time verification of standardized data, correct data errors, and supplement missing fields; S6. Transmit the processed data to a centralized processing platform for unified analysis, storage, and feedback control.
2. The method for analyzing and processing dynamic data of adaptively and actively adapting sensor signals according to claim 1, characterized in that: In step S2, the feature extraction includes signal waveform analysis, baud rate detection, check bit identification and data frame structure analysis. The feature extraction adopts the following steps: S21. Signal preprocessing: input the original binary signal stream, denoise the original binary signal stream, and perform sliding window mean filtering. The formula is: Eliminate high-frequency noise; Based on level jump detection, it is divided into independent data frames; S22. Baud rate detection, based on a combined analysis of time interval statistics and Fourier transform, counts the time intervals between adjacent rising / falling edges, takes the mode as the candidate baud rate, and extracts the signal's dominant frequency component through Fourier transform to verify the candidate baud rate. S23. Frame structure analysis: finite state machine combined with dynamic template matching to define states: start bit, data bit, check bit, stop bit, dynamically adjust the data bit length, and verify frame integrity through cyclic redundancy check; S24. Protocol type identification, rule-based feature hash and similarity matching, extract feature vector frame header flag, checksum mode, data length, calculate cosine similarity with pre-stored protocol templates, and select the highest match.
3. The method for analyzing and processing dynamic data of adaptively and actively adapting sensor signals according to claim 1, characterized in that: In step S3, the protocol library supports dynamic updating and can add or modify protocol templates through a remote server or a local interface. The method for adapting the parsing rules is as follows: S31 protocol library matching, input feature vector protocol type, frame structure, check method, search for matches in the preset protocol library, which is based on hash index acceleration. If the match is successful, load the corresponding parsing template; Otherwise, the rule generation process is triggered; S32. Machine learning-driven rule generation, based on a sequence-to-sequence model with a temporal attention mechanism, takes the raw signal sequence as input and outputs parsing rules including field segmentation positions and data type mapping tables; S33. Training data: Historical protocol samples of labeled raw signals and parsing rule pairs. Unrecognized signals are input, and the model outputs parsing rules. Rule validation is performed through simulated parsing and check digit verification to verify the validity of the rules. S34. The protocol library is dynamically updated. When a new protocol template is added, the feature vector is extracted and stored in the protocol library, and the hash index is updated at the same time.
4. The method for analyzing and processing dynamic data of adaptively and actively adapting sensor signals according to claim 1, characterized in that: In step S4, the standardized data format includes a unified timestamp, data unit, dimension, and data structure. The standardized data format mapping algorithm is as follows: S41. Field mapping table construction field mapping table construction, which parses the rules field name, data type, unit, according to the rules to define the field mapping relationship; S42. Data conversion, frame-by-frame parsing based on dynamic templates, splitting data bits according to parsing rules, extracting each field value, and performing unit conversion; S43. Timestamp synchronization: adds a unified timestamp to each piece of data with an accuracy of milliseconds, and synchronizes the system clock based on the NTP protocol.
5. The method for dynamic data analysis and processing of adaptive and active adaptation sensor signals according to claim 1, characterized in that: In step S5, the real-time verification includes data range verification, logical relevance check, and redundant data filtering. The real-time verification module detects and corrects data abnormalities, out-of-range values, and logical contradictions through data cleaning and error correction algorithms. The method is as follows: S51. Data range check: preset upper and lower limits for fields. If the limits are exceeded, the data is marked as invalid or interpolated using historical mean values. S52. Logical relevance check, defining the logical relationship between fields and performing real-time logic verification based on the rule engine; S53. Redundant data filtering and sliding window deduplication. If the difference between consecutive NN frame data is less than the threshold ∈∈, only the first frame is retained.
6. A device for the method for analyzing and processing dynamic data of adaptively and actively adapting sensor signals as described in any one of claims 1 to 5, characterized in that: include: Signal interface module, supporting RS-485, CAN, Modbus and TCP / IP protocols, used to receive multi-source heterogeneous sensor signals; A feature recognition module is connected to the signal interface module and implements signal segmentation, baud rate detection and frame structure analysis through a finite state machine and Fourier transform algorithm; The protocol adaptation module includes a protocol library unit and a machine learning unit. The protocol library unit stores preset parsing templates and accelerates matching through hash indexes. The machine learning unit generates parsing rules for unrecognized protocols based on the Transformer model. The data conversion module performs field mapping and unit conversion according to parsing rules and adds a unified timestamp; Verification and correction module, with built-in sliding window deduplication algorithm and rule engine, for data cleaning and logical error correction; The communication module uses a weighted round-robin scheduling strategy to optimize data transmission priority and supports a breakpoint-resume transmission mechanism; The self-diagnosis module monitors the health status of the device in real time through the isolation forest algorithm and triggers an alarm signal when an abnormality occurs.
7. The device for the dynamic data analysis and processing method of adaptively and actively adapting sensor signals according to claim 6, characterized in that: The protocol library of the protocol adaptation module supports remote OTA updates, and automatically extracts feature vectors and updates hash indexes when new protocol templates are added.
8. The device for the dynamic data analysis and processing method of adaptively and actively adapting sensor signals according to claim 6, characterized in that: The health indicators of the self-diagnosis module include CPU usage, protocol matching success rate and communication delay. Anomaly detection uses the isolation forest algorithm to calculate the anomaly score of real-time indicators. If the score exceeds the threshold, the faulty module is located.
9. The device for the dynamic data analysis and processing method of adaptively and actively adapting sensor signals according to claim 5, characterized in that: The breakpoint-resume transmission mechanism of the communication module is implemented based on the sliding window of the TCP protocol, which records the transmission progress and continues the transmission after the network is restored.
10. The device for the dynamic data analysis and processing method of adaptively and actively adapting sensor signals according to claim 5, characterized in that: The timestamp synchronization accuracy of the data conversion module is at the millisecond level and is synchronized with the system clock through the NTP protocol.
Citation Information
Cited By
Data persistence system and method based on dynamic interpreter
CN121070939A
A data persistence system and method based on dynamic interpreter
CN121070939B
System for protocol adaptation and unified access of multi-source heterogeneous industrial equipment
CN121691487A