A smart factory multi-source heterogeneous device collaborative management and control system test verification method

Abstract

This invention discloses a testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory, belonging to the field of testing and verification technology for collaborative management and control systems. The method includes the following steps: S1: Constructing a simulation test environment for multi-source heterogeneous equipment; S2: Inputting composite test data into the simulation test environment; S3: Verifying the core performance of the collaborative management and control system; S4: Generating a system defect report based on the verification results. The advantages are: by constructing a test environment close to the industrial site, simulating electromagnetic interference coupling scenarios and real protocols and PLC switching logic, idealization deviations are avoided, laying a solid foundation for defect investigation; injecting composite data matching the patterns of the industrial site fully exposes hidden defects under multiple anomalies, reducing the risk of system downtime; forming a verification-positioning-optimization closed loop, directly guiding system improvement, and effectively enhancing the reliability and practical value of the collaborative management and control system for multi-source heterogeneous equipment.

Description

Technical Field

[0001] This invention relates to the field of testing and verification technology for collaborative management and control systems, and in particular to a testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory. Background Technology

[0002] With the advancement of industry and intelligent manufacturing, smart factories have formed a collaborative management and control architecture for multi-source heterogeneous equipment, including industrial instruments, IoT sensors, and PLC controllers. Data interaction between devices is achieved through heterogeneous protocols such as Modbus, OPC UA, and LoRaWAN. Although basic testing methods already exist in the industry, which can perform preliminary verification on the range accuracy of a single device and the communication stability of a single protocol (such as using simulators to test the steady-state output of instruments and using simple frame injection tools to verify protocol transmission), providing basic support for system debugging, the most prominent technical shortcoming of existing testing and verification methods is that they cannot cover the complex anomalies and related effects in the collaborative scenario of multi-source heterogeneous equipment, and are seriously out of touch with actual industrial working conditions.

[0003] Specifically, existing methods neither simulate physical and operational scenarios such as electromagnetic interference coupling caused by the close-range deployment of IoT sensors and industrial instruments in industrial sites, nor inject complex abnormal data such as protocol anomaly frames and business scenarios. Furthermore, performance verification is limited to basic dimensions such as the accuracy of a single device and communication of a single protocol, which makes it impossible for testing to expose potential defects of the system under complex working conditions. After the system goes online, it is prone to misjudgment, interruption and other problems due to unforeseen complex anomalies. Summary of the Invention

[0004] The purpose of this invention is to solve the problems raised in the background art by proposing a test and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory.

[0005] In view of the above problems, the present invention provides a testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory, comprising the following steps: S1: Construct a multi-source heterogeneous device simulation test environment. The simulation test environment includes at least three types of devices: industrial instruments, IoT sensors, and PLC controllers. The devices communicate with each other using three or more heterogeneous protocols, such as Modbus, OPC UA, and LoRaWAN. S2: Input composite test data into the simulation test environment, the composite test data including: Normal operating data refers to steady-state data that conforms to the measurement range of each device. Timing deviation data causes the timestamp deviation between Modbus devices and OPC UA devices to be distributed between 20ms and 200ms. The sampling period of Modbus devices is 100ms, and the sampling period of OPC UA devices is 50ms. Protocol exception frame data, including Modbus CRC check error frames, OPC UA session timeout reconnection frames, and incomplete LoRaWAN uplink frames; Protocol semantic distortion data, including unit errors in the values corresponding to Modbus register addresses and range conflicts in OPC UA node data; IoT packet loss data, which causes the packet loss rate of IoT sensors to gradually increase as the signal strength weakens; Industrial instrument false data, including high-frequency pulse data caused by electromagnetic interference and baseline drift data caused by calibration deviation; Cross-validation comparison data, which are test values where the transmission deviation between industrial instruments and IoT sensors at the same monitoring point exceeds the preset threshold; S3: Verify the core performance of the collaborative control system, including: Heterogeneous protocol conversion fidelity, verifying the numerical, unit, and range semantic consistency of the converted data; Protocol exception frame fault tolerance, verifying the effective data acquisition rate of the system when injecting exception frames; Industrial instrument false data recognition rate, verifying the correct recognition ratio of the system for high-frequency pulse data and baseline drift data; Multi-source data cross-validation accuracy, verifying the accuracy of the system's credibility judgment of the deviation data between industrial instruments and IoT sensors; Multi-source data time series synchronization accuracy, verifying the timestamp calibration deviation of data from devices with different protocols; Device collaborative control response delay, verifying the closed-loop delay from instruction issuance to multi-device linkage feedback; S4: Generate a system defect report based on the verification results.

[0006] Compared with the existing technologies, the technical solution provided by this application has at least the following technical effects or advantages: 1: When constructing the simulation test environment, this method not only clarifies the specific types and parameters of industrial instruments, IoT sensors, and PLC controllers, but also simulates the electromagnetic interference coupling scenario in the industrial field by controlling the physical deployment distance of the devices. At the same time, it configures protocol parameters and PLC dual-protocol switching logic consistent with the actual situation. The way of constructing an environment close to the real working conditions avoids the problem of the disconnection between the idealized environment and the actual scenario in traditional tests, making subsequent data injection and performance verification accurately reflect the performance of the collaborative control system in actual operation, providing a reliable test basis for system defect troubleshooting.

[0007] 2. The composite test data injected by this method breaks through the limitations of traditional single abnormal data. It not only includes normal operation data, but also specifically designs multiple types of composite data such as time sequence deviation-equipment load correlation data, protocol abnormal frame-business scenario bound data, and IoT packet loss-spoof data coupled data. Moreover, each type of data matches the anomaly occurrence pattern in the industrial field. This comprehensive coverage of abnormal scenarios can accurately trigger hidden defects in the collaborative management system under multiple abnormal superposition conditions. Compared with traditional single data injection, it can more fully verify the system's anti-interference capability and fault tolerance capability, and reduce the risk of downtime caused by unforeseen anomalies after the system goes online.

[0008] 3. The performance verification process of this method sets quantitative indicators around core dimensions such as heterogeneous protocol conversion, fault tolerance of abnormal frames, identification of false data, timing synchronization, and collaborative control to ensure that the verification results are measurable. The generated defect report not only clearly identifies the location and cause of the problem, but also provides a targeted optimization solution that can be implemented. This closed-loop design from performance verification to defect localization and solution optimization avoids the traditional testing from merely finding problems to directly guiding technical personnel to make targeted improvements to the collaborative management and control system, effectively improving the accuracy of semantic conversion, fault tolerance of anomalies, and reliability of collaborative control. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the system architecture of a test and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory, according to an embodiment of the present invention. Detailed Implementation

[0010] The above technical solutions will now be described in detail with reference to the accompanying drawings and specific embodiments to provide a better understanding of them. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments used only to explain the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the drawings, not all of them.

[0011] Please see Figure 1 A testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory includes the following steps: S1: Construct a multi-source heterogeneous device simulation test environment. The simulation test environment should include at least three types of devices: industrial instruments, IoT sensors, and PLC controllers. Industrial instruments include thermocouple temperature instruments and capacitive pressure instruments. The measurement range of thermocouple temperature instruments is -50℃ to 300℃, and the measurement range of capacitive pressure instruments is 0 to 10MPa. IoT sensors include LoRa temperature and humidity sensors and NB-IoT vibration sensors. The sampling frequency of LoRa temperature and humidity sensors is 1Hz, and the sampling frequency of NB-IoT vibration sensors is 10Hz. The devices should communicate with each other using at least three heterogeneous protocols: Modbus, OPC UA, and LoRaWAN. The PLC controller should support both Modbus RTU and OPC UA protocols. A more detailed description includes hardware selection and connectivity as well as software configuration; Hardware selection and connection: Siemens S7-1200 series PLC was selected as the controller (supporting Modbus RTU / OPCUA dual protocols). Thermocouple temperature instrument (model: OMRON E5CC) and capacitive pressure instrument (model: SMC ISE30A) were connected via RS485 interface, and industrial switch was connected via Ethernet interface. The LoRa temperature and humidity sensor (model: SenseCAP S2100) and the NB-IoT vibration sensor (model: Quectel BC20) are wirelessly connected to the LoRa gateway (model: RAK7258) and the NB-IoT base station simulator (model: Keysight E7515A). The gateway and the base station simulator are connected to the same industrial switch via Ethernet, ultimately enabling network interconnection between all devices and the PLC controller. During physical deployment, the distance between the LoRa temperature and humidity sensor and the thermocouple temperature instrument is controlled within 5 meters, and the distance between the NB-IoT vibration sensor and the capacitive pressure instrument is controlled within 3 meters to simulate the electromagnetic interference coupling scenario between equipment in the industrial field.

[0012] Software configuration: TIA Portal V16 software is used to program the PLC, configuring Modbus RTU protocol parameters (baud rate 9600bps, data bits 8 bits, stop bits 1 bit, parity none) and OPC UA protocol parameters (port number 4840, security policy Basic256Sha256), and setting switching conditions (such as triggering when a "protocol switching command" is received) through the software's built-in "protocol switching trigger module". OPC Scout V10 software is used as the OPC UA client to monitor the communication status between the PLC and the OPC UA device; LoRaWAN Server (model: ChirpStack) is used to manage the data reception and forwarding of LoRa sensors. An electromagnetic interference simulator (model: EMTEST ESVS 200N) was used to set the interference source, with an interference frequency range of 10kHz-30MHz and an adjustable interference intensity (0-10V / m), for subsequent simulation of false data from industrial instruments. The PLC controller supports both Modbus RTU and OPC UA protocols, with a switching response time of no more than 100ms. The specific testing and verification process is as follows: Write a protocol switching test program in TIA Portal V16 software. The program logic is as follows: the PLC automatically triggers a protocol switch between Modbus RTU and OPC UA every 10 seconds and records the switch trigger time T1. The successful handshake moment T2 after PLC switching is collected in real time using the OPC Scout V10 client, and the successful register read moment T3 after Modbus RTU switching is collected using a Modbus debugging tool (such as Modbus Poll). Calculate the handover response time: OPC UA handover response time = T2 - T1, Modbus RTU handover response time = T3 - T1; Repeat the test multiple times (e.g., 30 times), remove the maximum and minimum values ​​and take the average value to ensure that the average response time does not exceed 100ms.

[0013] S2: Input composite test data into the simulation test environment. The composite test data includes: Normal operating data, which is steady-state data conforming to the measurement range of each device, is generated continuously using a steady-state data generation algorithm based on the measurement range and accuracy of each device. The normal data for thermocouple temperature instruments is 25℃±2℃, with a small fluctuation of ±0.5℃ added every 5 seconds (simulating natural changes in ambient temperature) to ensure the data remains within the 25℃±2℃ range. The normal data for capacitive pressure instruments is 1.0MPa±0.05MPa (with a fluctuation of ±0.02MPa added every 10 seconds, simulating small fluctuations in pipeline pressure, to ensure the data remains within the 1.0MPa±0.05MPa range). The normal data for LoRa temperature and humidity sensors is 50%±5% humidity (with a fluctuation of ±1% added every 30 seconds, simulating changes in ambient humidity, to ensure the data remains within the 50%±5% range). This is achieved using a device simulator (such as Keysight). The 34970A data acquisition unit injects the generated steady-state data into various devices. Industrial instruments inject data into thermocouple temperature instruments and capacitive pressure instruments through analog signal output interfaces (4-20mA). After receiving the data, the instruments upload it to the PLC via the Modbus RTU protocol. IoT sensors receive the pushed data through the data injection interface of the LoRa gateway / NB-IoT base station simulator and then upload it to the PLC via a wireless protocol. During the injection process, the PLC's real-time monitoring interface (TIA Portal online monitoring) observes whether the data is stable within the preset range. If fluctuations outside the range occur, the simulator output parameters are readjusted to ensure the validity of the steady-state data.

[0014] Timing deviation data causes the timestamp deviation between Modbus devices and OPC UA devices to be distributed between 20ms and 200ms. The sampling period of Modbus devices is 100ms, and the sampling period of OPC UA devices is 50ms. The generation of timing deviation data is related to the device load: When the number of devices connected to the PLC controller exceeds a certain number, the lower limit of the timestamp deviation between the Modbus device and the OPC UA device increases. For example, when the number exceeds 50 devices, the lower limit of the timestamp deviation between the Modbus device and the OPC UA device increases from 20ms to 50ms. When the load decreases to below a certain number of devices, the upper limit of the deviation decreases to simulate the impact of load changes on timing synchronization. For example, when the load decreases to below 20 devices, the upper limit of the deviation decreases from 200ms to 100ms to simulate the impact of load changes on timing synchronization.

[0015] Deploy a device connection monitoring module in the PLC to count the total number of currently connected devices (including industrial instruments and IoT sensors) in real time. The monitoring frequency is once per second, and the monitoring result is recorded as N. Based on the value of N, dynamically set the timestamp deviation range between Modbus devices and OPC UA devices using a timing deviation adjustment algorithm. The algorithm logic is as follows: When N > 50 units (high load): it is determined that the PLC processing capacity is close to the limit, and the lower limit of the timestamp deviation is increased from 20ms to 50ms, while the upper limit of the deviation is kept at 200ms (because the PLC processing delay increases under high load, the lower limit of the deviation needs to be expanded to simulate the real timing misalignment). When there are 20 units ≤ N ≤ 50 units (medium load): maintain a deviation range of 20ms-200ms (simulating timing characteristics under normal load); When N < 20 units (low load): it is determined that the PLC processing capacity is sufficient, the upper limit of deviation is reduced from 200ms to 100ms, and the lower limit of deviation is kept at 20ms (the PLC can process data quickly under low load, and the timing deviation is smaller). Using a timestamp manipulation tool (such as the timestamp editing plugin for Wireshark), add timestamp offset values ​​to each frame of data from a Modbus / OPC UA device based on the adjusted offset range. For example: Under high load, add a random deviation of 50ms-200ms to Modbus device data and a random deviation of 30ms-180ms to OPC UA device data (because the OPC UA protocol has higher processing efficiency than Modbus, the deviation is slightly smaller). Under low load, add a random bias of 20ms-100ms to Modbus device data and a random bias of 10ms-80ms to OPC UA device data; Deviation verification: Collect the actual deviation value of 100 frames of data through the PLC's timestamp recording function, and calculate whether the deviation range meets the adjusted settings. If it does not meet the settings (e.g., the lower limit of deviation is less than 50ms under high load), readjust the parameters of the tampering tool.

[0016] Protocol anomaly frame data includes Modbus CRC check error frames, OPC UA session timeout reconnection frames, and LoRaWAN incomplete uplink frames, with an injection probability of 1%-5%. The types of protocol anomaly frame data match protocol characteristics: Modbus CRC check error frames are generated by tampering with the last 2 bytes of the checksum; OPC UA session timeout reconnection frames are generated by forcibly terminating the session and reconnecting after a 500ms delay; and LoRaWAN incomplete uplink frames are generated by truncating the payload field to 60% of its original length. Protocol semantic distortion data includes unit errors in the values ​​corresponding to Modbus register addresses (e.g., kPa mislabeled as MPa) and range conflicts in OPC UA node data (e.g., the old device's range of 0-100℃ and the new device's range of 0-200℃ transmitting the same value). Specifically, the range conflict in the protocol semantic distortion data is as follows: the old device uses the Profinet V1 protocol with a temperature range of 0-100℃, while the new device uses Profinet... The V2 protocol has a temperature range of 0-200℃. When both transmit a value of 50, there is a difference of 2 times between the actual value of the physical quantity.

[0017] Protocol anomaly frames need to be injected in specific scenarios based on the communication characteristics of different protocols to avoid distortion of verification results caused by indiscriminate injection. The specific triggering and control steps are as follows: Abnormal frame type matches the triggering scenario: Modbus CRC check error frame: Injected only when the PLC sends a pressure regulation command to the capacitive pressure gauge. Because the instrument needs to respond quickly in this scenario, CRC errors will directly affect the control accuracy. OPC UA Session Timeout Reconnection Frame: Injected only when the LoRa temperature and humidity sensor uploads a humidity exceeding the limit alarm. Because alarm data needs to be transmitted in real time, session timeout will cause alarm delay. LoRaWAN incomplete uplink frame: Injected only when NB-IoT vibration sensor uploads vibration exceeding limit data (triggering condition: vibration acceleration > 5m / s²). 2 Because vibration data is related to equipment malfunctions, incomplete data can affect fault diagnosis. Injection probability control: The injection probability is set to 1%-5% through the abnormal frame probability control module. The specific logic is as follows: For every 100 frames of normal data collected, 1-5 frames are randomly selected for anomaly handling (such as tampering with CRC check bits or truncating payload fields). Record the injection time and device type of the abnormal frame to facilitate subsequent analysis of the impact of the abnormal frame on different devices; Details of abnormal frame generation: Modbus CRC check error frame: By tampering with the last 2 bytes of the CRC check bit of the message (changing the correct check bit 0x1234 to 0x5678), the instrument is ensured to fail the CRC check, triggering data retransmission; OPC UA Session Timeout Reconnect Frame: Forcefully terminates the current session through the OPC Server's session management interface, re-establishes the session after a 500ms delay, simulating session interruption caused by network fluctuations; LoRaWAN incomplete uplink frame: The original length of the payload field (e.g., 20 bytes) is truncated to 12 bytes (60%), and key fields such as vibration frequency and acquisition time are deleted to simulate data loss caused by wireless signal attenuation.

[0018] The packet loss data in the Internet of Things (IoT) causes the packet loss rate of IoT sensors to gradually increase as the signal strength decreases. The relationship between signal strength and packet loss rate in IoT packet loss data follows a non-linear pattern: when the signal strength drops from -50dBm to -90dBm, the packet loss rate increases linearly from 5% to 15%; when the signal strength drops from -90dBm to -120dBm, the packet loss rate jumps non-linearly from 15% to 30%, simulating the abrupt change characteristics of the critical value of wireless signals. The specific implementation steps are as follows: Signal strength control: The signal attenuation module of the NB-IoT base station simulator (Keysight E7515A) and LoRa gateway is used to gradually reduce the signal strength from -50dBm (strong signal) to -120dBm (weak signal), with each adjustment increment being 10dBm and a dwell time of 5 minutes (to ensure stable packet loss rate). Packet loss rate calculation: The packet loss rate at each signal strength level is calculated in real time using a packet loss rate statistical algorithm. The algorithm logic is as follows: Transmitter (sensor): Records the total number of data packets S transmitted within 5 minutes; Receiver (PLC): Records the total number R of data packets successfully received within 5 minutes; Packet loss rate P = (SR) / S × 100%; Nonlinear relationship verification: Based on the calculated packet loss rate, verify whether it conforms to the preset nonlinear relationship. When the signal strength drops from -50dBm to -90dBm, P increases linearly from 5% to 15% (P increases by 2.5% for every 10dBm decrease). When the signal strength drops from -90dBm to -120dBm, P increases nonlinearly from 15% to 30% (-90dBm→-100dBm, P increases to 20%; -100dBm→-110dBm, P increases to 25%; -110dBm→-120dBm, P increases to 30%). If the packet loss rate of a certain signal strength does not conform to the preset relationship (e.g., P is only 16% when it is -100dBm), adjust the parameters of the signal attenuation module (e.g., increase the attenuation amount) until it conforms to the nonlinear relationship.

[0019] Spurious data from industrial instruments includes high-frequency pulse data (pulse width 0.1s-1s, amplitude ±20% of normal value) caused by electromagnetic interference and baseline drift data (drift amplitude ±5% of normal value, duration 5s-10s) caused by calibration deviation. The verification of the spurious data identification rate for industrial instruments employs a dual-feature comparison: comparing the pulse width and amplitude change rate of the high-frequency pulse data with a preset feature library, and comparing the drift rate and duration of the baseline drift data with a preset feature library. A correct identification is determined when both match. The specific implementation steps are as follows: Construction of the interference source feature library: Data on common electromagnetic interference and calibration deviations in industrial settings are collected, characteristic parameters are extracted, and a feature library is established. The feature library contains feature items for two types of data: High-frequency pulse data characteristics: pulse width (0.1s-1s), amplitude change rate (amplitude change value / pulse width, such as 2℃ / 0.5s=4℃ / s), pulse interval (5s-10s, simulating intermittent interference); Baseline drift data characteristics: drift amplitude (normal value ±5%), drift rate (drift amplitude / duration, e.g., 0.05MPa / 5s = 0.01MPa / s), duration (5s-10s); The feature library is imported through the feature library storage module of the PLC, which supports subsequent updates of feature parameters based on new interference scenarios; Dual-feature comparison logic: When industrial instruments upload data, the PLC's fake data identification module performs the following comparison steps: Step 1: Extract real-time features of the data, such as the 25℃→30℃→25℃ data uploaded by the thermocouple (pulse width 0.5s, amplitude change rate 10℃ / s). Step 2: High-frequency pulse feature comparison. The pulse width of the real-time feature is compared with 0.1s-1s in the feature library, and the amplitude change rate is compared with ≥2℃ / s in the feature library (because the amplitude change rate of normal fluctuations is ≤0.5℃ / s). If both conditions are met, it is determined to be a high-frequency pulse feature match. Step 3: Baseline drift feature comparison. If the data is 25℃→26.25℃→26.25℃ (drift amplitude 1.25℃, duration 8s, drift rate 0.156℃ / s), compare the drift amplitude with ±5% (1.25℃) in the feature library, and compare the drift rate with ≤0.02℃ / s in the feature library (normal drift rate ≤0.01℃ / s). If both conditions are met, it is determined to be a baseline drift feature match. Step 4: Result determination. Data is only considered false data if both features of a certain type of data match, thus avoiding misjudgment caused by a single feature match (e.g., if the pulse width of normal data matches but the amplitude change rate does not, it is not considered false data).

[0020] Cross-validation comparison data consists of test values ​​where the transmission deviation between the industrial instrument and the IoT sensor at the same monitoring point exceeds a preset threshold (e.g., the instrument displays 100℃, and the sensor displays 80℃). The accuracy verification of multi-source data cross-validation is set with tiered thresholds: for temperature data, a confidence judgment is triggered when the deviation between the industrial instrument and the IoT sensor exceeds 5℃; for pressure data, a judgment is triggered when the deviation exceeds 0.5MPa. Different thresholds are used for different types of data to improve verification accuracy. The specific implementation steps are as follows: Determining the tiered thresholds: Based on the accuracy requirements of different data in a smart factory, tiered thresholds are set using a threshold analysis algorithm. The algorithm logic is as follows: Temperature data: The measurement accuracy of industrial instruments (thermocouples) is ±0.5℃, and the measurement accuracy of IoT sensors (LoRa temperature and humidity) is ±1℃. The maximum allowable deviation between the two is 4℃ (instrument accuracy + sensor accuracy + environmental interference error). Therefore, the trigger threshold is set to 5℃ (slightly higher than the maximum allowable deviation to avoid false triggering). Pressure data: The measurement accuracy of the industrial instrument (capacitive pressure) is ±0.02MPa, and the accuracy of the IoT sensor (no pressure sensor, here we assume the addition of an NB-IoT pressure sensor, accuracy ±0.03MPa) is 0.08MPa. Therefore, the trigger threshold is set to 0.5MPa (because the pressure data is related to equipment safety, it needs to be strictly judged, and the threshold is set to more than 6 times the maximum allowable deviation). Deviation Data Generation: The deviation data generation module generates data showing deviations exceeding a threshold for industrial instruments and IoT sensors at the same monitoring point. For example: Temperature monitoring points: Industrial instruments generate 100℃ data, and IoT sensors generate 80℃ data (deviation 20℃ > 5℃). When injecting, ensure that the timestamps of the two are consistent (to avoid the impact of timing deviation on cross-validation). Pressure monitoring points: Industrial instruments generate 2.0MPa data, and IoT sensors generate 1.4MPa data (deviation 0.6MPa > 0.5MPa). During injection, ensure that the measurement objects of both are the same pipe (to avoid deviation caused by different monitoring objects). Data injection order: First, inject normal cross-validation data (deviation < threshold). After the system stabilizes, inject data with deviation exceeding the threshold to ensure that the system can clearly distinguish between normal and abnormal deviations.

[0021] S3: Verify the core performance of the collaborative management and control system, including: Heterogeneous protocol conversion fidelity: Verify the semantic consistency of the converted data in terms of numerical values, units, and ranges, requiring the semantic distortion rate to be no more than 2%; clearly define the specific location of the distortion (e.g., Modbus register address 0x0001 corresponding to OPC UA node Pressure), the distortion type (unit error: kPa→MPa), and the deviation value (100kPa→100MPa, deviation 99900kPa). The protocol's fault tolerance for abnormal frames is verified by checking the system's effective data acquisition rate during abnormal frame injection, requiring an effective acquisition rate of no less than 95%. The interruption frequency caused by different abnormal frames is statistically analyzed (e.g., Modbus CRC error frames cause 5 interruptions per hour, OPC UA session timeouts cause 3 interruptions per hour), and the device types are associated (capacitive pressure gauges, LoRa temperature and humidity sensors). The causes of the interruptions are analyzed (e.g., CRC check retransmission mechanism is not enabled, OPC UA session timeout is set too short). The false data recognition rate of industrial instruments verifies the system's correct recognition rate of high-frequency pulse data and baseline drift data, requiring a recognition rate of no less than 90%. The system also analyzes the types of misjudgments (12 misjudgments of high-frequency pulses and 8 misjudgments of baseline drift), calculating the misjudgment rate as (12+8) / 200×100%=10% (out of a total of 200 test frames). The analysis also examines the causes of misjudgments (e.g., the threshold for the amplitude change rate of high-frequency pulses is set too low, causing normal fluctuations to be misjudged). Cross-validation of multi-source data accuracy: Verify the system's accuracy in judging the reliability of deviation data from industrial instruments and IoT sensors, requiring an accuracy of no less than 90%; Record failure scenarios (e.g., the system does not trigger reliability judgment when the temperature deviation is 6℃), and analyze the cause of failure (the threshold is set too high, the original threshold of 5℃ should actually be adjusted to 4℃). Multi-source data timing synchronization accuracy: Verify the timestamp calibration deviation of data from devices with different protocols, requiring the deviation to not exceed 30ms after calibration; The response latency of equipment collaborative control is verified to check the closed-loop latency from the issuance of the command to the feedback of multiple devices, and the latency is required to be no more than 500ms.

[0022] S4: Generate a system defect report based on the verification results. The system defect report includes the specific location and deviation value of unit and range semantic distortion in heterogeneous protocol conversion; the frequency of data acquisition interruption caused by abnormal protocol frames and the types of associated devices; the types and rates of misjudgment of false data from industrial instruments; scenarios of failure of multi-source data cross-validation and corresponding threshold setting suggestions; deviation analysis of each performance index from the preset threshold and targeted optimization schemes; the core components of the defect report are as follows: Heterogeneous protocol conversion defects: clearly define the specific location of semantic distortion (e.g., Modbus register 0x0001 corresponds to the OPC UA node "Pressure"), distortion type (unit error: kPa→MPa), and deviation value (e.g., 100kPa is mistakenly converted to 100MPa, with a deviation of 99900kPa). Impact of protocol anomaly frames: Statistics on the acquisition interruption frequency of different anomaly frames (e.g., Modbus CRC error frames causing 5 interruptions / hour) and the associated device type (capacitive pressure gauge). False data misjudgment: Record the misjudgment type (12 misjudgments of high-frequency pulse, 8 misjudgments of baseline drift), misjudgment rate = (12+8) / 200×100%=10%, analyze the reasons (such as the amplitude change rate threshold being too low); Cross-validation failure: Record the failure scenario (e.g., the judgment is not triggered when the temperature deviation is 6℃). The reason is mostly that the threshold is set too high (the original threshold of 5℃ needs to be adjusted to 4℃). Indicator deviation analysis: Compare the actual values ​​of each indicator with the preset threshold (e.g., semantic distortion rate 2.8% > 2%) to identify the root cause of the deviation.

[0023] Targeted optimization solutions include protocol conversion semantic verification rules: establishing a three-dimensional binding relationship between Modbus register addresses and OPC UA nodes—address-physical quantity-unit; if a conflict occurs between the unit field and the binding relationship during conversion, automatic unit correction is triggered and a correction log is recorded; the three-dimensional binding relationship is established as follows: Organize the mapping table between Modbus register addresses and OPC UA node addresses, physical quantities, and units (e.g., 0x0001 → pressure → kPa, 0x0002 → temperature → ℃). Deploy the binding module in the PLC protocol conversion module, import the mapping table, and set the update mechanism (new devices require manual review and automatic verification). Unit error correction and log recording: Error correction trigger: If the data unit conflicts with the binding relationship during conversion (e.g., 0x0001 data unit is MPa), it will be automatically corrected to kPa, and the value will remain unchanged (200MPa→200kPa). Log recording: Records the time of error correction, address, error / correction unit, and data value, and supports querying and tracing by time / address; Optimization results: After implementation, the semantic distortion rate of retesting decreased from 2.8% to 1.2%, meeting the requirement of ≤2%.

[0024] This detailed embodiment elaborates on the internal operating logic of each major functional module, aiming to provide a detailed basis and explanation for those skilled in the art to understand and implement it. It should be emphasized that the above description constitutes a specific, preferred embodiment, but the concept of the present invention is not limited thereto. Any equivalent transformations, modifications, or improvements based on the core spirit of the present invention, without departing from the technical principles and scope disclosed in this specification, should be considered to fall within the scope of protection claimed by the present invention, as long as they achieve the same or similar technical effects.

Claims

1. A testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in an intelligent factory, characterized in that, It includes the following steps: S1: Construct a multi-source heterogeneous device simulation test environment, which at least includes three types of devices: industrial instruments, Internet of Things sensors, and PLC controllers. More than three heterogeneous protocols, such as Modbus, OPC UA, and LoRaWAN, are used for communication between devices; S2: Input composite test data into the simulation test environment. The composite test data includes: Normal operation data, which are steady-state data that conform to the range of each device; Timing deviation data, such that the timestamp deviation between Modbus devices and OPC UA devices is distributed between 20 ms and 200 ms. The sampling period of Modbus devices is 100 ms, and the sampling period of OPC UA devices is 50 ms; Protocol abnormal frame data, including CRC check error frames of Modbus, session timeout reconnection frames of OPC UA, and incomplete uplink frames of LoRaWAN; Protocol semantic distortion data, including unit errors of the numerical values corresponding to Modbus register addresses and range conflicts of OPC UA node data; Internet of Things packet loss data, such that the packet loss rate of Internet of Things sensors gradually increases as the signal strength weakens; Industrial instrument false data, including high-frequency pulse data caused by electromagnetic interference and baseline drift data caused by calibration deviation; Cross-validation comparison data, which are test values where the transmission deviation between industrial instruments and Internet of Things sensors at the same monitoring point exceeds a preset threshold; S3: Verify the core performance of the collaborative control system, including: Heterogeneous protocol conversion fidelity, verifying the numerical value, unit, and range semantic consistency of the data after conversion; Protocol abnormal frame fault tolerance, verifying the effective data acquisition rate of the system when abnormal frames are injected; Industrial instrument false data recognition rate, verifying the correct recognition ratio of the system for high-frequency pulse data and baseline drift data; Multi-source data cross-validation accuracy, verifying the accuracy of the system's credibility judgment of the deviation data between industrial instruments and Internet of Things sensors; Multi-source data timing synchronization accuracy, verifying the timestamp calibration deviation of data from devices with different protocols; Device collaborative control response delay, verifying the closed-loop delay from instruction issuance to multi-device linkage feedback; 2. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, S4: Generate a system defect report based on the verification results. The industrial instruments include thermocouple temperature instruments and capacitive pressure instruments. The Internet of Things sensors include LoRa temperature and humidity sensors and NB-IoT vibration sensors. The PLC controller supports both Modbus RTU and OPC UA protocols; The measurement range of the thermocouple temperature instrument is -50°C to 300°C, and the measurement range of the capacitive pressure instrument is 0 to 10 MPa; The sampling frequency of the LoRa temperature and humidity sensor is 1 Hz, and the sampling frequency of the NB-IoT vibration sensor is 10 Hz; The switching response time of the PLC controller for switching between Modbus RTU and OPC UA protocols does not exceed 100 ms.

3. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 2, characterized in that, The normal readings for the thermocouple temperature instrument are 25℃±2℃, the normal readings for the capacitive pressure instrument are 1.0MPa±0.05MPa, and the normal readings for the LoRa temperature and humidity sensor are 50%±5% humidity.

4. The testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The generation of the timing deviation data is related to the device load: when the number of devices connected to the PLC controller exceeds a certain number, the lower limit of the timestamp deviation between the Modbus device and the OPC UA device increases; when the load decreases to below a certain number of devices, the upper limit of the deviation decreases, in order to simulate the impact of load changes on timing synchronization.

5. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The types of abnormal frame data in the protocol match the protocol characteristics: Modbus CRC check error frames are generated by tampering with the last 2 bytes of the check bit; OPC UA session timeout reconnection frames are generated by forcibly terminating the session and reconnecting after a 500ms delay; and LoRaWAN incomplete uplink frames are generated by truncating the payload field to 60% of its original length.

6. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The specific range conflict of the semantically distorted data in the protocol is as follows: the old device uses the Profinet V1 protocol, whose temperature range is 0-100℃, and the new device uses the Profinet V2 protocol, whose temperature range is 0-200℃. When both transmit a value of 50, a difference of 2 times occurs between the actual values ​​of the physical quantity and the actual values.

7. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The signal strength and packet loss rate of the lost IoT data satisfy a nonlinear relationship: when the signal strength drops from -50dBm to -90dBm, the packet loss rate increases linearly from 5% to 15%; when the signal strength drops from -90dBm to -120dBm, the packet loss rate jumps nonlinearly from 15% to 30%, simulating the abrupt change characteristics of the critical value of wireless signals.

8. The testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The verification of the false data recognition rate of the industrial instrument adopts a dual feature comparison: the pulse width and amplitude change rate of the high-frequency pulse data are compared with the preset feature library, and the drift rate and duration of the baseline drift data are compared with the preset feature library. When both match, it is determined to be a correct recognition.

9. The testing and verification method for a collaborative management and control system for multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The verification accuracy of the multi-source data cross-validation is set with graded thresholds: for temperature data, a confidence judgment is triggered when the deviation between the industrial instrument and the IoT sensor exceeds 5°C; for pressure data, a judgment is triggered when the deviation exceeds 0.5MPa. Different thresholds are used for different types of data to improve the verification accuracy.

10. The testing and verification method for a collaborative management and control system of multi-source heterogeneous equipment in a smart factory according to claim 1, characterized in that, The system defect report includes the specific location and deviation value of unit and range semantic distortion in heterogeneous protocol conversion; the frequency of data acquisition interruption caused by abnormal protocol frames and the type of associated equipment; the types and rates of misjudgment of false data from industrial instruments; and the scenarios of failure of multi-source data cross-validation and corresponding threshold setting suggestions. Deviation analysis of each performance indicator from the preset threshold and targeted optimization solutions; The targeted optimization scheme includes protocol conversion semantic verification rules: establish a three-dimensional binding relationship between Modbus register address and OPC UA node address-physical quantity-unit; if the unit field conflicts with the binding relationship during conversion, unit error correction is automatically triggered and a correction log is recorded.

Images