Video monitoring linkage method for realizing heterogeneous software system based on network communication

Abstract

The invention relates to the technical field of video monitoring linkage, and provides a video monitoring linkage method for realizing a heterogeneous software system based on network communication, and the method comprises the steps: building a dynamically loaded DLL plug-in architecture and a three-dimensional mapping power supply monitoring network in advance; the DLL plug-in architecture is configured with an associated industrial communication protocol stack; wherein the industrial communication protocol stack is provided with a protocol identification mechanism based on flow characteristics; the power supply monitoring network is configured with a distributed message bus constructed based on ZeroMQ and an XGBoost anomaly detection model; wherein the distributed message bus is provided with north-south data channels, and the north-south data channels are respectively connected with the video monitoring equipment; and performing anomaly detection on the equipment induction data of the video monitoring equipment through the XGBoost anomaly detection model, and converting the anomaly data into a streaming media interface.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance linkage, and particularly to a method for video surveillance linkage of heterogeneous software systems based on network communication. Background Art

[0002] With the rapid development of industrial automation and informatization, video surveillance systems have been widely used in industrial production, security monitoring and other fields. However, existing video surveillance systems face many challenges when dealing with heterogeneous devices and protocols. Traditional video surveillance systems usually adopt a static loading method, which is difficult to adapt to devices of different brands and models, resulting in poor system compatibility and insufficient scalability. In addition, existing systems also have deficiencies in anomaly detection and real-time response, and cannot detect and handle device anomalies in a timely manner.

[0003] In industrial environments, video surveillance devices and sensor nodes usually adopt a variety of different communication protocols, such as Modbus, OPC UA, RTSP, etc. Compatibility issues between these protocols limit the integration and interoperability of the system. At the same time, with the development of the Industrial Internet of Things (IIoT), the number of devices and the amount of data are increasing continuously, and it is impossible to achieve monitoring linkage.

[0004] Existing distributed message bus technologies, although capable of achieving efficient data transmission, still have deficiencies in the integration with video surveillance devices. In addition, the application of anomaly detection technologies in the field of video surveillance is not yet mature, and most systems rely on simple threshold judgments and cannot effectively handle complex anomaly situations. Summary of the Invention

[0005] The present invention proposes a method for video surveillance linkage of heterogeneous software systems based on network communication, which is used to solve the problems that existing systems also have deficiencies in anomaly detection and real-time response, cannot detect and handle device anomalies in a timely manner, with the development of the Industrial Internet of Things (IIoT), the number of devices and the amount of data are increasing continuously, and the system cannot achieve monitoring linkage; the application of anomaly detection technologies in the field of video surveillance is not yet mature, and most systems rely on simple threshold judgments and cannot effectively handle complex anomaly situations.

[0006] In a first aspect, the present invention proposes a method for video surveillance linkage of heterogeneous software systems based on network communication, including:

[0007] Pre-build a dynamically loaded DLL plugin architecture and a three-dimensional mapped power supply monitoring network;

[0008] Configure an industrial communication protocol stack in the DLL plugin architecture; wherein, the industrial communication protocol stack is provided with a protocol recognition mechanism based on traffic characteristics;

[0009] Configure a distributed message bus and an XGBoost anomaly detection model based on ZeroMQ in the power supply monitoring network; among them, the distributed message bus is provided with north-south data channels and is respectively connected to video monitoring devices;

[0010] Perform anomaly detection on the device sensing data of the video monitoring devices through the XGBoost anomaly detection model, and convert the anomaly data into a streaming media interface.

[0011] In the embodiment of the present application, the present application mainly proposes a technical solution in which monitoring data collected by different video monitoring devices under different devices and different software can be displayed on the same streaming media interface. In order to solve the compatibility problem of different devices and different software, the present application adopts a DLL plug-in architecture that can be dynamically loaded. In order to achieve a higher monitoring data perception ability, a three-dimensional mapping power supply monitoring network is configured. In order to achieve key data conversion, a protocol recognition mechanism based on traffic characteristics is configured. In order to make the recognition of anomaly data faster, a lightweight message middleware ZeroMQ is used to build a north-south data channel to achieve high-throughput and low-latency data transmission. Finally, in order to achieve real-time conversion of anomaly data and eliminate post-event analysis, an embedded monitoring grid of the XGBoost anomaly detection model is configured to achieve anomaly detection optimized based on feature engineering and visualize the display.

[0012] Combined with the first aspect, the DLL plug-in architecture includes a service responder, which is used to generate a plug-in service request when anomaly data appears in the video monitoring device, and determine the target communication protocol called by the plug-in service request in the industrial communication protocol stack among the DLL plug-in architectures to be loaded.

[0013] The service responder of the present application triggers a plug-in service request based on anomaly data, and only loads specific protocol plug-ins when needed, reducing the memory occupancy of the protocol plug-ins. It does not need to integrate all protocols by means of full protocol stack residency. The service responder matches the target protocol in the industrial communication protocol stack according to the priority. When the service responder fails to call the target protocol, it automatically switches to the standby protocol, and can combine the redundant paths of the three-dimensional mapping network to accelerate the power failure recovery speed.

[0014] Combined with the first aspect, the power supply monitoring network includes a physical topology layer, a virtual data layer, and an environment modeling layer; among them, the physical topology layer is used to determine the real-time data transmission nodes of the video monitoring devices, the virtual data layer is used to configure the target transmission channels of the device sensing data according to the real-time transmission nodes, and the environment modeling layer is used to generate a three-dimensional heat map that is linked and mapped with the video monitoring devices through a preset digital twin engine.

[0015] The physical topology layer of this application can dynamically allocate the optimal transmission path, the virtual data layer can ensure the key data permissions and configure dedicated transmission channels, and the environment modeling layer can generate a three-dimensional heat map and map the coordinates of faulty devices.

[0016] Combined with the first aspect, the multi-layer collaborative architecture of the industrial communication protocol stack executes end-to-end trusted communication link establishment; wherein, the multi-layer collaborative architecture includes a dynamic evolution protocol feature layer for converting the traffic characteristics of device sensing data into multi-dimensional quantum state vectors, and a protocol mimicry interface layer for simulating the industrial communication protocol and capable of converting and adapting the multi-dimensional quantum state vectors for streaming media interface display.

[0017] The dynamic evolution protocol feature layer of this application converts the traffic characteristics into multi-dimensional quantum state vectors, uses the quantum no-cloning theorem to prevent attacks. The protocol mimicry interface layer converts the quantum state vectors into mimicry communication flows by simulating the interaction logic of the industrial protocol, making it impossible for network sniffing tools to identify the real protocol type and improving the protocol fingerprint hiding rate. The protocol mimicry interface layer adapts the quantum state vectors into a format that can be parsed by the streaming media interface and losslessly restores the multi-dimensional industrial data.

[0018] Combined with the first aspect, the protocol recognition mechanism includes the following recognition process:

[0019] Real-time capture the device sensing data of the video surveillance device and divide it into traffic segment samples through a sliding time window;

[0020] Load the traffic segment samples into phase space reconstruction, determine the time series sensitivity quantization value, and based on the time series sensitivity quantization value, determine the multi-dimensional quantum state vectors representing the traffic characteristics;

[0021] Based on the quantum entanglement intensity corresponding to the protocol cluster in the industrial communication protocol stack, determine the base representation information of the multi-dimensional quantum state vectors in the dynamic evolution protocol feature layer;

[0022] According to the base representation information, determine the standard industrial protocol data stream whose evolved quantum state vector is consistent with the time series jitter characteristics of the video surveillance device after inverse quantum Fourier transform decoding, and determine the target industrial communication protocol.

[0023] This application extracts the time series sensitivity quantization value of the traffic segment through phase space reconstruction, combines the quantum state vectors to represent the protocol characteristics, and improves the recognition accuracy of complex industrial protocols. The inverse quantum Fourier transform inversely maps the evolved quantum state vectors to the time domain, eliminates the time series jitter in network transmission, and restores the standard protocol stream consistent with the original time series of the device, improving the protocol parsing integrity. Dynamically select the protocol cluster base based on the quantum entanglement intensity, reducing the protocol recognition time consumption.

[0024] Combined with the first aspect, the distributed message bus built based on ZeroMQ includes:

[0025] Pre-set ZeroMQ and bind the device nodes corresponding to different video surveillance devices; wherein, the device nodes are used to determine the data structure and data content;

[0026] Dock the device nodes with the north-south data channel and configure the communication mode based on the data type of the device nodes; wherein the communication modes include: data request mode, data distribution mode and data queue mode;

[0027] Generate a distributed message bus according to the communication mode.

[0028] Based on the dynamic data request mode of the device nodes, this application can achieve data distribution and data queue, increase data throughput, reduce the control instruction response delay, and perform north-south isolation control, which can achieve two-way traffic physical isolation and avoid data loops.

[0029] Combined with the first aspect, the XGBoost anomaly detection model includes a lightweight XGBoost classifier and a deep XGBoost classifier; wherein, the lightweight XGBoost classifier is used to determine the first screened data with abnormal features in the device sensing data, and the deep XGBoost classifier performs time series classification on the first screened data and determines the second screened data based on time series grouping with time series anomalies.

[0030] This application can identify periodic anomalies that cannot be captured by traditional static features, dynamically adjust the computing power allocation of the two-level classifiers according to the device load, and can turn off the deep classifier and only retain the lightweight detection in resource-constrained scenarios to improve system stability.

[0031] Combined with the first aspect, the anomaly detection further includes:

[0032] A lightweight identity proxy is embedded in the video surveillance device. The lightweight identity proxy is used to generate a two-way association identifier by combining the hash value of the abnormal feature in the first screened data with the device ID number of the video surveillance device; wherein, when there is a time series anomaly in the first screened data, a joint signature of the hardware fingerprint of the video surveillance device and the watermark data of the device sensing data is generated based on the two-way association identifier.

[0033] The two-way association identifier of this application binds the hash value of the abnormal feature to the device ID, ensuring that the data source is verifiable and preventing attackers from forging abnormal data. The joint signature combines the device hardware fingerprint and data watermark, which can prevent the data from being intercepted. The identity proxy generates the association identifier in real time on the device side and only needs to upload the lightweight signature instead of the original data.

[0034] Combined with the first aspect, the conversion of the abnormal data into a streaming media interface includes:

[0035] Receive abnormal data from video surveillance devices, where the abnormal data is generated by an abnormal detection model based on XGBoost;

[0036] Parse the abnormal data in real time and extract feature information related to video surveillance;

[0037] Map the parsed feature information to the streaming media interface to generate a dynamic visual display of abnormal events;

[0038] Push the streaming media interface to the user terminal in real time through the streaming media transmission protocol;

[0039] Add multi-dimensional annotation information to the streaming media interface, including timestamps, location information, abnormal types, etc., for users to quickly locate and analyze.

[0040] The dynamic visual display of abnormal events in this application maps the abstract abnormal data detected by XGBoost into a dynamic heat map in the streaming media interface, which can instantly locate abnormal devices. The three-dimensional annotation of timestamp - location information - abnormal type can be used to trace back the evolution process of abnormalities based on the time axis.

[0041] Combined with the first aspect, the conversion of abnormal data into a streaming media interface includes:

[0042] Classify and cluster analyze the abnormal data to extract key features;

[0043] Generate an abnormal data chart according to the key features; among them, the abnormal data chart includes: dynamic chart, heat map or device geographical distribution map;

[0044] Map the abnormal data chart to the visualization interface and add interactive controls to the visualization interface; among them, the interactive controls are used to filter, sort and drill down the abnormal data under user instructions;

[0045] Through a dynamic update mechanism, the visualization interface is updated in real time to form a streaming media interface.

[0046] This application dynamically selects the chart form according to the feature type and associates the three-dimensional mapping network coordinates through the device geographical distribution map, improving the abnormal positioning accuracy from the meter level to the centimeter level. The interactive space can also reduce the fault diagnosis time.

[0047] Other features and advantages of the present invention will be described in the subsequent specification, and part of them will become obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained through the structures specifically pointed out in the written specification and the drawings.

[0048] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0050] Figure 1 It is a flowchart of a method for realizing video surveillance linkage of heterogeneous software systems based on network communication in an embodiment of the present invention;

[0051] Figure 2 It is a response process diagram of a service responder in an embodiment of the present invention;

[0052] Figure 3 It is a composition architecture diagram of a power supply monitoring network in an embodiment of the present invention;

[0053] Figure 4 It is an execution process diagram of a multi-layer collaborative architecture of an industrial communication protocol stack in an embodiment of the present invention;

[0054] Figure 5 It is an identification flowchart of a protocol identification mechanism in an embodiment of the present invention;

[0055] Figure 6 It is a bus structure diagram of a distributed message bus constructed by ZeroMQ in an embodiment of the present invention;

[0056] Figure 7 It is a function execution diagram of an XGBoost anomaly detection model in an embodiment of the present invention;

[0057] Figure 8 It is a generation diagram of two-way association identifiers in anomaly detection in an embodiment of the present invention;

[0058] Figure 9 It is an execution process diagram of converting anomaly data into a streaming media interface in an embodiment of the present invention;

[0059] Figure 10 It is an execution process diagram of converting anomaly data into a streaming media interface in an embodiment of the present invention. Specific Embodiments

[0060] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0061] In order to solve the problem in the traditional technical solution that data incompatibility occurs when different software and different hardware perform anomaly video surveillance and the same set of systems cannot be displayed, the present application proposes a method for realizing video surveillance linkage of heterogeneous software systems based on network communication. Refer to Figure 1 :

[0062] First, this application constructs a dynamically loaded DLL plugin architecture and a three-dimensional mapped power supply monitoring network. When data linkage occurs between heterogeneous software and video monitoring devices, the target communication protocol is determined based on the traffic characteristics of the data that needs to be communicated between the heterogeneous software and the video monitoring devices. The target communication protocol is loaded under the decoupled protocol through the dynamically loaded DLL plugin architecture to achieve data interaction communication. In the case of loading and using, it can quickly realize data interaction communication between future software and different video monitoring devices, between video monitoring devices and control devices, or with the cloud, enabling different video monitoring devices and heterogeneous software to be linked and supervising the entire area. The video monitoring devices monitor each physical power supply node, such as areas like distribution cabinets and substations. The heterogeneous software can display and control the monitoring data of each physical power supply node; the physical power supply nodes are mapped into a virtual three-dimensional network, and the management and control of physical power supply nodes and resource scheduling can be achieved in the spatial dimension, linking the physical locations and data processing of each physical power supply node to prevent response lag during communication.

[0063] The DLL plugin architecture of this application is configured with an associated industrial communication protocol stack; among them, the industrial communication protocol stack is provided with a protocol recognition mechanism based on traffic characteristics; during the process of network communication, the protocol recognition mechanism extracts traffic characteristics such as packet length and interaction period based on the data of the heterogeneous software and the data of the video monitoring devices, and determines the corresponding industrial communication protocols of the heterogeneous software and different video monitoring devices during network communication according to the device protocol types corresponding to the traffic characteristics, for example: TCP protocol, ONVIF protocol.

[0064] The protocol recognition mechanism and the dynamically loaded DLL plugin architecture of this application can automatically analyze the communication trust characteristics of devices and heterogeneous software during the process of network communication, and dynamically load the corresponding industrial communication protocols that can be loaded into the DLLD plugin architecture. For example, based on traffic characteristics, it is determined that the communication protocol of the current video monitoring device is the BACnet protocol, load the associated BACnet protocol, identify the bacnet.Dll file, and achieve automatic discovery and automatic loading of the protocol. During the process of new device access, the new device access time can be shortened.

[0065] The power supply monitoring network of this application is configured with a distributed message bus built based on ZeroMQ and an XGBoost anomaly detection model; among them, the distributed message bus is provided with north-south data channels and is respectively connected to video monitoring devices; the ZeroMQ distributed message bus mainly conducts two-way isolated communication of the network communication data of the heterogeneous software corresponding to the distributed nodes and the video monitoring devices through the north-south data channels to prevent data congestion. The southbound channel is used to process the data aggregation of video monitoring devices or heterogeneous software and transmit it to the control decision-making center, such as the cloud. The northbound channel is used to communicate with the control decision-making center and video monitoring devices or heterogeneous software to issue control instructions. The XGBoost anomaly detection model performs time series division on the network communication data of different video monitoring devices and heterogeneous software, that is, device sensing data, generates time series data, determines time series features, and sorts according to the importance of time series features to achieve anomaly detection. Anomalies include but are not limited to current mutations, short circuits, open circuits, etc., reducing the false alarm rate.

[0066] In this application, by switching different modes of the distributed message bus built based on ZeroMQ, such as the PUB-SUB mode, the original data of video monitoring devices is broadcast to multiple XGBoost detection nodes for parallel processing, and then through parallel computing, fast analysis and parallel analysis of device sensing data are realized, improving the discovery and sensing speed of abnormal data.

[0067] The XGBoost anomaly detection model performs anomaly detection on the device sensing data of video monitoring devices and converts the abnormal data into a streaming media interface.

[0068] After detecting abnormal data, the three-dimensional mapped power supply network can determine the location data of the faulty devices of the monitored video monitoring devices. Then, during the process of generating the streaming media interface, the monitoring data of the corresponding video monitoring devices is superimposed with the abnormal data for marking abnormal events, so that operation and maintenance management personnel can directly view the panoramic view of the abnormal area through the terminal display device, thereby improving the linkage response speed of video monitoring devices.

[0069] Through the dynamic link library (DLL) plugin architecture, the present invention can achieve flexible loading and unloading of different functional modules. This architecture allows the system to dynamically load plugins that support different industrial communication protocols as needed, thereby achieving compatibility with heterogeneous software systems. The configured industrial communication protocol stack includes a protocol recognition mechanism based on traffic characteristics, which can automatically identify and parse the communication protocols of different devices to ensure accurate data transmission. The power supply monitoring network combines the spatial positions of physical devices with monitoring data through three-dimensional mapping technology to achieve visual management of devices. The network is configured with a distributed message bus built on ZeroMQ, which supports efficient data transmission and communication between devices. ZeroMQ is a high-performance asynchronous messaging library that can achieve communication between multi-threads, multi-processes, and multi-nodes. XGBoost is an efficient machine learning algorithm suitable for anomaly detection tasks. It can analyze the sensed data of video surveillance devices through the gradient boosting tree algorithm to identify abnormal data. The model converts the abnormal data into a streaming media interface for real-time monitoring and alarm.

[0070] This application combines the dynamic plugin architecture with the protocol recognition mechanism to solve the compatibility problem of heterogeneous device access and overcome the defects of traditional monitoring systems that use a single protocol stack and a single query mechanism. Moreover, in terms of anomaly detection, the XGBoost model, an accurate model based on image recognition, is adopted. It is associated with the three-dimensional spatial data of power supply equipment anomaly detection, and can comprehensively analyze anomalies from the aspects of dynamic operation monitoring of software and full-scenario display of hardware monitoring, reducing the situation of false anomaly alarms.

[0071] Embodiment 2:

[0072] The DLL plugin architecture of this application includes dynamically responding to the target communication protocol. The method of loading the target communication protocol is shown in Figure 2 :

[0073] The DLL plugin architecture of this application includes a service responder. The service responder is used to generate a plugin service request when abnormal data appears in the video surveillance device, and determine the target communication protocol called by the plugin service request in the DLL plugin architecture to be loaded in the industrial communication protocol stack. The service responder of this application is a dynamic protocol adaptation mechanism driven by abnormal data and also a dynamic adaptation protocol driven by the sensed data of devices with traffic characteristics. The service responder is triggered when abnormal data exists, and then generates a plugin service request. When needed, it loads specific protocol plugins. For example, when there is video stream data, based on the traffic characteristics of the video stream data, the corresponding ONVIF protocol is determined, and at the same time, the traffic characteristics of the sensor data of the monitoring device are adapted to determine the corresponding Modbus protocol. It can achieve dynamic loading of industrial communication protocols while avoiding the traditional protocol stack.

[0074] The service of the present application corresponds to its position in the industrial communication protocol stack and implements the matching of the target industrial protocol according to the request priority based on the plug-in service request. For example, when the abnormal type is current overload, based on the corresponding traffic characteristics, the power-specific protocol is preferentially called, so that during network communication, the device communication can be accurately achieved, and the power failure can be quickly processed. When the call to the target industrial communication protocol fails, the standby protocol can be automatically switched, and combined with the three-dimensional mapped power supply monitoring network, it is judged whether there is a redundant power path, and the power failure is solved through the redundant power path.

[0075] The present application uses abnormal data as the trigger of the event trigger mechanism, dynamically loads the industrial communication protocol according to requirements, realizes on-demand adjustment, rather than loading all protocols all the time, and avoids the chaotic interference effect between different communication protocols.

[0076] The present application can determine the abnormal type according to the abnormal data, and matches the abnormal type with the industrial communication protocol, which is applicable to the complex scenarios of multiple devices and multiple protocols, and is more in line with the device operating environment of the Internet of Things.

[0077] The service responder of the present application can combine the REQ-REP mode of ZeroMQ, send a plug-in call request in the industrial communication protocol stack, and receive the protocol ready state feedback, realizing the atomic operation of protocol loading and message communication, and avoiding the signaling conflict of the traditional polling mechanism.

[0078] When the present application is specifically implemented: The DLL plug-in architecture also includes

[0079] A dynamic loader for loading and unloading the target communication protocol;

[0080] A dependency resolver for checking the loaded target communication protocol to determine the dependency relationship of the communication protocol;

[0081] A protocol state machine for monitoring the full process state of the target communication protocol.

[0082] The dynamic loader is responsible for loading and unloading the target communication protocol. Based on the dynamic link library (DLL) mechanism of the operating system, the dynamic loader can dynamically load and unload plug-ins. When the service responder generates a plug-in service request, the dynamic loader will load the corresponding target communication protocol plug-in according to the request. When it is not needed, the dynamic loader can unload the plug-in, release system resources, and improve the flexibility and resource utilization rate of the system.

[0083] The dependency resolver is used to check the loaded target communication protocol and determine the dependency relationships of the communication protocol. The dependency resolver analyzes the metadata of the target communication protocol plugin to determine other modules or libraries it depends on. By parsing the dependency relationships, it ensures that all necessary dependencies are correctly loaded when the target communication protocol is loaded. If a missing dependency or version mismatch is found, the dependency resolver will prevent the plugin from loading and report an error.

[0084] The protocol state machine is used to monitor the full - process state of the target communication protocol. The protocol state machine is a finite - state machine (FSM) used to manage the various states of the target communication protocol (such as initialization, running, exception, termination, etc.). Through the state machine, the running state of the target communication protocol can be monitored in real - time to ensure its normal operation. If an abnormal state is detected, the protocol state will trigger corresponding processing mechanisms, such as re - loading the plugin, sending an alert, etc.

[0085] Embodiment 3:

[0086] The power supply monitoring network of this application has a three - layer structure. Refer to Figure 3 :

[0087] The physical topology layer of this application consists of heterogeneous sensor nodes and video monitoring devices deployed in the power supply area. The sensor nodes achieve multi - protocol adaptation through a dynamically loaded DLL plugin architecture, and the video monitoring devices are configured with a streaming media forwarding module based on H.265 encoding. The physical topology layer is mainly used to track the status of device nodes in real - time, and then dynamically allocate the optimal transmission path, such as bypassing a faulty switch node, thereby reducing the packet loss rate of the video stream. It dynamically discovers the connection status of devices such as switches and routers in the network device based on the DLL plugin architecture, constructs a real - time topology map, and provides data parameters for underlying path planning to the virtual data layer, solving the problems of inability to sense network connection changes, disconnection between transmission channel configuration and the physical network.

[0088] The virtual data layer constructs a distributed message bus based on ZeroMQ, including north - south data channels and east - west control channels. Among them, the north - south data channel is equipped with a protocol conversion gateway based on FPGA to achieve real - time conversion from the Modbus / OPC UA protocol to the RTSP streaming media protocol. The virtual data layer of this application can, based on the device type, for example: the video monitoring device is a 4K camera, the video monitoring device is a monitoring device configured with a temperature sensor, and configure dedicated transmission channels according to the data types corresponding to the video stream / sensor data. For example: the video stream uses the UDP high - speed channel, and the sensor data uses the TCP reliable channel. In the case of key data priority, the video frame rate stability rate will automatically increase in the network congestion scenario. It mainly solves the problems that traditional single channels cannot transmit heterogeneous data, and video frame freezing and data loss occur during data transmission.

[0089] The environmental modeling layer integrates a 3D geographic information engine and a digital twin engine. The digital twin engine dynamically generates a 3D thermal map of power supply equipment by receiving the anomaly detection results of the virtual data layer in real time, and establishes a linkage mapping relationship with the spatial coordinates of video monitoring equipment. The environmental modeling layer of this application generates a 3D thermal map through the digital twin engine, maps the abnormal data detected by XGBoost to the physical device coordinates, and the operation and maintenance personnel determine the fault area based on the physical device coordinates and quickly process the defects in the corresponding area. The 3D thermal map generated by this application is a power equipment status display map based on the digital twin engine, and can generate a thermal map displayed based on color gradients according to the fault type and degree of the equipment.

[0090] In the actual implementation of this application, the anomaly probability value output by the XGBoost anomaly detection model is used as the coloring weight of the thermal map. The high-probability area is displayed as dark red. The operation and maintenance personnel call the streaming media interface corresponding to power equipment with different color gradients according to the color gradient, and accurately confirm the anomaly through the video stream of the associated camera.

[0091] In the actual implementation of this application, the physical topology bus can update the node information, update the newly added nodes or the deleted nodes, so as to realize the reconfiguration and reconfiguration of the channel and reduce the end-to-end delay;

[0092] In the actual implementation of this application, independent channels are allocated to different target industrial communication channels through the virtual data layer to avoid interference between different protocols and increase the integrity of communication.

[0093] Embodiment 4:

[0094] The industrial communication protocol stack of this application realizes the establishment of an end-to-end trusted communication link through a multi-layer collaborative architecture; refer to Figure 4 , the multi-layer collaborative architecture includes a dynamic evolution protocol feature layer for converting the traffic characteristics of device sensing data into multi-dimensional quantum state vectors, and a protocol mimic interface layer that simulates industrial communication protocols and can convert and adapt the multi-dimensional quantum state vectors for display on the streaming media interface.

[0095] This application integrates the dynamically evolving protocol feature layer and quantum state conversion, so as to cut the original data stream into two pieces of quantum states based on the generation of quantum random numbers. After the calculation of the dynamic evolution operator, the unpredictability of communication characteristics is realized. Quantum state evolution can hide protocol characteristics such as port numbers and handshake protocols at the same time, preventing data leakage.

[0096] The protocol mimicry interface layer of this application can implement quantum state vectors through a protocol behavior simulator and encapsulate them into mimicry protocol packets. By extracting key data from the data of the mimicry protocol packets, when the multimedia display device shows an abnormal state, overlay data for the streaming media interface can be generated, and corresponding curves can also be rendered on the multimedia interface.

[0097] During the dynamic evolution process of this application, based on molecular circuits and combined with the corresponding network threat information of abnormal data, the evolution parameters can be automatically adjusted to enhance the adaptive ability of the protocol. Using quantum state vectors for the dynamic evolution of protocol features can solve the problem that each protocol in the protocol stack has a protocol fingerprint and may expose characteristics.

[0098] In actual implementation, the multi-layer collaborative architecture of this application also includes:

[0099] A protocol adaptation layer, configured with a dynamic protocol loading engine, which dynamically calls the adapted industrial communication protocol according to the data type of the abnormal data;

[0100] An identification embedding layer, coupled with the protocol adaptation layer, used to encode the device identification of the video surveillance device and the service identification of the target service into structured metadata and embed them into the header extension field of the industrial communication protocol;

[0101] A security authentication layer, integrating a lightweight cryptographic module based on PKI, performing asymmetric encryption signature processing on the device identification to generate an enhanced device identification carrying signature information;

[0102] A slice routing layer, including a network slice selector, parsing the feature vector encoded by the service identification, and establishing a policy mapping with the network slice resource pool through a pre-set SDN controller.

[0103] The protocol adaptation layer configures a dynamic protocol loading engine: According to the data type of the abnormal data, it dynamically calls the adapted industrial communication protocol. This mechanism can flexibly handle different types of abnormal data, ensuring that the system can quickly adapt to and process data from different devices. The identification embedding layer embeds through structured metadata: Encodes the device identifier of the video surveillance device and the service identifier of the target service into structured metadata, and embeds them into the header extension field of the industrial communication protocol. This design can ensure that the data carries sufficient context information during transmission, facilitating subsequent processing and analysis. Asymmetric encryption signature: Integrates a lightweight cryptographic module based on PKI (Public Key Infrastructure) to perform asymmetric encryption signature processing on the device identifier, generating an enhanced device identifier carrying signature information. Asymmetric encryption ensures the security and integrity of the data through public key encryption and private key decryption. Network slice selector: Parses the feature vector encoded by the service identifier, and establishes a policy mapping with the network slice resource pool through a pre-set SDN controller. Network slice technology can allocate independent network resources for different services, ensuring service isolation and quality of service.

[0104] The dynamic protocol loading engine of the present invention can dynamically select the adapted communication protocol according to different abnormal data types, improving the flexibility and adaptability of the system. The asymmetric encryption signature processing ensures the security and integrity of the device identifier, preventing the data from being tampered with or forged during transmission. The network slice technology realizes flexible resource allocation and service isolation through the SDN controller, improving the utilization rate of network resources and the reliability of the system. The embedding of structured metadata and the introduction of a security authentication mechanism enhance the overall reliability of the system, ensuring the accuracy and security of the data during transmission and processing.

[0105] Embodiment 5:

[0106] The protocol recognition mechanism of this application includes the following recognition process:

[0107] This application captures the device sensing data of the video surveillance device in real time and divides it into traffic segment samples through a sliding time window; for the device sensing data, it will be temporally divided through a sliding time window to generate multiple traffic segment samples.

[0108] This application loads traffic segment samples into phase space reconstruction, determines the quantization value of temporal sensitivity, and based on the quantization value of temporal sensitivity, determines a multi-dimensional quantum state vector representing traffic characteristics. To distinguish between steady-state protocols and burst protocols, this application maps traffic segment samples to a high-dimensional phase space, and then calculates the temporal sensitivity, such as the maximum Lyapunov exponent; thereby, through the temporal sensitivity, quantitative determination of temporal characteristics is performed to determine the degree of chaos of temporal characteristics during quantization. If the degree of chaos is close to 0, it represents a time-love protocol. If the degree of chaos is greater than or equal to 1 or greater than 0.5, a burst protocol is determined. Compared with the feature extraction of traditional sliding windows, this application can capture the characteristics of temporal dynamics and reduce the misjudgment of similar protocols. The multi-dimensional quantum state vector is the vector value after quantitative determination. The quantization of temporal sensitivity in this application can be combined with the XGBoost anomaly modeling model, and the output anomaly probability is used as the weight parameter for phase space reconstruction, increasing the high-anomaly probability analysis ability in the anomaly determination dimension and enhancing the probability of identifying key protocols.

[0109] This application determines the base representation information of the multi-dimensional quantum state vector in the dynamic evolution protocol feature layer based on the quantum entanglement strength corresponding to the protocol cluster in the industrial communication protocol stack; a preset entanglement strength threshold is set for the protocol cluster. For example, if the entanglement value required for the power protocol cluster is not lower than a certain preset value, the corresponding quantum base will be matched. The quantum base is based on a highly practical industrial communication protocol. Only when the entanglement value between the selected industrial communication protocol and the power protocol cluster is not lower than the preset value can the matching be successful, thereby avoiding resource losses caused by traversing and processing invalid industrial communication protocols. The base identification information represents the function display, protocol type, practicality, and other descriptive information of the highly practical industrial communication protocol. The quantum base parsing plugin dynamically loaded by the quantum base is based on the target industrial communication protocol, which gives the industrial communication protocol stack of this application the ability to mix quantum and classical parsing, is applicable to the quantum encryption of power equipment data, and is applicable to highly encrypted power projects.

[0110] Based on the base marking information, this application determines a standard industrial protocol data stream in which the evolved quantum state vector is consistent with the timing jitter characteristics of the video surveillance device after the inverse quantum Fourier transform decoding, and determines the target industrial communication protocol. During the inverse quantum Fourier transform decoding, the frequency-domain quantum state is inversely transformed to the time domain. For example, the protocol features of a specific frequency are converted into one or more combinations of protocols in the noise-resistant standard protocol stream. The dynamic evolution of the multi-dimensional quantum state vector is to set a controlled quantum phase rotation gate in the protocol feature layer to enhance the function of the key industrial communication protocol and improve the quantum state distinguishability. In this application, the ZeroMQ message bus can aggregate quantum state vectors, construct a ZAP authentication channel, and only transmit quantum state vectors. Combining with the quantum key distribution mechanism, it realizes end-to-end quantum encryption function to prevent attacks during the protocol recognition process. Moreover, under the quantized protocol recognition process, the industrial heterogeneous network can achieve security assurance while also being able to achieve the fusion of classical protocols and post-quantum secure protocols in terms of protocol compatibility extension for high-interference scenarios, with huge compatibility extension.

[0111] Example 6:

[0112] This application targets the physical power nodes of different monitoring devices and, through the distributed message bus constructed by ZeroMQ, refers to Figure 6 :

[0113] This application pre-sets ZeroMQ and binds the device nodes corresponding to different video surveillance devices; among them, the device nodes define the data structure and data content, and each device node has a predefined data format, which can prevent data from being mis-transmitted due to cross-device parsing errors during network communication; for example: video streams will use Protobuf binary encoding, and sensor data will use the JSON protocol;

[0114] Dock the device nodes with the north-south data channels and configure the communication mode based on the data types of the device nodes; the communication modes include: data request mode, data distribution mode, and data queue mode;

[0115] Generate a distributed message bus according to the communication mode.

[0116] When the device nodes are docked with the north-south data channels, the southbound channel will force the use of one-to-many broadcasting in the PUB-SUB mode, and the northbound channel will control the use of the REQ-QEP mode to ensure that the instructions can be achieved, avoiding the problem that the traditional full-duplex TCP mode or other single modes cannot adapt to the huge differences in data and control traffic in industrial scenarios.

[0117] In actual implementation, the initialization of ZeroMQ is bound to device nodes by pre-setting ZeroMQ and binding the device nodes corresponding to different video surveillance devices. Each device node defines a data structure and data content to ensure that the format and content of messages can be correctly parsed and processed. Through its flexible socket abstraction, ZeroMQ can support multiple communication modes, including request-response, publish-subscribe, push-pull, etc.

[0118] Configuration of communication mode: The device node is docked with the north-south data channel, and the communication mode is configured based on the data type of the device node.

[0119] Specifically, the communication modes include:

[0120] Data request mode: Suitable for scenarios where the client requests data from the server, using the request-response mode.

[0121] Data distribution mode: Suitable for scenarios where data is broadcast or distributed to multiple receivers, using the publish-subscribe mode.

[0122] Data queue mode: Suitable for scenarios of task distribution and load balancing, using the push-pull mode.

[0123] The communication modes of ZeroMQ are implemented through its socket types (such as REQ / REP, PUB / SUB, PUSH / PULL). These modes can meet the communication requirements in different scenarios. According to the configured communication mode, a distributed message bus is generated. Through its high-performance message passing mechanism, ZeroMQ can achieve low-latency and high-throughput message transmission. ZeroMQ supports multiple transport protocols (such as TCP, IPC, and inproc), and provides features such as automatic reconnection, queue management, and load balancing to ensure reliable message transmission.

[0124] Through ZeroMQ's lock-free queue model, batch processing algorithm, and multi-core thread binding technology, high-performance and low-latency message passing are achieved. This high-performance feature enables the distributed message bus to efficiently process large-scale data streams, especially suitable for real-time data processing and video surveillance scenarios. It supports multiple communication modes and can meet the communication requirements in different scenarios. For example, the publish-subscribe mode is suitable for real-time data broadcasting, and the push-pull mode is suitable for task distribution.

[0125] Reliable fault-tolerant mechanism provides an efficient, reliable, and easily extensible message passing solution for the video surveillance system.

[0126] In this application, by configuring a high-priority queue for key devices, such as cameras for fire monitoring, wired transmission can be achieved in case of sudden traffic. When a device node is abnormal, the REQ-REP mode can be automatically downgraded to the PUSH-PULL mode, enabling data transmission still.

[0127] In this application, by combining the DLL plug-in architecture with the communication mode, when loading a new industrial communication protocol, a dedicated communication mode will be automatically allocated. After the plug-in is loaded, there will be no delay in transmission.

[0128] By combining XGBoost anomaly detection with the data queue mode, when detecting abnormal data, by allocating an independent queue for abnormal devices, the traceability of abnormal data can be realized.

[0129] By combining the three-dimensional mapped power supply network with the topological structure of device nodes, the three-dimensional coordinates and topological representation of device nodes are realized at the environmental modeling layer, physical regions and communication regions are divided, and the traffic resource occupation across regions is reduced.

[0130] Example 7:

[0131] The XGBoost anomaly detection model of this application includes a lightweight XGBoost classifier and a deep XGBoost classifier; refer to Figure 7 :

[0132] The lightweight XGBoost classifier is used to determine the first screening data with abnormal features in the device sensing data. The deep XGBoost classifier performs time-series classification on the first screening data and determines the second screening data based on time-series grouping with time-series anomalies.

[0133] In this application, through the initial screening of abnormal features by the lightweight XGBoost classifier, that is, through the shallow decision tree method to perform initial screening with low-dimensional features, high-throughput processing is achieved to prevent industrial delays. Then, based on the time-series grouping analysis of the deep XGBoost classifier, high-order features are extracted through the time-series grouping method, such as the Hurst exponent and LSTM hidden state, and the gradual anomalies during the device operation can be determined to prevent misjudging transient noise, such as electromagnetic interference, as continuous abnormal conditions.

[0134] The two-stage classifier of this application can monitor the operating state of the device with different weight allocations to prevent device aging and reduce the monitoring accuracy of the device.

[0135] By combining the DLL plugin architecture with the dynamic loading of a two - level classifier, this application can, when detecting an abnormal pattern that has never occurred, dynamically load a dedicated deep classifier plugin to perform an online upgrade of the detection model without maintenance. The ZeroMQ message bus combined with hierarchical data transmission can report different modes of dual data, occupying bandwidth. By combining three - dimensional heat maps and temporal anomalies, joint early warnings can be issued in terms of time and space.

[0136] Example 8:

[0137] When applying for abnormal detection, a joint signature of a lightweight identity proxy and a two - way associated identity will also be introduced to ensure the reliability of the data source. Refer to Figure 8 :

[0138] This application embeds a lightweight identity proxy at the video surveillance device end. The lightweight identity proxy is used to generate a two - way associated identity from the abnormal feature hash value in the first - screened data and the device ID number of the video surveillance device. Among them, when there is a temporal anomaly in the first - screened data, a joint signature of the hardware fingerprint of the video surveillance device and the watermark data of the device sensing data is generated based on the two - way associated identity.

[0139] By binding the device ID with the abnormal features through the hash function, a mechanism for two - way associated identity can be realized, forming an irreversible two - way association to judge whether there is an attack during the intermediate transmission process, resulting in the inability to distinguish different types of data and forged data.

[0140] Through the joint signature method of hardware fingerprint and watermark, the device has the property of physical unclonability, thus generating a composite signature resistant to quantum cracking to prevent being bypassed by the software layer signature.

[0141] Example 9:

[0142] This application converts abnormal data into a streaming media interface. Refer to Figure 9 :

[0143] This application will receive abnormal data from the video surveillance device. The abnormal data is generated by an abnormal detection model based on XGBoost. Through a lightweight parser, key features such as abnormal confidence, device temperature, and current phase angle can be extracted to prevent interface data overload;

[0144] This application performs real-time parsing on abnormal data, extracts feature information related to video monitoring, maps the parsed feature information into the streaming media interface, and generates a dynamic and visual display of abnormal events; during the dynamic mapping process, by establishing binding rules between feature fields and interface components, for example, binding the current value to the pointer angle of the dashboard and binding the temperature to the color scale of the heat map, it realizes "don't" rendering to prevent traditional static mapping from being unable to handle multiple types of abnormalities.

[0145] Push the streaming media interface to the user terminal in real time through the streaming media transmission protocol.

[0146] Add multi-dimensional annotation information to the streaming media interface, including timestamps, location information, abnormal types, etc., so that users can quickly locate and analyze.

[0147] This application dynamically switches between the HLS or WebRTC protocol according to the network RTT, realizes the format in which the streaming media interface on the user terminal can be pushed in real time, enables the interface to be forwarded by means of screen capture, and can also generate a two-way associated identifier through the fusion of the identification proxy and real-time annotation, and use it as the unique index of the annotation data. Judge whether the original video, that is, the device sensing data, is abnormal; the abnormal probability value output by XGBoost drives the interface alarm level (red flashing when the probability > 90%, yellow warning when > 70%), and is linked with the three-dimensional heat map for zooming and focusing, and can be combined with the three-dimensional heat map to realize dynamic playback.

[0148] Embodiment 10:

[0149] This application will convert abnormal data into a streaming media interface. In specific implementation, refer to Figure 10 :

[0150] Receive model abnormal data from the abnormal detection based on XGBoost;

[0151] Classify and cluster analyze the abnormal data, and extract key features;

[0152] Map the abnormal data into the visualization interface to generate dynamic charts, heat maps or device geographical distribution maps;

[0153] Add interactive controls to the visualization interface to support users in filtering, sorting and drilling down on abnormal data;

[0154] Through the dynamic update mechanism, the change trends and correlation relationships of abnormal data are displayed in real time.

[0155] The technical principle of the above technical solution is:

[0156] The present invention receives abnormal data generated from an XGBoost-based anomaly detection model. XGBoost, with its powerful feature screening and classification capabilities, can effectively handle imbalanced data and identify anomalies. Classify and cluster analyze the abnormal data to extract key features. Classification and clustering analysis can help identify patterns and structures in the abnormal data, thereby extracting the most valuable features for anomaly detection. Map the abnormal data to a visualization interface to generate dynamic charts, heatmaps, or device geographical distribution maps. This visualization method can intuitively display the distribution and changes of abnormal events. Add interactive controls to the visualization interface to support users in filtering, sorting, and drilling down on the abnormal data. The interactive controls enhance the operational flexibility of users, enabling them to analyze the abnormal data more deeply. Through a dynamic update mechanism, the changing trends and correlation relationships of the abnormal data are displayed in real time. The dynamic update mechanism ensures that users can see the latest changes in the abnormal data and thus respond in a timely manner.

[0157] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A video surveillance linkage method for heterogeneous software systems based on network communication, characterized in that: include: Pre-build a dynamically loaded DLL plug-in architecture and a three-dimensional mapped power supply monitoring network; An industrial communication protocol stack is configured in the DLL plug-in architecture; wherein the industrial communication protocol stack is provided with a protocol identification mechanism based on traffic characteristics; A distributed message bus based on ZeroMQ and an XGBoost anomaly detection model are configured in the power supply monitoring network. The distributed message bus is equipped with north-south data channels and is connected to video monitoring devices respectively. The XGBoost anomaly detection model is used to detect anomalies in the device sensing data of the video surveillance device, and the anomaly data is converted into a streaming media interface.

2. A video surveillance linkage method for implementing heterogeneous software systems based on network communication as claimed in claim 1, characterized in that: The DLL plug-in architecture includes a service responder, which is used to generate a plug-in service request when abnormal data appears in the video monitoring device, and determine the target communication protocol called by the plug-in service request in the DLL plug-in architecture to be loaded in the industrial communication protocol stack.

3. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 1, characterized in that: The power supply monitoring network includes a physical topology layer, a virtual data layer and an environmental modeling layer; wherein the physical topology layer is used to determine the real-time data transmission nodes of the video surveillance equipment, the virtual data layer is used to configure the target transmission channel of the device sensing data according to the real-time transmission nodes, and the environmental modeling layer is used to generate a three-dimensional thermal map that is linked and mapped with the video surveillance equipment through a preset digital twin engine.

4. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 1, characterized in that: The multi-layer collaborative architecture of the industrial communication protocol stack executes end-to-end trusted communication link establishment; wherein, the multi-layer collaborative architecture includes a dynamically evolving protocol feature layer for converting traffic characteristics of device sensing data into multi-dimensional quantum state vectors, and a protocol mimicry interface layer that simulates the industrial communication protocol and can convert multi-dimensional quantum state vectors to adapt to streaming media interface display.

5. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 1, characterized in that: The protocol identification mechanism includes the following identification process: Capture device sensing data of video surveillance devices in real time and segment it into traffic segment samples through sliding time windows; Load the traffic segment samples into the phase space for reconstruction, determine the timing sensitivity quantization value, and determine the multi-dimensional quantum state vector representing the traffic characteristics based on the timing sensitivity quantization value; Determine the base representation information of the multi-dimensional quantum state vector in the dynamic evolution protocol feature layer based on the quantum entanglement strength corresponding to the protocol cluster in the industrial communication protocol stack; According to the basis representation information, the standard industrial protocol data stream whose evolved quantum state vector is consistent with the timing jitter characteristics of the video surveillance equipment after inverse quantum Fourier transform decoding is determined, and the target industrial communication protocol is determined.

6. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 1, characterized in that: The distributed message bus built on ZeroMQ includes: Pre-set ZeroMQ and bind the device nodes corresponding to different video surveillance devices; the device nodes are used to determine the data structure and data content; Connect the device node to the north-south data channel and configure the communication mode based on the data type of the device node; the communication modes include: data request mode, data distribution mode and data queue mode; Generate a distributed message bus based on the communication mode.

7. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 1, characterized in that: The XGBoost anomaly detection model includes a lightweight XGBoost classifier and a deep XGBoost classifier; wherein the lightweight XGBoost classifier is used to determine the first screening data with abnormal features in the device sensing data, and the deep XGBoost classifier performs time series classification on the first screening data and determines the second screening data based on time series grouping with time series anomalies.

8. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 7, characterized in that: The anomaly detection further includes: A lightweight identification agent is embedded in the video surveillance device, and is used to generate a bidirectional association identifier by combining the abnormal feature hash value in the first screening data with the device ID number of the video surveillance device; wherein, when there is a timing anomaly in the first screening data, a joint signature of the hardware fingerprint of the video surveillance device and the watermark data of the device sensing data is generated based on the bidirectional association identifier.

9. A video surveillance linkage method for implementing heterogeneous software systems based on network communication as claimed in claim 8, characterized in that: The converting of abnormal data into a streaming media interface includes: Receive abnormal data from video surveillance equipment, which is generated by an anomaly detection model based on XGBoost; Analyze abnormal data in real time and extract feature information related to video surveillance; Map the parsed feature information to the streaming media interface to generate a dynamic and visual display of abnormal events; Push the streaming media interface to the user terminal in real time through the streaming media transmission protocol; Add multi-dimensional annotation information in the streaming interface, including timestamp, location information, anomaly type, etc., so that users can quickly locate and analyze.

10. The method for realizing video surveillance linkage of heterogeneous software systems based on network communication according to claim 9, characterized in that: The converting of abnormal data into a streaming media interface includes: Classify and cluster abnormal data to extract key features; Generate abnormal data charts based on key features; wherein the abnormal data charts include: dynamic charts, heat maps or device geographical distribution maps; Mapping the abnormal data chart to a visualization interface and adding interactive controls in the visualization interface; wherein the interactive controls are used to filter, sort and drill the abnormal data under user instructions; Through the dynamic update mechanism, the visual interface is updated in real time to form a streaming media interface.

Images