A video monitoring linkage method for realizing heterogeneous software systems based on network communication

Abstract

The application relates to the technical field of video monitoring linkage, and provides a video monitoring linkage method for realizing a heterogeneous software system based on network communication, which comprises a pre-built DLL plug-in architecture and a three-dimensional mapping power supply monitoring network with dynamic loading; the DLL plug-in architecture is configured with an associated industrial communication protocol stack; the industrial communication protocol stack is provided with a protocol identification mechanism based on flow characteristics; the power supply monitoring network is configured with a distributed message bus and an XGBoost abnormality detection model which are built based on ZeroMQ; the distributed message bus is provided with south-north data channels and is connected with video monitoring equipment respectively; the XGBoost abnormality detection model is used for performing abnormality detection on device sensing data of the video monitoring equipment, and abnormal data is converted into a streaming media interface.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video monitoring linkage, and particularly relates to a video monitoring linkage method for realizing heterogeneous software systems based on network communication. BACKGROUND

[0002] With the rapid development of industrial automation and informatization, video monitoring systems have been widely applied in industrial production, security monitoring and other fields. However, the existing video monitoring systems face many challenges in processing heterogeneous devices and protocols. Traditional video monitoring systems usually adopt a static loading method, which is difficult to adapt to different brands and models of devices, resulting in poor system compatibility and insufficient scalability. In addition, the existing systems also have deficiencies in anomaly detection and real-time response, and cannot timely discover and handle abnormal situations of devices.

[0003] In industrial environments, video monitoring devices and sensor nodes usually use multiple different communication protocols such as Modbus, OPC UA, RTSP, etc. The compatibility problem between these protocols limits the integration and interoperability of the system. At the same time, with the development of Industrial Internet of Things (IIoT), the number of devices and the amount of data are increasing, and it is impossible to realize monitoring linkage.

[0004] The existing distributed message bus technology, although it can realize efficient data transmission, still has deficiencies in the integration with video monitoring devices. In addition, the application of anomaly detection technology in the field of video monitoring is not mature enough, and most systems rely on simple threshold judgment, which cannot effectively handle complex abnormal situations. SUMMARY

[0005] The present application proposes a video monitoring linkage method for realizing heterogeneous software systems based on network communication, which is used to solve the problems that the existing systems have deficiencies in anomaly detection and real-time response, cannot timely discover and handle abnormal situations of devices, with the development of Industrial Internet of Things (IIoT), the number of devices and the amount of data are increasing, and it is impossible to realize monitoring linkage; the application of anomaly detection technology in the field of video monitoring is not mature enough, and most systems rely on simple threshold judgment, which cannot effectively handle complex abnormal situations.

[0006] In the first aspect, the present application proposes a video monitoring linkage method for realizing heterogeneous software systems based on network communication, comprising:

[0007] Pre-installing a dynamically loaded DLL plug-in architecture and a three-dimensional mapping power supply monitoring network;

[0008] Configuring an industrial communication protocol stack in the DLL plug-in architecture; wherein the industrial communication protocol stack is provided with a protocol recognition mechanism based on traffic characteristics;

[0009] The power supply monitoring network is configured with a distributed message bus based on ZeroMQ and an XGBoost anomaly detection model; wherein the distributed message bus is provided with a north-south data channel and is connected with the video monitoring equipment respectively;

[0010] The XGBoost anomaly detection model is used to detect the abnormality of the device sensing data of the video monitoring equipment, and the abnormal data is converted into a streaming media interface.

[0011] In the embodiments of the present application, a technical solution is provided, in which the monitoring data collected by different video monitoring equipment under different devices and different software can be displayed on the same streaming media interface. In order to realize the compatibility of different devices and different software, the DLL plug-in architecture capable of dynamic loading is adopted. In order to realize higher monitoring data sensing capability, the power supply monitoring network with three-dimensional mapping is configured. In order to realize key data conversion, the protocol recognition mechanism based on traffic characteristics is configured. In order to make the identification of abnormal data faster, the north-south data channel is constructed by using the lightweight message middleware ZeroMQ to realize high-throughput and low-latency data transmission. Finally, in order to realize real-time conversion of abnormal data and avoid post-analysis, the embedded monitoring grid of the XGBoost anomaly detection model is configured to realize anomaly detection based on feature engineering optimization and visual display.

[0012] In combination with the first aspect, the DLL plug-in architecture includes a service responder, which is used to generate a plug-in service request when abnormal data occurs in the video monitoring equipment, and determine a target communication protocol in the DLL plug-in architecture to be called by the plug-in service request in the industrial communication protocol stack.

[0013] The service responder of the present application triggers the plug-in service request based on abnormal data, loads specific protocol plug-ins only when needed, reduces the memory occupation of the protocol plug-in, and does not need to integrate all protocols through the full protocol stack resident mode. The service responder matches the target protocol in the industrial communication protocol stack according to the priority, automatically switches the standby protocol when the target protocol fails to be called, and can combine the redundant path of the three-dimensional mapping network to speed up the power failure recovery.

[0014] In combination with the first aspect, the power supply monitoring network includes a physical topology layer, a virtual data layer and an environment modeling layer; wherein the physical topology layer is used to determine the real-time data transmission node of the video monitoring equipment, the virtual data layer is used to configure the target transmission channel of the device sensing data according to the real-time transmission node, and the environment modeling layer is used to generate a three-dimensional heat map linked and mapped with the video monitoring equipment through a preset digital twin engine.

[0015] The physical topology layer of the application can dynamically allocate the optimal transmission path, the virtual data layer can ensure the rights of key data, configure a dedicated transmission channel, and the environment modeling layer can generate a three-dimensional heat map to map the coordinates of the faulty equipment.

[0016] In combination with the first aspect, the multi-layer cooperative architecture of the industrial communication protocol stack performs end-to-end trusted communication link establishment; wherein the multi-layer cooperative architecture comprises a dynamic evolution protocol feature layer for converting traffic features of device sensing data into a multi-dimensional quantum state vector, and a protocol quasi-state interface layer for simulating an industrial communication protocol and being capable of converting the multi-dimensional quantum state vector into a quasi-state communication stream for display on a streaming media interface.

[0017] The dynamic evolution protocol feature layer of the application converts traffic features into a multi-dimensional quantum state vector, utilizes the quantum no-cloning theorem to prevent attacks, and the protocol quasi-state interface layer converts the quantum state vector into a quasi-state communication stream by simulating the interaction logic of the industrial protocol, so that network sniffing tools cannot identify the real protocol type, and the protocol fingerprint hiding rate is improved. The protocol quasi-state interface layer adapts the quantum state vector into a format that can be parsed by a streaming media interface, and losslessly restores multi-dimensional industrial data.

[0018] In combination with the first aspect, the protocol identification mechanism comprises the following identification process:

[0019] Real-time capture of device sensing data of a video monitoring device and division into traffic segment samples through a sliding time window;

[0020] Loading the traffic segment samples into a phase space reconstruction, determining a time-sensitive quantization value, and based on the time-sensitive quantization value, determining a multi-dimensional quantum state vector representing traffic features;

[0021] Determining, based on the quantum entanglement strength corresponding to the protocol cluster in the industrial communication protocol stack, the base representation information of the multi-dimensional quantum state vector in the dynamic evolution protocol feature layer;

[0022] According to the base representation information, determining a standard industrial protocol data stream whose time jitter feature is consistent with that of the video monitoring device after inverse quantum Fourier transform decoding of the evolved quantum state vector, and determining the target industrial communication protocol.

[0023] The application extracts the time-sensitive quantization value of the traffic segment through phase space reconstruction, and combines the quantum state vector to represent the protocol features, so that the identification accuracy of complex industrial protocols is improved. The inverse mapping of the evolved quantum state vector to the time domain by the quantum inverse Fourier transform eliminates the time jitter in network transmission, and restores the standard protocol stream consistent with the original time sequence of the device, thereby improving the protocol parsing integrity. The protocol cluster base is dynamically selected based on the quantum entanglement strength, and the protocol identification time is reduced.

[0024] In combination with the first aspect, the distributed message bus constructed based on ZeroMQ comprises:

[0025] The ZeroMQ is preset and is bound to device nodes corresponding to different video monitoring devices; the device nodes are used to determine data structures and data contents;

[0026] The device nodes are connected to the north-south data channel, and a communication mode is configured based on a data type of the device nodes; the communication mode includes a data request mode, a data distribution mode and a data queue mode;

[0027] A distributed message bus is generated according to the communication mode.

[0028] Based on the dynamic data request mode of the device nodes, the application can realize data distribution and data queue, increase data throughput, reduce response delay of control instructions, and realize physical isolation of bidirectional flow, thereby avoiding data loop.

[0029] In combination with the first aspect, the XGBoost anomaly detection model includes a lightweight XGBoost classifier and a deep XGBoost classifier; the lightweight XGBoost classifier is used to determine first screening data with abnormal features in the device sensing data, and the deep XGBoost classifier performs time sequence classification on the first screening data and determines second screening data with time sequence grouping based on time sequence anomalies.

[0030] The application can identify periodic anomalies that cannot be captured by traditional static features, dynamically adjust the power allocation of the two-level classifiers according to the device load, and in a resource-limited scenario, the deep classifier can be closed and only the lightweight detection is retained, thereby improving system stability.

[0031] In combination with the first aspect, the anomaly detection further includes:

[0032] The video monitoring device is embedded with a lightweight identification agent, and the lightweight identification agent is used to generate bidirectional association identification by associating abnormal feature hash values in the first screening data with device ID numbers of the video monitoring devices; when there is a time sequence anomaly in the first screening data, a joint signature of a hardware fingerprint of the video monitoring device and watermark data of the device sensing data is generated based on the bidirectional association identification.

[0033] The bidirectional association identification binds abnormal feature hash values with device IDs, ensures that the data source is verifiable, prevents attackers from forging abnormal data, and fuses the device hardware fingerprint and the data watermark in the joint signature, thereby preventing the data from being intercepted. The identification agent generates the association identification in real time at the device end, and only needs to upload the lightweight signature instead of the original data.

[0034] In combination with the first aspect, the conversion of the abnormal data into a streaming media interface includes:

[0035] receiving abnormal data from the video monitoring device, the abnormal data being generated by an XGBoost-based anomaly detection model;

[0036] real-time analyzing the abnormal data and extracting feature information related to video monitoring;

[0037] mapping the analyzed feature information into a streaming interface to generate a dynamically visualized abnormal event display;

[0038] pushing the streaming interface to a user terminal in real time through a streaming transmission protocol;

[0039] adding multi-dimensional labeling information, including timestamps, location information, and abnormal types, in the streaming interface to facilitate quick positioning and analysis by the user.

[0040] The dynamic visualized abnormal event display of the application maps the abstract abnormal data detected by XGBoost into a dynamic heat map in the streaming interface, enabling instant abnormal device positioning, three-dimensional labeling of timestamps, location information, and abnormal types, and tracing the abnormal evolution process based on the time axis.

[0041] In combination with the first aspect, the conversion of the abnormal data into the streaming interface includes:

[0042] classifying and clustering the abnormal data to extract key features;

[0043] generating an abnormal data chart based on the key features; wherein the abnormal data chart includes a dynamic chart, a heat map, or a device geographic distribution chart;

[0044] mapping the abnormal data chart to a visualized interface and adding interactive controls in the visualized interface; wherein the interactive controls are used to filter, sort, and drill down the abnormal data under user instructions;

[0045] updating the visualized interface in real time through a dynamic updating mechanism to form the streaming interface.

[0046] The application dynamically selects chart forms according to feature types and associates three-dimensional mapping network coordinates through device geographic distribution charts, improving the abnormal positioning accuracy from meters to centimeters. Interactive space can also reduce fault diagnosis time.

[0047] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims.

[0048] The technical solutions of the application will be further described in detail below with reference to the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are used to explain the application, but do not limit the application. In the drawings:

[0050] Figure 1 A method flow chart of a video monitoring linkage method based on network communication to realize a heterogeneous software system in an embodiment of the application;

[0051] Figure 2 A response process chart of a service responder in an embodiment of the application;

[0052] Figure 3 A constituent architecture chart of a power supply monitoring network in an embodiment of the application;

[0053] Figure 4 A multi-layer coordination architecture execution process chart of an industrial communication protocol stack in an embodiment of the application;

[0054] Figure 5 A recognition flow chart of a protocol recognition mechanism in an embodiment of the application;

[0055] Figure 6 A bus structure chart of a distributed message bus constructed by ZeroMQ in an embodiment of the application;

[0056] Figure 7 A function execution chart of an XGBoost anomaly detection model in an embodiment of the application;

[0057] Figure 8 A bidirectional association identification generation chart in anomaly detection in an embodiment of the application;

[0058] Figure 9 An execution process chart of converting anomaly data into a streaming media interface in an embodiment of the application;

[0059] Figure 10 An execution process chart of converting anomaly data into a streaming media interface in an embodiment of the application. DETAILED DESCRIPTION

[0060] The preferred embodiments of the application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to explain and illustrate the application, and do not limit the application.

[0061] To solve the problem of data incompatibility and the inability to realize the same system display when different software and different hardware perform abnormal video monitoring in the prior art, the application provides a video monitoring linkage method based on network communication to realize a heterogeneous software system, which is described in detail in the specification. Figure 1 :

[0062] The application first builds a dynamically loaded DLL plug-in architecture and a three-dimensional mapping power supply monitoring network. When heterogeneous software and video monitoring devices are linked, the target communication protocol is determined by the flow characteristics of the data that needs to be communicated between the heterogeneous software and the video monitoring devices. The target communication protocol is loaded under the decoupling protocol through the dynamically loaded DLL plug-in architecture, and the data interaction communication is realized. In the case of ready-to-use, different video monitoring devices and control devices, or and the cloud are quickly realized to realize data interaction communication, so that different video monitoring devices and heterogeneous software realize linkage and supervise the entire area. The video monitoring device monitors each physical power supply node, such as a power distribution cabinet, a substation, etc. The heterogeneous software can monitor and control each physical power supply node. The physical power supply node is mapped to a virtual three-dimensional network, and the physical position and data processing of each physical power supply node are linked to prevent response lag during communication.

[0063] The DLL plug-in architecture of the application is configured with an associated industrial communication protocol stack. The industrial communication protocol stack is provided with a protocol recognition mechanism based on flow characteristics. During network communication, the protocol recognition mechanism extracts the packet length, interaction period and other flow characteristics according to the data of the heterogeneous software and the data of the video monitoring device, and determines the corresponding industrial communication protocol of the heterogeneous software and the different video monitoring devices during network communication according to the corresponding device protocol type of the flow characteristics, such as TCP protocol and ONVIF protocol.

[0064] The protocol recognition mechanism and the dynamically loaded DLL plug-in architecture of the application can automatically analyze the fusion characteristics of the device and the heterogeneous software during network communication, dynamically load the corresponding industrial communication protocol that can be loaded into the DLLD plug-in architecture, for example, based on the flow characteristics, determine that the communication protocol of the current video monitoring device is BACnet protocol, load the associated BACnet protocol, identify the bacnet.Dll file, realize the automatic discovery and automatic loading of the protocol, and shorten the new device access time during the process of new device access.

[0065] The power supply monitoring network of the application is configured with a distributed message bus based on ZeroMQ and an XGBoost anomaly detection model; wherein the distributed message bus is provided with north-south data channels and is connected with video monitoring devices respectively; the ZeroMQ distributed message bus will mainly perform bidirectional isolation communication on the network communication data of the heterogeneous software and video monitoring devices corresponding to the distributed nodes through the north-south data channels, prevent data congestion, the southbound channel is used for processing the data aggregation of the video monitoring devices or the heterogeneous software, and transmitting to the control decision center, such as the cloud. The northbound channel is used for communication through the control decision center and the video monitoring devices or the heterogeneous software, and issuing control instructions. The XGBoost anomaly detection model performs time sequence division on the network communication data of different video monitoring devices and heterogeneous software, i.e. device sensing data, generates time sequence data, determines time sequence features, sorts according to the importance of the time sequence features, realizes anomaly detection, and the anomaly includes but is not limited to current mutation, short circuit, open circuit and the like, and reduces the false positive rate.

[0066] In the application, different modes of the distributed message bus built by ZeroMQ are switched, for example: PUB-SUB mode, the original data of the video monitoring device is broadcast to multiple XGBoost detection nodes for parallel processing, then through the way of parallel computing, the rapid analysis and parallel analysis of the device sensing data are realized, and the discovery sensing speed of abnormal data is improved.

[0067] The device sensing data of the video monitoring device is detected by the XGBoost anomaly detection model, and the abnormal data is converted into a streaming media interface.

[0068] After detecting the abnormal data, the three-dimensional mapping power supply network can determine the positioning data of the fault device of the monitored video monitoring device, then in the process of generating the streaming media interface, the monitoring data of the corresponding video monitoring device is superimposed with the abnormal data, the identification of the abnormal event is performed, so that the operation and management personnel can directly watch the panoramic picture of the abnormal area through the terminal display device, thereby improving the linkage response speed of the video monitoring device.

[0069] The application can achieve flexible loading and unloading of different functional modules through a dynamic link library (DLL) plug-in architecture. This architecture allows the system to dynamically load plug-ins supporting different industrial communication protocols as needed, thereby achieving compatibility with heterogeneous software systems. The configured industrial communication protocol stack includes a protocol recognition mechanism based on traffic characteristics, which can automatically recognize and analyze the communication protocols of different devices to ensure accurate data transmission. The power supply monitoring network combines the spatial position of physical devices with monitoring data through three-dimensional mapping technology to achieve visual management of the devices. The network is configured with a distributed message bus based on ZeroMQ, which supports efficient data transmission and communication between devices. ZeroMQ is a high-performance asynchronous message library that can enable communication between multiple threads, processes, and nodes. XGBoost is a high-efficiency machine learning algorithm suitable for anomaly detection tasks. It can analyze the sensing data of video monitoring devices through gradient boosting tree algorithms to identify abnormal data. The model converts abnormal data into a streaming media interface for real-time monitoring and alarm.

[0070] The application combines a dynamic plug-in architecture with a protocol recognition mechanism to solve the compatibility of heterogeneous device access and the defects of traditional monitoring systems using a single protocol stack and single query mechanism. In terms of anomaly monitoring, the application uses the XGBoost model, a precise model based on image recognition, which is associated with three-dimensional spatial data of power supply device anomaly monitoring. It can analyze anomalies from the dynamic operation monitoring of software and the full-scene display of hardware monitoring, reducing false positives.

[0071] Embodiment 2

[0072] The DLL plug-in architecture of the application includes dynamically responding to target communication protocols, loading target communication protocols, and referring to Figure 2 :

[0073] The DLL plug-in architecture of the application includes a service responder that generates a plug-in service request when abnormal data occurs in a video monitoring device and determines the target communication protocol in the DLL plug-in architecture to be loaded by the plug-in service request in the industrial communication protocol stack. The service response of the application is a dynamic protocol adaptation mechanism driven by abnormal data, which is also a dynamic adaptation protocol driven by device sensing data with traffic characteristics. The service responder is triggered when abnormal data exists, then generates a plug-in service request, and loads specific protocol plug-ins when needed, for example: when video stream data exists, based on the traffic characteristics of the video stream data, the corresponding ONVIF protocol is determined, and the traffic characteristics of the sensor data of the monitoring device are adapted to determine the corresponding Modbus protocol; it can achieve dynamic loading of industrial communication protocols while avoiding traditional protocol stacks.

[0074] The service of the present application corresponds to its in the industrial communication protocol stack, according to the plug-in service request based on the request priority, the matching of the target industrial protocol is realized, for example: when the abnormal type is current overload, based on the corresponding flow characteristics, the power special protocol is called preferentially, so that the power failure can be quickly handled in the case of accurate realization of device communication in network communication, when the target industrial communication protocol is called, the standby protocol can also be automatically switched, combined with the three-dimensional mapping of the power supply monitoring network, whether there is a redundant power path is judged, through the redundant power path, the power failure is solved.

[0075] The present application produces according to the demand, dynamically loads the industrial communication protocol through the abnormal data as the trigger of the event trigger mechanism, realizes the on-demand adjustment, instead of the full-period full-protocol loading, and the chaotic interference effect between different communication protocols.

[0076] According to the abnormal data, the present application can determine the abnormal type, match the abnormal type and the industrial communication protocol, be suitable for the complex scene of multiple devices and multiple protocols, and be more in line with the device operation environment of the Internet of Things.

[0077] The service responder of the present application can combine the REQ-REP mode of ZeroMQ, send plug-in calling requests in the industrial communication protocol stack, and receive protocol ready state feedback, realize the atomic operation of protocol loading and message communication, and avoid the signaling conflict of the traditional polling mechanism.

[0078] In the specific implementation of the present application: the DLL plug-in architecture further includes

[0079] The dynamic loader is used for loading and unloading the target communication protocol;

[0080] The dependency resolver is used for checking the loaded target communication protocol and determining the dependency relationship of the communication protocol;

[0081] The protocol state machine is used for full-process state monitoring of the target communication protocol.

[0082] The dynamic loader is used for dynamically loading and unloading the target communication protocol. The dynamic loader is based on the dynamic link library (DLL) mechanism of the operating system, and can dynamically load and unload the plug-in. After the service responder generates the plug-in service request, the dynamic loader will load the corresponding target communication protocol plug-in according to the request. When it is not needed, the dynamic loader can unload the plug-in, release the system resources, and improve the flexibility and resource utilization of the system.

[0083] The dependency resolver is used to check the loaded target communication protocol and determine its dependencies. The dependency resolver analyzes the metadata of the target communication protocol plugin to determine its dependencies on other modules or libraries. By resolving dependencies, it ensures that all necessary dependencies are properly loaded when the target communication protocol is loaded. If a missing dependency or version mismatch is found, the dependency resolver prevents the plugin from loading and reports an error.

[0084] The protocol state machine is used for full-flow state monitoring of the target communication protocol. The protocol state machine is a finite state machine (FSM) that manages the various states of the target communication protocol (such as initialization, running, exception, termination, etc.). Through the state machine, the running state of the target communication protocol can be monitored in real time to ensure its normal operation. If an abnormal state is detected, the protocol state will trigger the corresponding processing mechanism, such as reloading the plugin, sending an alarm, etc.

[0085] Embodiment 3:

[0086] The power supply monitoring network of the present application is a three-layer structure, as shown in Figure 3 :

[0087] The physical topology layer of the present application is composed of heterogeneous sensor nodes and video monitoring devices deployed in the power supply area. The sensor nodes realize multi-protocol adaptation through dynamically loaded DLL plugin architecture, and the video monitoring devices are configured with a stream media forwarding module based on H.265 encoding. The physical topology layer is mainly used for real-time tracking of device node state, and then dynamically allocating the optimal transmission path, such as bypassing a faulty switch node, thereby reducing the packet loss rate of video stream. It dynamically discovers the connection state of switches, routers and other devices in the network device based on the DLL plugin architecture, constructs a real-time topology graph, provides data parameters for bottom path planning to the virtual data layer, and solves the problem of inability to perceive network connectivity changes, transmission channel configuration and physical network disconnection.

[0088] The virtual data layer constructs a distributed message bus based on ZeroMQ, including north-south data channels and east-west control channels. The north-south data channel is equipped with a protocol conversion gateway based on FPGA, which realizes real-time conversion of Modbus / OPC UA protocol to RTSP stream media protocol. The virtual data layer of the present application can configure dedicated transmission channels based on the type of device, such as a 4K camera for video monitoring device, a monitoring device equipped with a temperature sensor, and according to the data type corresponding to the video stream / sensor data. For example, video stream goes through a UDP high-speed channel, sensor data goes through a TCP reliable channel, and in the case of critical data priority, the video frame rate stability will automatically increase in a network congestion scenario. It mainly solves the problem that traditional single channel cannot transmit heterogeneous data, and the problem of video frame freezing and data loss during data transmission.

[0089] The environment modeling layer integrates a three-dimensional geographic information engine and a digital twin engine, and the digital twin engine dynamically generates a three-dimensional heat map of the power supply device by receiving the anomaly detection result of the virtual data layer in real time, and establishes a linkage mapping relationship with the spatial coordinates of the video monitoring device. The environment modeling layer of the present application generates a three-dimensional heat map through the digital twin engine, maps the abnormal data detected by XGBoost to the physical device coordinates, and the operation and maintenance personnel determine the fault area through the physical device coordinates, process the defects in the corresponding area at high speed, and the three-dimensional heat map generated by the present application is a power equipment state display map based on the digital twin engine. According to the fault type and fault degree of the equipment, a heat map based on color gradient display can be generated.

[0090] In actual implementation, the present application outputs the abnormal probability value of the XGBoost anomaly detection model, and uses it as the coloring weight of the heat map. The high probability area is displayed as dark red. The operation and maintenance personnel call the corresponding streaming media interface of the power equipment with different color gradients according to the color gradient, and accurately confirm the anomaly through the video stream of the associated camera.

[0091] In actual implementation, the physical topology bus can update node information, update newly added nodes or deleted nodes, so as to realize reconfiguration and reconfiguration of the channel, and realize end-to-end delay reduction.

[0092] In actual implementation, the present application allocates independent channels for different target industrial communication channels through the virtual data layer, avoids interference between different protocols, and increases the integrity of communication.

[0093] Embodiment 4:

[0094] The industrial communication protocol stack of the present application realizes end-to-end trusted communication link establishment through a multi-layer cooperative architecture. Referring to Figure 4 , the multi-layer cooperative architecture includes a dynamic evolution protocol feature layer for converting the traffic characteristics of device sensing data into a multi-dimensional quantum state vector, and a protocol quasi-state interface layer for simulating industrial communication protocols and being able to convert and adapt the multi-dimensional quantum state vector to a streaming media interface display.

[0095] The present application fuses the dynamically evolved protocol feature layer and the quantum state conversion, so as to cut the original data stream into two pieces of quantum state based on the generation of quantum random numbers, and realize unpredictable communication characteristics after the calculation of the dynamic evolution operator. The quantum state evolution can simultaneously hide the port number and handshake protocol and other protocol features, preventing data leakage.

[0096] The protocol isomorphism interface layer of the application can realize quantum state vectors through a protocol behavior simulator, encapsulated as a pseudo-state protocol package, and the data of the pseudo-state protocol package is extracted through key data to generate superimposed layer data of a streaming media interface when an abnormal state is displayed on a multimedia display device, and the corresponding curve can also be rendered on the multimedia interface.

[0097] In the dynamic evolution process, the application can automatically adjust the evolution parameters based on the molecular weight circuit combined with the corresponding network threat information of the abnormal data, and enhance the adaptive ability of the protocol. The quantum state vector is used for dynamic evolution of protocol characteristics, which solves the problem that each protocol in the protocol stack has a protocol fingerprint and may have an exposure characteristic.

[0098] In actual implementation, the multi-layer cooperative architecture of the application further includes:

[0099] The protocol adaptation layer is configured with a dynamic protocol loading engine, which dynamically calls the adapted industrial communication protocol according to the data type of the abnormal data;

[0100] The identification embedding layer is coupled with the protocol adaptation layer, and is used for encoding the device identification of the video monitoring device and the business identification of the target business into structured metadata, and embedding the header extension field of the industrial communication protocol;

[0101] The security authentication layer integrates a lightweight cryptographic module based on PKI, performs asymmetric encryption signature processing on the device identification, and generates an enhanced device identification carrying signature information;

[0102] The slice routing layer includes a network slice selector, which analyzes the feature vector encoded by the business identification, and establishes a policy mapping with the network slice resource pool through a preinstalled SDN controller.

[0103] The protocol adaptation layer configures a dynamic protocol loading engine: according to the data type of abnormal data, the adaptive industrial communication protocol is dynamically called. This mechanism can flexibly cope with different types of abnormal data, and ensure that the system can quickly adapt and process data from different devices. The identification embedding layer is embedded through structured metadata: the device identification of the video monitoring device and the business identification of the target business are encoded as structured metadata and embedded in the packet header extension field of the industrial communication protocol. This design can ensure that the data carries enough context information during transmission, facilitating subsequent processing and analysis. Asymmetric encryption signature: integrate a lightweight cryptographic module based on PKI (Public Key Infrastructure), perform asymmetric encryption signature processing on the device identification, and generate an enhanced device identification carrying signature information. Asymmetric encryption ensures data security and integrity through public key encryption and private key decryption. Network slice selector: analyze the feature vector encoded by the business identification, and establish a policy mapping with the network slice resource pool through the pre-installed SDN controller. Network slicing technology can allocate independent network resources for different businesses, ensuring business isolation and service quality.

[0104] The dynamic protocol loading engine of the application can dynamically select the adaptive communication protocol according to different types of abnormal data, improving the flexibility and adaptability of the system. The asymmetric encryption signature processing ensures the security and integrity of the device identification, preventing data from being tampered with or forged during transmission. Network slicing technology realizes flexible allocation of resources and business isolation through the SDN controller, improving the utilization rate of network resources and the reliability of the system. The embedding of structured metadata and the introduction of the security authentication mechanism enhance the overall reliability of the system, ensuring the accuracy and security of data during transmission and processing.

[0105] Embodiment 5:

[0106] The protocol identification mechanism of the application includes the following identification process:

[0107] The application captures the device sensing data of the video monitoring device in real time, and divides it into traffic segment samples through a sliding time window. For device sensing data, a sliding time window is used for time sequence division to generate multiple traffic segment samples.

[0108] The application loads the traffic segment sample into a phase space reconstruction, determines a timing sensitive quantization value, and determines a multi-dimensional quantum state vector representing the traffic characteristics based on the timing sensitive quantization value. In order to distinguish between steady-state protocols and burst protocols, the application maps the traffic segment sample to a high-dimensional phase space, and then calculates the timing sensitivity, such as the maximum Lyapunov index. Thus, by timing sensitivity, the timing characteristics are quantitatively determined, and the degree of chaos in the quantization of the timing characteristics is determined. If the degree of chaos is close to 0, it indicates a time protocol. If the degree of chaos is greater than or equal to 1 or greater than 0.5, it is determined to be a burst protocol. Compared with the traditional sliding window feature extraction, the application can capture the timing dynamic characteristics and reduce the misjudgment of similar protocols. The multi-dimensional quantum state vector is the vector value after the quantitative determination. The timing sensitivity quantization of the application can be combined with the XGBoost abnormal modeling model to let the output abnormal probability serve as the weight parameter of the phase space reconstruction, increase the high abnormal probability analysis ability in the abnormal judgment dimension, and improve the probability of key protocol recognition.

[0109] The application determines the base representation information of the multi-dimensional quantum state vector in the dynamic evolution protocol characteristic layer based on the quantum entanglement strength corresponding to the protocol cluster in the industrial communication protocol stack. The protocol cluster has a preset entanglement strength threshold. For example, the entanglement value of the power protocol cluster should not be lower than a certain preset value, which will match the corresponding quantum base. The quantum base is based on the high practicality industrial communication protocol. The entanglement value of the selected industrial communication protocol and the power protocol cluster will not be lower than the preset value, which will match successfully. Thus, the application avoids the existence of invalid industrial communication protocols being processed, causing resource loss. The base indication information is the functional display and protocol type of the high practicality industrial communication protocol, and the description information of practicality. The quantum base dynamically loads the quantum base analysis plug-in based on the target industrial communication protocol. This makes the industrial communication protocol stack of the application have a hybrid analysis capability of quantum and classical, which is suitable for quantum encryption of power equipment data and high-encryption power projects.

[0110] The application determines the target industrial communication protocol according to the standard industrial protocol data stream consistent with the time jitter characteristics of the video monitoring device after the evolved quantum state vector is decoded by the quantum inverse Fourier transform. The quantum inverse Fourier transform decodes the frequency domain quantum state into the time domain, for example: converting the protocol characteristics of a specific frequency into any one or a combination of multiple protocols in the noise-resistant standard protocol stream. The dynamic evolution of the multi-dimensional quantum state vector sets a controlled quantum phase rotation gate for the protocol feature layer, enhances the function of the key industrial communication protocol, and improves the quantum state discrimination degree. In the application, the ZeroMQ message bus can collect quantum state vectors and construct a ZAP authentication channel to only transmit quantum state vectors. Combined with the quantum key distribution mechanism, the end-to-end quantum encryption function is realized to prevent attacks during protocol identification. In the quantum protocol identification process, the industrial heterogeneous network can ensure security while expanding protocol compatibility. In high-interference scenarios, the fusion of classical protocols and post-quantum secure protocols can be realized, and the compatibility expansion is huge.

[0111] Embodiment 6:

[0112] The application is directed to the physical power nodes of different monitoring devices, and a distributed message bus is constructed by ZeroMQ. Referring to Figure 6 :

[0113] The application pre-sets ZeroMQ and binds the device nodes corresponding to different video monitoring devices. The device nodes define data structures and data contents, and the device nodes have pre-defined data formats. During network communication, data transmission is prevented due to cross-device parsing errors. For example, video streams use Protobuf binary encoding, and sensor data use JSON protocol.

[0114] The device nodes are connected to the north-south data channels, and the communication mode is configured based on the data type of the device nodes. The communication mode includes data request mode, data distribution mode, and data queue mode.

[0115] According to the communication mode, a distributed message bus is generated.

[0116] When the device nodes are connected to the north-south data channels, the south channel will use the PUB-SUB mode for one-to-many broadcasting, and the north channel will use the REQ-REQ mode to ensure that the instructions can be achieved, avoiding the problem that the traditional full-duplex TCP mode or other single mode cannot adapt to the large difference between data and control traffic in industrial scenarios.

[0117] In actual implementation, the initialization of ZeroMQ is bound to the device node, and ZeroMQ is set in advance and bound to the device node corresponding to different video monitoring devices. Each device node defines a data structure and data content, ensuring that the format and content of the message can be correctly parsed and processed. ZeroMQ can support multiple communication modes through its flexible socket abstraction, including request-reply, publish-subscribe, push-pull, etc.

[0118] Configuration of communication mode: according to the connection of device node and north-south data channel, and based on the data type of device node, the communication mode is configured.

[0119] Specifically, the communication mode includes:

[0120] Data request mode: suitable for the scenario of client requesting data from server, using request-reply mode.

[0121] Data distribution mode: suitable for the scenario of broadcasting or distributing data to multiple recipients, using publish-subscribe mode.

[0122] Data queue mode: suitable for task distribution and load balancing scenarios, using push-pull mode.

[0123] The communication mode of ZeroMQ is realized through its socket types (such as REQ / REP, PUB / SUB, PUSH / PULL), which can meet the communication needs in different scenarios. According to the configured communication mode, a distributed message bus is generated. ZeroMQ can realize low-latency and high-throughput message transmission through its high-performance message passing mechanism. ZeroMQ supports multiple transmission protocols (such as TCP, IPC and inproc), and provides features such as automatic reconnection, queue management and load balancing, ensuring reliable message transmission.

[0124] Through the lock-free queue model, batch processing algorithm and multi-core thread binding technology of ZeroMQ, high-performance and low-latency message passing is realized. This high-performance feature enables the distributed message bus to efficiently process large-scale data streams, especially suitable for real-time data processing and video monitoring scenarios. Support multiple communication modes to meet the communication needs in different scenarios. For example, the publish-subscribe mode is suitable for real-time data broadcasting, and the push-pull mode is suitable for task distribution.

[0125] With fault-tolerant mechanism, it provides a high-efficiency, reliable and easy-to-expand message passing solution for video monitoring system.

[0126] The application can realize wired transmission when burst traffic occurs, and automatically downgrade the REQ-REP mode to the PUSH-PULL mode when the device node is abnormal, so that data can still be transmitted, by configuring a high-priority queue for a key device such as a fire monitoring camera.

[0127] The application combines the DLL plug-in architecture with the communication mode, and automatically allocates a dedicated communication mode when a new industrial communication protocol is loaded. After the plug-in is loaded, the transmission will not be delayed.

[0128] By combining XGBoost anomaly detection with data queue mode, abnormal data can be traced back to the source by allocating an independent queue for the abnormal device when abnormal data is detected.

[0129] By combining the three-dimensional mapping of the power supply network with the topology of the device nodes, the three-dimensional coordinates in the environment modeling layer and the topology representation of the device nodes are realized, the physical area and the communication area are divided, and the cross-area traffic resource occupation is reduced.

[0130] Embodiment 7:

[0131] The XGBoost anomaly detection model of the application includes a lightweight XGBoost classifier and a deep XGBoost classifier; see Figure 7 :

[0132] The lightweight XGBoost classifier is used to determine the first screening data with abnormal features in the device sensing data, and the deep XGBoost classifier classifies the first screening data in time sequence and determines the second screening data based on time sequence grouping with time sequence anomalies;

[0133] The application performs initial screening of abnormal features by the lightweight XGBoost classifier, that is, initial screening of low-dimensional features by shallow decision trees, realizes high-throughput processing, and prevents delays in industry. Then, based on the time sequence grouping analysis of the deep XGBoost classifier, high-order features such as Hurst index and LSTM hidden state are extracted by time sequence grouping, which can determine the gradual abnormality in the device running process, and prevent mistaking transient noise such as electromagnetic interference for continuous abnormality.

[0134] The two-level classifier of the application can monitor the running state of the device with different weights to prevent device aging and reduce the accuracy of device monitoring.

[0135] The application can dynamically load a dedicated deep classifier plug-in through the DLL plug-in architecture combined with the dynamic loading of the two-level classifier when detecting an abnormal pattern that has never appeared, and perform online upgrade of the detection model without maintenance. The ZeroMQ message bus combined with hierarchical data transmission can realize different mode reporting of double data and occupy bandwidth. Through the combination of three-dimensional heat maps and time sequence anomalies, joint early warning is realized in time and space.

[0136] Embodiment 8

[0137] When the application anomaly is detected, a joint signature of a lightweight identification agent and a bidirectional association identification is introduced to ensure the reliability of the data source. See Figure 8 :

[0138] The application embeds a lightweight identification agent in the video monitoring device. The lightweight identification agent is used to generate a bidirectional association identification between the abnormal feature hash value in the first screening data and the device ID number of the video monitoring device. When there is a time sequence anomaly in the first screening data, a joint signature of the watermark data of the hardware fingerprint and the device sensing data of the existing video monitoring device is generated based on the bidirectional association identification.

[0139] By binding the ID of the device with the abnormal feature through hash washing, the mechanism of bidirectional association identification is realized, which can form an irreversible bidirectional association to determine whether an attack occurs in the intermediate transmission process, resulting in the inability to distinguish different types of data and fake data.

[0140] Through the joint signature method of the hardware fingerprint and the watermark, the device has the property of physical unclonable, thereby generating a composite signature resistant to quantum cracking, preventing the signature from being bypassed by the software layer.

[0141] Embodiment 9

[0142] The application converts abnormal data into a streaming media interface. See Figure 9 :

[0143] The application receives abnormal data from the video monitoring device. The abnormal data is generated by an XGBoost-based anomaly detection model. Through a lightweight parser, key features such as abnormal confidence, device temperature, and current phase angle can be extracted to prevent interface data overload.

[0144] The application analyzes abnormal data in real time, extracts feature information related to video monitoring, maps the analyzed feature information to a streaming interface, and generates a dynamically visualized abnormal event display. During dynamic mapping, by establishing binding rules between feature fields and interface components, such as binding current values and instrument panel pointer angles, and binding temperature and heat map color scales, the rendering is prevented, and the traditional static mapping cannot support multiple types of abnormalities

[0145] The streaming interface is pushed to the user terminal in real time through the streaming media transmission protocol.

[0146] Multi-dimensional annotation information is added to the streaming interface, including timestamps, location information, and abnormal types, to facilitate quick positioning and analysis by users.

[0147] The application dynamically switches between HLS and WebRTC protocols based on network RTT, and realizes the format of the streaming interface that can be pushed to the user terminal in real time, so that the interface is forwarded in a screenshot manner. It can also generate a two-way association identifier by fusing an identification agent and real-time annotation, and use it as the unique index of annotation data. It is determined whether the original video, i.e. device sensing data, is abnormal; the XGBoost output abnormal probability value drives the interface alarm level (red flashing when the probability is > 90%, yellow warning when the probability is > 70%), and is linked and scaled with the three-dimensional heat map, and can realize dynamic projection combined with the three-dimensional heat map.

[0148] Embodiment 10

[0149] The application converts abnormal data into a streaming interface. In specific implementation, refer to Figure 10 :

[0150] Receive abnormal data from the XGBoost-based anomaly detection model;

[0151] Classify and cluster analyze the abnormal data to extract key features;

[0152] Map the abnormal data to a visual interface to generate dynamic charts, heat maps, or device geographic distribution maps;

[0153] Add interactive controls to the visual interface to support user filtering, sorting, and drilling of abnormal data;

[0154] Through a dynamic updating mechanism, the changing trend and correlation of abnormal data are displayed in real time.

[0155] The technical principle of the above technical solution is:

[0156] The application receives abnormal data generated from an XGBoost-based anomaly detection model. XGBoost can effectively handle imbalanced data and identify anomalies through its powerful feature selection and classification capabilities. The abnormal data is classified and clustered to extract key features. Classification and clustering analysis can help identify patterns and structures in abnormal data, extracting the most valuable features for anomaly detection. Abnormal data is mapped into a visualization interface to generate dynamic charts, heat maps, or device geographic distribution maps. This visualization method can intuitively show the distribution and changes of abnormal events. Interactive controls are added to the visualization interface to support user filtering, sorting, and drilling of abnormal data. Interactive controls enhance user flexibility, allowing users to analyze abnormal data more deeply. Through a dynamic updating mechanism, real-time trends and correlations of abnormal data are displayed. The dynamic updating mechanism ensures that users can see the latest changes in abnormal data, allowing for timely responses.

[0157] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video monitoring linkage method for heterogeneous software systems based on network communication, characterized in that, The application relates to a power supply monitoring network, and comprises the following steps: a dynamic loading DLL plug-in architecture and a three-dimensional mapping power supply monitoring network are pre-built; an industrial communication protocol stack is configured in the DLL plug-in architecture, wherein the industrial communication protocol stack is provided with a protocol identification mechanism based on flow characteristics; the DLL plug-in architecture comprises a service responder, which is used for generating a plug-in service request when abnormal data of a video monitoring device occurs, and determining a target communication protocol called by the plug-in service request in the industrial communication protocol stack; a multi-layer collaborative architecture of the industrial communication protocol stack performs end-to-end trusted communication link establishment; wherein the multi-layer collaborative architecture comprises a dynamic evolution protocol feature layer used for converting flow characteristics of device sensing data into a multi-dimensional quantum state vector, and a protocol quasi-state interface layer used for simulating an industrial communication protocol and capable of converting the multi-dimensional quantum state vector into a flow media interface display; the protocol identification mechanism comprises the following identification process: device sensing data of a video monitoring device is captured in real time and is divided into flow segment samples through a sliding time window; the flow segment samples are loaded into a phase space reconstruction, time-sensitive quantization values are determined, and based on the time-sensitive quantization values, a multi-dimensional quantum state vector representing flow characteristics is determined; based on the quantum entanglement strength corresponding to the protocol cluster in the industrial communication protocol stack, base representation information of the multi-dimensional quantum state vector in the dynamic evolution protocol feature layer is determined; based on the base representation information, standard industrial protocol data streams with time sequence jitter characteristics consistent with the video monitoring device are determined after inverse quantum Fourier transform decoding of the evolved quantum state vector, and a target industrial communication protocol is determined; a distributed message bus based on ZeroMQ and an XGBoost anomaly detection model are configured in the power supply monitoring network; wherein the distributed message bus is provided with south-north data channels and is connected with the video monitoring device; abnormal detection of device sensing data of the video monitoring device is performed through the XGBoost anomaly detection model, and abnormal data is converted into a flow media interface.

2. The video monitoring linkage method for realizing heterogeneous software systems based on network communication according to claim 1, characterized in that, The power supply monitoring network comprises a physical topology layer, a virtual data layer and an environment modeling layer; wherein the physical topology layer is used for determining real-time data transmission nodes of the video monitoring device, the virtual data layer is used for configuring a target transmission channel of the device sensing data according to the real-time transmission nodes, and the environment modeling layer is used for generating a three-dimensional thermal map linked and mapped with the video monitoring device through a preset digital twin engine.

3. The video monitoring linkage method for realizing heterogeneous software systems based on network communication according to claim 1, characterized in that, The distributed message bus based on ZeroMQ comprises the following steps: ZeroMQ is preset and is bound with device nodes corresponding to different video monitoring devices; wherein the device nodes are used for determining data structures and data contents; the device nodes are connected with the south-north data channels, and communication modes are configured based on the data types of the device nodes; wherein the communication modes comprise a data request mode, a data distribution mode and a data queue mode; the distributed message bus is generated according to the communication modes.

4. The video monitoring linkage method for realizing heterogeneous software systems based on network communication according to claim 1, characterized in that, The XGBoost anomaly detection model comprises a lightweight XGBoost classifier and a deep XGBoost classifier; wherein the lightweight XGBoost classifier is used to determine the first screening data with abnormal features in the device sensing data, and the deep XGBoost classifier performs time series classification on the first screening data and determines the second screening data with time series anomalies based on time series grouping.

5. The video monitoring linkage method for realizing heterogeneous software systems based on network communication according to claim 4, characterized in that, The anomaly detection further comprises: A lightweight identification agent is embedded in the video monitoring device, and the lightweight identification agent is used to generate a bidirectional association identification between the abnormal feature hash value in the first screening data and the device ID number of the video monitoring device; wherein when there is a time series anomaly in the first screening data, a joint signature of the hardware fingerprint of the video monitoring device and the watermark data of the device sensing data is generated based on the bidirectional association identification.

6. The video monitoring linkage method for realizing heterogeneous software systems based on network communication according to claim 5, characterized in that, The conversion of the abnormal data into the streaming media interface comprises: Receiving abnormal data from the video monitoring device, the abnormal data being generated by the XGBoost-based anomaly detection model; Real-time analysis of the abnormal data to extract feature information related to video monitoring; Mapping the analyzed feature information into the streaming media interface to generate a dynamically visualized abnormal event display; Real-time pushing of the streaming media interface to the user terminal through the streaming media transmission protocol; Adding multi-dimensional labeling information in the streaming media interface, including timestamp, location information, and abnormal type, so as to quickly locate and analyze the abnormal data.

7. The video monitoring linkage method for implementing heterogeneous software systems based on network communication according to claim 6, characterized in that, The conversion of the abnormal data into the streaming media interface comprises: Classifying and clustering the abnormal data to extract key features; Generating an abnormal data chart according to the key features; wherein the abnormal data chart comprises a dynamic chart, a heat map, or a device geographic distribution chart; Mapping the abnormal data chart to a visual interface and adding interactive controls in the visual interface; wherein the interactive controls are used to filter, sort, and drill down the abnormal data under user instructions; Real-time updating of the visual interface through a dynamic updating mechanism to form a streaming media interface.

Images