Video signal security transmission method and device based on video interface protocol conversion

Abstract

This application provides a method and apparatus for secure transmission of video signals based on video interface protocol conversion. The method includes: acquiring at least one network video stream from a management area and determining the target video channel corresponding to the network video stream; performing a first protocol conversion process on the network video stream to convert it into a video interface signal conforming to the video display interface specification; transmitting the video interface signal through a physical video transmission medium crossing the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side; performing a second protocol conversion process on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network; and connecting the network video stream generated on the production control area side to the production control area network. This application can reduce transformation costs and implementation risks, improve the reliability of cross-domain video transmission, and enhance the ability to fuse and analyze video and process data.

Description

Technical Field

[0001] This application relates to the field of industrial automation control technology, and in particular to a method and apparatus for secure transmission of video signals based on video interface protocol conversion. Background Technology

[0002] In process industries such as thermal power plants, combined heat and power plants, waste-to-energy plants, and chemical plants, critical production processes typically rely on video cameras for real-time monitoring. Video signals are usually transmitted to the management area in the form of streaming media carried by Ethernet, where they are decoded and projected onto a large screen for operators to inspect and handle. To meet industrial information security requirements, the production control area and the management area are generally isolated via network gateways, firewalls, and other security devices. Data is typically only allowed to flow unidirectionally from the production control area to the management area.

[0003] In existing technologies, video signals are difficult to transmit from the management area to the production control area. This prevents video data from being correlated and integrated with process data such as timing data and control loop status within the production control area within the same network domain, hindering intelligent analysis and closed-loop control for production control. Common solutions to this problem include independently deploying a video system in the production control area or adjusting the network topology to migrate the video network to the production control area and output it unidirectionally to the management area. The former requires rewiring and equipment procurement, resulting in high costs; the latter requires large-scale network and software modifications and places higher demands on transmission throughput, posing significant implementation risks. Another solution involves performing video analysis in the management area and transmitting the results back via serial port, but the amount of information transmitted is limited, and the production control area cannot directly view the original video, resulting in insufficient application scalability. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method and apparatus for secure transmission of video signals based on video interface protocol conversion, in order to solve the problems existing in the prior art, such as the difficulty in securely transmitting video from the management area to the production control area, the inability to intuitively view the original video in the control area, and the difficulty in associating and integrating video with process data.

[0005] A first aspect of this application provides a secure video signal transmission method based on video interface protocol conversion, applied in an industrial network environment where production control area and management area are isolated from each other. The method includes: acquiring at least one network video stream from the management area and determining a target video channel corresponding to the network video stream; performing a first protocol conversion process on the network video stream to convert it into a video interface signal conforming to the video display interface specification; transmitting the video interface signal through a physical video transmission medium crossing the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side; performing a second protocol conversion process on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network; connecting the network video stream generated on the production control area side to the production control area network, and providing the network video stream to an operator station or intelligent analysis server for video viewing or analysis, and associating the video data used for analysis with process data within the production control area.

[0006] A second aspect of this application provides a secure video signal transmission device based on video interface protocol conversion, applied in an industrial network environment where production control area and management area are isolated from each other. The device includes: an acquisition module for acquiring at least one network video stream from the management area and determining a target video channel corresponding to the network video stream; a first protocol conversion module for performing a first protocol conversion process on the network video stream to convert it into a video interface signal conforming to the video display interface specification; a transmission module for transmitting the video interface signal through a physical video transmission medium spanning the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side; a second protocol conversion module for performing a second protocol conversion process on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network; and an access module for accessing the network video stream generated on the production control area side to the production control area network, providing the network video stream to an operator station or intelligent analysis server for video viewing or analysis, and associating the video data used for analysis with process data within the production control area.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This application involves acquiring at least one network video stream from the management area and identifying the target video channel corresponding to the network video stream; performing a first protocol conversion process on the network video stream to convert it into a video interface signal conforming to the video display interface specification; transmitting the video interface signal through a physical video transmission medium crossing the boundary between the production control area and the management area, enabling the video interface signal to reach the production control area side; performing a second protocol conversion process on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network; connecting the network video stream generated on the production control area side to the production control area network and providing the network video stream to the operator station or intelligent analysis server for video viewing or analysis, and associating the video data used for analysis with process data within the production control area. This application can reduce transformation costs and implementation risks, improve the reliability of cross-domain video transmission, and enhance the ability to fuse and analyze video and process data. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the overall system architecture involved in the video signal secure transmission method provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the secure transmission method for video signals based on video interface protocol conversion provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of the video signal secure transmission device based on video interface protocol conversion provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] In process industries (such as thermal power plants, combined heat and power plants, waste-to-energy plants, and chemical industries), critical production processes (such as boiler flames, furnace combustion, liquid level interfaces, and material crushing) typically require real-time monitoring via video cameras. Current deployment methods are as follows: Video signals (such as RTSP / RTP over Ethernet) are transmitted to the management area via Ethernet, decoded in the main control room, and projected onto a large screen. Production processes are provided for operator reference, and some management area pages are projected onto the large screen via Ethernet. The video acquisition cameras include personnel security cameras, equipment process cameras, etc.

[0014] Due to information security requirements, the production control area (such as DCS and PLC systems) and the management area are isolated by network. Data can only flow unidirectionally from the control area to the management area, which usually requires information security equipment such as network gateways / firewalls.

[0015] The inability to directly transmit video signals back to the control area makes it impossible to perform multimodal fusion analysis with real-time production process data (such as timing data and control loops), which restricts the realization of intelligent closed-loop control.

[0016] Current common solutions and their problems: Existing technical solution 1 (independent deployment of control area video system): This solution is costly and requires rewiring and equipment procurement; Existing technical solution two (modifying network topology): The entire video network is migrated to the control area and transmitted unidirectionally to the management area via a network gateway. This solution requires large-scale modifications to the software and network structure, carries high implementation risks, and has extremely high requirements for gateway throughput. Since it involves passing through a network gateway, the RTSP protocol needs to be converted to a UDP-based RTP protocol to penetrate the gateway. The management side needs to receive the software, resulting in significant custom development costs. Existing technical solution three (management area analysis + result feedback): Video intelligent analysis is performed in the management area, and the results are fed back to the control area via serial port (such as RS485 / RS232). This solution deploys the production control intelligent analysis software and server in the management area, making it unable to provide services to the production intelligent analysis. This solution has limited information transmission capacity, operators cannot view the original video, and intelligent applications are limited.

[0017] In view of the problems existing in the prior art, this application provides a video signal transmission method based on video interface protocol conversion. This application belongs to the field of industrial automation control and industrial Internet technology, specifically involving the secure transmission and intelligent application technology of video signals between the production control area and the management area in process industries (such as power, chemical, waste-to-energy, etc.). It is particularly suitable for realizing the reliable transmission of video data from the management area to the production control area under the premise of meeting the requirements of industrial information security.

[0018] This application presents a video signal transmission method based on video interface protocol conversion, which enables secure and reliable transmission of video signals from the management area to the production control area without altering the existing network topology and software system. It allows operators in the control area to directly view video feeds and achieves multimodal intelligent control of video and process data. This method reduces system modification costs and implementation risks, and complies with industrial information security standards.

[0019] The core of this application's technical solution lies in using high-definition video display interfaces (such as HDMI, DP, DVI, and VGA) as protocol conversion intermediaries to convert the Ethernet-based video stream from the management area into a display interface signal, and then convert it back into a network video stream that the control area can receive. Specifically, this may include the following operations: 1. Video capture and conversion in the management area: The network video stream (such as RTSP protocol) is obtained from video sources in the management area (such as security cameras, NVRs, decoders); the network video stream is converted into a high-definition display interface signal (HDMI / DP / DVI / VGA) through a protocol conversion device (such as an RTSP-to-HDMI converter).

[0020] 2. Signal isolation and transmission: The display interface signal is transmitted to the boundary of the control area via physical video cables (such as HDMI cables and DP fiber optic cables); the physical isolation characteristics of the display interface are utilized to avoid direct network connection and meet information security requirements.

[0021] 3. Control area video reconstruction: On the control area side, video acquisition devices (such as HDMI-to-RTSP converters) are used to convert the display interface signal back into a network video stream (such as RTSP); the converted video stream is then distributed to the operator station or intelligent analysis system via the control area switch.

[0022] 4. Video application integration: Operators can directly view video feeds on the control area interface; video data is integrated with DCS and time-series data to achieve multimodal intelligent analysis and closed-loop control.

[0023] Before providing a detailed description of the embodiments of the method of this application, the overall system architecture of the video signal transmission method of this application in a real-world scenario will first be described in conjunction with the accompanying drawings and embodiments. Figure 1 This is a schematic diagram of the overall system architecture involved in the video signal secure transmission method provided in this application embodiment. The system includes the following: like Figure 1As shown, this system, while maintaining the network isolation boundary between the production control area and the management area, achieves controlled transmission of video from the management area to the production control area and service access by setting up protocol conversion links on both sides of the boundary and crossing the boundary with physical video interface links.

[0024] The management area side includes the video network and the management application side. The video network deploys video source devices such as furnace cameras, security cameras, thermal imaging cameras, and network video recorders. Each video source device outputs at least one Ethernet-bearing network video stream, which can be centrally accessed and decoded for display via a management application server or a large-screen decoder. On the management area side, a first protocol conversion device is installed. This device receives the network video stream and decodes it into a video interface signal conforming to the video display interface specification. The video display interface is at least one of a high-definition video multimedia interface, a display port interface, a digital video interface, or an analog video interface. Therefore, the data flow on the management area side can be summarized as: video source outputs network video stream → first protocol conversion device outputs video interface signal.

[0025] The production control area side includes the production control network and its access equipment. The production control network deploys switching equipment, operator stations, intelligent analysis servers, control application servers, and control system controllers and input / output devices. A second protocol conversion device is installed on the production control area side. This device receives video interface signals from the boundary side, performs video frame acquisition and encoding / encapsulation, generates a network video stream for transmission through the production control network, and outputs network access parameters for the network video stream to the switching equipment. This allows operator stations or intelligent analysis servers to acquire the network video stream on demand within the production control network. Therefore, the data flow on the production control area side can be summarized as follows: second protocol conversion device receives video interface signals → generates network video stream → distributed via switching equipment → displayed on operator stations or analyzed by intelligent analysis servers.

[0026] The cross-domain transmission link is located at the boundary between the production control area and the management area, using a physical video transmission medium as the carrying channel across the boundary. Specifically, the video interface signal output by the first protocol conversion device on the management area side is transmitted to the input port of the second protocol conversion device on the production control area side via the physical video transmission medium. The physical video transmission medium includes at least one of high-definition multimedia cables, display port cables, digital video cables, or analog video cables; in scenarios requiring long-distance transmission, the physical video transmission medium can be fiber optic cables or transmission media with video interface extension capabilities. This cross-domain link does not establish a network interconnection between the management area and the production control area during transmission; the boundary crossing is completed through physical interface signal transmission, thus... Figure 1 The network isolation boundaries shown remain consistent.

[0027] Furthermore, in Figure 1 Within the production control network shown, the intelligent analysis server, after acquiring the network video stream, can parse and process the video data and associate it with the process data generated by the control system controller and input / output. The control application server can execute control logic processing based on the associated data objects, issuing control commands to the control system controller and input / output or generating alarm information. Thus, the system of this application forms an overall architecture with "network video stream acquisition and first protocol conversion in the management area, cross-domain transmission of physical video interfaces, and second protocol conversion and network distribution in the production control area" as the main link, and connected to the operator station display link and intelligent analysis and processing link within the production control area.

[0028] The following examples illustrate the hardware solutions involved in the above system in real-world scenarios, which may include the following solutions: Hardware Option 1: Purchase existing mature equipment. The physical cables for HDMI / DP / DVI / VGA are only for isolation purposes; long-distance transmission can be achieved using Ethernet cables or fiber optic cables. 1) Equipment in the management area: Video source: Cameras, NVRs, and decoders that support RTSP / Ethernet output; First-level protocol converter: Network video stream → HDMI / DP / DVI / VGA; 2) Transmission medium: High-definition video cables (such as HDMI 2.1, DP 2.1 fiber optic); 3) Control area equipment: Second-level protocol converter: HDMI / DP / DVI / VGA → Network video stream (RTSP); Control area switches, operator stations, and intelligent analysis servers; 4) Optional modules: Video channel selector: Supports on-demand selection of specific video channels for transmission; Protocol Adaptive Module: Compatible with multiple video encoding formats (H.264 / H.265).

[0029] Hardware Solution 2: If the large screen decoder on site has HDMI / DP / DVI / VGA expansion capabilities, an HDMI / DP / DVI / VGA expansion card can be installed on the large screen decoder. HDMI / DP / DVI / VGA cables can be used for connection and isolation. The encoder converts the signal to RTSP over Ethernet and uses Ethernet to access the control network for analysis, processing, and display by the operator station and intelligent analysis server.

[0030] Hardware Solution 3: Develop an integrated hardware and software product with RTSP over Ethernet input and output. The encoding chip, decoding chip, and HDMI cable are integrated into the PCB board, and a software configuration page is provided.

[0031] The following examples illustrate the practical application of the above system: Specific Example 1 (Flame Monitoring in a Waste-to-Energy Plant Furnace) Management area: The furnace flame camera outputs an RTSP stream via the NVR, which is then connected to the HDMI output port of the large screen decoder via an RTSP-to-HDMI converter; Transmission: Utilize a large-screen decoder to expand the HDMI interface, and use an HDMI cable to connect the decoder RTSP over Ethernet across area boundaries; Control area: RTSP over Ethernet connects to the production control network via Ethernet; Application: Video cameras in waste-to-energy incinerators use large-screen decoders and intelligent algorithm analysis servers in the production control area to intelligently identify combustion status using the RTSP protocol. Combined with data collected by the DCS, comprehensive analysis is performed to automatically issue control commands, reducing operator workload and increasing automation rates.

[0032] Specific Example 2 (Monitoring of Critical Equipment) Management area: Video from key equipment, instruments, and cameras is output via the decoder's HDMI port; Conversion: A DP-to-RTSP converter converts a signal into an RTSP stream; Control area: Intelligent identification of readings from video streams or liquid level interfaces through fluoroscopy, etc., to provide DCS control or alarm to operators in case of abnormal situations.

[0033] It should be noted that the technical solution of this application is not limited to the above application scenarios, but is also applicable to scenarios such as biomass power generation silo monitoring and pyrolysis furnace flame monitoring. In actual operation, it is only necessary to adapt the corresponding video analysis algorithm and interface conversion equipment.

[0034] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0035] Figure 2 This is a flowchart illustrating a secure video signal transmission method based on video interface protocol conversion provided in an embodiment of this application. Figure 2 As shown, the method may specifically include: S201, Obtain at least one network video stream from the management area and determine the target video channel corresponding to the network video stream; S202, Perform a first protocol conversion process on the network video stream to convert the network video stream into a video interface signal that conforms to the video display interface specification; S203, transmits video interface signals through a physical video transmission medium that crosses the boundary between the production control area and the management area, so that the video interface signals reach the production control area side. S204, On the production control area side, a second protocol conversion process is performed on the arriving video interface signal to convert the video interface signal into a network video stream for transmission over the production control area network; S205 connects the network video stream generated on the production control area side to the production control area network, and provides the network video stream to the operator station or intelligent analysis server for video viewing or analysis processing, and associates the video data on which the analysis and processing are based with the process data in the production control area.

[0036] In some embodiments, obtaining at least one network video stream from the management area and determining the target video channel corresponding to the network video stream includes: Obtain channel description information of network video streams published externally by at least one video source within the management area. The channel description information includes video source identifier, channel identifier, and network access parameters corresponding to the channel identifier. Based on the preset video channel selection strategy, the target video channel is selected from the channel description information, and the corresponding streaming configuration is generated. Based on the pull stream configuration, establish a network video stream acquisition connection for the target video channel to obtain the network video stream corresponding to the target video channel.

[0037] Specifically, the management area first establishes a video source resource list to uniformly describe the available video sources and video channels on site. Furnace flame cameras, security cameras, thermal imaging cameras, or network video recorders can be used as video source entities. Each video source is assigned a video source identifier, and under each video source identifier, a channel identifier is maintained for at least one network video stream channel that the video source can output. The channel identifier is used to distinguish different viewpoints, different bitrates, or different resolution channels of the same video source. For example, the same network video recorder can simultaneously provide two channels: a furnace flame image and a silo image, or the same camera can simultaneously provide two channels: a main bitrate stream and a sub-bitrate stream.

[0038] In some examples, in order to enable the subsequent establishment of a network video stream acquisition connection, this embodiment further writes the network access parameters corresponding to the channel identifier into the channel description information. The network access parameters include at least one of the access address, port parameters and session parameters used to locate the network video stream, and optionally include authentication information or access credentials.

[0039] In some examples, the process of obtaining channel description information can be provided by the management application server, network video recorder, or decoder on the management area side. Specifically, when the video source is a network video recorder, the configuration interface of the network video recorder or its provided management interface can output a channel list, which includes the channel identifier of each channel and the network access parameters that can access that channel. When the video source is a camera, the network video stream parameters published by the camera can be read through the camera's configuration page, and the parameters can be associated and stored with the pre-configured video source identifier and channel identifier. When the video signal is first aggregated by the decoder and then output externally, the decoder can generate a channel identifier corresponding to each decoding output window and record the corresponding network access parameters for each channel identifier. In this way, the management area side forms a set of channel description information containing video source identifiers, channel identifiers, and network access parameters, providing an input basis for subsequent target video channel selection.

[0040] In some examples, to avoid consuming unnecessary transmission resources by introducing all video channels within the management area into the cross-domain link, this application selects target video channels from the channel description information according to a preset video channel selection strategy and generates a streaming configuration corresponding to the target video channel. Specifically, the video channel selection strategy can be jointly determined by production process monitoring requirements and security policies, and is configured and fixed on the management area side or in the configuration page associated with the protocol conversion device.

[0041] For example, taking the monitoring of furnace flames in a waste-to-energy plant as an example, the video channel selection strategy can prioritize the channel identifier corresponding to the furnace flame camera, or select the channel identifier corresponding to the furnace flame image from multiple channels of the network video recorder. In critical equipment monitoring scenarios, the video channel selection strategy can select the channel identifier corresponding to the critical equipment instrument camera, or select the channel identifier corresponding to the liquid level interface of the fluoroscopy lens. Furthermore, the video channel selection strategy can also be filtered in conjunction with constraints such as business priority, monitoring area, video resolution level, or bitstream type, so that the selected target video channel meets the input requirements for viewing by operators on the control side and processing by the intelligent analysis server.

[0042] In some examples, the pull-stream configuration encapsulates the channel identifier of the target video channel with its network access parameters, enabling the first protocol conversion device to initiate a network video stream acquisition connection. Specifically, the pull-stream configuration includes, but is not limited to, the video source identifier, channel identifier, and network access parameters corresponding to the target video channel, and optionally includes reconnection policy parameters and pull-stream session keep-alive parameters. The reconnection policy parameters are used to re-establish the acquisition connection in the event of network jitter or session interruption, while the keep-alive parameters are used to maintain the session's continued validity. By writing the above information into the pull-stream configuration, subsequent network video stream acquisition connections can be repeatedly established and configurably switched.

[0043] In some examples, a network video stream acquisition connection to the target video channel is established based on the pull stream configuration to obtain the network video stream corresponding to the target video channel. Specifically, after reading the pull stream configuration, the first protocol conversion device on the management area side initiates a session establishment request to the target video channel based on the network access parameters therein, and continuously receives the media payload data output by the target video channel after the session is successfully established, thereby forming the network video stream input corresponding to the target video channel.

[0044] For example, taking the monitoring of furnace flames in a waste-to-energy plant as an example, the video from the furnace flame camera can first be aggregated and output as a network video stream by a network video recorder. The first protocol conversion device then retrieves the network video stream corresponding to the furnace flame channel identifier from the network video recorder according to the pull-stream configuration. Similarly, taking the monitoring of critical equipment as an example, the video from the critical equipment instrument camera can be aggregated and output as a network video stream by a decoder. The first protocol conversion device then retrieves the network video stream corresponding to the critical equipment channel identifier from the decoder according to the pull-stream configuration. The acquired network video stream serves as the input for the subsequent first protocol conversion process, and is further converted into a video interface signal conforming to the video display interface specification before entering the cross-domain physical video transmission link.

[0045] Through the above embodiments, the management area side provides configurable descriptions of video sources and video channels based on channel description information, and generates streaming configurations based on video channel selection strategies. This enables the establishment of network video stream acquisition connections for target video channels, allowing subsequent protocol conversion and cross-domain transmission links to be continuously executed with the selected video channels as input. This improves the deployment flexibility and maintainability of cross-domain transmission links, while avoiding unnecessary video channel occupation of transmission resources.

[0046] In some embodiments, a first protocol conversion process is performed on the network video stream to convert the network video stream into a video interface signal conforming to the video display interface specification, including: The network video stream is input into the first protocol conversion device, and media payload data is obtained by parsing the session parameters of the network video stream. Decoding and frame reconstruction are performed on the media payload data to generate video frame data, and the video frame data is time-encapsulated according to the preset video output format. Based on the time-encapsulated data, a video interface signal conforming to the video display interface specification is generated, and the video interface signal is output through the output port corresponding to the video display interface. The video display interface is at least one of a high-definition video multimedia interface, a display port interface, a digital video interface, or an analog video interface.

[0047] Specifically, the network video stream on the management area side originates from the output channel of the furnace flame camera aggregated by the network video recorder, or from the output channel of the key equipment instrument camera aggregated by the decoder. The management area side first determines the network access parameters based on the pull configuration generated in the previous embodiment, and writes the network access parameters into the input configuration item of the first protocol conversion device, enabling the first protocol conversion device to establish a stable acquisition connection to the target video channel. The network access parameters include, but are not limited to, one or more of the following: network video stream access address, port parameters, and authentication information. The session parameters corresponding to the network video stream also include one or more of the following: encoding format identifier, stream type identifier, resolution parameter, frame rate parameter, and timestamp reference parameter.

[0048] In some examples, when establishing a connection, the first protocol conversion device completes session negotiation and media reception of the network video stream based on session parameters. Specifically, the first protocol conversion device initiates a session establishment request to the network video stream publisher, obtains the transmission parameters of the media channel, and then starts the media reception process, continuously receiving network packets carrying media data. To adapt to network jitter and out-of-order risks in the field, the first protocol conversion device can establish a media buffer on the receiving side and perform order reordering and packet loss determination on the received media packets according to the packet sequence number and timestamp, thereby outputting continuous media payload data. The media payload data can be understood as video encoded data segments parsed from network packets and their associated time information. Subsequent decoding and frame reconstruction processes are continuously executed with this media payload data as input.

[0049] In some examples, the first protocol conversion device performs decoding and frame reconstruction processing on the media payload data to generate video frame data for display output. Specifically, the first protocol conversion device selects the corresponding decoding component according to the encoding format identifier in the session parameters, converts the media payload data into pixel-domain video frame data, and completes inter-frame prediction recovery based on keyframe markers and reference frame relationships during the decoding process. When the media payload data contains resolution switching, bitstream switching, or parameter set updates, the first protocol conversion device resets the parameters of the decoding component based on the parameter set update information and performs frame boundary verification on the decoded output to ensure the integrity and displayability of the video frame data.

[0050] Furthermore, for furnace flame monitoring scenarios where a network video recorder serves as the video source aggregation point, the first protocol conversion device can directly receive the target channel bitstream output by the network video recorder and decode and output video frame data; for critical equipment monitoring scenarios where a decoder serves as the aggregation point, the first protocol conversion device can receive the target channel bitstream released by the decoder and decode and output video frame data.

[0051] In some examples, after obtaining video frame data, the first protocol conversion device performs timing encapsulation on the video frame data according to a preset video output format to generate continuous timing signals that meet the requirements of the video display interface. The video output format is used to define the set of output parameters on the display interface side. The set of output parameters includes one or more of the following: output resolution, output refresh rate, pixel arrangement format, color space parameters, and synchronization timing parameters.

[0052] For example, the first protocol conversion device establishes a display clock reference based on the output refresh rate, writes video frame data into the display buffer according to the frame rate mapping rule, and outputs pixel data in pixel scan order within each display cycle. Simultaneously, it generates line synchronization, field synchronization, and blanking zone timing that match the output format. When the network video stream frame rate is inconsistent with the output refresh rate, the first protocol conversion device can perform rate adaptation using frame hold or frame drop methods to maintain continuous and stable display timing. Through the above timing encapsulation process, the video frame data is converted into a timing data stream that can directly drive the display interface output port, and this timing data stream carries synchronization control information consistent with the output format.

[0053] In some examples, the first protocol conversion device generates a video interface signal conforming to the video display interface specification based on the time-encapsulated data, and outputs the video interface signal via the corresponding output port. The video display interface can be at least one of a high-definition video multimedia interface, a display port interface, a digital video interface, or an analog video interface.

[0054] Specifically, when using a high-definition video multimedia interface for output, the first protocol conversion device maps the timing data stream into a physical layer encoded signal specified by the interface and outputs it to the high-definition video multimedia interface port; when using a display port interface for output, the first protocol conversion device outputs the corresponding physical layer signal according to the link training and data channel mapping rules of the display port interface; when using a digital video interface or an analog video interface for output, the first protocol conversion device outputs a signal waveform that matches the electrical characteristics of the corresponding interface.

[0055] In combination with the aforementioned hardware solutions, under certain field conditions, the first protocol conversion device can be implemented using a commercial network video stream to video interface signal converter; under other field conditions, the expansion interface capability of the large screen decoder in the management area can be utilized. By configuring a video display interface expansion card for the large screen decoder, the large screen decoder can undertake the functions of network video stream decoding and video interface signal output, thereby outputting the target channel image to the cross-domain physical video transmission medium through the video interface port.

[0056] In a specific example, taking the monitoring of furnace flames in a waste-to-energy plant as an example, the video from the furnace flame camera is first aggregated by a network video recorder and then released as a network video stream. The first protocol conversion device reads the network access parameters of the target channel, establishes an acquisition connection, parses the media payload data, and completes decoding and frame reconstruction. It generates a continuous display timing sequence according to the preset output resolution and refresh rate, and finally outputs the video interface signal through the high-definition video multimedia interface port and connects to the cross-domain physical video cable. Taking the monitoring of critical equipment as an example, the video from the critical equipment instrument camera is aggregated by a decoder. The first protocol conversion device pulls the network video stream based on the network access parameters corresponding to the channel, completes decoding and timing encapsulation, and then outputs the video interface signal through the display port interface or digital video interface to provide input for subsequent cross-domain transmission and control area reconstruction.

[0057] Through the first protocol conversion process in this embodiment, the network video stream carried by Ethernet can be stably converted into a video interface signal that conforms to the video display interface specification on the management area side, and the display timing can be maintained continuously under the condition that the output parameters are configurable. This reduces the dependence of cross-domain links on network penetration and protocol modification, reduces the development workload of customized receiving software on site, and provides reliable input for subsequent isolated transmission based on physical video transmission media and video reconstruction on the control area side.

[0058] In some embodiments, transmitting video interface signals via a physical video transmission medium that crosses the boundary between the production control area and the management area, so that the video interface signals reach the production control area side, includes: On the management area side, the video interface signal is connected to a physical video transmission medium that matches the video display interface type, and the physical video transmission medium is deployed as a direct link that crosses the network isolation boundary between the production control area and the management area. Physical layer transmission is performed on the video interface signal in the direct link to transmit the video interface signal to the input port on the production control area side, wherein the physical video transmission medium is at least one of high-definition multimedia cable, display port cable, digital video cable or analog video cable; On the production control area side, video interface signals are received based on the input port, and the received video interface signals are used as input for the second protocol conversion process.

[0059] Specifically, after the management area completes the first protocol conversion process, it obtains a video interface signal that conforms to the video display interface specification. The video interface signal is output from the expansion interface output port of the first protocol conversion device or the large-screen decoder. To ensure that the video interface signal maintains interface type consistency during cross-domain transmission, the management area selects a matching physical video transmission medium according to the video display interface type and directly connects the output port to the input end of the physical video transmission medium.

[0060] For example, the physical video transmission medium can be selected from at least one of high-definition multimedia cable, display port cable, digital video cable or analog video cable; in the scenario where the field needs to cross the factory area passage or cable tray over a long distance, the physical video transmission medium can be selected from display port cable in the form of fiber optic cable or high-definition multimedia cable with long-distance transmission capability, so as to reduce transmission attenuation and improve anti-interference capability.

[0061] In some examples, to establish a direct link between the physical video transmission medium and the network isolation boundary, physical access points are installed on both sides of the isolation boundary on the management area and the production control area sides. These physical access points can be located in cabinets adjacent to network gateways or firewalls, but they do not establish a network connection with the gateways or firewalls. On the management area side, the physical video transmission medium is laid along existing cable channels or independent low-voltage channels, and physical connection is achieved at the points crossing the isolation boundary using wall sleeves or isolation cable conduits, structurally forming a direct link from the output port on the management area side to the input port on the production control area side. To avoid accidental connection of network devices or the introduction of unnecessary branches, Ethernet switching and routing devices are not configured in the middle section of the direct link, and no network address configuration is performed.

[0062] In some examples, the physical layer transmission of video interface signals in a direct link is manifested as the continuous transmission of interface electrical or optical signals in the physical video transmission medium. To ensure transmission stability, the management area can install interface latches or port fixing components at the output end of the first protocol conversion device to reduce interface loosening caused by vibration or maintenance operations. In industrial sites with strong electromagnetic interference, the physical video transmission medium can be shielded, and high-voltage cables and high-power frequency converter wiring areas should be avoided when routing cables across cabinets. In scenarios requiring long distances, signal equalization or extension components can be used at both ends of the direct link, but the extension components only perform physical layer signal shaping or multiplexing and do not introduce network protocol processing.

[0063] On the production control area side, the end of the direct link is connected to the input port of the second protocol conversion device or the input port of the video acquisition and encoding device. The interface type of the input port matches the interface type of the output port on the management area side, thereby enabling the production control area side to directly receive video interface signals.

[0064] For example, taking the monitoring of furnace flames in a waste-to-energy plant as an example, the management area can use a large-screen decoder to expand the interface and output video interface signals. These signals cross the isolation boundary via high-definition multimedia cables and then enter the input port of the second protocol conversion device on the production control area side. Similarly, in the case of monitoring critical equipment, the video interface signals output by the critical equipment's instrument cameras after being aggregated by a decoder can cross the isolation boundary via display port cables or digital video cables and connect to the input port on the production control area side. The video interface signals received on the production control area side serve as input for subsequent second protocol conversion processing, used to generate a network video stream for transmission to the production control network and further distribute it to operator stations or intelligent analysis servers.

[0065] Through the physical video transmission medium direct link in this embodiment, cross-boundary transmission of video interface signals can be completed without establishing cross-domain network interconnection, reducing the dependence on network topology modification, protocol penetration and customized receiving software, and maintaining signal transmission stability in long-distance deployment and industrial interference environments, thereby reducing system modification costs and implementation risks, and providing continuous and reliable signal input for video reconstruction and subsequent intelligent analysis applications on the production control area side.

[0066] In some embodiments, a second protocol conversion process is performed on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network, including: The video interface signal arriving at the production control area is input into the second protocol conversion device, and video frame acquisition and timing recovery are performed based on the video interface signal to obtain the acquired video frame sequence; The acquired video frame sequence is encoded and compressed to generate an encoded bitstream corresponding to the preset video encoding format, and media session parameters of the network video stream are generated based on the encoded bitstream. The encoded bitstream is encapsulated according to a preset streaming media transmission protocol to generate a network video stream for transmission in the production control area network, and the network access parameters of the network video stream are output for the operator station or intelligent analysis server to obtain the network video stream. The preset video encoding format includes either video encoding format one or video encoding format two, and the preset streaming media transmission protocol includes the transmission protocol corresponding to the real-time streaming media session control protocol.

[0067] Specifically, a second protocol conversion device is installed near the network isolation boundary on the production control area side. This device includes a video interface input port, a frame acquisition and timing recovery unit, an encoding and compression unit, and a streaming media encapsulation unit. The interface type of the video interface input port matches the video display interface type output from the management area side, enabling it to directly receive video interface signals delivered by the physical video transmission medium. Based on the aforementioned hardware solution, the second protocol conversion device can be implemented using a commercially available video interface signal to network video stream converter, or it can be implemented using an encoder device with video acquisition and encoding capabilities. In scenarios utilizing a large-screen decoder to expand the interface, the second protocol conversion device can be deployed in conjunction with the encoder device, allowing the video interface signal output from the large-screen decoder to reach the production control area via a physical link, where it is then acquired by the encoder and output as a network video stream.

[0068] In some examples, after the video interface signal arriving at the production control area is input to the second protocol conversion device, the second protocol conversion device performs video frame acquisition and timing recovery based on the video interface signal to obtain the acquired video frame sequence. Specifically, the frame acquisition and timing recovery unit recovers the boundary of each frame according to the pixel data and synchronization timing information in the video interface signal, constructs a frame buffer, and divides the continuous pixel stream into video frames according to the frame boundaries. When the video interface signal adopts a digital interface form, the frame acquisition and timing recovery unit recovers the pixel scanning order and frame refresh cycle according to the horizontal synchronization, vertical synchronization, and blanking area timing. When the video interface signal adopts an analog interface form, the frame acquisition and timing recovery unit performs sampling quantization on the input signal and recovers the horizontal and vertical timing according to the synchronization pulse. To adapt to changes in field output parameters, the frame acquisition and timing recovery unit can automatically identify the input resolution and refresh rate, and refresh the timing recovery parameters when parameter changes are detected, keeping the acquired video frame sequence continuously available.

[0069] In some examples, the second protocol conversion device performs encoding and compression processing on the acquired video frame sequence to generate an encoded bitstream corresponding to a preset video encoding format, and generates media session parameters for the network video stream based on the encoded bitstream. The preset video encoding format includes either video encoding format one or video encoding format two. Video encoding format one can correspond to an inter-frame predictive coding format, while video encoding format two can correspond to a high-compression-ratio inter-frame predictive coding format.

[0070] Specifically, the encoding and compression unit selects a preset video encoding format based on the bandwidth constraints of the production control network and the decoding capabilities of the intelligent analysis server. It then inputs the video frame sequence into the encoder, sets the bitrate control parameters, keyframe period parameters, and resolution parameters, and outputs a continuous encoded bitstream. Simultaneously, the encoding and compression unit extracts encoding format identifiers, resolution parameters, frame rate parameters, bitrate parameters, and keyframe marker parameters from the encoding process to generate media session parameters for subsequent streaming media session establishment and decoding parameter configuration. Taking the monitoring of furnace flames in a waste-to-energy plant as an example, to ensure the detailed characteristics of combustion status identification, the encoding and compression unit can select a higher resolution and set a stable bitrate. Similarly, for monitoring critical equipment, to ensure the clarity of instrument character recognition and liquid level interface boundary recognition, the encoding and compression unit can increase the keyframe frequency or improve the encoding quality level, enabling the intelligent analysis server to obtain more stable image input.

[0071] In some examples, the second protocol conversion device encapsulates the encoded bitstream according to a preset streaming media transmission protocol, generating a network video stream for transmission over the production control area network, and outputs network access parameters for the network video stream to be accessed by the operator station or intelligent analysis server. The preset streaming media transmission protocol includes the transmission protocol corresponding to the real-time streaming media session control protocol.

[0072] Specifically, the streaming media encapsulation unit generates session description information based on media session parameters and encapsulates the encoded bitstream into media packets that can be transmitted over Ethernet, forming a network video stream output. Network access parameters include one or more of the following: access address for locating the network video stream, port parameters, and session identifier, and optionally, authentication information. The second protocol conversion device writes the network access parameters to the configuration page on the production control area side or publishes them to the business configuration service in the production control area, enabling the operator station to retrieve and display the network video stream based on the network access parameters. The intelligent analysis server can subscribe to or retrieve the network video stream based on the same network access parameters and perform intelligent recognition and analysis.

[0073] In a specific example, taking the monitoring of furnace flames in a waste-to-energy plant as an example, the video from the furnace flame camera on the management area side is output as a network video stream via a network video recorder. The first protocol conversion process outputs a video interface signal, which crosses the isolation boundary via a high-definition multimedia cable. The second protocol conversion device receives the video interface signal on the production control area side, recovers the video frame sequence, and performs encoding compression and streaming media encapsulation, outputting a network video stream for transmission to the production control network. The intelligent analysis server acquires the network video stream, performs intelligent combustion status identification, and combines it with process data collected within the production control area to generate the data output required for control applications. Taking the monitoring of critical equipment as an example, the video from the critical equipment instrument camera is output as a video interface signal via a decoder and crosses the isolation boundary to reach the production control area side. The second protocol conversion device converts this video interface signal into a network video stream. The intelligent analysis server performs instrument reading identification or fluoroscopy liquid level interface identification on the video stream and provides the identification results to the control application server for alarm or control logic processing.

[0074] Through the second protocol conversion process in this embodiment, the video interface signal arriving across domains can be reconstructed into a distributable network video stream within the production control network on the production control area side. This reduces the control area side's reliance on the original video direct-connection display device and enables the operator station and intelligent analysis server to obtain video input with unified network access parameters. This facilitates joint analysis and business integration with process data within the same network domain, thereby improving the deployability and scalability of cross-domain video access and reducing system transformation costs and implementation risks.

[0075] In some embodiments, the network video stream generated on the production control area side is connected to the production control area network, and the network video stream is provided to the operator station or intelligent analysis server for video viewing or analysis processing, including: The generated network video stream is connected to the switching equipment on the production control area side, and the network video stream is published to the production control area network based on the switching equipment; Send network access parameters corresponding to the network video stream to the operator station or intelligent analysis server so that the operator station or intelligent analysis server can initiate a pull request and obtain the network video stream based on the network access parameters; The operator station decodes and displays the acquired network video stream to output video images, or the intelligent analysis server performs parsing processing on the acquired network video stream to extract video analysis data; The video analytics data is aligned and correlated with the process data in the production control area using a unified time stamp, and the correlation results are output to the control application server to generate control commands or alarm information.

[0076] Specifically, the network video stream output by the second protocol conversion device is connected to the switching equipment on the production control area side via an Ethernet interface. This switching equipment can be an aggregation switch or an access switch within the production control network. To ensure the network video stream can be accessed by multiple nodes within the production control network, the production control network assigns a network address to the second protocol conversion device and connects the port of the second protocol conversion device to the service network segment of the production control network on the switching equipment. For production control networks with multi-service isolation, the port of the second protocol conversion device can also be assigned to a dedicated virtual LAN for video services, logically isolating video service traffic from control command traffic on the switching equipment side. After the network video stream is connected to the switching equipment, it is published within the production control network through the switching equipment's Layer 2 or Layer 3 forwarding capabilities, enabling operator stations, intelligent analysis servers, and control application servers to access the network video stream within the same network domain.

[0077] In some examples, to enable operator stations or intelligent analysis servers to accurately locate and acquire network video streams, the production control area sends network access parameters corresponding to the network video streams to the operator stations or intelligent analysis servers. These network access parameters include one or more of the following: network video stream access address, port parameters, session identifier, and authentication information. Specifically, network access parameters can be generated and displayed locally by the second protocol conversion device, or they can be aggregated and managed by the control application server or the configuration management service of the production control area before being sent out. For example, in furnace flame monitoring, the network access parameters corresponding to the furnace flame channel can be sent to the configuration items on the main control room operator station interface, and simultaneously to the video access configuration items on the intelligent analysis server. Similarly, in critical equipment monitoring, the network access parameters corresponding to the critical equipment instrument channels can be sent to the operator stations of the corresponding positions, and also to the input configuration items on the intelligent analysis server for reading recognition or level recognition tasks. For ease of subsequent maintenance, channel identifiers or channel names can also be included in the network access parameters, allowing the operator station interface to display selectable video sources in the form of channel names.

[0078] In some examples, the operator station initiates a streaming request based on network access parameters and acquires the network video stream. The operator station then decodes and displays the acquired video stream to output the video image. Specifically, the operator station's display component reads the network access parameters, establishes a media session, receives media packets from the network video stream, performs reordering and buffering control, and then calls the decoding component to decode the encoded stream. Finally, the decoded video frames are rendered to the operator station's human-machine interface window. Taking the monitoring of furnace flames in a waste-to-energy plant as an example, the operator station can directly open the furnace flame image on the production control area side for manual verification of the combustion status identification results. For monitoring critical equipment, the operator station can simultaneously display instrument screens or a liquid level interface on a fluoroscopy lens for on-site status confirmation when an alarm is triggered.

[0079] In some examples, the intelligent analysis server initiates a streaming request based on network access parameters and acquires the network video stream. The intelligent analysis server then performs parsing processing on the acquired network video stream to extract video analysis data. Specifically, after establishing a media session, the intelligent analysis server receives the network video stream, decapsulates the media packets to obtain the encoded bitstream, and then decodes it into a video frame sequence. Based on preset analysis tasks, the intelligent analysis server performs feature extraction, target region localization, and state recognition processing on the video frame sequence, outputting video analysis data.

[0080] For example, video analytics data may include one or more of the following: combustion state category, flame area ratio, flame center location, instrument reading, liquid level interface boundary location, or abnormal event marker. Taking the monitoring of furnace flames in a waste-to-energy plant as an example, the intelligent analysis server can output combustion state identification results and attach a sampling time stamp to each identification result; taking the monitoring of critical equipment as an example, the intelligent analysis server can output instrument readings or liquid level interface identification results and attach corresponding time stamps to the identification results for subsequent alignment with process data.

[0081] In some examples, video analytics data is aligned and correlated with process data within the production control area using a unified time stamp, and the correlation results are output to the control application server for generating control commands or alarm information. Specifically, when generating video analytics data, the intelligent analytics server records a time stamp corresponding to each video frame for each analysis result; the process data within the production control area is generated by the control system controller and input / output acquisition, and the process data carries a process time stamp corresponding to the acquisition time. Based on a unified time reference or time synchronization strategy, the control application server maps the time stamps of the video analytics data and the process time stamps of the process data to a correlation key, and performs time alignment and matching according to the correlation key, generating a multi-source correlated data object containing video analytics data and process data.

[0082] The control application server performs control logic processing based on multi-source associated data objects. For example, in the furnace flame monitoring scenario, the combustion status identification result is aligned with process quantities such as oxygen content, furnace negative pressure, and feed rate to generate control quantity adjustment instructions, or an alarm event is generated when the combustion status is abnormal. In the critical equipment monitoring scenario, the instrument reading identification result is consistent with the collected value of the corresponding measuring point. When the deviation exceeds the threshold, an alarm message is generated or the protection logic of the control loop is triggered. The liquid level interface identification result is aligned with the liquid level measuring point data for abnormal liquid level linkage handling.

[0083] Through the network video stream access, publishing, and application integration processing in this embodiment, the network video stream generated on the production control area side can be distributed in a standardized manner within the production control network. The operator station and the intelligent analysis server obtain video input through unified network access parameters and respectively complete the image viewing and intelligent analysis processing. At the same time, the video analysis data and process data are aligned and associated with a unified time stamp and output to the control application server, which improves the availability and business integration of video data in the production control area, enhances the comprehensive monitoring and linkage response capabilities of combustion status and key equipment status, thereby improving the deployability of automated applications and reducing the burden of manual inspection.

[0084] In some embodiments, the video data on which the analysis and processing are based is associated with process data within the production control area, including: When performing analysis and processing on network video streams, determine the timestamps corresponding to the video data and write the timestamps into the frame-level metadata of the video data or the data records associated with the video data; Acquire process data within the production control area, wherein the process data carries a process time stamp corresponding to the acquisition time; A correlation key is generated based on the time stamp and process time stamp, and the video data and process data are time-aligned and matched according to the correlation key to obtain the correlated multi-source data object; Output multi-source data objects to intelligent analysis servers or control application servers for analysis and processing based on the multi-source data objects or to generate data output related to analysis and processing.

[0085] Specifically, when the intelligent analysis server performs analysis and processing on the network video stream, it first determines the timestamp corresponding to the video data and writes the timestamp into the frame-level metadata of the video data or the data record associated with the video data. Specifically, when the intelligent analysis server retrieves the network video stream and completes decapsulation, it can parse the frame time information from the media packets and use the parsed frame time information as the timestamp; when the media packets do not carry directly usable frame time information, the intelligent analysis server can generate a timestamp based on the decoded output frame sequence number and the local synchronization clock, and write this timestamp into the analysis record of each frame.

[0086] In some examples, the timestamp can be written to a frame-level metadata field or an analysis record field associated with a video frame identifier. For instance, in monitoring the furnace flames of a waste-to-energy plant, the intelligent analysis server generates a combustion state category, flame area characteristics, and abnormal event markers for each frame or time window when performing combustion state identification on the furnace flame image, and writes a timestamp to this analysis record. Similarly, in monitoring critical equipment, the intelligent analysis server writes a timestamp to each identification result when performing reading identification on instrument screens or boundary identification on the liquid level interface of a fluoroscopy lens, so that it can be aligned with subsequent measurement data at the same time.

[0087] In some examples, when acquiring process data within the production control area, the process data carries a process timestamp corresponding to the acquisition time. Specifically, the control system controller and input / output acquire process measurement point data and loop status data according to the control cycle or sampling cycle. The process data carries a process timestamp at the acquisition time when it is generated. The process timestamp can be generated by the unified clock of the control system or provided by the time synchronization service within the production control network.

[0088] For example, process data may include time-series data related to the combustion process, such as furnace negative pressure, oxygen content, feed rate, air volume, and steam flow rate, as well as measurement data from key equipment such as pressure, temperature, flow rate, and liquid level, and status quantities such as control loop outputs, valve openings, and alarm statuses. Taking furnace flame monitoring as an example, process data is periodically collected by the control system controller and its inputs and outputs and output to the control application server or data acquisition service; taking key equipment monitoring as an example, process data includes the measurement values ​​corresponding to the monitored instruments and their associated interlocking states, which are collected and recorded by the control system controller and its inputs and outputs.

[0089] In some examples, association keys are generated based on time stamps and process time stamps, and video data and process data are time-aligned and matched according to the association keys to obtain associated multi-source data objects. Specifically, the association keys can be composed of time stamps with unified time granularity. Unified time granularity includes mapping video time stamps and process time stamps to the same time reference and rounding or windowing processing at the same sampling granularity.

[0090] For example, video time stamps can be mapped to time slices corresponding to the control cycle, and process time stamps can be aggregated according to the same time slice, thus using the time slice identifier as the association key. Alternatively, a nearest neighbor matching strategy can be used, with the video time stamp as the primary key, selecting the process time stamp of the closest video time stamp within a preset time deviation threshold, and generating the association key. After the association key is generated, the system merges video analysis records and process data records of the same time slice or near the same moment according to the association key, forming a multi-source data object. The multi-source data object contains at least a video analysis data field, a process data field, and an association key field.

[0091] For example, taking the monitoring of the furnace flame in a waste-to-energy plant as an example, the multi-source data object can include the combustion state identification result and process quantities such as oxygen content, furnace negative pressure, and feed rate within the same time slice; taking the monitoring of key equipment as an example, the multi-source data object can include the instrument reading identification result and the corresponding measurement point acquisition value and interlock status quantity within the same time slice, or include the liquid level interface boundary identification result and the liquid level measurement point value within the same time slice.

[0092] In some examples, multi-source data objects are output to an intelligent analysis server or a control application server for analysis and processing or to generate data outputs related to the analysis and processing. Specifically, multi-source data objects can be formed on the intelligent analysis server and directly output to the control application server, or they can be formed by the control application server itself after receiving video analysis data and process data and aligning and fusing them. The control application server performs control logic processing and event handling based on the multi-source data objects to generate data outputs, which include one or more of the following: control adjustment parameters, control commands, alarm events, or linkage handling strategies.

[0093] For example, taking furnace flame monitoring as an example, the control application server generates combustion control-related data outputs based on the combination relationship between combustion state identification results and process quantities in multi-source data objects, and generates alarm information when abnormal combustion states continue to occur; taking critical equipment monitoring as an example, the control application server generates consistency verification alarms based on the deviation relationship between instrument identification readings and measurement point acquisition values ​​in multi-source data objects, or generates abnormal liquid level linkage handling data outputs based on the correlation relationship between liquid level interface identification results and liquid level measurement point values.

[0094] Through the time stamping and association key mechanism of this embodiment, video data generated by network video streams can be aligned and fused with time-stamped process data in the production control area to form reusable multi-source data objects, which can then be output to intelligent analysis and control applications. This enables video analysis results such as combustion status recognition, instrument reading recognition, and liquid level interface recognition to establish stable associations with process measurement points and loop states within the control cycle, thereby improving the usability of analysis conclusions and the data consistency of control logic. This enhances the integration efficiency of multimodal fusion applications in the production control area and reduces manual review and processing costs.

[0095] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0096] Figure 3 This is a schematic diagram of the structure of a video signal secure transmission device based on video interface protocol conversion provided in an embodiment of this application. Figure 3 As shown, the device includes: The acquisition module 301 is used to acquire at least one network video stream from the management area and determine the target video channel corresponding to the network video stream; The first protocol conversion module 302 is used to perform a first protocol conversion process on the network video stream to convert the network video stream into a video interface signal that conforms to the video display interface specification. Transmission module 303 is used to transmit video interface signals through a physical video transmission medium that crosses the boundary between the production control area and the management area, so that the video interface signals reach the production control area side. The second protocol conversion module 304 is used to perform a second protocol conversion process on the incoming video interface signal on the production control area side, so as to convert the video interface signal into a network video stream for transmission over the production control area network. The access module 305 is used to connect the network video stream generated on the production control area side to the production control area network, and provide the network video stream to the operator station or intelligent analysis server for video viewing or analysis and processing, and associate the video data on which the analysis and processing is based with the process data in the production control area.

[0097] In some embodiments, Figure 3 The acquisition module 301 acquires the channel description information of the network video stream published by at least one video source in the management area. The channel description information includes the video source identifier, the channel identifier, and the network access parameters corresponding to the channel identifier. According to the preset video channel selection strategy, the module selects the target video channel from the channel description information and generates the pull stream configuration corresponding to the target video channel. Based on the pull stream configuration, the module establishes a network video stream acquisition connection to the target video channel to acquire the network video stream corresponding to the target video channel.

[0098] In some embodiments, Figure 3 The first protocol conversion module 302 inputs the network video stream into the first protocol conversion device and obtains media payload data by parsing the session parameters of the network video stream; it performs decoding and frame reconstruction processing on the media payload data to generate video frame data, and performs timing encapsulation on the video frame data according to the preset video output format; it generates a video interface signal that conforms to the video display interface specification based on the timing encapsulated data, and outputs the video interface signal through the output port corresponding to the video display interface, wherein the video display interface is at least one of a high-definition video multimedia interface, a display port interface, a digital video interface, or an analog video interface.

[0099] In some embodiments, Figure 3 The transmission module 303 connects the video interface signal to a physical video transmission medium matching the video display interface type on the management area side, and deploys the physical video transmission medium as a direct link across the network isolation boundary between the production control area and the management area; performs physical layer transmission on the video interface signal in the direct link to transmit the video interface signal to the input port on the production control area side, wherein the physical video transmission medium is at least one of high-definition multimedia cable, display port cable, digital video cable or analog video cable; on the production control area side, the video interface signal is received based on the input port, and the received video interface signal is used as the input for the second protocol conversion processing.

[0100] In some embodiments, Figure 3 The second protocol conversion module 304 inputs the video interface signal arriving at the production control area side into the second protocol conversion device, and performs video frame acquisition and timing recovery based on the video interface signal to obtain the acquired video frame sequence; it performs encoding and compression processing on the acquired video frame sequence to generate an encoded bitstream corresponding to the preset video encoding format, and generates media session parameters for the network video stream based on the encoded bitstream; it encapsulates the encoded bitstream according to the preset streaming media transmission protocol to generate a network video stream for transmission in the production control area network, and outputs the network access parameters of the network video stream for the operator station or intelligent analysis server to obtain the network video stream; wherein, the preset video encoding format includes video encoding format one or video encoding format two, and the preset streaming media transmission protocol includes the transmission protocol corresponding to the real-time streaming media session control protocol.

[0101] In some embodiments, Figure 3The access module 305 connects the generated network video stream to the switching equipment on the production control area side, and publishes the network video stream to the production control area network based on the switching equipment; it sends network access parameters corresponding to the network video stream to the operator station or intelligent analysis server, so that the operator station or intelligent analysis server can initiate a pull request and obtain the network video stream based on the network access parameters; the operator station decodes and displays the obtained network video stream to output video images, or the intelligent analysis server performs parsing processing on the obtained network video stream to extract video analysis data; the video analysis data is aligned and associated with the process data in the production control area according to a unified time stamp, and the association result is output to the control application server for generating control commands or alarm information.

[0102] In some embodiments, Figure 3 When performing analysis and processing on the network video stream, the access module 305 determines the timestamp corresponding to the video data and writes the timestamp into the frame-level metadata of the video data or the data record associated with the video data; it acquires process data within the production control area, wherein the process data carries a process timestamp corresponding to the acquisition time; it generates an association key based on the timestamp and the process timestamp, and performs time alignment and association matching between the video data and the process data according to the association key to obtain the associated multi-source data object; it outputs the multi-source data object to the intelligent analysis server or the control application server for performing analysis and processing based on the multi-source data object or generating data output related to the analysis and processing.

[0103] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0104] Figure 4 This is a schematic diagram of the electronic device 4 provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0105] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or different components.

[0106] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0107] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 4. The memory 402 can also include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which may be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0110] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for secure transmission of video signals based on video interface protocol conversion, characterized in that, Applied to industrial network environments where production control areas and management areas are isolated, including: Obtain at least one network video stream from the management area and determine the target video channel corresponding to the network video stream; Perform a first protocol conversion process on the network video stream to convert the network video stream into a video interface signal that conforms to the video display interface specification; The video interface signal is transmitted through a physical video transmission medium that crosses the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side. On the production control area side, a second protocol conversion process is performed on the arriving video interface signal to convert the video interface signal into a network video stream for transmission over the production control area network. The network video stream generated on the production control area side is connected to the production control area network and provided to the operator station or intelligent analysis server for video viewing or analysis. The video data on which the analysis is based is then associated with the process data within the production control area.

2. The method according to claim 1, characterized in that, The step of acquiring at least one network video stream from the management area and determining the target video channel corresponding to the network video stream includes: Obtain channel description information of network video streams published externally by at least one video source within the management area. The channel description information includes video source identifier, channel identifier, and network access parameters corresponding to the channel identifier. According to the preset video channel selection strategy, the target video channel is selected from the channel description information and a streaming configuration corresponding to the target video channel is generated. Based on the aforementioned streaming configuration, a network video stream acquisition connection is established for the target video channel to obtain the network video stream corresponding to the target video channel.

3. The method according to claim 1, characterized in that, The step of performing a first protocol conversion process on the network video stream to convert the network video stream into a video interface signal conforming to the video display interface specification includes: The network video stream is input into the first protocol conversion device, and media payload data is obtained by parsing the session parameters of the network video stream. Decoding and frame reconstruction processing are performed on the media payload data to generate video frame data, and the video frame data is time-encapsulated according to a preset video output format; Based on the time-encapsulated data, a video interface signal conforming to the video display interface specification is generated, and the video interface signal is output through the output port corresponding to the video display interface, wherein the video display interface is at least one of a high-definition video multimedia interface, a display port interface, a digital video interface, or an analog video interface.

4. The method according to claim 1, characterized in that, The step of transmitting the video interface signal through a physical video transmission medium spanning the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side, includes: On the management area side, the video interface signal is connected to a physical video transmission medium that matches the video display interface type, and the physical video transmission medium is deployed as a direct link that crosses the network isolation boundary between the production control area and the management area. Physical layer transmission is performed on the video interface signal in the direct link to transmit the video interface signal to the input port on the production control area side, wherein the physical video transmission medium is at least one of high-definition multimedia cable, display port cable, digital video cable or analog video cable; The video interface signal is received at the input port on the production control area side, and the received video interface signal is used as the input for the second protocol conversion process.

5. The method according to claim 1, characterized in that, The step of performing a second protocol conversion process on the arriving video interface signal on the production control area side to convert the video interface signal into a network video stream for transmission over the production control area network includes: The video interface signal arriving at the production control area is input into the second protocol conversion device, and video frame acquisition and timing recovery are performed based on the video interface signal to obtain the acquired video frame sequence; The acquired video frame sequence is encoded and compressed to generate an encoded bitstream corresponding to a preset video encoding format, and media session parameters of the network video stream are generated based on the encoded bitstream. The encoded stream is encapsulated according to a preset streaming media transmission protocol to generate a network video stream for transmission in the production control area network, and the network access parameters of the network video stream are output for the operator station or intelligent analysis server to obtain the network video stream. The preset video encoding format includes video encoding format one or video encoding format two, and the preset streaming media transmission protocol includes the transmission protocol corresponding to the real-time streaming media session control protocol.

6. The method according to claim 5, characterized in that, The step of connecting the network video stream generated on the production control area side to the production control area network and providing the network video stream to the operator station or intelligent analysis server for video viewing or analysis includes: The generated network video stream is connected to the switching equipment on the production control area side, and the network video stream is published to the production control area network based on the switching equipment; Send network access parameters corresponding to the network video stream to the operator station or intelligent analysis server, so that the operator station or intelligent analysis server can initiate a pull request and obtain the network video stream based on the network access parameters; The operator station decodes and displays the acquired network video stream to output video images, or the intelligent analysis server performs parsing processing on the acquired network video stream to extract video analysis data. The video analysis data is aligned and associated with the process data in the production control area using a unified time stamp, and the association result is output to the control application server for generating control commands or alarm information.

7. The method according to claim 1, characterized in that, The step of associating the video data used for the analysis and processing with the process data within the production control area includes: When performing analysis and processing on the network video stream, a timestamp corresponding to the video data is determined, and the timestamp is written into the frame-level metadata of the video data or the data record associated with the video data; Acquire process data within the production control area, wherein the process data carries a process time stamp corresponding to the acquisition time; An association key is generated based on the time stamp and the process time stamp, and the video data and the process data are time-aligned and matched according to the association key to obtain the associated multi-source data object; The multi-source data object is output to the intelligent analysis server or control application server so that the analysis process can be performed based on the multi-source data object or data output related to the analysis process can be generated.

8. A secure video signal transmission device based on video interface protocol conversion, characterized in that, Applied to industrial network environments where production control areas and management areas are isolated, including: The acquisition module is used to acquire at least one network video stream from the management area and determine the target video channel corresponding to the network video stream; The first protocol conversion module is used to perform a first protocol conversion process on the network video stream to convert the network video stream into a video interface signal that conforms to the video display interface specification. The transmission module is used to transmit the video interface signal through a physical video transmission medium that crosses the boundary between the production control area and the management area, so that the video interface signal reaches the production control area side. The second protocol conversion module is used to perform a second protocol conversion process on the incoming video interface signal on the production control area side, so as to convert the video interface signal into a network video stream for transmission over the production control area network. The access module is used to connect the network video stream generated on the production control area side to the production control area network, and provide the network video stream to the operator station or intelligent analysis server for video viewing or analysis processing, and associate the video data on which the analysis processing is based with the process data in the production control area.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images