Data transmission method and device, storage medium and computer equipment

Abstract

The embodiment of the invention provides a data transmission method and device, a storage medium and computer equipment. The method comprises the following steps: receiving an industrial data stream uploaded by industrial equipment and a video data stream uploaded by monitoring equipment in real time; converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of the specified protocol; performing clock synchronization on the industrial equipment and the monitoring equipment; allocating different transmission priorities to the industrial conversion data stream and the video conversion data stream; and controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gating list. Compared with the situation that a TSN scheduling mechanism has weak support on the real-time performance and the stability of the video data, resulting in lagging and frame loss of the finally presented video content, the industrial data and the video data can be considered, and lagging and frame loss of the finally presented video content are avoided.

Description

Technical Field

[0001] This application relates to the field of network technologies, and in particular, to a data transmission method, apparatus, storage medium, and computer device. Background Art

[0002] Existing industrial networks usually adopt a traditional hierarchical architecture and use multiple industrial protocols, such as Profinet, EtherCAT, etc. to complete data communication. However, the heterogeneity between these protocols leads to poor data interoperability between different devices. At the same time, with the increasing demand for video surveillance in industrial scenarios, the access of Internet Protocol Version 6 (IPv6) / Internet Protocol Version 4 (IPv4) video streams has become crucial.

[0003] In related technologies, as an efficient real-time communication technology, Time-Sensitive Networking (TSN) can ensure the real-time, reliable, and deterministic data transmission, and gradually becomes the mainstream of the next-generation industrial network. However, the existing TSN scheduling mechanism pays more attention to industrial data and has weak support for the real-time and stability of video data, resulting in frame drops and lags in the finally presented video content. Therefore, related technologies urgently need to propose a data transmission method to solve the above technical problems. Summary of the Invention

[0004] The main purpose of this application is to provide a data transmission method, apparatus, storage medium, and computer device, which can take into account both industrial data and video data and avoid frame drops and lags in the finally presented video content.

[0005] In a first aspect, an embodiment of this application provides a data transmission method, including:

[0006] Receiving in real time an industrial data stream uploaded by an industrial device and a video data stream uploaded by a monitoring device;

[0007] Converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0008] Performing clock synchronization on the industrial device and the monitoring device;

[0009] Assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0010] Controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gate control list.

[0011] In a second aspect, an embodiment of the present application provides a data transmission device, including:

[0012] a receiving unit, configured to receive in real time an industrial data stream uploaded by an industrial device and a video data stream uploaded by a monitoring device;

[0013] a data conversion unit, configured to convert the industrial data stream of an industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and convert the video data stream of an Internet protocol into a video conversion data stream of a specified protocol;

[0014] a clock synchronization unit, configured to perform clock synchronization on the industrial device and the monitoring device;

[0015] an allocation unit, configured to allocate different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0016] a control unit, configured to control the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified in the gating list.

[0017] In a third aspect, an embodiment of the present application provides a storage medium. The computer-readable storage medium stores multiple instructions, and these instructions are suitable for being loaded by a processor to execute the data transmission method as described in any one of the above.

[0018] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the data transmission method as described in any one of the above is implemented.

[0019] In the embodiments of the present application, by receiving the industrial data stream uploaded by industrial devices and the video data stream uploaded by monitoring devices in real time; converting the industrial data stream with the industrial Ethernet protocol into an industrial conversion data stream with a specified protocol, and converting the video data stream with the Internet protocol into a video conversion data stream with a specified protocol; performing clock synchronization on the industrial devices and the monitoring devices; assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream; and controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gate control list. Compared with the related art, the TSN scheduling mechanism pays more attention to industrial data and has weak support for the real-time performance and stability of video data, resulting in frame drops and stuttering in the finally presented video content. It is possible to specify the transmission period of the industrial conversion data stream through the gate control list, so that the industrial conversion data stream can be transmitted according to a fixed transmission period, and different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream, so that the industrial conversion data stream and the video conversion data stream are transmitted in turn according to the transmission priorities in each transmission period, thereby achieving the consideration of both industrial data and video data and avoiding frame drops and stuttering in the finally presented video content.

[0020] Other features and advantages of the present disclosure will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present disclosure. The objectives and other advantages of the present disclosure can be achieved and obtained by the structures specifically pointed out in the specification, the claims, and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 Scenario schematic diagram of the data transmission system provided by the embodiments of the present application.

[0023] Figure 2 Flow schematic diagram of the data transmission method provided by the embodiments of the present application.

[0024] Figure 3 Architecture schematic diagram of the data transmission method provided by the embodiments of the present application.

[0025] Figure 4 Structure schematic diagram of the data transmission device provided by the embodiments of the present application.

[0026] Figure 5 Schematic diagram of the computer device provided by the embodiment of the present application. Detailed implementation manners

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0028] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings, there are multiple steps that appear in a specific order. However, it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish different steps, and the numbers themselves do not represent any execution order. In addition, descriptions such as "first", "second" or "target" in this document are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence.

[0029] Before further elaborating on the embodiments of the present disclosure, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are applicable to the following explanations:

[0030] Time Sensitive Networking (TSN): A network technology used to solve real-time communication problems in scenarios such as industrial control. It improves traditional Ethernet through a series of standard protocols to ensure that data transmission has real-time performance, reliability, and determinacy, and has gradually become the mainstream technology for the next-generation industrial network.

[0031] Tunnel mode conversion: Generally refers to encapsulating a network protocol or data format in another protocol or format for transmission in network communication, so as to achieve communication between different network environments or protocols.

[0032] Tunnel mode conversion encapsulates the original data packet by adding a new header to it and enclosing it in a data packet of another protocol, enabling the original data to be transmitted in an incompatible network. The new header contains information required by the target network, such as routing information, address information, etc. It's like putting a "coat" on the original data, allowing it to be correctly identified and transmitted in different network environments. After reaching the target network, the encapsulated data packet is then unpacked to restore the original data.

[0033] Common tunnel modes include:

[0034] 1. IPSec Tunnel Mode: Used for network security communication, it encrypts the entire IP data packet, including the header and the payload. In the ESP tunnel mode, the IP data packet is encapsulated through the ESP protocol, the IP header, and the ESP authentication trailer. The packet signature part is after the ESP header, and the encrypted part covers all contents except the ESP authentication trailer. The original header is placed after the ESP header, and the ESP trailer is appended before encryption. The entire ESP payload is encapsulated in a new unencrypted tunnel header. It is often used to protect the communication between different networks. For example, when the enterprise headquarters and branch offices communicate through an untrusted Internet, the IPSec tunnel mode is used to ensure data security.

[0035] 2. GRE Tunnel: The Generic Routing Encapsulation protocol can encapsulate data packets of multiple protocols, such as IP, IPX, etc. It provides a logical connection between different subnets without encryption. In an enterprise network, if there are multiple office areas with different network segments, the GRE tunnel can be used to achieve interconnection and facilitate data transmission and resource sharing.

[0036] 3. SRv6 Tunnel: A new network technology based on IPv6, which utilizes the flexibility and scalability of IPv6 to achieve network programming and traffic engineering. By configuring the SRv6TEPolicy tunnel or the SRv6BE tunnel, a cross-site VPN interconnection can be dynamically established. It is suitable for scenarios that require fine control and flexible scheduling of network traffic, such as large data center networks, which can optimize the traffic path and improve the utilization rate of network resources.

[0037] Gate Control List (GCL): A key concept in Time-Sensitive Networking (TSN), it is a traffic scheduling mechanism used to achieve precise scheduling and control of different types of data in the network, ensuring that time-sensitive data can be reliably transmitted within the specified time.

[0038] The gate control list works based on the principle of Time Division Multiplexing (TDM). The time axis is divided into periodic time intervals called cycles. Each cycle is further divided into multiple time slots. A gate control list is configured for each port or link, and the list specifies the traffic class or queue allowed to be transmitted in each time slot. By controlling the opening and closing of the port in each time slot, it determines which data can be transmitted at a specific time.

[0039] However, the existing TSN scheduling mechanisms pay more attention to industrial data and have weak support for the real-time performance and stability of video data, resulting in frame drops and stuttering in the finally presented video content.

[0040] In order to solve the above problems, the embodiments of the present application receive the industrial data stream uploaded by industrial devices and the video data stream uploaded by monitoring devices in real time; convert the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and convert the video data stream of the Internet protocol into a video conversion data stream of a specified protocol; perform clock synchronization on the industrial devices and the monitoring devices; assign different transmission priorities to the industrial conversion data stream and the video conversion data stream; based on the transmission period of the industrial conversion data stream specified by the gating list, control the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities. Compared with the related technology, the TSN scheduling mechanism pays more attention to industrial data and has weak support for the real-time performance and stability of video data, resulting in frame drops and lags in the finally presented video content. By specifying the transmission period of the industrial conversion data stream through the gating list, the industrial conversion data stream can be transmitted according to a fixed transmission period, and different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream, so that the industrial conversion data stream and the video conversion data stream are transmitted in turn according to the transmission priorities in each transmission period, thereby achieving the consideration of both industrial data and video data and avoiding frame drops and lags in the finally presented video content.

[0041] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the scenario of the data transmission system provided by the embodiments of the present application. It includes a device layer, an intermediate layer, a control layer, etc.

[0042] The device layer consists of industrial controllers, industrial monitoring cameras, and various industrial controllers. Devices such as robotic arms and industrial monitoring camera devices (such as cameras) belong to the devices in this layer. The industrial controllers communicate with the TSN gateway in the intermediate layer through industrial Ethernet protocols such as Profinet and EtherCAT, and the industrial monitoring camera devices communicate with the TSN gateway in the intermediate layer through Internet protocols such as Internet Protocol Version 6 (IPv6) / Internet Protocol Version 4 (IPv4). Its role is to serve as the basis of the industrial system, responsible for collecting various data in the production site, such as physical quantities sensed by sensors such as temperature and pressure; at the same time, executing control instructions, such as the actuator performing mechanical actions according to the instructions to achieve production operations.

[0043] The middle layer (TSN device) consists of a TSN gateway and a TSN switch. The TSN gateway is responsible for converting industrial protocol data and video streams into data streams that conform to the TSN standard, and the TSN switch performs priority scheduling on the data streams according to the 802.1Qbv and 802.1Qbu standards. Its role is the hub of data transmission. The switching device is responsible for data forwarding and switching, and the access device is used to connect various devices in the device layer, enabling the data in the device layer to be stably and real - time transmitted to the control layer, and also conveying the control layer instructions to the device layer to ensure smooth and efficient network communication.

[0044] The control layer covers modules such as intelligent applications, quality management, data analysis, and process management. Its role is at the top of the industrial system, performing in - depth analysis and processing on the data collected by the device layer. Through intelligent applications, the production process is optimized, the product quality is controlled using the quality management module, potential problems are mined through data analysis, and the stability of the production process is ensured based on process management, thereby realizing intelligent control of the entire industrial production process, and simultaneously processing video streams in real - time for monitoring and display.

[0045] The data transmission method of the embodiments of the present disclosure can be implemented on a TSN device.

[0046] It should be noted that Figure 1 The schematic diagram of the scenario of the data transmission system shown is only an example. The data transmission system and scenario described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art know that with the evolution of image processing technology and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0047] In this embodiment, it will be described from the perspective of a data transmission device, which can be specifically integrated in a computer device with a storage unit and installed with a microprocessor and having computing capabilities.

[0048] Please refer to Figure 2 , Figure 2 which is the flowchart of the data transmission method provided by the embodiments of the present application. The data transmission method includes:

[0049] In step 201, the industrial data stream uploaded by industrial devices and the video data stream uploaded by monitoring devices are received in real - time.

[0050] Among them, industrial equipment refers to the equipment used to achieve various production functions in the industrial production process, such as machine tools, robots, sensors, etc. They can generate data related to the production process, such as temperature, pressure, rotational speed, position information, etc. Industrial data stream is a set of data generated by industrial equipment during operation, and these data reflect the real-time status and operating parameters of industrial production, and are transmitted in the form of a stream through industrial Ethernet protocols such as Profinet and EtherCAT.

[0051] Monitoring equipment is used to monitor the industrial production environment, equipment operating status, etc. in real time. Common examples include industrial cameras. Video data stream is the continuous video image data collected by monitoring equipment (such as cameras), and is transmitted in the form of a stream through Internet protocols such as IPv6 / IPv4 for real-time monitoring of the production site situation.

[0052] In step 202, the industrial data stream of the industrial Ethernet protocol is converted into an industrial conversion data stream of a specified protocol, and the video data stream of the Internet protocol is converted into a video conversion data stream of a specified protocol.

[0053] Among them, in order to support the industrial data streams uploaded by various industrial Ethernet protocol devices such as Profinet and EtherCAT, and at the same time be compatible with the video data streams of the IPv6 / IPv4 protocol, it is necessary to convert the data streams of different protocols into data streams of the same protocol. Since this application uses TSN equipment, the industrial data streams of different industrial Ethernet protocols are converted into industrial conversion data streams of a specified protocol (TSN standard data stream protocol) through the TSN gateway in the TSN equipment, and the video data streams of the Internet protocol are converted into video conversion data streams of a specified protocol (TSN standard data stream protocol), so that the protocols of the industrial conversion data stream and the video conversion data stream are both the specified protocol for subsequent data stream control.

[0054] Specifically, the data stream converted to the specified protocol can be mapped to the specified protocol (TSN standard data stream protocol) in tunnel mode. For the industrial data streams of industrial Ethernet protocols such as Profinet and EtherCAT, they are parsed to extract the valid data payloads. Then, according to the format requirements of the TSN standard data stream protocol, new header information (including TSN-related parameters such as timestamps and priority identifiers) is added, and the valid data is encapsulated into data frames in TSN format. For example, for the real-time data of Profinet, the key control instruction data part is extracted and encapsulated into the corresponding data segment in the TSN frame. For the video data streams of Internet protocols such as IPv6 / IPv4, the video data streams of the IPv6 / IPv4 protocol are parsed to separate the valid payload part of the video data, such as video coding data, timestamps, etc. According to the requirements of the TSN standard data stream protocol, new headers are added to the parsed video data. The headers contain TSN-related information such as timestamps (for clock synchronization and transmission scheduling), etc. The video data after adding the headers is encapsulated into TSN data frames according to the TSN protocol specifications so that it can be transmitted in the TSN network.

[0055] In step 203, clock synchronization is performed for industrial devices and monitoring devices.

[0056] Among them, to ensure the timestamp consistency between the industrial conversion data stream and the video conversion data stream, it is necessary to implement clock synchronization between devices based on the IEEE802.1AS protocol.

[0057] The specific clock synchronization method is as follows: Designate one device as the master clock, and other industrial devices and / or monitoring devices as slave clocks. The master clock provides an accurate time reference, and the slave clocks adjust their own clocks to be synchronized with the master clock by communicating with the master clock. The specific clock synchronization process is that the master clock periodically sends time synchronization messages. After receiving the messages, the slave clocks measure the transmission delay of the messages and adjust their own clocks according to the delay information. Eventually, through multiple interactions and adjustments, accurate clock synchronization is gradually achieved.

[0058] In step 204, different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream.

[0059] Among them, different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream according to factors such as the importance and real-time requirements of the data. Data with high transmission priority has the right to be processed and transmitted preferentially in network transmission to ensure that it can reach the destination in time.

[0060] During the data encapsulation process, priority tags (VLAN tags) are added to the industrial conversion data stream and the video conversion data stream.

[0061] In some embodiments, allocating different transmission priorities to the industrial conversion data stream and the video conversion data stream includes:

[0062] (1) Screening out key frame conversion data and non-key frame conversion data from the video conversion data stream;

[0063] (2) Allocating a first transmission priority to the industrial conversion data stream;

[0064] (3) Allocating a second transmission priority to the key frame conversion data;

[0065] (4) Allocating a third transmission priority to the non-key frame conversion data.

[0066] Among them, in the embodiments of the present application, in order to ensure the real-time performance and stability of video display, key frame recognition is performed on the video conversion data stream to screen out key frame conversion data and non-key frame conversion data from the video conversion data stream. A key frame is a term in computer animation, referring to the frame at which a key action occurs during the movement or change of a character or object, which is equivalent to the original painting in two-dimensional animation. In video processing, it is a video frame with representational significance, which can be used to concisely express video content and is widely used in fields such as video compression storage and structured indexing. The following are some common key frame recognition methods:

[0067] Based on shot boundary detection

[0068] Principle: A video is composed of different shots. The frames at the shot transition points often have significant content changes, and such frames are regarded as key frames. By calculating the differences between adjacent frames, such as changes in visual features such as color, texture, and shape, if the difference exceeds a certain threshold, it is determined as a shot boundary, and the corresponding frame is a key frame.

[0069] Based on motion analysis

[0070] Principle: Calculate the motion vectors of the objects in the video frames. The frames with large changes in motion vectors indicate significant changes in the motion state of the scene or object and may be key frames. For example, in a sports event video, actions such as a player running fast and suddenly changing direction will result in significant changes in the motion vectors of the corresponding frames.

[0071] Based on image features

[0072] Principle: Extract the image features of the video frames, such as color histograms, texture features, edge features, etc. Compare the features of adjacent frames, and the frames with large feature differences are identified as key frames. For example, when the scene in the video changes from day to night, there will be obvious changes in the color histogram.

[0073] Based on deep learning

[0074] Principle: Construct a deep neural network model, such as a convolutional neural network (CNN). Train the model with a large amount of video data with labeled key frames so that it can learn the key frame feature patterns. The trained model can then identify key frames in new video frames.

[0075] Specifically, according to the characteristics and application requirements of the industrial conversion data stream and the video conversion data stream, determine the priority evaluation criteria. For example, the industrial conversion data stream has a high requirement for real-time performance and should be assigned a higher priority (i.e., the first transmission priority); while the video frame data in the video conversion data stream has a relatively low requirement for real-time performance and can be assigned a lower priority. For the key frame conversion data, its importance is lower than that of the industrial conversion data stream but higher than that of the non-key frame conversion data. Therefore, it is assigned the second transmission priority second only to the first transmission priority; for the non-key frame conversion data, its importance is the lowest, so it is assigned the third transmission priority lower than the second transmission priority.

[0076] In this way, different transmission priorities are assigned according to the importance and real-time requirements of the data, making reasonable use of the network resource bandwidth. For non-key frame conversion data with low real-time requirements, it is transmitted when the network resource bandwidth is sufficient, avoiding competing for bandwidth with high-priority data and improving the utilization rate of network resources. In industrial production, the network traffic may fluctuate. Through this transmission priority allocation mechanism, the data transmission strategy can be flexibly adjusted under different network load conditions, enabling more effective utilization of network resources.

[0077] In step 205, based on the transmission period of the industrial conversion data stream specified by the gating list, control the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities.

[0078] Among them, the gating list specifies the transmission period of the industrial conversion data stream with high real-time requirements. After limiting the transmission period of the industrial conversion data stream, the video conversion data stream can be transmitted within the remaining transmission duration of the transmission period.

[0079] Specifically, the gating list is a core component of the time-aware shaping mechanism (TAS) in a time-sensitive network (TSN), which is a mechanism for controlling the queue transmission switch based on time. Each port has an ordered list of gating operation states, which contains several nodes. Each node specifies the gating state (0 represents closed, 1 represents open) of each queue and its respective execution time. During the execution time, according to the transmission selection algorithm (such as the strict priority algorithm), the data packets in the queues with a state of 1 are forwarded one by one in order of decreasing priority. When the current node finishes running, it moves to the next node for execution. After all nodes have been executed, it returns to the first node and loops. In this way, the "isolation" of the transmission of different priority queues is achieved, ensuring the accurate forwarding of high-priority services and providing ultra-low latency and jitter guarantees for services with strong real-time requirements such as industrial control. First, clarify the parameters such as the data frame length and transmission rate of the industrial conversion data stream. Consider the requirements of the actual industrial production business for data timeliness and accuracy. If the industrial equipment control instructions need to be conveyed in a timely manner to ensure the precise operation of the equipment, it is required that the industrial conversion data stream complete the transmission within a short time interval. Evaluate the network bandwidth, load, etc. If the network bandwidth is limited and the load is high, to avoid congestion, it may be necessary to appropriately extend the transmission cycle; if the bandwidth is sufficient and the load is low, the transmission cycle can be shortened. According to the determined transmission cycle above, set it in the gating list. During the cycle of the gating list, allocate a dedicated transmission time slot for the industrial conversion data stream to ensure its transmission within the specified cycle.

[0080] In some embodiments, before controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities, it further includes:

[0081] (1) Perform lossless encoding or low-compression ratio encoding on the key frame conversion data to obtain the encoded key frame conversion data;

[0082] (2) Perform high-compression ratio encoding on the non-key frame conversion data to obtain the encoded non-key frame conversion data;

[0083] Controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission cycle of the industrial conversion data stream specified by the gating list includes:

[0084] (3) Control the industrial conversion data stream to be preferentially transmitted within the transmission cycle of the industrial conversion data stream specified by the gating list;

[0085] (4) Obtain the first remaining transmission duration between the target time point when the industrial conversion data stream finishes transmission and the next transmission cycle;

[0086] (5) Control the transmission of the encoded key-frame conversion data within the first remaining transmission duration;

[0087] (6) When the encoded key-frame conversion data is transmitted within the first remaining transmission duration and there is a second remaining transmission duration, control the transmission of the encoded non-key-frame conversion data within the second remaining transmission duration.

[0088] Among them, lossless encoding or low-compression-ratio encoding of the key-frame conversion data is to minimize the data volume while retaining the information of the key frames to the greatest extent, so as to ensure that the key content of the video can be accurately displayed and meet the real-time and stability requirements of video display. Some classic lossless encoding algorithms or low-compression-ratio encoding algorithms can be used, such as lossless encoding methods like Huffman encoding and arithmetic encoding, or low-compression-ratio encoding algorithms that can better retain image details during the compression process. Taking Huffman encoding as an example, it constructs a Huffman tree by statistically analyzing the frequencies of different data values in the key-frame conversion data, represents the data with a high frequency with a shorter code, and represents the data with a low frequency with a longer code, thereby realizing data compression while ensuring information losslessness.

[0089] High-compression-ratio encoding of the non-key-frame conversion data is to minimize the data volume of the non-key frames as much as possible without affecting the overall understanding and fluency of the video, so as to save network resource bandwidth and make it more flexible in network transmission. Some efficient high-compression-ratio encoding algorithms can be selected, such as the JPEG compression algorithm based on the discrete cosine transform (DCT) or the high-compression-ratio mode of video compression algorithms based on motion estimation and compensation (such as H.264 / AVC, H.265 / HEVC, etc.). Taking H.264 as an example, it performs intra-frame prediction and inter-frame prediction on non-key frames, utilizes the spatial redundancy and temporal redundancy information in the video, divides the image into small blocks for encoding, and compresses the regions with less change using a larger quantization parameter, thereby achieving a high compression ratio.

[0090] After encoding the key-frame conversion data and the non-key-frame conversion data, it is necessary to control the transmission of three types of data with different transmission priorities: industrial conversion data stream, key-frame conversion data, and non-key-frame conversion data. The transmission period of the industrial conversion data stream is specified by the gating list, so control the industrial conversion data stream to be transmitted within the transmission period specified by the gating list. For example, transmit the industrial conversion data stream every 100 milliseconds. After transmitting the industrial conversion data stream within the current transmission period, obtain the remaining transmission duration between the current time period and the next transmission period, and transmit the key-frame conversion data and the non-key-frame conversion data within the remaining transmission duration.

[0091] Specifically, since the second transmission priority of the key-frame conversion data is higher than the third transmission priority of the non-key-frame conversion data, the encoded key-frame conversion data is controlled to be transmitted within the first remaining transmission duration. When the encoded key-frame conversion data is transmitted within the first remaining transmission duration and there is a second remaining transmission duration, the encoded non-key-frame conversion data is controlled to be transmitted within the second remaining transmission duration.

[0092] For example, when a 100-millisecond transmission cycle starts, the system sends the industrial conversion data stream to the network for transmission. Assume that in this example, the industrial conversion data stream is transmitted within 80 milliseconds. At this time, the system obtains the remaining transmission duration between the current time period (at 80 milliseconds) and the next transmission cycle (the start of the next 100-millisecond cycle, that is, 20 milliseconds later), which is 20 milliseconds (the first remaining transmission duration). Since the priority of the key-frame conversion data is higher than that of the non-key-frame conversion data, the system preferentially controls the encoded key-frame conversion data to be transmitted within the 20-millisecond first remaining transmission duration. Assume that the amount of key-frame conversion data is small and it is transmitted within 15 milliseconds. At this time, there is still a remaining transmission duration of 5 milliseconds (the second remaining transmission duration). Detecting the existence of the second remaining transmission duration, the encoded non-key-frame conversion data is then controlled to be transmitted within the 5-millisecond second remaining transmission duration. When the next 100-millisecond transmission cycle starts, the industrial conversion data stream will be preferentially transmitted again, repeating the above process.

[0093] In this way, by strictly controlling its transmission within the transmission cycle specified in the gating list, the timely and accurate transmission of key data such as control instructions and device status feedback in industrial production is ensured. This helps to maintain the normal operation of industrial equipment and avoid production accidents, equipment failures, or product quality problems caused by data transmission delays or errors, guaranteeing the continuity and stability of the industrial production process. Lossless encoding or low-compression-ratio encoding is performed on the key-frame conversion data, maximizing the retention of key-frame information and enabling the accurate display of the key content of the video, meeting the real-time and stability requirements of video display. In scenarios such as industrial monitoring, key frames often contain important information such as the key status and abnormal conditions of equipment operation. Accurately displaying this content helps operators to detect problems in a timely manner and take corresponding measures. High-compression-ratio encoding is used for non-key-frame conversion data, reducing the amount of data and saving network and bandwidth resources without affecting the overall understanding and fluency of the video, making the video data more flexible and efficient in network transmission while also ensuring the basic viewing experience of the video.

[0094] According to the different priorities and characteristics of the data, the transmission time and method are reasonably allocated, effectively utilizing the network resource bandwidth. The industrial conversion data stream is preferentially transmitted within the specified period to ensure that it is not interfered; within the remaining time after the industrial conversion data stream is transmitted, the transmission is carried out in the priority order of the key-frame conversion data and the non-key-frame conversion data. This method avoids the bandwidth competition between high-priority data and low-priority data, enabling a more reasonable allocation of the network resource bandwidth among different types of data, improving the utilization rate of network resources. Especially in the case of limited network resource bandwidth, it can better meet the transmission requirements of various data.

[0095] In some embodiments, after controlling the transmission of the encoded non-key-frame conversion data within the second remaining transmission duration, it further includes:

[0096] When it is detected that the encoded non-key-frame conversion data has not been transmitted completely and there are other industrial conversion data streams that need to be transmitted, the transmission of the encoded non-key-frame conversion data is prohibited, and the other industrial conversion data streams are controlled to be transmitted.

[0097] Among them, for the situation where the encoded non-key-frame conversion data has not been transmitted within the second remaining transmission duration due to low network resource bandwidth or a large data stream of non-key-frame conversion data, resulting in some encoded non-key frames remaining in the transmission queue, if the next transmission cycle arrives and other industrial conversion data streams need to be transmitted at this time, the transmission of the encoded non-key-frame conversion data is prohibited in the priority order of the transmission priority, and other industrial conversion data streams are preferentially controlled to be transmitted, so as to ensure the transmission of the industrial conversion data stream even under high load.

[0098] Thus, when it is detected that the encoded non-key-frame conversion data has not been transmitted completely and there are other industrial conversion data streams that need to be transmitted, the industrial conversion data stream is preferentially controlled to be transmitted. The industrial conversion data stream usually includes key control instructions, equipment status information, etc. in industrial production, and has extremely high requirements for real-time performance and accuracy. In the case of limited network resource bandwidth or high load, this strategy ensures that the data related to industrial production can be transmitted in a timely manner, avoids the impact on industrial production caused by non-key data occupying the bandwidth, guarantees the continuity and stability of industrial production, and reduces the risk of production accidents and equipment failures.

[0099] In some embodiments, the method further includes:

[0100] When it is detected that the encoded non-key-frame conversion data has not been transmitted completely and there are other encoded key-frame conversion data that need to be transmitted, the transmission of the encoded non-key-frame conversion data is prohibited, and the other encoded key-frame conversion data is controlled to be transmitted.

[0101] Among them, in the case where the encoded non-key frame conversion data fails to be transmitted within the second remaining transmission duration due to low network resource bandwidth or a large data stream of the encoded non-key frame conversion data after encoding, resulting in some encoded non-key frame conversion data remaining in the transmission queue, if the next transmission cycle arrives and other encoded key frame conversion data streams need to be transmitted at this time, the transmission of the encoded non-key frame conversion data is prohibited in the order of the transmission priority, and the transmission of other encoded key frame conversion data is preferentially controlled, so as to ensure the video quality even under high load.

[0102] In this way, when the network resource bandwidth is limited, especially under high load. Prioritizing the transmission of key frame data can concentrate the limited bandwidth resources on the most important data, avoiding non-key data from occupying too much bandwidth and causing delays in the transmission of key data. This dynamic bandwidth allocation strategy can improve the utilization efficiency of network resources and enable the network to still operate stably under high load. Prohibiting the unfinished non-key frames from continuing to occupy the transmission time can avoid further backlog of data in the transmission queue. This helps to maintain the high efficiency of network transmission and prevent network congestion and data loss caused by an overly long queue.

[0103] In some embodiments, controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration includes:

[0104] (1.1) Obtaining the current network resource bandwidth in real time;

[0105] (1.2) When the current network resource bandwidth reaches the preset network resource bandwidth, during the process of controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration, the encoded non-key frame conversion data is discarded according to a preset discard period.

[0106] Among them, during the process of controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration, the current network resource bandwidth is obtained in real time. If the current network resource bandwidth reaches the preset network resource bandwidth, it indicates that the network load is large at this time. During the process of controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration, the encoded non-key frame conversion data is discarded according to a preset discard period. Thus, the effect of reducing the frame rate is achieved, for example, reducing from the original 30 frames per second to 15 frames per second.

[0107] As a result, when the network load is large, network resources become tight. Industrial conversion data streams and key frame conversion data have extremely high requirements for real-time and accuracy, and their transmission needs to be guaranteed first. By discarding some of the encoded non-key frame conversion data, more network bandwidth can be freed up for key data, ensuring that key data such as industrial production control instructions and key monitoring images can be transmitted in a timely and stable manner, thereby ensuring the normal operation of industrial production and the timely presentation of key video information. Although some of the encoded non-key frame conversion data is discarded, the basic coherence of the video can still be maintained to a certain extent because the encoded non-key frames themselves have a relatively small impact on the overall understanding of the video. For example, from 30 frames per second to 15 frames per second, although the frame rate has dropped, viewers can still roughly understand the content of the video, and there will be no serious video freezes or incomprehensible situations.

[0108] In some embodiments, the method further comprises:

[0109] When the current network resource bandwidth does not reach the preset network resource bandwidth, discarding the encoded non-key frame conversion data according to the preset discarding period is prohibited.

[0110] Among them, if the current network resource bandwidth does not reach the preset network resource bandwidth, it means that the network load is small at this time, and there is no need to transmit the encoded non-key frame conversion data by reducing the frame rate. Therefore, it is prohibited to discard the encoded non-key frame conversion data according to the preset discard period.

[0111] Industrial equipment commissioning and maintenance support

[0112] Video streams are used to monitor the operating status of industrial equipment in real time, and to record the debugging process by combining industrial control instructions. The consistency of timestamps between video streams and industrial data enables accurate location of the time when a fault occurs. Through log analysis and video backtracking, the root cause of the problem can be quickly discovered.

[0113] Therefore, when the network load is small, that is, the current network resource bandwidth does not reach the preset network resource bandwidth, not discarding non-key frame conversion data can ensure the stability of the video frame rate, thereby providing a higher quality video display effect. For example, in a video surveillance system, it can present a smoother and clearer picture, which helps operators to observe and judge the on-site situation more accurately. For some application scenarios with high requirements for video quality, not reducing the frame rate can ensure the integrity and details of the video picture and improve the user experience.

[0114] As described above, in the embodiments of the present application, industrial data streams uploaded by industrial devices and video data streams uploaded by monitoring devices are received in real time; the industrial data streams in the industrial Ethernet protocol are converted into industrial conversion data streams in a specified protocol, and the video data streams in the Internet protocol are converted into video conversion data streams in a specified protocol; clock synchronization is performed on the industrial devices and the monitoring devices; different transmission priorities are assigned to the industrial conversion data streams and the video conversion data streams; based on the transmission period of the industrial conversion data streams specified in the gate control list, the industrial conversion data streams and the video conversion data streams are controlled to be transmitted according to the corresponding transmission priorities. Compared with the related technologies, the TSN scheduling mechanism pays more attention to industrial data and has weak support for the real-time performance and stability of video data, resulting in frame freezing and loss in the finally presented video content. By specifying the transmission period of the industrial conversion data streams through the gate control list, the industrial conversion data streams can be transmitted according to a fixed transmission period, and different transmission priorities are assigned to the industrial conversion data streams and the video conversion data streams, so that the industrial conversion data streams and the video conversion data streams are transmitted in turn according to the transmission priorities in each transmission period, thereby achieving the consideration of both industrial data and video data and avoiding frame freezing and loss in the finally presented video content.

[0115] The following describes the compression, decompression, and frame rate adjustment mechanisms for video frame data provided by the embodiments of the present application.

[0116] (1) Video Compression and Decompression

[0117] The H.265 / HEVC encoding technology is adopted, which can save about 50% of the bandwidth compared with H.264 while maintaining the same picture quality.

[0118] Low-latency optimization is performed within the TSN network, combined with a fast motion estimation algorithm, to improve the compression efficiency and reduce the encoding delay.

[0119] Combined with the frame priority scheduling strategy, different compression ratios are applied to video data streams with different priorities:

[0120] P2 (key frame data): Lossless encoding or low-compression ratio encoding is adopted to ensure the picture clarity.

[0121] P3 (non-key frame data): High-compression ratio encoding is adopted to reduce the bandwidth occupancy.

[0122] (2) Frame Rate Dynamic Adjustment

[0123] Adaptive adjustment based on network load

[0124] Monitor the bandwidth occupancy of the TSN network and automatically adjust the frame rate of the video stream.

[0125] Reduce the frame rate of non-critical frames under high load conditions (e.g., from 30 FPS to 15 FPS) to ensure the priority transmission of key frames.

[0126] Restore full frame rate transmission under low load conditions to improve the smoothness of the video stream.

[0127] Key frame priority strategy

[0128] Combine with TSN time synchronization (IEEE 802.1AS) to ensure the timing consistency of key frames and industrial control data.

[0129] Key frames remain at a high priority during network congestion to ensure the integrity of video data.

[0130] The following describes the priority scheduling strategy provided by the embodiments of the present application:

[0131] (1) Priority division of data streams

[0132] To meet the real-time requirements of industrial conversion data and optimize the transmission quality of the video stream, the embodiments of the present application divide the data into three categories according to priority:

[0133] Highest priority (P1): Industrial conversion data to ensure millisecond-level real-time transmission of control instructions;

[0134] Medium priority (P2): Key frame conversion data to ensure that the core data of the video stream is not lost;

[0135] Lowest priority (P3): Non-critical frame conversion data (ordinary video stream frames), which can sacrifice part of the frame rate when the bandwidth is tight.

[0136] (2) Optimization mechanism of TAS + frame preemption

[0137] Optimization of Time-Aware Scheduling (TAS):

[0138] Use the Gate Control List (GCL) of IEEE 802.1Qbv to control the transmission windows of data streams with different priorities.

[0139] P1 (industrial conversion data) uses a fixed transmission cycle and ensures its priority transmission in all transmission cycles.

[0140] P2 (key frame conversion data) is scheduled during the transmission cycles not occupied by P1 to ensure that key frames can be transmitted with the minimum delay.

[0141] P3 (non-critical frame conversion data) is only transmitted when the bandwidth is sufficient to avoid affecting the real-time performance of industrial conversion data and key frame conversion data.

[0142] Optimization of the frame preemption mechanism:

[0143] Combined with IEEE 802.1Qbu, it allows high-priority data (P1 / P2) to be inserted during the transmission of low-priority data (P3), thereby avoiding low-priority frames from blocking high-priority data.

[0144] Specific strategy:

[0145] When P1 arrives, immediately preempt the transmission of the current P3 frame.

[0146] When P2 arrives, if the currently transmitted frame is a P3 frame, preempt it preferentially to ensure that the priority of critical-frame conversion data is higher than that of non-critical-frame conversion data.

[0147] When P3 adopts the fragmentation transmission method, it is transmitted intermittently in the idle time slice to improve the bandwidth utilization rate.

[0148] Specifically, please refer to Figure 3 , Figure 3 which is the schematic architecture diagram of the data transmission method provided by the embodiment of this application. Figure 3Between Zhongyuan industrial control equipment and video equipment, the video surveillance system and industrial cameras communicate through IPv4 / 6 data streams, and the industrial control PLC interacts with servo motors and IO modules based on EtherCAT and PROFINET data streams respectively. In the solution, the data streams of these different protocols need to be converted. For example, IPv4 / 6 video data streams and industrial data streams of industrial Ethernet protocols need to be converted into TSN standard data stream protocols through tunnel mode for efficient transmission under a unified network architecture. Different devices use different protocols, generating heterogeneous data, such as industrial conversion data streams and video conversion data streams. This requires an architecture like the industrial heterogeneous data fusion measurement platform to integrate and process various types of data, and the TSN network plays a key role in it, enabling the fusion transmission of data of different protocols. The TSN device converts the data streams of multiple industrial Ethernet protocols and Internet protocols into TSN standard data stream protocols, and the TSN network in the figure is the key to realizing this conversion and fusion. As the core hub, it connects various devices, allowing data of different protocols to be transmitted under a unified standard, improving network compatibility and data transmission efficiency. The network line load schematic diagram in the figure shows the usage of IPv4 / 6 bandwidth and industrial control bandwidth. The control strategy for data transmission in the solution is closely related to network load. For example, when controlling the transmission of non-critical frame conversion data, the network resource bandwidth is obtained in real time. When the bandwidth reaches the preset value (i.e., the network load is large), data is discarded according to the preset discard period to reduce the frame rate and avoid network congestion, ensuring the transmission of critical data. From the load schematic diagram, the industrial control bandwidth communicates in a constant cycle, indicating a high real-time requirement for industrial control data transmission. Prioritizing the guarantee of the transmission of industrial conversion data streams and the transmission strategy of critical frame conversion data when the network load changes are in line with the importance of industrial control data reflected in the figure, ensuring the stable transmission of industrial production and video critical information.

[0149] For the specific implementation of each of the above steps, reference may be made to the previous embodiments and will not be elaborated here.

[0150] To facilitate the better implementation of the data transmission method provided in the embodiments of the present application, the embodiments of the present application also provide a device based on the above data transmission method. The meanings of the nouns are the same as those in the above data transmission method, and the specific implementation details can refer to the description in the method embodiments.

[0151] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of the data transmission device provided in the embodiments of the present application. The data transmission device is applied to a computer device. The data transmission device may include a receiving unit 601, a data conversion unit 602, a clock synchronization unit 603, a distribution unit 604, and a control unit 605, etc.

[0152] A receiving unit 601 for receiving in real time the industrial data stream uploaded by industrial devices and the video data stream uploaded by monitoring devices;

[0153] A data conversion unit 602 for converting the industrial data stream with an industrial Ethernet protocol into an industrial conversion data stream with a specified protocol, and converting the video data stream with an Internet protocol into a video conversion data stream with a specified protocol;

[0154] A clock synchronization unit 603 for performing clock synchronization on the industrial devices and the monitoring devices;

[0155] An allocation unit 604 for allocating different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0156] A control unit 605 for controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified in the gating list.

[0157] In some embodiments, the allocation unit 604 includes:

[0158] A screening subunit for screening out key frame conversion data and non-key frame conversion data from the video conversion data stream;

[0159] A first allocation subunit for allocating a first transmission priority to the industrial conversion data stream;

[0160] A second allocation subunit for allocating a second transmission priority to the key frame conversion data;

[0161] A third allocation subunit for allocating a third transmission priority to the non-key frame conversion data.

[0162] In some embodiments, the device further includes:

[0163] A first encoding unit for performing lossless encoding or low compression ratio encoding on the key frame conversion data to obtain encoded key frame conversion data;

[0164] A second encoding unit for performing high compression ratio encoding on the non-key frame conversion data to obtain encoded non-key frame conversion data;

[0165] The control unit 605 includes:

[0166] A first control subunit for controlling the industrial conversion data stream to be preferentially transmitted within the transmission period of the industrial conversion data stream specified in the gating list;

[0167] An acquisition subunit, configured to acquire a first remaining transmission duration between a target time point when the industrial conversion data stream transmission is completed and the next transmission cycle;

[0168] A second control subunit, configured to control the encoded key-frame conversion data to be transmitted within the first remaining transmission duration;

[0169] A third control subunit, configured to, when the encoded key-frame conversion data is transmitted within the first remaining transmission duration and there is a second remaining transmission duration, control the encoded non-key-frame conversion data to be transmitted within the second remaining transmission duration.

[0170] In some embodiments, the control unit 605 further includes:

[0171] A fourth control subunit, configured to, when it is detected that the encoded non-key-frame conversion data is not transmitted completely and there are other industrial conversion data streams to be transmitted, prohibit the encoded non-key-frame conversion data from being transmitted and control the other industrial conversion data streams to be transmitted.

[0172] In some embodiments, the control unit 605 further includes:

[0173] A fifth control subunit, configured to, when it is detected that the encoded non-key-frame conversion data is not transmitted completely and there are other encoded key-frame conversion data to be transmitted, prohibit the encoded non-key-frame conversion data from being transmitted and control the other encoded key-frame conversion data to be transmitted.

[0174] In some embodiments, the third control subunit is configured to:

[0175] Acquire the current network resource bandwidth in real time;

[0176] When the current network resource bandwidth reaches the preset network resource bandwidth, during the process of controlling the encoded non-key-frame conversion data to be transmitted within the second remaining transmission duration, discard the encoded non-key-frame conversion data according to a preset discard period.

[0177] In some embodiments, the third control subunit is further configured to:

[0178] When the current network resource bandwidth does not reach the preset network resource bandwidth, prohibit discarding the encoded non-key-frame conversion data according to the preset discard period.

[0179] For the specific implementation of each of the above units, reference may be made to the previous embodiments and will not be elaborated herein.

[0180] As can be seen from the above, in the embodiment of the present application, the receiving unit 601 receives in real time the industrial data stream uploaded by the industrial device and the video data stream uploaded by the monitoring device; the data conversion unit 602 converts the industrial data stream with the industrial Ethernet protocol into an industrial conversion data stream with a specified protocol, and converts the video data stream with the Internet protocol into a video conversion data stream with a specified protocol; the clock synchronization unit 603 synchronizes the clocks of the industrial device and the monitoring device; the allocation unit 604 allocates different transmission priorities to the industrial conversion data stream and the video conversion data stream; the control unit 605 controls the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gating list. Compared with the related art, the TSN scheduling mechanism pays more attention to industrial data and has weak support for the real-time performance and stability of video data, resulting in frame drops and lags in the finally presented video content. It is possible to specify the transmission period of the industrial conversion data stream through the gating list, so that the industrial conversion data stream can be transmitted according to a fixed transmission period, and different transmission priorities are allocated to the industrial conversion data stream and the video conversion data stream, so that the industrial conversion data stream and the video conversion data stream are transmitted in turn according to the transmission priorities in each transmission period, thereby achieving both industrial data and video data and avoiding frame drops and lags in the finally presented video content.

[0181] For the specific implementation of each of the above units, reference may be made to the previous embodiments and will not be elaborated here.

[0182] Refer to Figure 5 , Figure 5 FIG. is a block diagram of a part of a computer device 1000 for implementing the embodiment of the present disclosure. The computer device 1000, as a TSN device, may vary greatly due to configuration or performance differences, and may include one or more central processing units (CPUs) 622 (for example, one or more processors) and a memory 632, and one or more storage media 630 (for example, one or more mass storage devices) for storing application programs 642 or data 644. Among them, the memory 632 and the storage media 630 may be transient storage or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server 600. Further, the central processing unit 622 may be configured to communicate with the storage media 630 and execute a series of instruction operations in the storage media 630 on the server 600.

[0183] The computer device 1000 may further include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and so on.

[0184] The central processing unit 622 in the computer device 1000 may be used to execute the data transmission method according to the embodiments of the present disclosure. For example:

[0185] Receiving in real time the industrial data stream uploaded by the industrial device and the video data stream uploaded by the monitoring device;

[0186] Converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0187] Performing clock synchronization on the industrial device and the monitoring device;

[0188] Assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0189] Based on the transmission period of the industrial conversion data stream specified by the gating list, controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities.

[0190] The embodiments of the present disclosure further provide a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the data transmission methods in the foregoing respective embodiments.

[0191] The embodiments of the present disclosure further provide a computer program product, which includes a computer program. The processor of the computer device reads and executes the computer program, so that the computer device executes to implement the above data transmission method. For example:

[0192] Receiving in real time the industrial data stream uploaded by the industrial device and the video data stream uploaded by the monitoring device; converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0193] Performing clock synchronization on the industrial device and the monitoring device;

[0194] Assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0195] Based on the transmission period of the industrial conversion data stream specified by the gating list, control the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities.

[0196] In addition, the terms "comprise" and "include" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these process, method, product or apparatus.

[0197] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (one) of the following" or similar expressions refer to any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0198] It should be understood that in the description of the embodiments of this application, the meaning of "a plurality (or multiple)" is more than two. Understandings such as greater than, less than, exceeding, etc. do not include the present number, and understandings such as above, below, within, etc. include the present number.

[0199] In several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, in each embodiment of the present application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0203] It should also be understood that the various embodiments provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0204] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of the module or unit.

[0205] The above is a specific description of the embodiments of the present application, but the present application is not limited to the above embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present application, and these equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A data transmission method, characterized in that, Including: Receiving in real time the industrial data stream uploaded by industrial devices and the video data stream uploaded by monitoring devices; Converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of a specified protocol; Performing clock synchronization on the industrial devices and the monitoring devices; Assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream; Based on the transmission period of the industrial conversion data stream specified by the gating list, controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities.

2. The data transmission method according to claim 1, wherein The assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream includes: Screening out key frame conversion data and non-key frame conversion data from the video conversion data stream; Assigning a first transmission priority to the industrial conversion data stream; Assigning a second transmission priority to the key frame conversion data; Assigning a third transmission priority to the non-key frame conversion data.

3. The data transmission method according to claim 2, wherein Before controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities, it further includes: Performing lossless encoding or low compression ratio encoding on the key frame conversion data to obtain encoded key frame conversion data; Performing high compression ratio encoding on the non-key frame conversion data to obtain encoded non-key frame conversion data; The controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gating list includes: Controlling the industrial conversion data stream to be preferentially transmitted within the transmission period of the industrial conversion data stream specified by the gating list; Obtaining a first remaining transmission duration between the target time point when the industrial conversion data stream is transmitted completely and the next transmission period; Controlling the encoded key frame conversion data to be transmitted within the first remaining transmission duration; When the encoded key frame conversion data is transmitted completely within the first remaining transmission duration and there is a second remaining transmission duration, controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration.

4. The data transmission method according to claim 3, wherein After controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration, it further includes: When it is detected that the encoded non-key frame conversion data is not transmitted completely and there are other industrial conversion data streams that need to be transmitted, prohibiting the encoded non-key frame conversion data from being transmitted and controlling the other industrial conversion data streams to be transmitted.

5. The data transmission method according to claim 4, wherein The method further includes: When it is detected that the encoded non-key frame conversion data is not transmitted completely and there are other encoded key frame conversion data that need to be transmitted, prohibiting the encoded non-key frame conversion data from being transmitted and controlling the other encoded key frame conversion data to be transmitted.

6. The data transmission method according to claim 3, characterized in that The controlling the encoded non-key frame conversion data to be transmitted within the second remaining transmission duration includes: Obtaining the current network resource bandwidth in real time; When the current network resource bandwidth reaches the preset network resource bandwidth, during the process of controlling the transmission of the encoded non-key frame conversion data within the second remaining transmission duration, the encoded non-key frame conversion data is discarded according to a preset discard period.

7. The data transmission method according to claim 6, wherein The method further includes: When the current network resource bandwidth does not reach the preset network resource bandwidth, discarding the encoded non-key frame conversion data according to a preset discard period is prohibited.

8. A data transmission device, characterized in that, It includes: A receiving unit, configured to receive in real time the industrial data stream uploaded by the industrial device and the video data stream uploaded by the monitoring device; A data conversion unit, configured to convert the industrial data stream with the industrial Ethernet protocol into an industrial conversion data stream with a specified protocol, and convert the video data stream with the Internet protocol into a video conversion data stream with a specified protocol; A clock synchronization unit, configured to perform clock synchronization on the industrial device and the monitoring device; An allocation unit, configured to allocate different transmission priorities to the industrial conversion data stream and the video conversion data stream; A control unit, configured to control the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified in the gating list.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the data transmission method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the data transmission method according to any one of claims 1 to 7 is implemented.

Images