Data transmission method and device, storage medium and computer device

Abstract

Embodiments of the present application provide a data transmission method and device, a storage medium and a computer device, comprising: receiving industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time; converting the industrial data streams of the industrial Ethernet protocol into industrial conversion data streams of a specified protocol, and converting the video data streams of the Internet protocol into video conversion data streams of the specified protocol; performing clock synchronization on the industrial equipment and the monitoring equipment; assigning different transmission priorities to the industrial conversion data streams and the video conversion data streams; and based on a transmission cycle of the industrial conversion data streams specified by a gating list, controlling the industrial conversion data streams and the video conversion data streams to be transmitted according to the corresponding transmission priorities. Compared with the TSN scheduling mechanism, the embodiments can take into account both the industrial data and the video data, and avoid the final video content from being frozen and losing frames, in terms of the real-time and stability support for video data being relatively weak, resulting in the final video content being frozen and losing frames.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network, and particularly relates to a data transmission method and device, a storage medium and a computer device. BACKGROUND

[0002] The existing industrial network usually adopts a traditional layered architecture, and uses various industrial protocols, such as Profinet, EtherCAT and the like, to complete data communication. However, the heterogeneity among these protocols leads to poor data interoperability among different devices, and meanwhile, with the increasing demand of industrial scenarios for video monitoring, the access of Internet Protocol Version 6 (IPv6) / Internet Protocol Version 4 (IPv4) video stream becomes a key.

[0003] In the related art, as an efficient real-time communication technology, Time-Sensitive Network (TSN) can ensure the real-time performance, reliability and determinacy of data transmission, and gradually becomes the mainstream of the next generation of industrial network. However, the existing TSN scheduling mechanism pays more attention to industrial data, and has weak support for the real-time performance and stability of video data, which leads to the lag and frame loss of the final presented video content, so the related art urgently needs to provide a data transmission method to solve the above technical problems. SUMMARY

[0004] The main purpose of the present application is to provide a data transmission method, device, storage medium and computer device, which can take into account industrial data and video data, and avoid the lag and frame loss of the final presented video content.

[0005] In a first aspect, an embodiment of the present application provides a data transmission method, comprising:

[0006] receiving an industrial data stream uploaded by an industrial device and a video data stream uploaded by a monitoring device in real time;

[0007] converting the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and converting the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0008] synchronizing the clock of the industrial device and the monitoring device;

[0009] allocating different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0010] controlling the industrial conversion data stream and the video conversion data stream to be transmitted according to the corresponding transmission priorities based on the transmission period of the industrial conversion data stream specified by the gate list.

[0011] Secondly, embodiments of this application provide a data transmission apparatus, including:

[0012] The receiving unit is used to receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time.

[0013] A data conversion unit is used to convert the industrial data stream of the Industrial Ethernet protocol into an industrial converted data stream of a specified protocol, and to convert the video data stream of the Internet protocol into a video converted data stream of a specified protocol.

[0014] A clock synchronization unit is used to synchronize the clocks of the industrial equipment and the monitoring equipment.

[0015] An allocation unit is used to assign different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0016] The control unit is used to control the transmission of the industrial conversion data stream and the video conversion data stream according to their corresponding transmission priorities based on the transmission cycle specified in the gating list.

[0017] Thirdly, embodiments of this application provide a storage medium that stores multiple instructions adapted for loading by a processor to execute any of the above data transmission methods.

[0018] Fourthly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data transmission method as described above.

[0019] In this embodiment, industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment are received in real time. The industrial data streams using the Industrial Ethernet protocol are converted into industrial conversion data streams using a specified protocol, and the video data streams using the Internet protocol are converted into video conversion data streams using a specified protocol. Clock synchronization is performed on the industrial equipment and the monitoring equipment. Different transmission priorities are assigned to the industrial conversion data streams and the video conversion data streams. Based on the transmission cycle of the industrial conversion data streams specified by the gating list, the transmission of the industrial conversion data streams and the video conversion data streams is controlled according to their corresponding transmission priorities. Compared to related technologies where the TSN scheduling mechanism focuses more on industrial data and has weaker support for the real-time performance and stability of video data, resulting in stuttering and frame drops in the final video content, the gating list can specify the transmission cycle of the industrial conversion data streams, allowing them to be transmitted according to a fixed cycle. Different transmission priorities are assigned to the industrial conversion data streams and the video conversion data streams, ensuring that they are transmitted sequentially in each transmission cycle according to their transmission priorities. This achieves a balance between industrial data and video data, avoiding stuttering and frame drops in the final video content.

[0020] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This application provides a schematic diagram of a data transmission system scenario.

[0023] Figure 2 This is a flowchart illustrating the data transmission method provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram of the architecture of the data transmission method provided in the embodiments of this application.

[0025] Figure 4 This is a schematic diagram of the structure of the data transmission device provided in the embodiments of this application.

[0026] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It should be noted that while some processes described in the specification, claims, and accompanying drawings contain multiple steps that appear in a specific order, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not represent any particular order of execution. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0029] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0030] Time-Sensitive Networking (TSN) is a network technology used to solve real-time communication problems in scenarios such as industrial control. It improves upon traditional Ethernet through a series of standard protocols, ensuring real-time, reliable, and deterministic data transmission, and is gradually becoming the mainstream technology for next-generation industrial networks.

[0031] Tunnel mode conversion: This usually refers to the process in network communication of encapsulating one network protocol or data format into another protocol or format for transmission, thereby enabling communication between different network environments or protocols.

[0032] Tunneling mode conversion encapsulates the original datagram with a new header within a data packet of another protocol, enabling the original data to be transmitted over incompatible networks. The new header contains information required by the target network, such as routing and address information; it's like giving the original data a "clothes," allowing it to be correctly identified and transmitted in different network environments. Upon arrival at the target network, the encapsulated data packet is decapsulated, restoring the original data.

[0033] Common tunneling patterns include:

[0034] 1. IPSec Tunnel Mode: Used for secure network communication, it encrypts the entire IP packet, including the header and payload. In ESP tunnel mode, the IP packet is encapsulated using the ESP protocol, IP header, and ESP authentication trailer. The packet signature follows the ESP header, and the encryption covers everything except the ESP authentication trailer. The original header is placed after the ESP header, and the ESP trailer is appended before encryption. The entire ESP payload is encapsulated in a new, unencrypted tunnel header. It is commonly used to protect communication between different networks, such as when corporate headquarters and branch offices communicate over an untrusted internet connection, using IPSec tunnel mode to ensure data security.

[0035] 2. GRE Tunnel: A general routing encapsulation protocol that can encapsulate various protocol data packets, such as IP and IPX. It provides logical connections between different subnets without encryption. In an enterprise network, if there are multiple office areas with different network segments, GRE tunnels can be used to achieve interconnection, facilitating data transmission and resource sharing.

[0036] 3. SRv6 Tunnel: A new network technology based on IPv6, leveraging the flexibility and scalability of IPv6 to achieve network programming and traffic engineering. By configuring SRv6TEPolicy tunnels or SRv6BE tunnels, cross-site VPN interconnection can be dynamically established. It is suitable for scenarios requiring fine-grained control and flexible scheduling of network traffic, such as large data center networks, where it can optimize traffic paths and improve network resource utilization.

[0037] Gate Control List (GCL): A key concept in Time-Sensitive Networking (TSN), it is a traffic scheduling mechanism used to achieve precise scheduling and control of different types of data in the network, ensuring that time-sensitive data can be reliably transmitted within a specified time.

[0038] Gated lists operate on the principle of Time Division Multiplexing (TDM), dividing the time axis into periodic time intervals called cycles. Each cycle is further divided into multiple time slots. A gated list is configured for each port or link, specifying the traffic categories or queues allowed to be transmitted in each time slot. By controlling the opening and closing of ports in each time slot, it determines which data can be transmitted within a specific time period.

[0039] However, the existing TSN scheduling mechanism focuses more on industrial data and has weak support for the real-time performance and stability of video data, resulting in stuttering and frame drops in the final video content.

[0040] To address the aforementioned problems, this application embodiment receives industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time; converts the industrial data streams (industrial Ethernet protocol) to industrial conversion data streams of a specified protocol, and converts the video data streams (in Internet protocol) to video conversion data streams of a specified protocol; synchronizes the clocks of the industrial equipment and the monitoring equipment; assigns different transmission priorities to the industrial conversion data streams and the video conversion data streams; and controls the transmission of the industrial conversion data streams and video conversion data streams according to their corresponding transmission priorities based on the transmission cycle specified by the gating list. Compared to related technologies where the TSN scheduling mechanism focuses more on industrial data and has weaker support for the real-time performance and stability of video data, resulting in stuttering and frame drops in the final video content, this application uses a gating list to specify the transmission cycle of the industrial conversion data streams. This allows the industrial conversion data streams to be transmitted according to a fixed transmission cycle, and different transmission priorities are assigned to the industrial conversion data streams and video conversion data streams. This ensures that the industrial conversion data streams and video conversion data streams are transmitted sequentially in each transmission cycle according to their transmission priorities, thereby balancing industrial data and video data and avoiding stuttering and frame drops in the final video content.

[0041] Please see Figure 1 , Figure 1 This is a schematic diagram of a data transmission system provided in an embodiment of this application. It includes a device layer, an intermediate layer, and a control layer, etc.

[0042] The equipment layer consists of industrial controllers, industrial surveillance cameras, and various other industrial controllers. Devices like robotic arms and industrial surveillance cameras (such as webcams) belong to this layer. Industrial controllers communicate with the middle-layer TSN gateway via industrial Ethernet protocols such as Profinet and EtherCAT. Industrial surveillance cameras communicate with the middle-layer TSN gateway via Internet protocols such as Internet Protocol Version 6 (IPv6) / Internet Protocol Version 4 (IPv4). Its function is to serve as the foundation of the industrial system, responsible for collecting various data from the production site, such as sensors sensing physical quantities like temperature and pressure; and executing control commands, such as actuators performing mechanical actions according to instructions to achieve production operations.

[0043] The middle layer (TSN devices) consists of TSN gateways and TSN switches. TSN gateways are responsible for converting industrial protocol data and video streams into data streams compliant with TSN standards. TSN switches prioritize and schedule data streams according to the 802.1Qbv and 802.1Qbu standards. Their role is that of a data transmission hub. Switching devices are responsible for data forwarding and switching. Access devices connect various devices at the device layer, enabling stable and real-time transmission of device layer data to the control layer, and also conveying control layer commands to the device layer, ensuring smooth and efficient network communication.

[0044] The control layer encompasses modules such as intelligent applications, quality management, data analysis, and process management. Its role is to operate at the top of the industrial system, performing in-depth analysis and processing of data collected from the equipment layer. Intelligent applications optimize production processes, the quality management module controls product quality, data analysis uncovers potential problems, and process management ensures stable production processes, thereby achieving intelligent control over the entire industrial production process. Simultaneously, it processes video streams in real time for monitoring and display.

[0045] The data transmission method of this disclosure can be implemented in a TSN device.

[0046] It should be noted that, Figure 1 The schematic diagram of the data transmission system shown is merely an example. The data transmission system and scenario described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of image processing technology and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0047] In this embodiment, the description will be from the perspective of a data transmission device, which can be integrated into a computer device that has a storage unit and a microprocessor and thus computing capabilities.

[0048] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data transmission method provided in an embodiment of this application. The data transmission method includes:

[0049] In step 201, industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment are received in real time.

[0050] Industrial equipment refers to devices used in industrial production processes to achieve various production functions, such as machine tools, robots, and sensors. These devices generate data related to the production process, such as temperature, pressure, rotational speed, and position information. Industrial data streams are collections of data generated by industrial equipment during operation. This data reflects the real-time status and operating parameters of industrial production and is transmitted in stream form using industrial Ethernet protocols such as Profinet and EtherCAT.

[0051] Monitoring equipment is used to monitor industrial production environments and equipment operating status in real time; common examples include industrial cameras. Video data streams are continuous video image data collected by monitoring equipment (such as cameras) and transmitted in stream form via Internet protocols such as IPv6 / IPv4 for real-time monitoring of the production site.

[0052] In step 202, the industrial data stream of the Industrial Ethernet protocol is converted into an industrial conversion data stream of a specified protocol, and the video data stream of the Internet protocol is converted into a video conversion data stream of a specified protocol.

[0053] To support industrial data streams uploaded by devices using various industrial Ethernet protocols such as Profinet and EtherCAT, and to be compatible with video data streams using IPv6 / IPv4 protocols, it is necessary to convert data streams from different protocols into data streams using the same protocol. Since this application uses a TSN device, the TSN gateway within the TSN device converts industrial data streams from different industrial Ethernet protocols into industrial conversion data streams using a specified protocol (TSN standard data stream protocol), and converts video data streams using Internet protocols into video conversion data streams using the specified protocol (TSN standard data stream protocol). This ensures that both the industrial conversion data streams and the video conversion data streams use the specified protocol for subsequent data stream management.

[0054] Specifically, data streams converted to a specified protocol can be mapped to that protocol (TSN standard data stream protocol) using tunneling mode. For industrial data streams using industrial Ethernet protocols such as Profinet and EtherCAT, the data is parsed to extract the valid data payload. Then, according to the format requirements of the TSN standard data stream protocol, new header information (including timestamps, priority identifiers, and other TSN-related parameters) is added, and the valid data is encapsulated into TSN format data frames. For example, for Profinet real-time data, the key control command data portion is extracted and encapsulated into the corresponding data segment in a TSN frame. For video data streams using Internet protocols such as IPv6 / IPv4, the video data streams are parsed to separate the valid payload portion of the video data, such as video encoding data and timestamps. According to the requirements of the TSN standard data stream protocol, a new header is added to the parsed video data. The header contains TSN-related information, such as timestamps (used for clock synchronization and transmission scheduling). The video data with the added header is then encapsulated into TSN data frames according to the TSN protocol specification, enabling it to be transmitted in the TSN network.

[0055] In step 203, clock synchronization is performed on the industrial equipment and monitoring equipment.

[0056] To ensure the consistency of timestamps between industrial conversion data streams and video conversion data streams, clock synchronization between devices needs to be achieved based on the IEEE 802.1AS protocol.

[0057] The specific clock synchronization method is as follows: one device is designated as the master clock, and other industrial equipment and / or monitoring equipment act as slave clocks. The master clock provides a precise time reference, and the slave clocks adjust their own clocks to maintain synchronization by communicating with the master clock. The specific clock synchronization process involves the master clock periodically sending time synchronization messages. Upon receiving a message, the slave clock measures the transmission delay and adjusts its own clock accordingly. Ultimately, through multiple interactions and adjustments, precise clock synchronization is gradually achieved.

[0058] In step 204, different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream.

[0059] Based on factors such as data importance and real-time requirements, different transmission priorities are assigned to industrial conversion data streams and video conversion data streams. High-priority data has priority in network processing and transmission to ensure it reaches its destination in a timely manner.

[0060] During the data encapsulation process, priority tags (VLAN tags) are added to the industrial conversion data stream and the video conversion data stream.

[0061] In some implementations, assigning different transmission priorities to the industrial conversion data stream and the video conversion data stream includes:

[0062] (1) Filter out keyframe conversion data and non-keyframe conversion data from the video conversion data stream;

[0063] (2) Assign a first transmission priority to the industrial conversion data stream;

[0064] (3) Assign a second transmission priority to the keyframe conversion data;

[0065] (4) Assign a third transmission priority to the non-critical frame conversion data.

[0066] In this embodiment, to ensure the real-time performance and stability of video display, keyframe identification is performed on the video conversion data stream to filter out keyframe conversion data and non-keyframe conversion data. A keyframe, a computer animation term, refers to the frame containing a crucial action in the movement of a character or object, equivalent to a keyframe in 2D animation. In video processing, it is a video frame with representative meaning, used to concisely describe video content, and widely applied in video compression storage, structured indexing, and other fields. The following are some common keyframe identification methods:

[0067] Lens boundary detection

[0068] Principle: Videos are composed of different shots, and the frames at shot transitions often have significant content changes; these frames are considered keyframes. By calculating the differences between adjacent frames, such as changes in visual features like color, texture, and shape, if the difference exceeds a certain threshold, it is determined to be a shot boundary, and the corresponding frame is a keyframe.

[0069] Based on motion analysis

[0070] Principle: Calculate the motion vectors of objects in video frames. Frames with large changes in motion vectors indicate significant changes in the scene or the motion state of objects, and may be key frames. For example, in sports event videos, actions such as players running quickly or suddenly changing direction result in significant changes in the motion vectors of the corresponding frames.

[0071] Based on image features

[0072] Principle: Extract image features from video frames, such as color histograms, texture features, and edge features. Compare the features of adjacent frames; frames with significant feature differences are identified as keyframes. For example, when a scene in a video switches from daytime to nighttime, the color histogram will show a noticeable change.

[0073] Based on deep learning

[0074] Principle: Construct a deep neural network model, such as a convolutional neural network (CNN). Train the model using a large amount of video data with labeled keyframes, allowing it to learn keyframe feature patterns. The trained model can then perform keyframe recognition on new video frames.

[0075] Specifically, priority evaluation criteria are determined based on the characteristics and application requirements of industrial conversion data streams and video conversion data streams. For example, industrial conversion data streams have high real-time requirements and should be assigned a higher priority (i.e., first transmission priority); while video frame data in video conversion data streams have relatively lower real-time requirements and can be assigned a lower priority. For keyframe conversion data, its importance is lower than that of industrial conversion data streams but higher than that of non-keyframe conversion data, therefore it is assigned a second transmission priority, just below the first priority; for non-keyframe conversion data, its importance is the lowest, therefore it is assigned a third transmission priority, below the second priority.

[0076] In this way, different transmission priorities are allocated based on the importance and real-time requirements of the data, making rational use of network bandwidth resources. For non-critical frame conversion data with low real-time requirements, transmission is carried out when network bandwidth is sufficient, avoiding competition for bandwidth with high-priority data and improving the utilization rate of network resources. In industrial production, network traffic may fluctuate. Through this transmission priority allocation mechanism, data transmission strategies can be flexibly adjusted under different network load conditions, making network resources more effectively utilized.

[0077] In step 205, based on the transmission cycle of the industrial conversion data stream specified in the gating list, the industrial conversion data stream and the video conversion data stream are controlled to be transmitted according to their corresponding transmission priorities.

[0078] The gating list specifies the transmission period for industrial conversion data streams with high real-time requirements. After limiting the transmission period of the industrial conversion data stream, the video conversion data stream can be transmitted within the remaining transmission time of the transmission period.

[0079] Specifically, the gating list is a core component of the Time-Aware Shaping (TAS) mechanism in Time-Sensitive Networking (TSN), a mechanism for controlling queue transmission switching based on time. Each port has an ordered list of gate operation states containing several nodes. Each node specifies the gating state (0 for off, 1 for on) of each queue and its execution time. During the execution time, data packets in the queue with a gating state of 1 are forwarded sequentially from highest to lowest priority according to a transmission selection algorithm (such as a strict priority algorithm). When the current node finishes execution, execution moves to the next node, and so on, returning to the first node for a loop. In this way, "isolation" of transmissions in queues with different priorities is achieved, ensuring accurate forwarding of high-priority services and providing ultra-low latency and jitter protection for real-time demanding services such as industrial control. First, the data frame length, transmission rate, and other parameters of the industrial conversion data stream are defined. The real-time and accuracy requirements of actual industrial production operations are considered. If industrial equipment control commands need to be transmitted promptly to ensure accurate equipment operation, the industrial conversion data stream must be transmitted within a short time interval. Assess network bandwidth and load. If network bandwidth is limited and load is high, the transmission cycle may need to be appropriately extended to avoid congestion; if bandwidth is sufficient and load is low, the transmission cycle can be shortened. Based on the determined transmission cycle, configure it in the gating list. Within the cycle of the gating list, allocate dedicated transmission time slots for the industrial conversion data stream to ensure its transmission within the specified period.

[0080] In some embodiments, before controlling the industrial conversion data stream and the video conversion data stream to transmit according to corresponding transmission priorities, the method further includes:

[0081] (1) Perform lossless encoding or low compression ratio encoding on the keyframe conversion data to obtain the encoded keyframe conversion data;

[0082] (2) The non-key frame conversion data is encoded with a high compression ratio to obtain the encoded non-key frame conversion data;

[0083] The transmission cycle of the industrial conversion data stream, based on the gating list, controls the transmission of the industrial conversion data stream and the video conversion data stream according to their corresponding transmission priorities, including:

[0084] (3) Control the industrial conversion data stream to be transmitted preferentially within the transmission period of the industrial conversion data stream specified in the gating list;

[0085] (4) Obtain the first remaining transmission time between the target time point when the industrial conversion data stream transmission is completed and the next transmission cycle;

[0086] (5) Control the transmission of the encoded keyframe conversion data within the first remaining transmission time;

[0087] (6) When the encoded key frame conversion data is transmitted within the first remaining transmission time and there is a second remaining transmission time, the encoded non-key frame conversion data is controlled to be transmitted within the second remaining transmission time.

[0088] Lossless or low-compression encoding of keyframe transformation data aims to minimize data volume while preserving keyframe information to the greatest extent possible. This ensures accurate display of critical video content and meets the real-time and stability requirements of video display. Classic lossless or low-compression algorithms can be employed, such as Huffman coding and arithmetic coding, or low-compression algorithms that better preserve image details during compression. For example, Huffman coding constructs a Huffman tree by statistically analyzing the frequency of different data values ​​in the keyframe transformation data. Frequently occurring data is represented with shorter codes, while infrequently occurring data is represented with longer codes, thus achieving data compression while maintaining information integrity.

[0089] Encoding non-keyframe transformation data with high compression ratio aims to minimize the amount of data in non-keyframes without affecting the overall understanding and smoothness of the video, thereby saving network bandwidth resources and making it more flexible in network transmission. Several efficient high-compression-ratio coding algorithms can be selected, such as the JPEG compression algorithm based on Discrete Cosine Transform (DCT) or the high-compression-ratio modes of video compression algorithms based on motion estimation and compensation (such as H.264 / AVC, H.265 / HEVC, etc.). Taking H.264 as an example, it performs intra-frame prediction and inter-frame prediction on non-keyframes, utilizing spatial and temporal redundancy information in the video to segment the image into small blocks for encoding. For regions with minimal changes, a larger quantization parameter is used for compression, thus achieving a high compression ratio.

[0090] After encoding the keyframe and non-keyframe conversion data, it's necessary to control the transmission of three data streams with different priorities: industrial conversion data, keyframe conversion data, and non-keyframe conversion data. A gating list defines the transmission period of the industrial conversion data stream; therefore, the industrial conversion data stream is controlled to be transmitted within the specified period. For example, the industrial conversion data stream is transmitted every 100 milliseconds. After the industrial conversion data stream is transmitted within this transmission period, the remaining transmission time between the current time period and the next transmission period is obtained. Keyframe and non-keyframe conversion data are then transmitted within the remaining transmission time.

[0091] Specifically, since the second transmission priority of key frame conversion data is higher than the third transmission priority of non-key frame conversion data, the encoded key frame conversion data is transmitted within the first remaining transmission time. When the encoded key frame conversion data is transmitted within the first remaining transmission time and there is a second remaining transmission time, the encoded non-key frame conversion data is transmitted within the second remaining transmission time.

[0092] For example, when a 100-millisecond transmission cycle begins, the system sends an industrial conversion data stream to the network for transmission. In this example, suppose the industrial conversion data stream is transmitted after 80 milliseconds. At this point, the system obtains the remaining transmission time between the current time period (80 milliseconds) and the next transmission cycle (the start of the next 100-millisecond cycle, i.e., 20 milliseconds later), which is 20 milliseconds (the first remaining transmission time). Because keyframe conversion data has higher priority than non-keyframe conversion data, the system prioritizes the transmission of the encoded keyframe conversion data within this 20-millisecond first remaining transmission time. Assuming the keyframe conversion data is small and is transmitted within 15 milliseconds, there is still a 5-millisecond remaining transmission time (the second remaining transmission time). Detecting the existence of the second remaining transmission time, the system controls the transmission of the encoded non-keyframe conversion data within this 5-millisecond second remaining transmission time. When the next 100-millisecond transmission cycle begins, the industrial conversion data stream is again prioritized for transmission, and the above process repeats.

[0093] By strictly controlling transmission within the specified transmission cycle of the gating list, the timely and accurate transmission of critical data such as control commands and equipment status feedback in industrial production is ensured. This helps maintain the normal operation of industrial equipment, avoids production accidents, equipment failures, or product quality problems caused by data transmission delays or errors, and guarantees the continuity and stability of industrial production processes. Lossless or low-compression encoding of keyframe conversion data preserves keyframe information to the greatest extent, enabling accurate display of key video content and meeting the real-time and stability requirements of video display. In scenarios such as industrial monitoring, keyframes often contain important information such as critical equipment operating states and abnormal situations; accurately displaying this content helps operators promptly identify problems and take appropriate measures. High-compression encoding is used for non-keyframe conversion data, reducing data volume and saving network bandwidth resources without affecting the overall understanding and smoothness of the video. This makes video data transmission more flexible and efficient over the network, while also ensuring a basic viewing experience.

[0094] Based on the different priorities and characteristics of the data, transmission time and methods are rationally allocated, effectively utilizing network bandwidth resources. Industrial conversion data streams are transmitted first within a specified period to ensure they are not interfered with. In the remaining time after the industrial conversion data stream transmission is completed, key frame conversion data and non-key frame conversion data are transmitted in priority order. This method avoids bandwidth competition between high-priority and low-priority data, resulting in a more rational allocation of network bandwidth among different types of data, improving network resource utilization, and better meeting the transmission needs of diverse data, especially when network bandwidth is limited.

[0095] In some implementations, after controlling the transmission of the encoded non-key frame conversion data within the second remaining transmission duration, the method further includes:

[0096] When it is detected that the encoded non-critical frame conversion data has not been fully transmitted and there are other industrial conversion data streams that need to be transmitted, the transmission of the encoded non-critical frame conversion data is prohibited, and the transmission of the other industrial conversion data streams is controlled.

[0097] Specifically, in cases where the encoded non-critical frame conversion data is not fully transmitted within the second remaining transmission time due to low network bandwidth or a large data stream of non-critical frame conversion data, resulting in some encoded non-critical frames remaining in the transmission queue, if the next transmission cycle is reached and other industrial conversion data streams need to be transmitted, the transmission of the encoded non-critical frame conversion data is prohibited according to the priority order of transmission priority, and other industrial conversion data streams are controlled to be transmitted first, so that the transmission of industrial conversion data streams can be guaranteed even under high load conditions.

[0098] Therefore, when it is detected that the encoded non-critical frame conversion data has not been fully transmitted and other industrial conversion data streams need to be transmitted, the transmission of industrial conversion data streams is prioritized. Industrial conversion data streams typically contain critical control commands and equipment status information in industrial production, and have extremely high requirements for real-time performance and accuracy. Under conditions of limited network bandwidth or high load, this strategy ensures that industrial production-related data can be transmitted in a timely manner, avoiding disruptions to industrial production due to bandwidth consumption by non-critical data, thus guaranteeing the continuity and stability of industrial production and reducing the risk of production accidents and equipment failures.

[0099] In some embodiments, the method further includes:

[0100] When it is detected that the encoded non-key frame conversion data has not been fully transmitted, and there is other encoded key frame conversion data that needs to be transmitted, the transmission of the encoded non-key frame conversion data is prohibited, and the transmission of the other encoded key frame conversion data is controlled.

[0101] Specifically, in cases where the encoded non-key frame conversion data is not fully transmitted within the second remaining transmission time due to low network bandwidth or a large data stream of encoded non-key frame conversion data, resulting in some encoded non-key frames remaining in the transmission queue, if the next transmission cycle arrives and other encoded key frame conversion data streams need to be transmitted, the transmission of the encoded non-key frame conversion data is prohibited according to the priority order of transmission priority, and other encoded key frame conversion data is controlled to be transmitted first, so that video quality can be guaranteed even under high load conditions.

[0102] Therefore, when network bandwidth is limited, especially under high load, prioritizing the transmission of critical frames concentrates limited bandwidth resources on the most important data, preventing non-critical data from consuming too much bandwidth and causing delays in critical data transmission. This dynamic bandwidth allocation strategy improves the utilization efficiency of network resources, enabling the network to operate stably under high load. Prohibiting untransmitted non-critical frames from continuing to occupy transmission time avoids further data backlog in the transmission queue. This helps maintain the efficiency of network transmission and prevents network congestion and data loss due to excessively long queues.

[0103] In some implementations, controlling the transmission of the encoded non-key frame conversion data within the second remaining transmission duration includes:

[0104] (1.1) Obtain current network resource bandwidth in real time;

[0105] (1.2) When the current network resource bandwidth reaches the preset network resource bandwidth, the encoded non-critical frame conversion data is controlled to be transmitted within the second remaining transmission time, and the encoded non-critical frame conversion data is discarded according to the preset discarding period.

[0106] During the transmission of the non-critical frame conversion data after control encoding within the second remaining transmission time, the current network resource bandwidth is acquired in real time. If the current network resource bandwidth reaches the preset network resource bandwidth, it indicates that the network load is high. During the transmission of the non-critical frame conversion data after control encoding within the second remaining transmission time, the encoded non-critical frame conversion data is discarded according to a preset discard period. This achieves the effect of reducing the frame rate, for example, from 30 frames per second to 15 frames per second.

[0107] Therefore, network resources become strained under heavy network load. Industrial conversion data streams and keyframe conversion data have extremely high requirements for real-time performance and accuracy, necessitating priority in their transmission. By discarding some encoded non-keyframe conversion data, more network bandwidth can be freed up for critical data, ensuring the timely and stable transmission of key data such as industrial production control commands and critical monitoring images, thereby guaranteeing the normal operation of industrial production and the timely presentation of critical video information. Although some encoded non-keyframe conversion data is discarded, the impact of encoded non-key frames on the overall understanding of the video is relatively small, thus maintaining basic video continuity to a certain extent. For example, reducing the frame rate from 30 frames per second to 15 frames per second, although the frame rate decreases, viewers can still roughly understand the video content without experiencing severe stuttering or incomprehensibility.

[0108] In some embodiments, the method further includes:

[0109] When the current network resource bandwidth does not reach the preset network resource bandwidth, it is prohibited to discard the encoded non-critical frame conversion data according to the preset discarding period.

[0110] If the current network resource bandwidth does not reach the preset network resource bandwidth, it means that the network load is small at this time. Therefore, there is no need to transmit the encoded non-critical frame conversion data by reducing the frame rate. Thus, it is prohibited to discard the encoded non-critical frame conversion data according to the preset discarding period.

[0111] Industrial equipment commissioning and maintenance support

[0112] Video streams are used for real-time monitoring of industrial equipment operation, recording the debugging process in conjunction with industrial control commands. The consistency of timestamps between the video stream and industrial data allows for precise location of the fault's occurrence. Log analysis and video playback enable rapid identification of the root cause of problems.

[0113] Therefore, when network load is low, i.e., the current network bandwidth has not reached the preset bandwidth, not discarding non-critical frame conversion data can ensure stable video frame rate, thereby providing higher quality video display. For example, in video surveillance systems, it can present smoother and clearer images, helping operators to observe and judge the situation on site more accurately. For some application scenarios with high video quality requirements, not reducing the frame rate can ensure the integrity and detail of the video image, improving the user experience.

[0114] As described above, this embodiment of the application receives industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time; converts the industrial data streams of the Industrial Ethernet protocol into industrial conversion data streams of a specified protocol, and converts the video data streams of the Internet protocol into video conversion data streams of a specified protocol; synchronizes the clocks of the industrial equipment and the monitoring equipment; assigns different transmission priorities to the industrial conversion data streams and the video conversion data streams; and controls the industrial conversion data streams and the video conversion data streams to be transmitted according to their corresponding transmission priorities based on the transmission cycle of the industrial conversion data streams specified by the gating list. Compared with related technologies, where the TSN scheduling mechanism focuses more on industrial data and has weaker support for the real-time performance and stability of video data, resulting in stuttering and frame drops in the final video content, this application can use a gating list to specify the transmission cycle of the industrial conversion data streams, allowing the industrial conversion data streams to be transmitted according to a fixed transmission cycle. By assigning different transmission priorities to the industrial conversion data streams and the video conversion data streams, the transmission of the industrial conversion data streams and the video conversion data streams is carried out sequentially in each transmission cycle according to the transmission priority, thereby achieving a balance between industrial data and video data and avoiding stuttering and frame drops in the final video content.

[0115] The following describes the compression, decompression, and frame rate adjustment mechanisms for video frame data provided in the embodiments of this application.

[0116] (1) Video compression and decompression

[0117] Using H.265 / HEVC encoding technology, it can save about 50% of bandwidth compared to H.264 while maintaining the same image quality.

[0118] Low-latency optimization is performed within the TSN network, combined with a fast motion estimation algorithm to improve compression efficiency and reduce encoding latency.

[0119] By combining frame priority scheduling strategies, different compression rates are applied to video data streams of different priorities:

[0120] P2 (Keyframe Data): Lossless encoding or low compression ratio encoding is used to ensure image clarity.

[0121] P3 (Non-keyframe data): Employs high compression ratio encoding to reduce bandwidth usage.

[0122] (2) Frame rate dynamic adjustment

[0123] Based on network load adaptive adjustment

[0124] Monitor TSN network bandwidth usage and automatically adjust the frame rate of the video stream.

[0125] Reduce the frame rate of non-critical frames under high load conditions (e.g., from 30 FPS to 15 FPS) to ensure that critical frame transmission is prioritized.

[0126] Restore full frame rate transmission under low load conditions to improve the smoothness of video streaming.

[0127] Keyframe priority strategy

[0128] Combined with TSN time synchronization (IEEE 802.1AS), ensure that key frames are in sync with industrial control data.

[0129] Keyframes maintain high priority even during network congestion to ensure the integrity of video data.

[0130] The priority scheduling strategy provided in the embodiments of this application is described below:

[0131] (1) Priority division of data streams

[0132] To meet the real-time requirements of industrial data conversion and optimize the transmission quality of video streams, this application embodiment divides the data into three categories according to priority:

[0133] Highest priority (P1): Industrial conversion data, ensuring millisecond-level real-time transmission of control commands;

[0134] Medium priority (P2): Keyframe conversion data to ensure that the core data of the video stream is not lost;

[0135] Lowest priority (P3): Non-key frame conversion data (normal video stream frames), which can sacrifice some frame rate when bandwidth is tight.

[0136] (2) Optimization mechanism of TAS + frame preemption

[0137] Time-Aware Scheduling (TAS) Optimization:

[0138] The IEEE 802.1Qbv Gate Control List (GCL) is used to control the transmission windows of data streams with different priorities.

[0139] P1 (industrial conversion data) uses a fixed transmission cycle, ensuring its priority transmission in all transmission cycles.

[0140] P2 (keyframe transition data) is scheduled during transmission cycles not occupied by P1 to ensure that keyframes can be transmitted with minimal delay.

[0141] P3 (non-keyframe conversion data) is transmitted only when bandwidth is sufficient to avoid affecting the real-time performance of industrial conversion data and keyframe conversion data.

[0142] Frame preemption mechanism optimization:

[0143] In conjunction with IEEE 802.1Qbu, high-priority data (P1 / P2) is allowed to be inserted during the transmission of low-priority data (P3), thereby preventing low-priority frames from blocking high-priority data.

[0144] Specific strategies:

[0145] When P1 arrives, it immediately preempts the transmission of the current P3 frame.

[0146] When P2 arrives, if the current transmission is of frame P3, it will take priority to preempt the frame, ensuring that the priority of key frame conversion data is higher than that of non-key frame conversion data.

[0147] When P3 uses fragmented transmission, it transmits intermittently within idle time slices to improve bandwidth utilization.

[0148] For details, please refer to Figure 3 , Figure 3 This is a schematic diagram of the architecture of the data transmission method provided in the embodiments of this application. Figure 3In the Zhongyuan industrial control equipment and video equipment system, the video surveillance system communicates with industrial cameras via IPv4 / 6 data streams, while the industrial control PLC interacts with servo motors and I / O modules based on EtherCAT and PROFINET data streams, respectively. In this solution, these data streams using different protocols require conversion processing. For example, IPv4 / 6 video data streams and industrial data streams using industrial Ethernet protocols need to be converted to the TSN standard data stream protocol via tunneling mode for efficient transmission under a unified network architecture. Different devices use different protocols, generating heterogeneous data, such as industrial conversion data streams and video conversion data streams. This necessitates an architecture like an industrial heterogeneous data fusion measurement platform to integrate and process various types of data. The TSN network plays a crucial role in this, enabling the fusion and transmission of data from different protocols. TSN devices convert data streams from various industrial Ethernet protocols and Internet protocols into the TSN standard data stream protocol. The TSN network shown in the diagram is key to achieving this conversion and fusion. As a core hub, it connects various devices, allowing data from different protocols to be transmitted under a unified standard, improving network compatibility and data transmission efficiency. The network line load diagram in the figure illustrates the usage of IPv4 / 6 bandwidth and industrial control bandwidth. The data transmission control strategy in this solution is closely related to network load. For example, when controlling the transmission of non-critical frame conversion data, network bandwidth is acquired in real time. When the bandwidth reaches a preset value (i.e., high network load), data is discarded according to a preset discarding cycle to reduce the frame rate, avoid network congestion, and ensure the transmission of critical data. As shown in the load diagram, the constant-cycle cyclic communication of industrial control bandwidth indicates high real-time requirements for industrial control data transmission. Prioritizing the transmission of industrial conversion data streams and the transmission strategy for critical frame conversion data when network load changes aligns with the importance of industrial control data as illustrated in the diagram, ensuring stable transmission of critical information in industrial production and video.

[0149] For details on the implementation of each of the above steps, please refer to the previous examples, which will not be repeated here.

[0150] To facilitate better implementation of the data transmission method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above data transmission method. The meanings of the terms used are the same as in the above data transmission method, and specific implementation details can be found in the descriptions in the method embodiments.

[0151] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application. The data transmission device is applied to a computer device. The data transmission device may include a receiving unit 601, a data conversion unit 602, a clock synchronization unit 603, a distribution unit 604, and a control unit 605, etc.

[0152] The receiving unit 601 is used to receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time.

[0153] The data conversion unit 602 is used to convert the industrial data stream of the industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and to convert the video data stream of the Internet protocol into a video conversion data stream of a specified protocol.

[0154] The clock synchronization unit 603 is used to synchronize the clock of the industrial equipment and the monitoring equipment.

[0155] The allocation unit 604 is used to assign different transmission priorities to the industrial conversion data stream and the video conversion data stream;

[0156] The control unit 605 is used to control the industrial conversion data stream and the video conversion data stream to be transmitted according to their corresponding transmission priorities based on the transmission cycle of the industrial conversion data stream specified in the gating list.

[0157] In some embodiments, the allocation unit 604 includes:

[0158] A filtering subunit is used to filter out keyframe conversion data and non-keyframe conversion data from the video conversion data stream;

[0159] The first allocation subunit is used to allocate a first transmission priority to the industrial conversion data stream;

[0160] The second allocation subunit is used to allocate a second transmission priority to the keyframe conversion data;

[0161] The third allocation subunit is used to allocate a third transmission priority to the non-critical frame conversion data.

[0162] In some embodiments, the apparatus further includes:

[0163] The first encoding unit is used to perform lossless encoding or low compression ratio encoding on the keyframe conversion data to obtain encoded keyframe conversion data.

[0164] The second encoding unit is used to perform high compression ratio encoding on the non-key frame conversion data to obtain encoded non-key frame conversion data.

[0165] Control unit 605 includes:

[0166] The first control subunit is used to control the industrial conversion data stream to be transmitted preferentially within the transmission cycle of the industrial conversion data stream specified in the gating list;

[0167] The acquisition subunit is used to acquire the first remaining transmission duration between the target time point when the industrial conversion data stream transmission is completed and the next transmission cycle.

[0168] The second control subunit is used to control the transmission of the encoded keyframe conversion data within the first remaining transmission time.

[0169] The third control subunit is used to control the encoded non-keyframe conversion data to be transmitted within the second remaining transmission time when the encoded keyframe conversion data has been transmitted within the first remaining transmission time and there is a second remaining transmission time.

[0170] In some embodiments, the control unit 605 further includes:

[0171] The fourth control subunit is used to prevent the transmission of the encoded non-critical frame conversion data and control the transmission of the other industrial conversion data streams when it is detected that the encoded non-critical frame conversion data has not been fully transmitted and there are other industrial conversion data streams that need to be transmitted.

[0172] In some embodiments, the control unit 605 further includes:

[0173] The fifth control subunit is used to prevent the transmission of the encoded non-key frame conversion data and control the transmission of the other encoded key frame conversion data when it is detected that the encoded non-key frame conversion data has not been completely transmitted and there is other encoded key frame conversion data that needs to be transmitted.

[0174] In some embodiments, the third control subunit is configured to:

[0175] Get the current network resource bandwidth in real time;

[0176] When the current network resource bandwidth reaches the preset network resource bandwidth, during the transmission of the encoded non-critical frame conversion data within the second remaining transmission time, the encoded non-critical frame conversion data is discarded according to a preset discarding period.

[0177] In some embodiments, the third control subunit is further configured to:

[0178] When the current network resource bandwidth does not reach the preset network resource bandwidth, it is prohibited to discard the encoded non-critical frame conversion data according to the preset discarding period.

[0179] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0180] As described above, in this embodiment, the receiving unit 601 receives industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time; the data conversion unit 602 converts the industrial data streams of the industrial Ethernet protocol into industrial conversion data streams of a specified protocol, and converts the video data streams of the Internet protocol into video conversion data streams of a specified protocol; the clock synchronization unit 603 synchronizes the clocks of the industrial equipment and the monitoring equipment; the allocation unit 604 assigns different transmission priorities to the industrial conversion data streams and the video conversion data streams; and the control unit 605 controls the industrial conversion data streams and the video conversion data streams to be transmitted according to their corresponding transmission priorities based on the transmission cycle of the industrial conversion data streams specified in the gating list. Compared to other related technologies, the TSN scheduling mechanism focuses more on industrial data and has weaker support for the real-time performance and stability of video data, resulting in stuttering and frame drops in the final video content. In contrast, a gating list can be used to specify the transmission cycle of the industrial conversion data stream, allowing it to be transmitted according to a fixed cycle. Different transmission priorities can be assigned to the industrial conversion data stream and the video conversion data stream, ensuring that they are transmitted sequentially in each transmission cycle according to their priority. This achieves a balance between industrial data and video data, avoiding stuttering and frame drops in the final video content.

[0181] The specific implementation of each of the above units can be found in the previous embodiments, and will not be repeated here.

[0182] Reference Figure 5 , Figure 5 This is a partial structural block diagram of a computer device 1000 implementing an embodiment of the present disclosure. The computer device 1000, as a TSN device, can vary considerably due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 622 (e.g., one or more processors) and a memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) storing application programs 642 or data 644. The memory 632 and storage media 630 may be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the server 600. Furthermore, the CPU 622 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the server 600.

[0183] Computer device 1000 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0184] The central processing unit 622 in the computer device 1000 can be used to execute the data transmission method of the embodiments of this disclosure, for example:

[0185] It can receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time.

[0186] Convert the industrial data stream of the Industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and convert the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0187] The industrial equipment and the monitoring equipment are synchronized with clocks.

[0188] Different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream;

[0189] Based on the transmission cycle of the industrial conversion data stream as specified in the gating list, the industrial conversion data stream and the video conversion data stream are controlled to be transmitted according to their corresponding transmission priorities.

[0190] This disclosure also provides a computer-readable storage medium for storing program code for executing the data transmission methods of the foregoing embodiments.

[0191] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the data transmission method described above. For example:

[0192] It can receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time.

[0193] Convert the industrial data stream of the Industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and convert the video data stream of the Internet protocol into a video conversion data stream of a specified protocol;

[0194] The industrial equipment and the monitoring equipment are synchronized with clocks.

[0195] Different transmission priorities are assigned to the industrial conversion data stream and the video conversion data stream;

[0196] Based on the transmission cycle of the industrial conversion data stream as specified in the gating list, the industrial conversion data stream and the video conversion data stream are controlled to be transmitted according to their corresponding transmission priorities.

[0197] Furthermore, the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0198] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0199] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0200] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0201] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0202] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0203] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0204] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0205] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0206] The above is a detailed description of the embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A data transmission method, characterized in that, include: It can receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time. Convert the industrial data stream of the Industrial Ethernet protocol into an industrial conversion data stream of a specified protocol, and convert the video data stream of the Internet protocol into a video conversion data stream of a specified protocol; The industrial equipment and the monitoring equipment are synchronized with clocks. Keyframe conversion data and non-keyframe conversion data are filtered from the video conversion data stream; Assign a first transmission priority to the industrial conversion data stream; Assign a second transmission priority to the keyframe conversion data; Assign a third transmission priority to the non-critical frame conversion data; The keyframe conversion data is subjected to lossless encoding or low compression ratio encoding to obtain encoded keyframe conversion data. The non-key frame conversion data is encoded with a high compression ratio to obtain the encoded non-key frame conversion data; The industrial conversion data stream is controlled to be transmitted preferentially within the transmission cycle specified in the gating list; Obtain the first remaining transmission duration between the target time point when the industrial conversion data stream transmission is completed and the next transmission cycle; The encoded keyframe conversion data is transmitted within the first remaining transmission time. When the encoded keyframe conversion data is transmitted within the first remaining transmission time and there is a second remaining transmission time, the encoded non-keyframe conversion data is controlled to be transmitted within the second remaining transmission time.

2. The data transmission method according to claim 1, characterized in that, After controlling the transmission of the encoded non-critical frame conversion data within the second remaining transmission duration, the method further includes: When it is detected that the encoded non-critical frame conversion data has not been fully transmitted and there are other industrial conversion data streams that need to be transmitted, the transmission of the encoded non-critical frame conversion data is prohibited, and the transmission of the other industrial conversion data streams is controlled.

3. The data transmission method according to claim 2, characterized in that, The method further includes: When it is detected that the encoded non-key frame conversion data has not been fully transmitted, and there is other encoded key frame conversion data that needs to be transmitted, the transmission of the encoded non-key frame conversion data is prohibited, and the transmission of the other encoded key frame conversion data is controlled.

4. The data transmission method according to claim 1, characterized in that, The control of transmitting the encoded non-key frame conversion data within the second remaining transmission time includes: Get the current network resource bandwidth in real time; When the current network resource bandwidth reaches the preset network resource bandwidth, during the transmission of the encoded non-critical frame conversion data within the second remaining transmission time, the encoded non-critical frame conversion data is discarded according to a preset discarding period.

5. The data transmission method according to claim 4, characterized in that, The method further includes: When the current network resource bandwidth does not reach the preset network resource bandwidth, it is prohibited to discard the encoded non-critical frame conversion data according to the preset discarding period.

6. A data transmission device, characterized in that, include: The receiving unit is used to receive industrial data streams uploaded by industrial equipment and video data streams uploaded by monitoring equipment in real time. A data conversion unit is used to convert the industrial data stream of the Industrial Ethernet protocol into an industrial converted data stream of a specified protocol, and to convert the video data stream of the Internet protocol into a video converted data stream of a specified protocol. A clock synchronization unit is used to synchronize the clocks of the industrial equipment and the monitoring equipment. The allocation unit includes: A filtering subunit is used to filter out keyframe conversion data and non-keyframe conversion data from the video conversion data stream; The first allocation subunit is used to allocate a first transmission priority to the industrial conversion data stream; The second allocation subunit is used to allocate a second transmission priority to the keyframe conversion data; The third allocation subunit is used to allocate a third transmission priority to the non-critical frame conversion data; The first encoding unit is used to perform lossless encoding or low compression ratio encoding on the keyframe conversion data to obtain encoded keyframe conversion data. The second encoding unit is used to perform high compression ratio encoding on the non-key frame conversion data to obtain encoded non-key frame conversion data. Control unit, including: The first control subunit is used to control the industrial conversion data stream to be transmitted preferentially within the transmission cycle of the industrial conversion data stream specified in the gating list; The acquisition subunit is used to acquire the first remaining transmission duration between the target time point when the industrial conversion data stream transmission is completed and the next transmission cycle. The second control subunit is used to control the transmission of the encoded keyframe conversion data within the first remaining transmission time. The third control subunit is used to control the encoded non-keyframe conversion data to be transmitted within the second remaining transmission time when the encoded keyframe conversion data has been transmitted within the first remaining transmission time and there is a second remaining transmission time.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the data transmission method according to any one of claims 1 to 5.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data transmission method according to any one of claims 1 to 5.

Images