A method for audio and video scheduling across physically isolated networks

Abstract

The present invention discloses a method for audio and video scheduling across physically isolated networks. The method relates to the field of electrical communication technology, and includes the following steps: converting audio and video into audio and video signals through an acquisition device, compressing and encoding the audio and video signals and encapsulating them into data packets, encrypting the data packets using an encryption algorithm, installing a scheduling machine on each cross-physical network, establishing a secure tunnel between cross-physical isolated networks through the scheduling machine, designing a scheduling strategy based on time synchronization, testing the secure tunnel in an actual network environment, and adjusting and optimizing the scheduling strategy based on time synchronization according to the test results. The present invention first uses an asymmetric encryption algorithm to securely exchange keys, and then uses a symmetric encryption algorithm to encrypt the data packet itself, thereby achieving the effect of improving the security of audio and video data transmission across physically isolated networks, and effectively solving the problem of insufficient security of audio and video data transmission across physically isolated networks in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of electrical communication technology, and in particular to an audio and video scheduling method across a physically isolated network. Background Art

[0002] With the advancement of electronic technology, digital signal processing technology has become the basis of audio and video content processing. It includes the sampling, quantization, encoding and compression of audio and video. In audio and video processing and transmission, the scheduling algorithm can optimize the use of resources. In order to improve the transmission efficiency and quality of audio and video content, content distribution network technology is widely used. It reduces the delay and bandwidth consumption of data transmission by deploying a large number of cache servers in the network.

[0003] For example, an audio and video scheduling method and system announced by the invention patent with announcement number: CN116437116B includes: establishing a data communication relationship with a terminal; collecting the video stream and audio stream collected by the terminal and dividing the video stream and audio stream in terms of time length; compressing the audio stream segment; grayscale processing the image in the video stream segment to obtain a reference image and divide the reference image into regions to obtain multiple sub-reference images; gridding the sub-reference image to obtain multiple sub-reference image units, and the area of ​​the sub-reference image unit is less than or equal to the set unit area; compressing the sub-reference image unit and sending the compressed video stream segment and audio stream segment to the terminal.

[0004] For example, the invention patent announcement number is CN113179426B, which discloses a highly confidential audio and video command and dispatch system, including: a front-end user terminal, a back-end command and dispatch center, a confidential communication module, and a communication networking device. Among them, the front-end user terminal includes a handheld terminal, a mobile vehicle-mounted device, and an auxiliary monitoring device; the back-end command and dispatch center includes a system server and a command and dispatch platform, and the command and dispatch platform includes a dispatch device, a video decoding device, and a monitoring display device; the confidential communication module includes an encryption submodule and a decryption submodule; the communication networking device is used to establish communication between the front-end user terminal and the back-end command and dispatch center based on the communication network of the communication operator to realize the transmission of instructions or data;

[0005] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0006] In the prior art, when transmitting audio and video data, the security of data transmission cannot be guaranteed, and the data may be tampered with during transmission, resulting in the problem of insufficient security of transmitting audio and video data. Summary of the invention

[0007] The embodiments of the present application provide an audio and video scheduling method across physically isolated networks, thereby solving the problem of insufficient security of audio and video data transmitted across physically isolated networks in the prior art, and achieving the effect of improving the security of audio and video data transmitted across physically isolated networks.

[0008] The embodiment of the present application provides an audio and video scheduling method across physically isolated networks, comprising the following steps: converting audio and video into audio and video signals through an acquisition device, and compressing and encoding the audio and video signals using an audio and video encoder; encapsulating the encoded audio and video data into data packets, and encrypting the data packets using an encryption algorithm to obtain encrypted data packets; installing a scheduling machine on each cross-physical network, establishing a secure tunnel between cross-physical isolated networks through the scheduling machine, and transmitting the encrypted data packets through the secure tunnel; designing a scheduling strategy based on time synchronization, testing the secure tunnel in an actual network environment, and adjusting and optimizing the scheduling strategy based on time synchronization according to the test results.

[0009] Furthermore, the specific compression encoding process of using the audio and video encoder to compress the audio and video signals is: compressing the data volume of the audio signal, compressing the video signal using motion compensation technology, and obtaining the preprocessed audio and video signals.

[0010] Furthermore, the specific encapsulation process of encapsulating the encoded audio and video data into data packets is: encode the preprocessed audio and video signals through an encoder, encode the video data into H.264 format, and encode the audio data into AAC format to obtain the encoded audio and video data, divide the encoded audio and video data into multiple small data packets of fixed size, add timestamp information and data packet sequence number to each data packet, and encapsulate the data packets using the TCP network protocol.

[0011] Furthermore, the specific encryption process of encrypting a data packet using an encryption algorithm is as follows: using the timestamp information of the data packet as an initial vector, selecting a key generation algorithm generated by a pseudo-random number to generate a key, first using an asymmetric encryption algorithm to securely exchange the key, and then using a symmetric encryption algorithm to encrypt the data packet itself to obtain an encrypted data packet.

[0012] Furthermore, the specific process of establishing a secure tunnel across physically isolated networks is as follows: configuring an IP address, subnet mask and gateway for the scheduler, using the IPsec tunnel protocol to establish multiple tunnel endpoints based on the IP address, subnet mask and gateway of the scheduler, and building a secure tunnel between the tunnel endpoints.

[0013] Furthermore, the specific design process of designing a scheduling strategy based on time synchronization is as follows: determine the lowest delay as the scheduling goal, determine the highest available bandwidth as the scheduling constraint, select the Network Time Protocol as the time synchronization method for real-time scheduling, and use the received timestamp to determine the time of sending and receiving audio and video data packets.

[0014] Furthermore, the specific test process of testing the secure tunnel in the actual network environment is as follows: prepare two physically isolated networks, deploy the scheduling machine, audio and video encoding, transmission and decoding software in the two networks; use the IPsec tunnel protocol to establish a secure tunnel between the two physically isolated networks, and configure the tunnel parameters according to the encoded audio and video data; transmit the encrypted data packets through the secure tunnel, simulate different network environments and monitor the synchronization error, delay and jitter of the transmission, and obtain the synchronization error, delay and jitter under different network environments; evaluate the performance of the audio and video scheduling system running under the secure tunnel, compare it with the performance when the secure tunnel is not used, and evaluate the impact of the secure tunnel on the audio and video scheduling system.

[0015] Furthermore, the specific optimization process of adjusting and optimizing the scheduling strategy based on time synchronization is carried out according to the test results: the scheduling strategy is adjusted according to the synchronization error, delay and jitter under different network environments, and a redundant transmission path is introduced on the main transmission path. When the synchronization error, delay and jitter of the main path are higher than the test results, switch to the redundant transmission path; the scheduling strategy is adjusted according to the evaluation results of the performance of the audio and video scheduling system running in a secure tunnel, the bandwidth utilization of each tunnel endpoint load is collected in real time, and the scheduling strategy is dynamically adjusted according to the bandwidth utilization of each tunnel endpoint load.

[0016] Furthermore, a specific evaluation process is provided to evaluate the performance of the audio and video scheduling system running under a secure tunnel: the network environment of the secure tunnel is evaluated using the synchronization error, delay, and jitter obtained from the test to obtain a network fluctuation evaluation coefficient; the bandwidth utilization obtained from the test is used to evaluate the tunnel endpoint load of the secure tunnel to obtain a tunnel endpoint load evaluation coefficient; the network fluctuation evaluation coefficient and the tunnel endpoint load evaluation coefficient are used to comprehensively evaluate the secure tunnel to obtain a secure tunnel evaluation coefficient.

[0017] Furthermore, the specific constraint formula of the safety tunnel evaluation coefficient is:

[0018]

[0019] In the formula, Represents the security tunnel assessment coefficient, which is used to evaluate the comprehensive impact of network fluctuations and tunnel endpoint load on the security tunnel. represents the network fluctuation assessment coefficient, represents the tunnel endpoint load assessment coefficient, and e represents a natural constant.

[0020] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0021] 1. By first using an asymmetric encryption algorithm to securely exchange keys, and then using a symmetric encryption algorithm to encrypt the data packet itself, an encrypted data packet is obtained, thereby achieving the effect of improving the security of audio and video data transmission across physically isolated networks, and effectively solving the problem of insufficient security of audio and video data transmission across physically isolated networks in the prior art.

[0022] 2. By using the IPsec tunnel protocol to establish a secure tunnel, the tunnel parameters are configured according to the encoded audio and video data, thereby achieving the effect of improving the stability of audio and video data transmission across physically isolated networks, and effectively solving the problem of insufficient stability of audio and video data transmission across physically isolated networks in the prior art.

[0023] 3. By simulating different network environments and monitoring the synchronization error, delay and jitter of the transmission, the synchronization error, delay and jitter in different network environments are obtained, so that the scheduling strategy can be adjusted according to the synchronization error, delay and jitter in different network environments, thereby achieving the effect of improving the real-time transmission of audio and video data across physically isolated networks, and effectively solving the problem of insufficient real-time transmission of audio and video data across physically isolated networks in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flow chart of an audio and video scheduling method across a physically isolated network provided in an embodiment of the present application;

[0025] Figure 2 An image of a network fluctuation assessment coefficient in an audio and video scheduling method across a physically isolated network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application solve the problem of insufficient security of audio and video data transmission across physically isolated networks in the prior art by providing an audio and video scheduling method across physically isolated networks. The method first uses an asymmetric encryption algorithm to securely exchange keys, and then uses a symmetric encryption algorithm to encrypt the data packet itself to obtain an encrypted data packet, thereby achieving the effect of improving the security of audio and video data transmission across physically isolated networks.

[0027] The technical solution in the embodiment of the present application is to solve the above-mentioned problem of insufficient security of audio and video data transmission across physically isolated networks. The overall idea is as follows:

[0028] By designing a scheduling strategy based on time synchronization, testing the secure tunnel in an actual network environment, and adjusting and optimizing the scheduling strategy based on time synchronization according to the test results, the security of transmitting audio and video data across physically isolated networks is improved.

[0029] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] like Figure 1 As shown, it is a flow chart of an audio and video scheduling method across physically isolated networks provided in an embodiment of the present application, the method comprising the following steps: converting audio and video into audio and video signals through an acquisition device, and compressing and encoding the audio and video signals using an audio and video encoder; encapsulating the encoded audio and video data into data packets, and encrypting the data packets using an encryption algorithm to obtain encrypted data packets; installing a scheduling machine on each cross-physical network, establishing a secure tunnel between cross-physical isolated networks through the scheduling machine, and transmitting the encrypted data packets through the secure tunnel; designing a scheduling strategy based on time synchronization, testing the secure tunnel in an actual network environment, and adjusting and optimizing the scheduling strategy based on time synchronization according to the test results.

[0031] Furthermore, the specific compression encoding process of using the audio and video encoder to compress the audio and video signals is: compressing the data volume of the audio signal, compressing the video signal using motion compensation technology, and obtaining the preprocessed audio and video signals.

[0032] In this embodiment, the audio signal is compressed using data volume compression technology to reduce the 16-bit depth to 8 bits or convert the stereo signal into a mono signal, and the motion compensation technology is used to detect and compensate for the motion between video frames to reduce the amount of data in the video compression to obtain the preprocessed audio and video signal. The compressed audio and video data only needs to use less bandwidth for transmission, which can reduce the bandwidth demand in a limited network bandwidth environment.

[0033] Furthermore, the specific encapsulation process of encapsulating the encoded audio and video data into data packets is: encode the preprocessed audio and video signals through an encoder, encode the video data into H.264 format, and encode the audio data into AAC format to obtain the encoded audio and video data, divide the encoded audio and video data into multiple small data packets of fixed size, add timestamp information and data packet sequence number to each data packet, and encapsulate the data packets using the TCP network protocol.

[0034] In this embodiment, the preprocessed audio and video signals of other coding formats are encoded by the encoder, the video data is encoded in H.264 format, and the audio data is encoded in AAC format. The use of H.264 and AAC encoding ensures high compression efficiency and quality of video and audio. The encoded audio and video data are divided into fixed-size data packets for network transmission and processing. A timestamp and sequence number are added to the header of each data packet to ensure the order and synchronization of the data during transmission. The specific encapsulation process of encapsulating the data packet using the TCP network protocol (Transmission Control Network Protocol) is as follows: if the data is larger than the MSS (maximum segment size) of TCP, the data will be divided into multiple TCP segments. Each TCP segment includes a TCP header and a portion of data. The data to be identified is added to the TCP header and encapsulated into the data packet. The receiving party will parse the received data packet, extract the TCP segment and verify the key of the data packet.

[0035] Furthermore, the specific encryption process of encrypting a data packet using an encryption algorithm is as follows: using the timestamp information of the data packet as an initial vector, selecting a key generation algorithm generated by a pseudo-random number to generate a key, first using an asymmetric encryption algorithm to securely exchange the key, and then using a symmetric encryption algorithm to encrypt the data packet itself to obtain an encrypted data packet.

[0036] In this embodiment, when the audio and video scheduling system is started, the timestamp of the data packet is used as the initial vector. Even if the data packets sent continuously in a short period of time have different initial vectors, using the timestamp as the initial vector can prevent replay attacks because the IV of each data packet is unique, which can improve the security of the audio and video scheduling system. A pseudo-random number generation algorithm pre-set by the system is selected to create a key, and an asymmetric encryption algorithm is used to allow the sender and the receiver to securely exchange keys. The key and the initial vector are transmitted to the receiver through a secure tunnel. The receiver uses a symmetric encryption algorithm to encrypt the data packet itself to obtain an encrypted data packet.

[0037] Furthermore, the specific process of establishing a secure tunnel across physically isolated networks is as follows: configuring an IP address, subnet mask and gateway for the scheduler, using the IPsec tunnel protocol to establish multiple tunnel endpoints based on the IP address, subnet mask and gateway of the scheduler, and building a secure tunnel between the tunnel endpoints.

[0038] In this embodiment, an IP address, subnet mask and gateway are configured for the dispatcher according to the network environment. Based on the IP address, subnet mask and gateway configured by the dispatcher, an IPsec tunnel protocol is used to establish multiple tunnel endpoints, and a secure tunnel is built between the tunnel endpoints to ensure that audio and video data is not illegally accessed, tampered with or leaked during transmission, thereby ensuring the confidentiality, integrity and availability of data transmission.

[0039] In this embodiment, the audio and video scheduling system can also be implemented by the following method: in a system of multiple scheduling machines, one scheduling machine is selected as the main scheduling machine and the others are slave scheduling machines. The scheduling console is located on the main scheduling machine, and the audio, video, and data services on the slave scheduling machine are automatically forwarded to the main scheduling machine; the status information on the slave scheduling machine is also forwarded to the main scheduling machine, and the audio and video scheduling is uniformly performed on the main scheduling machine.

[0040] The slave dispatch machine uses a virtual dispatch desk to map its dispatch function to the master dispatch desk, and maps the audio and video of the left and right telephones, gooseneck microphones, and video intercom functions on the virtual dispatch desk to the corresponding audio interface on the dispatch desk of the master dispatch machine through the E1 audio channel and the video channel of the video optical terminal. The UI dispatch interface and operation commands of the master and slave dispatch machines are synchronized through the serial port information in E1, just like directly operating the dispatch desk on the slave dispatch machine.

[0041] There must be enough audio time slots and video channels between the dispatchers of the two networks to ensure that when audio conferences and video conferences are held, the number of concurrent meetings between the systems can be met. When there are many concurrent meeting members, conference rooms are opened on their respective dispatchers and conference rooms on multiple dispatchers are connected together.

[0042] Furthermore, the specific design process of designing a scheduling strategy based on time synchronization is as follows: determine the lowest delay as the scheduling goal, determine the highest available bandwidth as the scheduling constraint, select the Network Time Protocol as the time synchronization method for real-time scheduling, and use the received timestamp to determine the time of sending and receiving audio and video data packets.

[0043] In this embodiment, the minimum delay is determined as the main goal of the audio and video scheduling system, the maximum available bandwidth is determined as the constraint condition of the audio and video scheduling system, the network time protocol is selected as the time synchronization method for real-time scheduling, and ntpclient (NTP client software) is installed on each audio and video playback device to communicate with the NTP server, and an NTP server is configured to provide time synchronization services for all devices. The NTP server can be a dedicated time server, or a device with NTP service function, to ensure that all audio and video playback devices can access the NTP server. This may require setting up appropriate proxies or VPN connections between network isolation zones, starting ntpclient, and synchronizing the time of each device with the NTP server. The devices keep the time consistent with each other through the NTP protocol, and the NTP protocol can provide microsecond-level time synchronization accuracy. When the audio and video are playing, each device controls the playback progress according to the synchronized timestamp to ensure the synchronous playback of the audio and video.

[0044] Furthermore, the specific test process of testing the secure tunnel in the actual network environment is as follows: prepare two physically isolated networks, deploy the scheduling machine, audio and video encoding, transmission and decoding software in the two networks; use the IPsec tunnel protocol to establish a secure tunnel between the two physically isolated networks, and configure the tunnel parameters according to the encoded audio and video data; transmit the encrypted data packets through the secure tunnel, simulate different network environments and monitor the synchronization error, delay and jitter of the transmission, and obtain the synchronization error, delay and jitter under different network environments; evaluate the performance of the audio and video scheduling system running under the secure tunnel, compare it with the performance when the secure tunnel is not used, and evaluate the impact of the secure tunnel on the audio and video scheduling system.

[0045] In this embodiment, two physically isolated networks, network A and network B, are prepared. The audio and video signals of network A need to be scheduled to network B for playback. In network A, an audio and video encoder is set to encode the signal and send it to the dispatcher. In network B, the dispatcher forwards the signal to the decoder for playback. An IPsec secure tunnel is established between network A and network B using the IPsec tunnel protocol. The IPsec tunnel improves the security of data transmission and protects the audio and video content from unauthorized access. By actually transmitting and monitoring data packets, it is found that despite the addition of a secure tunnel, the synchronization error, delay and jitter of the audio and video signals are within an acceptable range. Performance evaluation shows that the performance of the audio and video scheduling system under the secure tunnel is almost the same as when the secure tunnel is not used, indicating that the method is effective.

[0046] Furthermore, the specific optimization process of adjusting and optimizing the scheduling strategy based on time synchronization is carried out according to the test results: the scheduling strategy is adjusted according to the synchronization error, delay and jitter under different network environments, and a redundant transmission path is introduced on the main transmission path. When the synchronization error, delay and jitter of the main path are higher than the test results, switch to the redundant transmission path; the scheduling strategy is adjusted according to the evaluation results of the performance of the audio and video scheduling system running in a secure tunnel, the bandwidth utilization of each tunnel endpoint load is collected in real time, and the scheduling strategy is dynamically adjusted according to the bandwidth utilization of each tunnel endpoint load.

[0047] In this embodiment, audio and video signals are transmitted between two physically isolated networks, and an IPsec secure tunnel is established using the IPsec tunnel protocol. Through testing, it is found that the synchronization error, delay, and jitter on the main path remain within an acceptable range during normal teaching time, but will increase during peak hours. According to the test results, a redundant transmission path is introduced into the scheduling strategy, that is, a backup secure tunnel is used during peak hours. Through real-time monitoring, it is found that the performance index of the main path exceeds the threshold during peak hours, and the system automatically switches to the redundant path, ensuring the stability of audio and video transmission. At the same time, the bandwidth utilization data of each tunnel endpoint is collected in real time, and the scheduling strategy is dynamically adjusted according to this data, such as increasing the transmission priority at the endpoint with lower bandwidth utilization, to ensure high-quality audio and video transmission.

[0048] Furthermore, a specific evaluation process is provided to evaluate the performance of the audio and video scheduling system running under a secure tunnel: the network environment of the secure tunnel is evaluated using the synchronization error, delay, and jitter obtained from the test to obtain a network fluctuation evaluation coefficient; the bandwidth utilization obtained from the test is used to evaluate the tunnel endpoint load of the secure tunnel to obtain a tunnel endpoint load evaluation coefficient; the network fluctuation evaluation coefficient and the tunnel endpoint load evaluation coefficient are used to comprehensively evaluate the secure tunnel to obtain a secure tunnel evaluation coefficient.

[0049] In this embodiment, two physically isolated networks are prepared to transmit audio and video signals, and an IPsec secure tunnel is established using the IPsec tunnel protocol. Through testing, it is found that the synchronization errors, delays, and jitters of different secure tunnels are different. Some tunnels have higher delays during peak hours. According to the test results, network fluctuation assessment coefficients of different secure tunnels are obtained. The higher the coefficient, the worse the network environment. At the same time, the bandwidth utilization data of each tunnel endpoint is collected in real time. The tunnel endpoint load assessment coefficient is obtained based on the bandwidth utilization data. The lower the coefficient, the lighter the endpoint load. The network fluctuation assessment coefficient and the tunnel endpoint load assessment coefficient are used to comprehensively evaluate each secure tunnel to obtain a secure tunnel assessment coefficient. According to the secure tunnel assessment coefficient, the scheduling strategy is optimized, and secure tunnels with higher comprehensive assessment coefficients are preferentially used for audio and video transmission.

[0050] Furthermore, the specific constraint formula of the safety tunnel evaluation coefficient is:

[0051]

[0052] In the formula, Indicates the security tunnel assessment coefficient, which is used to evaluate the comprehensive impact of network fluctuations and tunnel endpoint load on the security tunnel. Comprehensive evaluation using the security tunnel assessment coefficient can more accurately understand the actual load of network fluctuations and tunnel endpoints, avoid resource waste, and ensure smooth transmission of audio and video data. represents the network fluctuation assessment coefficient, represents the tunnel endpoint load assessment coefficient, and e represents a natural constant.

[0053] In this embodiment, the specific constraint formula of the network fluctuation assessment coefficient is:

[0054]

[0055] In the formula, It represents the network fluctuation assessment coefficient, which is used to assess the impact of network fluctuation on the secure tunnel. Several dispatcher monitoring points are set for the secure tunnel. Z = 1, 2, 3, ..., a, where a represents the total number of dispatchers, and A Z represents the network bandwidth rate at the monitoring point of the Zth dispatcher, It is represented by the average network bandwidth rate of a scheduling machine, B represents the network packet loss rate, C represents the network response time, and D Z represents the amount of transmission data under the monitoring point of the Zth dispatcher, It is expressed as the average amount of data transmitted by a scheduling machines, and π is expressed as a natural constant.

[0056] In a specific embodiment, the network fluctuation assessment coefficient data is exemplified in the following table.

[0057] Table 1 Network fluctuation assessment coefficient data example

[0058]

[0059] From the data in Table 1, it can be seen that when the network bandwidth rate, the amount of transmitted data, and the network response time remain unchanged, the greater the network packet loss rate, the higher the network fluctuation assessment coefficient, and the network packet loss rate is positively correlated with the network fluctuation assessment coefficient.

[0060] Figure 2An image of a network fluctuation assessment coefficient in an audio and video scheduling method across a physically isolated network provided in an embodiment of the present application, wherein the network packet loss rate B is the horizontal axis independent variable of the function, function a is a curve showing how the network fluctuation assessment coefficient changes with the network packet loss rate when the network bandwidth rate is 10Mbps, the network response time is 90ms, and the transmission data volume is 10Mb, the network fluctuation assessment coefficient increases with the increase of the network packet loss rate, 90 is the network response time, 100 is the network bandwidth rate multiplied by the transmission data volume, and the fixed value 90 / 100 is calculated by the formula as the coefficient of the horizontal axis independent variable of function a, function b is a curve showing how the network fluctuation assessment coefficient changes with the network packet loss rate when the network bandwidth rate is 20Mbps, the network response time is 90ms, and the transmission data volume is 10Mb, the network fluctuation assessment coefficient increases with the increase of the network packet loss rate, 90 is the network response time, 200 is the network bandwidth rate and the transmission data volume The data volume is multiplied, and the fixed value 90 / 200 is calculated by the formula as the coefficient of the horizontal axis independent variable of function b. Function c is a curve of the network fluctuation assessment coefficient changing with the network packet loss rate when the network bandwidth rate is 30Mbps, the network response time is 90ms and the transmission data volume is 10Mb. The network fluctuation assessment coefficient increases with the increase of the network packet loss rate. 90 is the network response time, and 300 is the network bandwidth rate multiplied by the transmission data volume. The fixed value 90 / 300 is calculated by the formula as the coefficient of the horizontal axis independent variable of function c. Curves a, b and c represent the function images when the network bandwidth rates are 10Mbps, 20Mbps and 30Mbps. When the network packet loss rate takes the same value, the network fluctuation assessment coefficient of curve a is the largest, followed by curve b, and curve c is the smallest. By observing the changes in the network fluctuation assessment coefficient under different function images, the problem that the network fluctuation cannot be accurately assessed in the prior art is solved.

[0061] Network bandwidth rate refers to the ability of a network to transmit data per unit time. Network bandwidth rate can be tested using network analysis tools (such as speedtest.net) to measure upload and download rates. The bandwidth rate of home broadband is usually between 10Mbps-100Mbps, and commercial-grade broadband is between 100Mbps-10Gbps.

[0062] The network packet loss rate refers to the ratio of data packets lost due to network problems during data transmission. The network packet loss rate can be calculated by sending specific data packets and monitoring the number of lost data packets. The ideal network packet loss rate should be 0%, but in the actual network environment, the packet loss rate may be between 0% and 1%.

[0063] The network response time refers to the time from sending a request to receiving a response. The network response time can be measured by a ping test. In a high-quality network environment, the response time ranges from 10-100ms.

[0064] Data transmission volume refers to the total amount of data sent through the network during audio and video transmission. Data transmission volume can be measured by network traffic monitoring tools. Data transmission volume may range from 10MB to 100MB.

[0065] In this embodiment, the specific constraint formula of the tunnel endpoint load evaluation coefficient is:

[0066]

[0067] In the formula, It represents the tunnel endpoint load assessment coefficient, which is used to assess the impact of the tunnel endpoint load on the secure tunnel. H represents the tunnel traffic. G represents the number of concurrent connections. F represents the packet delay. J represents the resource utilization of the tunnel endpoint. e represents a natural constant.

[0068] Tunnel traffic refers to the amount of data that passes through a secure tunnel per unit time. The amount of data entering and leaving the tunnel is measured using network monitoring tools or traffic analyzers. The tunnel traffic value range is 100-1000kbps.

[0069] The number of concurrent connections refers to the number of network connections established simultaneously through the tunnel. The number of concurrent connections can be tracked using network management software or tools provided by the operating system. The value range of the number of concurrent connections is 10-100.

[0070] Packet delay refers to the time required for a packet to travel from the sender to the receiver. Packet delay can be measured using a network test tool. The value range of packet delay is 1-10ms.

[0071] The resource utilization of a tunnel endpoint refers to the ratio of system resources (such as CPU, memory) consumed by the tunnel endpoint to the total resource capacity. The CPU and memory usage can be monitored through the operating system or specialized network management software. The resource utilization of a tunnel endpoint ranges from 50% to 70%.

[0072] The technical solutions in the above-mentioned embodiments of the present application have at least the following technical effects or advantages: relative to an audio and video scheduling method and system announced in the invention patent with publication number: CN116437116B, the embodiments of the present application first use an asymmetric encryption algorithm to securely exchange keys, and then use a symmetric encryption algorithm to encrypt the data packet itself, thereby obtaining an encrypted data packet, thereby achieving the effect of improving the security of audio and video data transmission across physically isolated networks; relative to a highly confidential audio and video command and scheduling system announced in the invention patent with publication number: CN113179426B, the embodiments of the present application establish a secure tunnel by using the IPsec tunnel protocol, thereby configuring tunnel parameters according to the encoded audio and video data, thereby achieving the effect of improving the stability of audio and video data transmission across physically isolated networks.

[0073] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0078] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for audio and video scheduling across physically isolated networks, characterized in that: The following steps are involved: Convert the audio and video into audio and video signals through the acquisition device, and use the audio and video encoder to compress and encode the audio and video signals; Encapsulating the encoded audio and video data into a data packet, and encrypting the data packet using an encryption algorithm to obtain an encrypted data packet; Install a dispatcher on each cross-physical network, establish a secure tunnel between cross-physical isolated networks through the dispatcher, and transmit encrypted data packets through the secure tunnel; Design a scheduling strategy based on time synchronization, test the secure tunnel in an actual network environment, and adjust and optimize the scheduling strategy based on time synchronization according to the test results; The specific encryption process of using the encryption algorithm to encrypt the data packet is as follows: The timestamp information of the data packet is used as the initial vector, and a key generation algorithm generated by a pseudo-random number is selected to generate a key. The key is first securely exchanged using an asymmetric encryption algorithm, and then the data packet itself is encrypted using a symmetric encryption algorithm to obtain an encrypted data packet; The specific process of establishing a secure tunnel across physically isolated networks by the dispatcher is as follows: Configure the IP address, subnet mask and gateway for the dispatcher, use the IPsec tunnel protocol to establish multiple tunnel endpoints based on the IP address, subnet mask and gateway of the dispatcher, and build a secure tunnel between the tunnel endpoints.

2. The method for scheduling audio and video across physically isolated networks as claimed in claim 1, characterized in that: The specific compression encoding process of using the audio and video encoder to compress and encode the audio and video signals is as follows: The audio signal is compressed, and the video signal is compressed using motion compensation technology to obtain preprocessed audio and video signals.

3. The audio and video scheduling method across a physically isolated network as claimed in claim 1, characterized in that: The specific encapsulation process of encapsulating the encoded audio and video data into a data packet is as follows: The preprocessed audio and video signals are encoded through an encoder, the video data is encoded into H.264 format, and the audio data is encoded into AAC format to obtain the encoded audio and video data, which are divided into multiple small data packets of fixed size, and timestamp information and data packet sequence number are added to each data packet, and the data packets are encapsulated using the TCP network protocol.

4. The method for scheduling audio and video across physically isolated networks as claimed in claim 1, characterized in that: The specific design process of the scheduling strategy based on time synchronization is as follows: The lowest delay is determined as the scheduling target, the highest available bandwidth is determined as the scheduling constraint, the Network Time Protocol is selected as the time synchronization method for real-time scheduling, and the received timestamp is used to determine the time of sending and receiving audio and video data packets.

5. The method for scheduling audio and video across physically isolated networks as claimed in claim 1, characterized in that: The specific test process of testing the secure tunnel in the actual network environment is as follows: Prepare two physically isolated networks and deploy dispatchers, audio and video encoding, transmission, and decoding software in the two networks; Use IPsec tunneling protocol to establish a secure tunnel between two physically isolated networks, and configure tunnel parameters based on the encoded audio and video data; Transmit encrypted data packets through secure tunnels, simulate different network environments and monitor transmission synchronization errors, delays, and jitters to obtain synchronization errors, delays, and jitters under different network environments; Evaluate the performance of the audio and video scheduling system running under the secure tunnel, compare it with the performance when the secure tunnel is not used, and evaluate the impact of the secure tunnel on the audio and video scheduling system.

6. The method for scheduling audio and video across physically isolated networks as claimed in claim 1, characterized in that: The specific optimization process of adjusting and optimizing the scheduling strategy based on time synchronization according to the test results is as follows: Adjust the scheduling strategy according to the synchronization error, delay and jitter in different network environments, introduce a redundant transmission path on the main transmission path, and switch to the redundant transmission path when the synchronization error, delay and jitter of the main path are higher than the test results; The scheduling strategy is adjusted according to the evaluation results of the performance of the audio and video scheduling system running under the secure tunnel, the bandwidth utilization of each tunnel endpoint load is collected in real time, and the scheduling strategy is dynamically adjusted according to the bandwidth utilization of each tunnel endpoint load.

7. The method for scheduling audio and video across physically isolated networks as claimed in claim 5, characterized in that: The specific evaluation process of evaluating the performance of the audio and video scheduling system running in a secure tunnel is as follows: Use the synchronization error, delay and jitter obtained in the test to evaluate the network environment of the secure tunnel and obtain the network fluctuation evaluation coefficient; Use the bandwidth utilization rate obtained in the test to evaluate the load on the endpoint of the secure tunnel, and obtain a tunnel endpoint load evaluation coefficient; The network fluctuation assessment coefficient and the tunnel endpoint load assessment coefficient are used to comprehensively assess the security tunnel and obtain the security tunnel assessment coefficient.

8. The method for scheduling audio and video across physically isolated networks as claimed in claim 7, characterized in that: The specific constraint formula of the safety tunnel evaluation coefficient is: In the formula, Represents the security tunnel assessment coefficient, which is used to evaluate the comprehensive impact of network fluctuations and tunnel endpoint load on the security tunnel. represents the network fluctuation assessment coefficient, represents the tunnel endpoint load assessment coefficient, and e represents a natural constant.

Images