An audio and video transmission method and system based on DDS technology

By utilizing the publish-subscribe mechanism and Liveliness strategy of DDS technology, high availability and real-time performance of the audio and video transmission system are achieved, solving the system crash problem caused by node failure in existing technologies and ensuring the reliability and efficiency of data transmission.

CN116405469BActive Publication Date: 2026-11-17BEIJING HUAYUTONGSOFT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310463032.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2026-11-17
Estimated Expiration
2043-04-26

AI Technical Summary

Technical Problem

Existing audio and video transmission protocols are prone to system crashes when there is network congestion or hardware failure, and cannot guarantee high availability and real-time performance. In particular, in point-to-point communication mode, the failure of a single node can lead to transmission failure.

Method used

The publish-subscribe mechanism using DDS technology transforms the audio and video transmission relationship into a publish-subscribe relationship mediated by topics. Multiple data publishers are set up as primary and backup nodes, and the Liveliness strategy enables real-time monitoring and rapid switching to ensure continued transmission in the event of node failure.

Benefits of technology

It improves the availability and real-time performance of the audio and video transmission system, with low resource consumption and high transmission efficiency during master-slave switching. The node failure switching time can be less than 10ms, ensuring the stability and real-time performance of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116405469B_ABST
    Figure CN116405469B_ABST
Patent Text Reader

Abstract

This invention discloses an audio and video transmission method and system based on DDS technology, relating to the field of audio and video transmission. The method includes: a first data publisher sending encapsulated format data and compressed structure data; a data receiver decoding the compressed structure data; determining whether the first data publisher has failed; if not, determining whether all compressed structure data has been sent; if yes, the program terminates; otherwise, the first data publisher continues sending compressed structure data; if yes, a second data publisher sends compressed structure data; the data receiver decodes the compressed structure data and determines whether all compressed structure data has been sent; if yes, the program terminates; otherwise, it continues to determine whether the first data publisher has failed. This invention solves the problem of the entire transmission system becoming unavailable due to the failure of one node through primary / backup switching, thereby improving the availability of the audio and video transmission system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of audio and video transmission, and more specifically, to an audio and video transmission method and system based on DDS technology. Background Technology

[0002] With the rapid development of information technology, audio and video technology applications are now ubiquitous, such as short videos, live streaming, autonomous driving, and aerospace, all of which establish communication with users based on audio and video streaming media.

[0003] During audio and video transmission, network congestion or hardware failure at a certain node may occur, leading to the collapse of the entire audio and video transmission system and reducing its availability. Currently, mainstream audio and video transmission protocols such as RTMP (Real Time Messaging Protocol), RTSP (Real Time Streaming Protocol), and HLS (HTTP Live Streaming) are based on the traditional TCP / UDP model at the transport layer, aiming to improve the stability of streaming media during network transmission.

[0004] Of the three mainstream audio and video transmission protocols mentioned above, RTSP offers the best real-time performance, using UDP at the transport layer. However, it relies on other protocols and cannot guarantee high availability. RTMP is highly adaptable with low latency, and its cluster implementation allows for node switching. However, its complexity leads to poor performance. HLS suffers from high latency and poor real-time performance, making it suitable only for on-demand playback and not for real-time audio and video transmission. Furthermore, all three protocols rely on point-to-point communication for audio and video transmission. A failure in any node will cause audio and video transmission to fail, compromising the high availability of the entire audio and video transmission system. Summary of the Invention

[0005] This invention provides an audio and video transmission method and system based on DDS technology to overcome at least one technical problem existing in the prior art.

[0006] On one hand, embodiments of the present invention provide an audio and video transmission method based on DDS technology, including:

[0007] Establish data publishers and data receivers; the data publishers include at least a first data publisher and a second data publisher;

[0008] Configure QoS policies for the data publisher and the data receiver; the QoS policy of the data publisher includes ownership, ownership strength, and liveness, and the QoS policy of the data receiver includes ownership and liveness; wherein, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher;

[0009] The original audio and video files are parsed using a decoding protocol to obtain the encapsulated format data.

[0010] The encapsulated data is decapsulated to obtain compressed structure data;

[0011] The first data publisher sends the encapsulated format data and the compressed structure data to the data receiver;

[0012] The data receiver decodes the compressed structure data according to the encapsulation format data to obtain uncompressed data;

[0013] The system determines whether the first data publisher has malfunctioned. If not, it determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, the first data publisher continues to send the compressed structure data to the data receiver. If yes, the second data publisher sends the compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the encapsulation format data to obtain uncompressed data, and determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, it continues to determine whether the first data publisher has malfunctioned.

[0014] Optionally, the compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data.

[0015] Decoding the compressed structure data yields uncompressed data, specifically:

[0016] When the data receiver receives audio compressed encoded data, it decodes the audio compressed encoded data according to the encapsulation format data to obtain the original audio data;

[0017] When the data receiver receives video compressed encoded data, it decodes the video compressed encoded data according to the encapsulation format data to obtain the original video data.

[0018] Optionally, the QoS policies for the data publisher and the data receiver are as follows: ownership is set to exclusive, liveness is set to manual confirmation by topic, and the live lease period of the data publisher is shorter than that of the data receiver.

[0019] Optionally, after obtaining the uncompressed data, the following may also be included:

[0020] The data receiver uses the cross-platform multimedia development library SDL to play the uncompressed data.

[0021] Optionally, the data publisher may further include a third data publisher, wherein the ownership strength of the first data publisher is greater than that of the third data publisher, and the ownership strength of the third data publisher is greater than that of the second data publisher;

[0022] When the first data publisher fails, the third data publisher sends the compressed structure data to the data receiver.

[0023] Optionally, the data receiver includes at least a first data receiver and a second data receiver; it also includes:

[0024] The data receiver's malfunction is determined based on its liveness policy. When the first data receiver malfunctions, the second data receiver receives the encapsulation format data and the compressed structure data. When the second data receiver malfunctions, the first data receiver receives the encapsulation format data and the compressed structure data. When neither the first nor the second data receiver malfunctions, both the first and second data receivers simultaneously receive the encapsulation format data and the compressed structure data.

[0025] On the other hand, the present invention also provides an audio and video transmission system based on DDS technology, comprising:

[0026] A module is established to establish data publishers and data receivers; the data publishers include at least a first data publisher and a second data publisher.

[0027] The configuration module is used to configure the QoS policies of the data publisher and the data receiver; the QoS policy of the data publisher includes ownership, ownership strength, and liveness, and the QoS policy of the data receiver includes ownership and liveness; wherein, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher;

[0028] The parsing module is used to parse the original audio and video files according to the decoding protocol to obtain the encapsulated format data;

[0029] The decapsulation module is used to decapsulate the encapsulated format data to obtain compressed structure data;

[0030] A sending module is used to enable the first data publisher to send the encapsulated format data and the compressed structure data to the data receiver;

[0031] A decoding module is used to enable the data receiver to decode the compressed structure data according to the encapsulation format data to obtain uncompressed data;

[0032] The judgment module is used to determine whether the first data publisher has failed. If not, it determines whether all compressed structure data has been sent. If yes, the program ends; if no, the first data publisher continues to send the compressed structure data to the data receiver. If yes, the second data publisher sends the compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the encapsulation format data to obtain uncompressed data, and determines whether all compressed structure data has been sent. If yes, the program ends; if no, it continues to determine whether the first data publisher has failed.

[0033] Optionally, the compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data.

[0034] When the data receiver receives audio compressed encoded data, the decoding module is used to decode the audio compressed encoded data according to the encapsulation format data to obtain the original audio data;

[0035] When the data receiver receives video compressed encoded data, the decoding module is used to decode the video compressed encoded data according to the encapsulation format data to obtain the original video data.

[0036] Optionally, it also includes:

[0037] The cross-platform multimedia development library SDL is used to play the uncompressed data.

[0038] Optionally, the data receiver includes at least a first data receiver and a second data receiver;

[0039] The judgment module is further configured to determine whether the data receiver has failed based on the data receiver's liveness strategy. When the first data receiver fails, the second data receiver receives the encapsulation format data and the compressed structure data. When the second data receiver fails, the first data receiver receives the encapsulation format data and the compressed structure data. When neither the first data receiver nor the second data receiver fails, both the first data receiver and the second data receiver simultaneously receive the encapsulation format data and the compressed structure data.

[0040] The innovative aspects of this invention include:

[0041] 1. In this embodiment, the communication relationship between audio and video transmission is transformed from the original point-to-point direct communication to a publish-subscribe relationship mediated by a Topic through the publish-subscribe mechanism provided by DDS technology. This decouples the publisher and receiver, which is beneficial to improving transmission efficiency and is one of the innovative points of this embodiment.

[0042] 2. In this embodiment, the publish-subscribe mechanism of DDS is adopted, which can flexibly set multiple data publishers as primary and backup nodes. When the primary node fails, the primary and backup switching can solve the problem of the entire transmission system being unavailable due to the failure of one node, thereby improving the availability of the audio and video transmission system. This is one of the innovations of this embodiment.

[0043] 3. In this embodiment, by adopting the Liveliness strategy, real-time monitoring of the data publisher is achieved, thereby obtaining the data publisher's own activity status. When the software or hardware of a certain node in the system fails, the primary and backup nodes will be quickly switched, which is beneficial to improving the availability of the audio and video transmission system. This is one of the innovations of this embodiment.

[0044] 4. In this embodiment, the overhead of computing and storage resources during primary-backup switching is small. Therefore, the impact on audio and video transmission and playback is small. Moreover, when a higher-priority data publisher fails, the switching time of the backup node can be less than 10ms, which can ensure the real-time performance of the entire system. This is one of the innovative points of this embodiment. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart of a transmission method provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a switching method provided in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of audio and video decoding provided in an embodiment of the present invention;

[0049] Figure 4Another schematic diagram of the switching method provided in an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of a transmission system provided in an embodiment of the present invention;

[0051] Figure 6 This is another schematic diagram of the transmission system provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0054] This invention discloses an audio and video transmission method and system based on DDS technology. These will be described in detail below.

[0055] Figure 1 This is a flowchart of a transmission method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a switching method provided in an embodiment of the present invention. Please refer to it. Figures 1-2 The audio and video transmission method based on DDS technology provided in this embodiment includes:

[0056] Step 1: Establish data publishers and data receivers; data publishers should include at least a first data publisher and a second data publisher;

[0057] Step 2: Set the QoS policies for data publishers and data receivers; the QoS policies for data publishers include ownership, ownership strength, and liveness, and the QoS policies for data receivers include ownership and liveness; wherein, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher;

[0058] Step 3: Parse the original audio and video files using the decryption protocol to obtain the encapsulation format data;

[0059] Step 4: Decapsulate the encapsulated data to obtain compressed structure data;

[0060] Step 5: The first data publisher sends the encapsulated format data and compressed structure data to the data receiver; the data receiver decodes the compressed structure data according to the encapsulated format data to obtain the uncompressed data;

[0061] Step 6: Determine if the first data publisher has malfunctioned. If not, determine if all compressed structure data has been sent. If yes, the program ends. If no, the first data publisher continues to send compressed structure data to the data receiver. If yes, the second data publisher sends compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the encapsulation format data to obtain uncompressed data, and determines if all compressed structure data has been sent. If yes, the program ends. If no, continue to determine if the first data publisher has malfunctioned.

[0062] For details, please refer to Figure 1 The audio and video transmission method based on DDS technology provided in this embodiment of the invention first establishes a data publisher and a data receiver in step 1. Here, the data publisher and data receiver are established by the publisher and subscriber of DDS (Data Distribution Service).

[0063] In existing audio and video transmission methods, most are point-to-point communication, meaning that a failure in any one node will cause the audio and video transmission to fail. To avoid the problem of the entire transmission failing due to the failure of a single node, in this embodiment, the publishing end establishes at least two data publishers, such as a first data publisher and a second data publisher. In this way, when a node such as the first data publisher fails, another node such as the second data publisher can be used to transmit data, ensuring reliable data transmission.

[0064] This invention employs the publish-subscribe mechanism of DDS, which allows for the flexible configuration of multiple data publishers as redundant nodes. When the primary node fails, a primary-backup switchover is implemented to resolve the issue of the entire transmission system becoming unavailable due to the failure of a single node, thereby improving the availability of the audio and video transmission system.

[0065] During DDS communication, publishers and subscribers send and receive messages based on the Topic and QoS (Quality of Service) policy. Therefore, it is necessary to initialize both the data publisher and the data receiver.

[0066] In this embodiment, the QoS policies for the data publisher and data receiver are set in step 2. Since this invention relates to communication for data transmission during primary / backup node switching, and primary / backup node switching is related to node ownership, ownership strength, and liveliness, the QoS policy configured for the data publisher in this embodiment includes ownership, ownership strength, and liveness, while the QoS policy for the data receiver includes ownership and liveness.

[0067] The Ownership policy controls whether multiple data writers are allowed to publish data to the same topic. Ownership can be EXCLUSIVE or SHARED. In this embodiment, the ownership of the data publisher and data receiver is set to EXCLUSIVE, meaning that only the owner of the topic is allowed to update the data. When ownership is set to EXCLUSIVE, the ownership of the topic is determined by OWNERSHIP_STRENGTH. The data writer with the largest value in the OWNERSHIP_STRENGTH member variable is the owner of the topic. In this embodiment, the ownership strength of the first data publisher is set to be greater than that of the second data publisher. Therefore, the first data publisher has priority in owning the topic, and the data receiver receives the data sent by the first data publisher.

[0068] In addition to ownership strength, ownership also depends on whether the highest-priority data writer is still alive; that is, it is related to the data writer's LIVELINESS (liveliness) policy. For example, if the data writer with the highest ownership strength fails, the second-highest-priority data writer will acquire ownership of the topic, and the data receiver will receive the data sent by the second-highest-priority data writer.

[0069] This invention transforms the communication relationship between audio and video transmission from direct point-to-point communication to a publish-subscribe relationship mediated by topics through the publish-subscribe mechanism provided by DDS technology. The sender of audio and video only needs to publish streaming media data to the topic without the need for a response from the receiver, and the receiver only needs to subscribe to data from the topic. This decoupling between the publisher and receiver is beneficial to improving transmission efficiency.

[0070] After setting the QoS policy, in steps 3 and 4, FFmpeg (Fast Forward Mpeg, a multimedia video processing tool) is used to process the audio and video files.

[0071] First, in step 3, the original audio and video files, such as those using streaming media protocols, are parsed using a decryption protocol to obtain the encapsulation format data. For example, data transmitted using the RTMP protocol, after decryption, outputs FLV format data. Then, in step 4, the encapsulation format data is decapsulated to obtain compressed structure data; that is, the encapsulation format data is separated into audio stream compressed encoded data and video stream compressed encoded data. For example, FLV format data, after decapsulation, outputs an H.264 encoded video stream and an AAC encoded audio stream.

[0072] Please refer to Figure 1 After obtaining the encapsulation format data and compressed structure data, in step 5, the first data publisher with the strongest ownership sends the compressed structure data to the data receiver. Upon receiving the compressed structure data, the data receiver needs to decode it to obtain the original audio and video data. Decoding requires the corresponding decoder. Therefore, in step 5, the encapsulation format data also needs to be sent to the data receiver; the encapsulation format data must contain at least decoder information.

[0073] Please refer to Figure 1 Each time the first data publisher sends data, it needs to determine its liveness, that is, whether it has failed. Therefore, in step 6 of this embodiment, it is determined whether the first data publisher has failed. If the first data publisher has not failed, it is determined whether all compressed structure data has been sent. If it has been sent, it means that all audio and video data has been sent and the program ends; if it has not been sent, the first data publisher continues to send compressed structure data to the data receiver.

[0074] Conversely, please refer to Figure 2 If the first data publisher fails, the second-highest priority data publisher acquires ownership of the topic. The second data publisher then sends compressed structured data to the data receiver. The data receiver decodes the received compressed structured data according to the encapsulation format to obtain the uncompressed data. After sending data once, the second data publisher checks if the transmission is complete. If it is, all audio and video data has been sent, and the program ends. If not, it checks if the first data publisher has recovered, i.e., checks again for a potential failure, and so on, until all compressed structured data has been sent.

[0075] It should be noted that, Figure 2 In this diagram, W1 represents the first data publisher, W2 represents the second data publisher, R1 represents the first data receiver, and R2 represents the second data receiver. Furthermore, Figure 2This is merely an illustrative statement to show that when the data publisher with the strongest ownership fails, the backup data publisher can continue to send data. It does not represent the actual number of data publishers and data receivers. In practical applications, the number of data publishers and data receivers can be specifically set as needed, and this application does not impose any specific limitations on this.

[0076] This invention employs a Liveliness strategy to achieve real-time monitoring of data publishers, thereby acquiring their activity status. When a software or hardware failure occurs at a node in the system, a rapid switchover between primary and backup nodes is initiated. Since the overhead of computing and storage resources during primary-backup switching is minimal, the impact on audio and video transmission and playback is small. Furthermore, when a higher-priority data publisher fails, the backup node switchover time can be less than 10ms, ensuring the real-time performance of the entire system.

[0077] Optionally, Figure 3 This is a schematic diagram of audio and video decoding provided in an embodiment of the present invention. Please refer to it. Figure 3 The compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data; the uncompressed data is obtained by decoding the compressed structure data, specifically: when the data receiver receives audio compressed encoded data, the audio compressed encoded data is decoded according to the encapsulation format data to obtain the raw audio data; when the data receiver receives video compressed encoded data, the video compressed encoded data is decoded according to the encapsulation format data to obtain the raw video data.

[0078] For details, please refer to Figure 3 After decapsulating the encapsulated data, the resulting compressed structure data includes audio compressed encoded data and video compressed encoded data. When the receiving end decodes, if it receives audio compressed encoded data, it obtains the original audio data; if it receives video compressed encoded data, it obtains the original video data. After the data receiver decodes the video compressed encoded data or audio compressed encoded data, it can use SDL (Simple DirectMedia Layer, a cross-platform multimedia development library) for playback.

[0079] Optionally, the QoS policies for data publishers and data receivers are as follows: ownership is set to exclusive, liveness is set to manual confirmation by topic, and the live lease period of the data publisher is shorter than that of the data receiver.

[0080] Specifically, in this embodiment, the ownership of data publishers and data receivers is set to exclusive, meaning that only the owner of the topic is allowed to update the data. When ownership is set to exclusive, the ownership of the topic is determined by OWNERSHIP_STRENGTH (ownership strength). The data writer with the largest value in the OWNERSHIP_STRENGTH member variable is the owner of the topic.

[0081] In addition to ownership strength, ownership also requires determining whether the highest-priority data writer is still alive, which is related to the data writer's LIVELINESS (liveliness) policy. This embodiment sets liveness to MANUAL_BY_TOPIC_LIVELINESS_QOS (manual confirmation by topic), meaning the signal confirming a topic's liveness is at the topic level. Only when a message is published on that topic or an explicit signal asserting liveness on that topic from an application will the topic be marked as active.

[0082] If the data publisher is manually confirmed to be still active, the system will consider the data publisher active for the remaining lease duration. During the lease duration, the data publisher must publish data; otherwise, the system will consider it to have stopped working. To ensure that the data receiver can receive the data sent by the data publisher, the data receiver's liveness time must be greater than the data publisher's liveness time. Therefore, in this embodiment, the liveness time of the data publisher is set to be less than that of the data receiver. For example, the liveness time of the data publisher can be set to liveliness.lease_duration = 100, and the liveness time of the data receiver can be set to liveliness.lease_duration = 120.

[0083] This invention transforms the communication relationship between audio and video transmission from direct point-to-point communication to a publish-subscribe relationship mediated by topics through the publish-subscribe mechanism provided by DDS technology. The sender of audio and video only needs to publish streaming media data to the topic without the need for a response from the receiver, and the receiver only needs to subscribe to data from the topic. This decoupling between the publisher and receiver is achieved, which helps to improve transmission efficiency.

[0084] Optionally, Figure 4 For another schematic diagram of the switching method provided in this embodiment of the invention, please refer to... Figure 4The data publisher also includes a third data publisher. The ownership strength of the first data publisher is greater than that of the third data publisher, and the ownership strength of the third data publisher is greater than that of the second data publisher. When the first data publisher fails, the third data publisher sends compressed structured data to the data receiver.

[0085] For details, please refer to Figure 4 In this embodiment, DDS also establishes a third data publisher, wherein the ownership strength of the first data publisher, the ownership strength of the third data publisher, and the ownership strength of the second data publisher decrease in sequence. For example, the ownership strength of the first data publisher is 3, the ownership strength of the third data publisher is 2, and the ownership strength of the second data publisher is 1.

[0086] It should be noted that, Figure 4 In this diagram, W1 represents the first data publisher, W2 represents the second data publisher, W3 represents the third data publisher, R1 represents the first data receiver, and R2 represents the second data receiver.

[0087] Thus, during audio and video transmission, the first data publisher with the strongest ownership acquires ownership of the topic first. After sending data once, the first data publisher determines whether a failure has occurred based on its liveness. If the first data publisher fails, the third data publisher with the second-highest ownership strength is selected to send the compressed structure data to the data receiver, following the order of decreasing ownership strength. Only when both the first and third data publishers fail will the second data publisher with the weakest ownership strength send data. Furthermore, once the first data publisher recovers, it resumes sending data.

[0088] This invention employs a Liveliness strategy to achieve real-time monitoring of data publishers, thereby obtaining their own activity status. When a software or hardware failure occurs in a node of the system, a rapid switchover between primary and backup nodes will be performed.

[0089] Alternatively, please refer to Figure 2 The data receiver includes at least a first data receiver and a second data receiver; it also includes: determining whether a data receiver has failed based on the data receiver's liveness strategy; when the first data receiver fails, the second data receiver receives the encapsulated format data and the compressed structure data; when the second data receiver fails, the first data receiver receives the encapsulated format data and the compressed structure data; when neither the first nor the second data receiver fails, the first and second data receivers simultaneously receive the encapsulated format data and the compressed structure data.

[0090] For details, please refer to Figure 2 In this embodiment, DDS establishes two data receivers: a first data receiver and a second data receiver. After the data sender transmits data, it needs to determine whether the data receiver is functioning correctly; that is, it determines whether the data receiver has malfunctioned based on the data receiver's liveness policy.

[0091] When neither of them malfunctions, both the first and second data receivers will receive data and play audio and video data using SDL. If either of them malfunctions, only the one that did not malfunction will receive data. For example, if the first data receiver malfunctions, only the second data receiver will receive data; conversely, if the second data receiver malfunctions, only the first data receiver will receive data.

[0092] This invention adopts a DDS publish-subscribe mechanism. When one data receiver fails, the other data receivers are unaffected and can still operate normally, continue to receive and process data, which helps to further improve the availability of the audio and video transmission system.

[0093] Based on the same inventive concept, embodiments of the present invention also provide an audio and video transmission system based on DDS technology. Figure 5 Please refer to the schematic diagram of a transmission system provided in an embodiment of the present invention. Figure 5 The audio and video transmission system 100 based on DDS technology provided in this embodiment of the invention includes:

[0094] Establish module 10 to establish data publishers and data receivers; the data publishers include at least a first data publisher and a second data publisher;

[0095] The configuration module 20 is used to configure the QoS policies of data publishers and data receivers. The QoS policies of data publishers include ownership, ownership strength, and liveness, and the QoS policies of data receivers include ownership and liveness. Among them, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher.

[0096] Parsing module 30 is used to parse the original audio and video files through the decoding protocol to obtain the encapsulation format data;

[0097] The decapsulation module 40 is used to decapsulate the encapsulated format data to obtain compressed structure data;

[0098] The sending module 50 is used to enable the first data publisher to send encapsulated format data and compressed structure data to the data receiver.

[0099] The decoding module 70 is used to enable the data receiver to decode the compressed structure data according to the encapsulation format data to obtain uncompressed data;

[0100] The judgment module 60 is used to determine whether the first data publisher has failed. If not, it determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, the first data publisher continues to send compressed structure data to the data receiver. If yes, the second data publisher sends compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the encapsulation format data to obtain uncompressed data, and determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, it continues to determine whether the first data publisher has failed.

[0101] For details, please refer to Figure 5 The audio and video transmission system 100 based on DDS technology provided in this embodiment of the invention includes a setup module 10. The setup module 10 establishes a DDS (Data Distribution Service) publisher and subscriber. Then, the publisher establishes a data publisher and the subscriber establish a data receiver.

[0102] In existing audio and video transmission methods, most are point-to-point communication, meaning that a failure in any one node will cause the audio and video transmission to fail. To avoid the problem of the entire transmission failing due to the failure of a single node, in this embodiment, the publishing end establishes at least two data publishers, such as a first data publisher and a second data publisher. In this way, when a node such as the first data publisher fails, another node such as the second data publisher can be used to transmit data, ensuring reliable data transmission.

[0103] This invention employs the publish-subscribe mechanism of DDS, which allows for the flexible configuration of multiple data publishers as redundant nodes. When the primary node fails, a primary-backup switchover is implemented to resolve the issue of the entire transmission system becoming unavailable due to the failure of a single node, thereby improving the availability of the audio and video transmission system.

[0104] During DDS communication, publishers and subscribers send and receive messages based on the Topic and QoS (Quality of Service) policy. Therefore, it is necessary to initialize both the data publisher and the data receiver.

[0105] In this embodiment, the QoS policies for data publishers and data receivers are set through the setting module 20. Since this invention relates to communication for data transmission during primary / backup node switching, and primary / backup node switching is related to node ownership, ownership strength, and liveliness, the configured QoS policy for the data publisher in this embodiment includes ownership, ownership strength, and liveness, while the QoS policy for the data receiver includes ownership and liveness.

[0106] The Ownership policy controls whether multiple data writers are allowed to publish data to the same topic. Ownership can be EXCLUSIVE or SHARED. In this embodiment, the ownership of the data publisher and data receiver is set to EXCLUSIVE, meaning that only the owner of the topic is allowed to update the data. When ownership is set to EXCLUSIVE, the ownership of the topic is determined by OWNERSHIP_STRENGTH. The data writer with the largest value in the OWNERSHIP_STRENGTH member variable is the owner of the topic. In this embodiment, the ownership strength of the first data publisher is set to be greater than that of the second data publisher. Therefore, the first data publisher has priority in owning the topic, and the data receiver receives the data sent by the first data publisher.

[0107] In addition to ownership strength, ownership also depends on whether the highest-priority data writer is still alive; that is, it is related to the data writer's LIVELINESS (liveliness) policy. For example, if the data writer with the highest ownership strength fails, the second-highest-priority data writer will acquire ownership of the topic, and the data receiver will receive the data sent by the second-highest-priority data writer.

[0108] This invention transforms the communication relationship between audio and video transmission from direct point-to-point communication to a publish-subscribe relationship mediated by topics through the publish-subscribe mechanism provided by DDS technology. The sender of audio and video only needs to publish streaming media data to the topic without the need for a response from the receiver, and the receiver only needs to subscribe to data from the topic. This decoupling between the publisher and receiver is achieved, which helps to improve transmission efficiency.

[0109] After setting the QoS policy, use FFmpeg (Fast Forward Mpeg, a multimedia video processing tool) to process the audio and video files.

[0110] First, the parsing module 30 parses the original audio and video files, such as those using streaming media protocols, using a decryption protocol to obtain the encapsulation format data. For example, data transmitted using the RTMP protocol, after the decryption operation, outputs FLV format data. Then, the decapsulation module 40 decapsulates the encapsulation format data to obtain compressed structure data; that is, it separates the encapsulation format data into audio stream compressed encoded data and video stream compressed encoded data. For example, FLV format data, after the decapsulation operation, outputs an H.264 encoded video stream and an AAC encoded audio stream.

[0111] After obtaining the encapsulated format data and compressed structure data, the sending module 50 enables the first data publisher with greater ownership power to send the compressed structure data to the data receiver. Upon receiving the compressed structure data, the data receiver needs to decode it using the decoding module 70 to obtain the original audio and video data. Decoding requires the corresponding decoder. Therefore, the encapsulated format data, which contains at least decoder information, also needs to be sent to the data receiver.

[0112] After each data transmission, the first data publisher needs to determine its viability, i.e., whether it has malfunctioned. Therefore, in this embodiment, the determination module 60 determines whether the first data publisher has malfunctioned. If the first data publisher has not malfunctioned, it determines whether all compressed structure data has been transmitted. If it has been transmitted, the program ends; if it has not been transmitted, the first data publisher continues to transmit compressed structure data to the data receiver.

[0113] Conversely, please refer to Figure 2 If the first data publisher malfunctions, the second-highest priority data publisher sends compressed structure data to the data receiver. The data receiver decodes the received compressed structure data according to the encapsulation format data to obtain uncompressed data. After sending data once, the second data publisher checks whether the data transmission is complete. If it is, it means that all audio and video data has been sent, and the program ends. If it is not complete, it checks whether the first data publisher has recovered, that is, it checks again whether the first data publisher has malfunctioned, and so on, until all compressed structure data has been sent.

[0114] This invention employs a Liveliness strategy to achieve real-time monitoring of data publishers, thereby acquiring their activity status. When a software or hardware failure occurs at a node in the system, a rapid switchover between primary and backup nodes is initiated. Since the overhead of computing and storage resources during primary-backup switching is minimal, the impact on audio and video transmission and playback is small. Furthermore, when a higher-priority data publisher fails, the backup node switchover time can be less than 10ms, ensuring the real-time performance of the entire system.

[0115] Optionally, Figure 6 For another structural schematic diagram of the transmission system provided in this embodiment of the invention, please refer to... Figure 6 The audio and video transmission system based on DDS technology provided in this embodiment of the invention also includes a cross-platform multimedia development library SDL 80; the compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data; when the data receiver receives audio compressed encoded data, the decoding module 70 is used to decode the audio compressed encoded data according to the encapsulation format data to obtain the raw audio data; when the data receiver receives video compressed encoded data, the decoding module 70 is used to decode the video compressed encoded data according to the encapsulation format data to obtain the raw video data.

[0116] For details, please refer to Figure 6 The audio and video transmission system based on DDS technology provided in this embodiment of the invention also includes the cross-platform multimedia development library SDL 80. After decapsulating the encapsulated format data, the resulting compressed structure data includes audio compressed encoded data and video compressed encoded data. When the receiving end decodes, if it receives audio compressed encoded data, it obtains the original audio data after decoding; if it receives video compressed encoded data, it obtains the original video data after decoding. After the data receiver decodes the video compressed encoded data or audio compressed encoded data, it can use SDL80 (Simple DirectMedia Layer, cross-platform multimedia development library) to play it.

[0117] Alternatively, please refer to Figure 2 The data receivers include at least a first data receiver and a second data receiver. The judgment module 60 is also used to determine whether a data receiver has failed based on the data receiver's liveness strategy. When the first data receiver fails, the second data receiver receives the encapsulated format data and the compressed structure data. When the second data receiver fails, the first data receiver receives the encapsulated format data and the compressed structure data. When neither the first nor the second data receiver fails, the first and second data receivers simultaneously receive the encapsulated format data and the compressed structure data.

[0118] For details, please refer to Figure 2 In this embodiment, DDS establishes two data publishers: a first data receiver and a second data receiver. After sending data, the data publisher needs to determine whether the data receiver is functioning correctly; that is, it determines whether the data receiver has malfunctioned based on the data receiver's liveness policy.

[0119] When neither of them malfunctions, both the first and second data receivers will receive data and play audio and video data using SDL. If either of them malfunctions, only the one that did not malfunction will receive data. For example, if the first data receiver malfunctions, only the second data receiver will receive data; conversely, if the second data receiver malfunctions, only the first data receiver will receive data.

[0120] This invention adopts a DDS publish-subscribe mechanism. When one data receiver fails, the other data receivers are unaffected and can still operate normally, continue to receive and process data, which helps to further improve the availability of the audio and video transmission system.

[0121] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0122] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio and video transmission method based on DDS technology, characterized in that, include: Establish data publishers and data receivers; the data publishers include at least a first data publisher and a second data publisher; Configure QoS policies for the data publisher and the data receiver; the QoS policy of the data publisher includes ownership, ownership strength, and liveness, and the QoS policy of the data receiver includes ownership and liveness; wherein, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher; The original audio and video files are parsed using a decoding protocol to obtain encapsulation format data; the encapsulation format data contains at least decoder information. The encapsulated data is decapsulated to obtain compressed structure data; The first data publisher sends the encapsulated format data and the compressed structure data to the data receiver; The data receiver decodes the compressed structure data according to the decoder information to obtain uncompressed data; The system determines whether the first data publisher has malfunctioned. If not, it determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, the first data publisher continues to send the compressed structure data to the data receiver. If yes, the second data publisher sends the compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the decoder information to obtain uncompressed data, and then determines whether all compressed structure data has been sent. If yes, the program ends; otherwise, it continues to determine whether the first data publisher has malfunctioned.

2. The DDS technology-based audio and video transmission method according to claim 1, characterized in that, The compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data. Decoding the compressed structure data to obtain uncompressed data specifically involves: When the data receiver receives audio compressed encoded data, it decodes the audio compressed encoded data according to the encapsulation format data to obtain the original audio data; When the data receiver receives video compressed encoded data, it decodes the video compressed encoded data according to the encapsulation format data to obtain the original video data.

3. The audio and video transmission method based on DDS technology according to claim 1, characterized in that, The QoS policies for the data publisher and the data receiver are as follows: ownership is set to exclusive, liveness is set to manual confirmation by topic, and the live lease period of the data publisher is shorter than that of the data receiver.

4. The DDS-based audio and video transmission method of claim 1, wherein, After obtaining the uncompressed data, it also includes: The data receiver uses the cross-platform multimedia development library SDL to play the uncompressed data.

5. The DDS-based audio and video transmission method of claim 1, wherein, The data publisher also includes a third data publisher, wherein the ownership strength of the first data publisher is greater than that of the third data publisher, and the ownership strength of the third data publisher is greater than that of the second data publisher; When the first data publisher fails, the third data publisher sends the compressed structure data to the data receiver.

6. The DDS-based audio and video transmission method of claim 1, wherein, The data receivers include at least a first data receiver and a second data receiver; and also include: The data receiver's malfunction is determined based on its liveness policy. When the first data receiver malfunctions, the second data receiver receives the encapsulation format data and the compressed structure data. When the second data receiver malfunctions, the first data receiver receives the encapsulation format data and the compressed structure data. When neither the first nor the second data receiver malfunctions, both the first and second data receivers simultaneously receive the encapsulation format data and the compressed structure data.

7. An audio and video transmission system based on DDS technology, characterized in that, include: A module is established to establish data publishers and data receivers; the data publishers include at least a first data publisher and a second data publisher. The configuration module is used to configure the QoS policies of the data publisher and the data receiver; the QoS policy of the data publisher includes ownership, ownership strength, and liveness, and the QoS policy of the data receiver includes ownership and liveness; wherein, the ownership strength of the first data publisher is greater than the ownership strength of the second data publisher; The parsing module is used to parse the original audio and video files through a decoding protocol to obtain encapsulation format data; the encapsulation format data contains at least decoder information; The decapsulation module is used to decapsulate the encapsulated format data to obtain compressed structure data; A sending module is used to enable the first data publisher to send the encapsulated format data and the compressed structure data to the data receiver; A decoding module is used to enable the data receiver to decode the compressed structure data according to the decoder information to obtain uncompressed data; The judgment module is used to determine whether the first data publisher has failed. If not, it determines whether all compressed structure data has been sent. If yes, the program ends; if no, the first data publisher continues to send the compressed structure data to the data receiver. If yes, the second data publisher sends the compressed structure data to the data receiver. The data receiver decodes the compressed structure data according to the decoder information to obtain uncompressed data, and then determines whether all compressed structure data has been sent. If yes, the program ends; if no, it continues to determine whether the first data publisher has failed.

8. The DDS-based audio-video transmission system of claim 7, wherein, The compressed structure data includes audio compressed encoded data and video compressed encoded data; the uncompressed data includes raw video data and raw audio data. When the data receiver receives audio compressed encoded data, the decoding module is used to decode the audio compressed encoded data according to the encapsulation format data to obtain the original audio data; When the data receiver receives video compressed encoded data, the decoding module is used to decode the video compressed encoded data according to the encapsulation format data to obtain the original video data.

9. The DDS-based audio-video transmission system of claim 7, wherein, Also includes: The cross-platform multimedia development library SDL is used to play the uncompressed data.

10. The DDS-based audio / video transmission system of claim 7, wherein, The data receivers at least include a first data receiver and a second data receiver; The judging module is further configured to judge whether the data receivers fail according to a survival strategy of the data receivers, and when the first data receiver fails, the second data receiver receives the encapsulated format data and the compressed structure data; When the second data receiver fails, the first data receiver receives the encapsulated format data and the compressed structure data; and when neither the first data receiver nor the second data receiver fails, the first data receiver and the second data receiver simultaneously receive the encapsulated format data and the compressed structure data.

Citation Information

Patent Citations

  • Method for improving reliability of distributed system based on DDS technology

    CN107040403A

  • Data structure classification device and publishing and subscribing system and method of unstructured data

    CN107665237A