Unmanned vehicle video backhaul control method and system based on SIP communication protocol

Abstract

The application relates to the technical field of video backhaul, in particular to a unmanned vehicle video backhaul control method and system based on a SIP communication protocol. The control method comprises the following steps: sending, by a local end as a SIP client, a SIP connection request to a unmanned vehicle end through TLS encryption, explicitly indicating that the local end is a video decoding end and the unmanned vehicle end is a video encoding end, simultaneously negotiating audio bidirectional communication capability, and establishing an encrypted signaling connection. The SIP is a mature multimedia communication protocol, has perfect and safe session establishment, modification and termination mechanisms, has better network adaptability, supports NAT traversal and firewall penetration, can stably work in different network environments, and is suitable for unmanned vehicle mobile scenes. The application solves the problems of dynamic bandwidth adaptation and safe and reliable transmission control of unmanned vehicle video backhaul in a complex network environment by constructing a systematic control scheme based on the SIP protocol.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video backhaul, in particular to a method and system for unmanned vehicle video backhaul control based on SIP communication protocol. BACKGROUND

[0002] In the remote control scene, the SIP protocol is used as the signaling mechanism for multimedia session control. The traditional application scheme has limitations when facing complex network environments. Although the existing technology can realize video session establishment and media transmission, there are technical bottlenecks in dynamically adapting to network bandwidth changes and ensuring the quality of service (QoS) of transmission.

[0003] Specifically, the fixed code rate video encoding transmission mode cannot adjust the code stream parameters according to the real-time network state, resulting in network congestion intensifying or high-definition video transmission quality deteriorating in low-bandwidth scenarios. At the same time, the traditional scheme relies on fixed ports to manage media stream transmission, which is prone to signaling and media stream penetration failure in cross-NAT and firewall environments, and lacks systematic congestion control and link optimization mechanisms, making it difficult to cope with network jitter and packet loss in complex scenarios such as mines and the wild. In addition, the uneven design of bidirectional communication capability and the imperfection of encryption mechanism further limit the real-time performance of voice interaction and the security of signaling transmission in the remote control scene.

[0004] The above problems reflect the technical gap of the existing technology in dynamic bandwidth adaptation and reliable transmission control, which makes it difficult to meet the actual application requirements of the stability, efficiency and security of video backhaul in the remote control scene of unmanned vehicles. SUMMARY

[0005] The present disclosure proposes a method and system for unmanned vehicle video backhaul control based on SIP communication protocol, aiming to overcome at least one of the deficiencies in the prior art.

[0006] To achieve the above-mentioned purpose, the technical solutions disclosed in the present application are as follows:

[0007] According to one aspect of the present disclosure, a method for unmanned vehicle video backhaul control based on SIP communication protocol is provided, the steps of the control method comprising:

[0008] The local end as a SIP client sends a SIP connection request to the unmanned vehicle end through TLS encryption, the SDP in the request carrying the extended video capability attribute a=decode-only, and the unmanned vehicle end responding by carrying the extended video capability attribute a=encode-only in the SDP, explicitly indicating that the local end is a video-only decoding end and the unmanned vehicle end is a video-only encoding end, while negotiating the audio bidirectional communication capability, and establishing an encrypted signaling connection;

[0009] After the local end analyzes the unmanned vehicle end SDP response, the video receiving channel is activated according to the a=encode-only attribute, the audio sending / receiving channel is synchronously opened, the unmanned vehicle end keeps the video encoding sending state and activates the audio receiving channel based on the negotiation result, and a bidirectional communication link of "vehicle end video upload-local end audio download" is formed.

[0010] During the video decoding process, the local end sends an I frame request to the unmanned vehicle end through a SIP info message, the unmanned vehicle end triggers the video encoding module to generate and return a key frame after receiving the request, and the video transmission code rate is reduced to adapt to a low-bandwidth network.

[0011] The local end and the unmanned vehicle end are registered to a SIP server, network penetration is realized by using an ICE framework combined with STUN / TURN protocols, network packet loss state and packet arrival time are fed back through an RTCP packet, and the video transmission code rate is dynamically adjusted.

[0012] Further, the video capability attributes a=decode-only or a=encode-only of the SDP extension are self-defined negotiation fields, which are used to uniquely limit the video processing roles of the local end and the unmanned vehicle end, and the TLS encryption covers the whole SIP signaling transmission process.

[0013] Further, the activation mechanism of the video receiving channel is that when the local end detects the a=encode-only attribute in the SDP, an RTP receiving port is automatically configured for video stream analysis, and the audio sending / receiving channel supports G.711 or G.729 coding and decoding formats, which are used for remote voice interaction.

[0014] Further, the I frame request message includes current decoding timestamp and frame number information, the unmanned vehicle end encoding module locates the encoding position and generates a corresponding I frame according to the request message, the B / P frame redundancy is reduced by on-demand transmission of the I frame, and the I frame is sent to the local end after being encapsulated by RTP.

[0015] Further, the network penetration includes NAT and firewall penetration of signaling and media streams, and the implementation is that public network NAT mapping addresses of both ends are obtained through the STUN protocol, a TURN server is used as a relay node to forward media streams that cannot be directly penetrated, and an ICE framework is used to dynamically screen host addresses, reflection addresses or relay addresses as optimal transmission paths.

[0016] Further, the execution process of feeding back network packet loss status and packet arrival time through the RTCP packet is as follows: the local end records the RTP packet arrival timestamp and sequence number as a receiving end, encapsulates the RTCP feedback packet containing the DLEP and PLI fields, and sends the RTCP feedback packet to the unmanned vehicle end, and the unmanned vehicle end adjusts the resolution, frame rate or code rate of video encoding based on the feedback information of the RTCP feedback packet.

[0017] Further, the control method further comprises a session update mechanism, when the network bandwidth fluctuation exceeds a preset threshold or the device power is lower than a critical value, the local end initiates a session update request through the SIP signaling, reconfigures the encoding format and encapsulation strategy through SDP negotiation, and performs parameter switching by the media transmission module.

[0018] Further, the SIP signaling interaction complies with the IETF standard protocol, and the video capability attributes a=decode-only or a=encode-only extended by the SDP are transmitted through the SIP extension header field, forming a communication mode of "video one-way transmission-audio bidirectional interaction", which is different from the traditional bidirectional video session.

[0019] Further, after the unmanned vehicle end activates the audio receiving channel, the audio media stream sent by the local end can be supported to be received, so that the remote voice shouting function is realized, and the audio media stream and the video stream are transmitted through different RTP ports.

[0020] According to another aspect of the present disclosure, there is provided an unmanned vehicle video backhaul control system based on a SIP communication protocol, which is used to implement the unmanned vehicle video backhaul control method based on the SIP communication protocol as described above, and the control system comprises:

[0021] A signaling interaction module is configured to transmit the SIP signaling through TLS encryption, extend the a=decode-only or a=encode-only attribute in the SDP to negotiate the video processing role and the audio capability, and establish an encrypted signaling connection.

[0022] A media channel module is configured to activate the local end video receiving channel and the audio bidirectional channel, the unmanned vehicle end video sending channel and the audio receiving channel according to the SDP negotiation result, and support the RTP encapsulated media stream transmission.

[0023] An encoding control module is configured to generate an I frame request message in the SIP info format by the local end, generate and send the I frame by the unmanned vehicle end after responding to the request, and realize dynamic code rate adjustment.

[0024] A network adaptation module is configured to realize network penetration by integrating the ICE+STUN / TURN protocol, analyze the RTCP feedback packet to perform congestion control, and optimize the network transmission performance.

[0025] The signaling interaction module comprises an SDP analysis unit for identifying a=decode-only / a=encode-only attributes; the encoding control module comprises a quality monitoring unit and a frame generation unit for triggering I frame request and generating key frames respectively; the network adaptation module comprises a penetration processing unit and a congestion control unit for processing NAT penetration and dynamically adjusting bandwidth.

[0026] The present application has the following advantages:

[0027] The present application solves the problems of dynamic bandwidth adaptation and safe and reliable transmission control of unmanned vehicle video backhaul in complex network environment by constructing a systematic control scheme based on SIP protocol.

[0028] At the signaling interaction level, through the self-defined negotiation mechanism of SDP extension attributes a=decode-only and a=encode-only, the video processing roles of the local end and the unmanned vehicle end are clearly divided, and the security and uniqueness of signaling transmission are guaranteed while realizing the efficient communication mode of "video one-way transmission-audio two-way interaction" combined with the TLS full encryption technology.

[0029] At the media transmission level, a dynamic I frame request mechanism is introduced, the local end triggers the unmanned vehicle end to generate key frames on demand according to the decoding demand through the SIP info message, avoiding the redundant transmission of B / P frames in traditional fixed code rate encoding, reducing the video code rate from the source to adapt to the low bandwidth network, and SIP as a mature multimedia communication protocol has perfect and safe session establishment, modification and termination mechanism. Better network adaptability, support for NAT traversal and firewall penetration, can work stably in different network environments, suitable for unmanned vehicle mobile scenarios. Support for multiple media formats and codecs, adapt to different video formats and rates to realize low latency transmission requirements.

[0030] In terms of network adaptation, the ICE framework integrates STUN / TURN protocols to realize double penetration of signaling and media flow, deeply analyzes RTCP feedback packets, and adjusts video encoding parameters in real time to cope with network congestion, improving the transmission reliability in complex network environment.

[0031] In the system architecture, the signaling interaction, media channel, encoding control and network adaptation modules are cooperatively designed to form a closed-loop control system from session establishment, code stream optimization to link maintenance, not only solving the problems of complex port management and low penetration efficiency in traditional solutions, but also dynamically responding to bandwidth fluctuations and device state changes through the session update mechanism, ensuring the continuous stability of video backhaul in remote control scenarios, and constructing a video backhaul control system with dynamic adaptation capability, high reliability and security mechanism, providing efficient and stable technical support for remote monitoring of unmanned vehicles in complex environments.

[0032] The above description is only a summary of the technical solutions of the present application. In order to make the technical means of the present application more clearly understood and implemented according to the content of the description, the preferred embodiments of the present application are described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 Flow chart of a video return control method in an embodiment of the present application;

[0034] Figure 2 Flow chart of a media capability secure channel connection establishment in an embodiment of the present application;

[0035] Figure 3 Schematic diagram of a dynamic frame request and response mechanism in an embodiment of the present application;

[0036] Figure 4 Network penetration architecture based on ICE+STUN / TURN in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0038] The term “comprising” and any variation thereof in the specification and claims of the present application is intended to cover not exclusively including, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In addition, the use of “and / or” in the specification and claims means at least one of the connected objects, for example, A and / or B means three cases including A alone, B alone, and A and B both.

[0039] In the embodiments of the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of “exemplary” or “for example” is intended to present relevant concepts in a specific manner.

[0040] The present application provides the following preferred embodiments:

[0041] Embodiment one: in order to solve the problems of insufficient dynamic bandwidth adaptation and reliable transmission control in unmanned vehicle video backhaul in complex network environment, the embodiment discloses a kind of unmanned vehicle video backhaul control method based on SIP communication protocol, refines the implementation logic of each link of control method based on SIP protocol, as shown in Figure 1 The control method flow is as follows:

[0042] S100: the local end as SIP client sends SIP connection request to unmanned vehicle end by TLS encryption, the request carries the extended video capability attribute a=decode-only in SDP, and the unmanned vehicle end carries the extended video capability attribute a=encode-only in SDP when responding, which clearly indicates that the local end is a video decoding end and the unmanned vehicle end is a video encoding end, while negotiating the audio bidirectional communication capability, and establishing an encrypted signaling connection.

[0043] S200: after the local end parses the SDP response of the unmanned vehicle end, the video receiving channel is activated according to the a=encode-only attribute, and the audio sending / receiving channel is opened synchronously, the unmanned vehicle end keeps the video encoding and sending state based on the negotiation result and activates the audio receiving channel, forming a bidirectional communication link of "vehicle end video upload-local end audio download".

[0044] S300: during video decoding, the local end sends I frame request to the unmanned vehicle end through SIP info message, the unmanned vehicle end triggers the video encoding module to generate and return key frame after receiving, and reduces the video transmission code rate to adapt to low bandwidth network.

[0045] S400: the local end and the unmanned vehicle end register to SIP server, realize network penetration by using ICE framework combined with STUN / TURN protocol, and dynamically adjust the video transmission code rate by feeding back network packet loss state and packet arrival time through RTCP packet.

[0046] Specifically, when the local end as SIP client initiates session establishment, it sends INVITE request to the unmanned vehicle end through TLS encrypted channel, and embeds the custom extended attribute "a=decode-only" in the request body SDP to clearly declare that the local end only has video decoding function; the unmanned vehicle end carries "a=encode-only" attribute in SDP response after receiving the signaling, which indicates that it only undertakes video encoding role, and at the same time, both parties negotiate G.711 or G.729 audio codec parameters through m=audio field of SDP to complete the negotiation of bidirectional audio communication capability, thereby establishing a full-encryption signaling connection, ensuring the uniqueness of video processing role and the security of signaling transmission.

[0047] Further, when the local end analyzes the SDP of the unmanned vehicle end response, after detecting the "a=encode-only" attribute, the media channel initialization is triggered: an independent RTP receiving port is automatically assigned for video stream analysis, and an H.264 decoding thread is activated; the audio codec module is synchronized and initialized, and a bidirectional audio channel is opened to support real-time voice interaction. Based on the negotiation result, the unmanned vehicle end continuously runs the video encoding module and activates the audio receiving channel, establishing an asymmetric communication link of "vehicle end video upload-local end audio download", which meets the actual needs of one-way video monitoring and two-way voice interaction in the remote control scene.

[0048] Further, in the video transmission optimization link, as shown in Figure 2 , the local decoding module monitors the decoding state in real time. When it is detected that the decoding delay caused by B / P frame dependence or the network bandwidth is lower than the threshold, a SIP info message containing the current decoding timestamp and the target frame number is generated and sent to the unmanned vehicle end through the signaling channel. After the signaling processing module of the unmanned vehicle end analyzes the request, the video encoding module is triggered to locate the encoding sequence at the corresponding time point, and an I frame required for independent decoding is generated and sent to the local end after being encapsulated by RTP, replacing the traditional fixed-period I frame generation encoding mode, thereby reducing the transmission of redundant B / P frames and realizing dynamic adjustment of the code rate to adapt to low-bandwidth networks.

[0049] Further, at the network transmission level, as shown in Figure 3 , Figure 4 , the local end and the unmanned vehicle end complete registration with the SIP server in advance, and the ICE framework is started to perform network penetration process: the public network NAT mapping address of each host is obtained through the STUN protocol, and the available transmission addresses such as host address and reflection address are identified; for scenes that cannot be directly penetrated, such as symmetric NAT, a TURN server is enabled as a relay node to forward media streams, and the ICE framework dynamically selects the optimal transmission path (host address, reflection address or relay address) according to the network detection result, to realize the double penetration of signaling and media stream. At the same time, the local end records the arrival timestamp and sequence number of the RTP packet as a receiving end in real time, encapsulates the RTCP feedback packet containing the packet loss rate and average delay information, and sends it to the congestion control module of the unmanned vehicle end to adjust the quantization parameter of the video encoding or the key frame interval, forming a closed-loop control mechanism of "receiving end feedback-sending end adjustment", and realizing the adaptive optimization of the transmission code rate.

[0050] Through the embodiment, the communication architecture suitable for the remote control scene is constructed based on the role negotiation mechanism of the SDP extension attribute and the asymmetric media channel configuration; the dynamic I frame request and the ICE / STUN / TURN penetration technology are cooperatively applied, effectively solving the problems of low efficiency of fixed code rate transmission and network penetration, and providing a systematic technical solution for video backhaul in complex network environment.

[0051] Embodiment two: In order to solve the problems of ambiguity of role negotiation and insufficient security of signaling transmission in traditional video backhaul, the embodiment further refines the implementation logic of SDP extension attribute definition and TLS encryption mechanism.

[0052] Further, the SDP extended video capability attributes "a=decode-only" and "a=encode-only" are used as custom negotiation fields to limit roles in SIP signaling interaction. Specifically, when the local end initiates a session as a SIP client, it embeds "a=decode-only" in the SDP body of the INVITE request to explicitly declare that it only supports video decoding function; after the unmanned vehicle receives the signaling, it carries "a=encode-only" in the SDP of the 200 OK response to uniquely limit itself as a video encoding end. This custom field follows the SIP protocol extension specification, increases a proprietary attribute line in the session description, realizes explicit negotiation of the video processing roles of both ends, and avoids the functional conflicts that may be caused by traditional implicit role determination. It should be understood that the attribute field adopts a key-value pair format, the key "a=" follows the SIP standard attribute prefix, and the value "decode-only" and "encode-only" are custom function identifiers, ensuring compatibility with the existing protocol framework.

[0053] Further, the TLS encryption mechanism covers the whole process of SIP signaling transmission, including registration request, session establishment signaling (such as INVITE, ACK), and control messages (such as INFO, BYE). Before signaling interaction, both ends complete certificate exchange and encryption parameter negotiation through the TLS handshake protocol, perform identity verification using RSA or ECDSA algorithm, negotiate to determine encryption suites such as AES-256-GCM or ChaCha20-Poly1305, and perform end-to-end encryption on signaling data. During transmission, each SIP message is encapsulated in the TLS record layer, and a message authentication code is generated through a hash algorithm to ensure the integrity and tamper resistance of the signaling. For example, as shown in the SIP info message, the I frame request message sent by the local end needs to be encrypted by TLS before transmission, and the message content is parsed by the unmanned vehicle after receiving and verifying the certificate and decrypting the message, thereby ensuring the secure transmission of dynamic control signaling in public networks. Figure 2

[0054] Through this embodiment, the custom SDP extension attribute realizes the unique limitation of video processing roles in a standardized manner, avoiding the ambiguity of traditional negotiation mechanisms; the TLS encryption protects the whole process of signaling, improving the security of the communication process, and providing technical support for the standardization and security of signaling interaction in the remote control scene of unmanned vehicles.

[0055] ​Embodiment three: In order to solve the problem of unclear media channel configuration process and non-standard audio interaction capability negotiation, this embodiment further refines the video receiving channel activation mechanism and the implementation logic of audio communication capability.

[0056] Specifically, when the local end parses the SDP content of the unmanned vehicle end response, after detecting the "a=encode-only" attribute through syntax analysis, the automatic configuration process of the video receiving channel is triggered: first, the network module dynamically selects an unoccupied port as the RTP receiving port according to the system resource allocation strategy, which follows the RTP protocol standard, the port number is even, and the corresponding RTCP port is odd, which is used to receive the video stream data sent by the unmanned vehicle end; then activate the video decoding module, create a corresponding decoding thread according to the pre-negotiated encoding format (such as H.264 or H.265), and establish a decoding buffer to cache the RTP packets to be processed. It should be understood that the automatic configuration process of the RTP receiving port includes port availability detection and protocol stack binding, which ensures that the video stream data can be accurately directed to the decoding module.

[0057] Further, in terms of audio communication capability construction, the local end and the unmanned vehicle end negotiate the encoding format through the m=audio field in the SDP, supporting mainstream audio encoding standards such as G.711μ-law, G.711a-law or G.729. Specifically, the local end lists the supported encoding list in the initial INVITE request, the unmanned vehicle end selects one or more formats according to its own capability and confirms in the response, and finally the intersection part is determined as the actual use encoding scheme. After the negotiation is completed, the local end initializes the audio encoding module and opens the bidirectional audio channel: the sending channel compresses the local voice data into RTP packets through the selected encoding format, and transmits them to the unmanned vehicle end through the media channel; the receiving channel decodes the audio RTP packets sent by the unmanned vehicle end and restores them into analog audio signals. This mechanism ensures the real-time and compatibility of bidirectional voice interaction in the remote control scene.

[0058] Through this embodiment, the video receiving channel activation mechanism based on SDP attribute detection realizes the automatic initialization of the media processing module, avoiding the port conflict risk caused by manual configuration; the standardized audio encoding negotiation process and bidirectional channel configuration provide a stable communication foundation for remote voice interaction, forming a "one-way video monitoring-bidirectional voice interaction" composite communication link with the video upload channel, meeting the actual needs of the unmanned vehicle remote control scene.

[0059] Embodiment four: In order to solve the problem of redundant data transmission caused by traditional fixed period I frame encoding, this embodiment further refines the interactive mechanism of dynamic I frame request and response.

[0060] When the local video decoding module detects decoding delay or network bandwidth fluctuation, as shown in Figure 2 The decoding time point based on the RTP timestamp unit is used to identify the timing position of the current decoding process, and the relative position of the expected key frame in the encoding sequence, which together constitute the key parameters of the encoding positioning. The message is transmitted to the unmanned vehicle end through the encrypted signaling channel, triggering the dynamic response logic of the video encoding module: the encoding module retrieves the encoding timeline according to the decoding time point, locates to the GOP starting position corresponding to the relative position, skips the B / P frame generation process dependent on the previous frame, and directly generates the I frame required for independent decoding.

[0061] Further, the generated I frame complies with the RTP payload format standard, and when encapsulated, the load type field identifies the H.264 or H.265 encoding format, and the timestamp and sequence number fields ensure the inter-frame timing relationship. The unmanned vehicle end replies with an ACK confirmation message while sending the I frame RTP packet, completing the signaling interaction closed loop of request response. This mechanism triggers I frame generation on demand, avoiding the transmission of redundant B / P frames in fixed period encoding, and realizes the dynamic coordination of the encoding and decoding ends, reducing the transmission load of video streams from the data source.

[0062] Through this embodiment, the accurate encoding positioning technology based on timestamp and frame number information enables the unmanned vehicle end to respond to the key frame requirements of the decoding end in real time, and builds an intelligent interaction link of "decoding state awareness-encoding dynamic adjustment", providing technical support for efficient video transmission in low-bandwidth network environment.

[0063] Embodiment five: To solve the problem of signaling and media stream penetration in complex network environment, this embodiment further clarifies the network penetration implementation process based on ICE framework, as shown in Figure 4 .

[0064] Specifically, the local end and the unmanned vehicle end first complete identity registration through the SIP registration process to the registration server and obtain public network access identifiers. Then start the ICE candidate address collection phase: send a Binding request to the STUN server through the STUN protocol to obtain their public NAT mapping addresses, including host address (local network interface address), reflection address (NAT mapped public address), and server reflection address (STUN server relay address). For scenarios that cannot directly establish a connection such as symmetric NAT, a TURN server is introduced as a relay node, and both ends apply for media stream forwarding service to the server through the TURN protocol, generating a relay address.

[0065] Further, the ICE framework performs connectivity check on the collected candidate addresses (host address, reflection address and relay address): verifies address reachability by sending STUN probe packets, records parameters such as round-trip time (RTT) and packet loss rate, and selects the optimal transmission path based on a preset policy - preferentially uses the host address or reflection address with the lowest delay, and automatically switches to the TURN relay address when direct connection fails. At the signaling level, SIP messages are connected through the public network address resolved by STUN, ensuring the penetration of signaling interactions such as registration and session establishment; at the media level, RTP / RTCP data streams are transmitted according to the path selected by ICE, realizing the cross-NAT forwarding of video and audio streams.

[0066] Through this embodiment, the ICE framework and the cooperative mechanism of STUN / TURN protocol build a dual penetration scheme covering signaling and media stream, effectively solving the connection establishment problem in cross-firewall and multi-level NAT environment, and ensuring the stability and reachability of communication link in complex network scenarios.

[0067] Embodiment six: To realize real-time perception of network congestion and adaptive adjustment of code stream parameters, this embodiment further specifies the execution logic of feeding back network packet loss state and packet arrival time through RTCP packets, as shown in Figure 3 .

[0068] Specifically, the local end as the media receiving end records the arrival timestamp and sequence number of each packet in the RTP packet processing process in real time, calculates the average transmission delay based on the time sequence difference of consecutive packets, and calculates the packet loss rate through sequence number continuity detection. These state information is encapsulated into the RTCP feedback packet, the packet delay related extension DLEP field is used to carry the delay distribution statistics, and the picture layer retransmission request PLI field identifies whether the key frame needs to be retransmitted. The feedback packet is sent to the congestion control module of the unmanned vehicle end through a signaling channel independent of the media stream.

[0069] Further, after receiving the RTCP feedback packet, the unmanned vehicle end analyzes the network state parameters therein: when it is detected that the average transmission delay exceeds the preset threshold or the packet loss rate is higher than the packet loss tolerance, the encoding parameter adjustment logic is triggered - the video resolution is reduced to reduce the single frame data amount, or the key frame interval is increased to reduce the encoding complexity, or the code rate can also be adjusted through dynamic quantization parameter. The adjusted encoding parameters are synchronized to the local end through the SDP re-negotiation process, forming a closed-loop control link of "receiving end feedback - sending end adjustment". This mechanism ensures that the encoding end can dynamically optimize the output code stream according to the real-time network state, avoiding video lag or data loss caused by congestion.

[0070] Through the combination of congestion control and RTCP feedback mechanism, precise evaluation of network transmission quality and intelligent adaptation of encoding strategy are realized, providing technical support for media transmission stability in complex network environment.

[0071] Embodiment Seven: To cope with the influence of dynamic network environment and device state fluctuation on communication quality, the implementation process of session update mechanism is further improved in this embodiment.

[0072] Specifically, when the network bandwidth fluctuation exceeds the preset threshold, such as a 20% bandwidth drop, or the device power of the unmanned vehicle end is detected to be lower than the critical value, the SIP session update process is triggered: the local end initiates a parameter renegotiation request by sending a RE-INVITE signaling, and the signaling body carries an SDP description containing parameters such as new encoding format and packaging strategy. After receiving, the unmanned vehicle end checks the SDP parameters according to the current system resources and network conditions, and if it supports, it replies with a 200 OK response containing the confirmed parameters, and if it does not support, it proposes alternative solutions in the response, and after multiple interactions, an agreement is reached.

[0073] Further, after the negotiation is completed, the media transmission module performs configuration switching according to the new SDP parameters: if the encoding format is switched from H.264 to H.265, the codec instance of the decoding module is updated; if the packaging strategy is adjusted, such as changing the RTP payload type, the packet parsing rules of the network layer are reconfigured. During the parameter switching process, the signaling interaction and media stream transmission maintain continuous connection, and temporary buffering of undecoded data is used to avoid picture interruption. This mechanism ensures that both parties can dynamically adapt to environmental changes in runtime, maintaining reliable video backhaul and control link.

[0074] Through this embodiment, the real-time response to network and device state is realized based on the SIP signaling-based session update mechanism, and through the standardized parameter renegotiation process, a systematic solution is provided for dynamic adjustment of communication parameters in complex scenarios.

[0075] Embodiment Eight: To solve the problem of resource waste in traditional two-way video session in the remote control scene of unmanned vehicles, this embodiment further clarifies the one-way video capability negotiation mechanism based on SIP protocol.

[0076] Specifically, the signaling interaction process strictly follows the IETF RFC 3261 standard, and the video capability attribute is extended in the SDP description of the SIP message body: the local end as the control end carries the a=decode-only attribute in the initial INVITE request, indicating that it only has video decoding capability; the unmanned vehicle end as the video acquisition end returns the a=encode-only attribute in the response, declaring that it only supports video encoding transmission. The attribute is carried through the SIP extension header field Content-Type: application / sdp, ensuring that both parties explicitly indicate the video transmission role in the session establishment stage. Unlike the traditional two-way video session, both ends need to maintain the encoding and decoding modules at the same time, and in this embodiment, the attribute negotiation forms a "local one-way receiving-unmanned vehicle one-way sending" video transmission link, while retaining the two-way interaction capability of the audio channel. The audio negotiation completes the intersection matching of the encoding and decoding formats (such as G.711μ-law) through the m=audio field, ensuring the two-way flow of voice commands and environmental sounds.

[0077] Further, the parsing of the a=decode-only and a=encode-only attributes triggers differentiated module initialization: after detecting the a=encode-only of the other party, the local end activates the port allocation logic of the video receiving channel, follows the protocol specification of RTP port even number and RTCP port odd number, and disables the local video encoding module to save system resources; after recognizing the a=decode-only of the local end, the unmanned vehicle end only initializes the video encoding and sending module, and closes the decoding function to reduce device power consumption. This mechanism clearly defines the media capability boundary of the communication parties at the protocol level through standardized signaling extension, and builds an asymmetric media transmission mode that adapts to the remote monitoring scenario.

[0078] Through this embodiment, the capability negotiation mechanism based on SDP attribute extension realizes accurate control of the video transmission direction and on-demand allocation of resources, avoiding the running overhead of redundant modules in traditional two-way sessions, and providing a protocol-level optimization scheme for low-power remote communication of unmanned vehicles.

[0079] Embodiment Nine: To meet the real-time voice interaction and device shouting needs in the remote control scene, this embodiment further specifies the activation mechanism and media stream transmission strategy of the audio receiving channel.

[0080] When the unmanned vehicle end completes the SDP negotiation and parses the audio codec format (such as G.729) supported by the local end, the audio receiving channel initialization process is started: first, the network module independently allocates a set of unoccupied RTP / RTCP ports for the audio stream according to the port allocation rules, which is different from the video stream port. Among them, the RTP port is even, and the RTCP port is the corresponding odd number, ensuring the separate transmission of audio and video streams at the network layer. The voice data collected by the microphone of the local end is compressed by the encoding module and packaged into a data packet conforming to the RTP payload format, and then sent to the unmanned vehicle end through the independent port; the audio RTP packet received by the unmanned vehicle end receiving channel is subjected to sequence number checking and packet loss compensation processing, and then restored to a PCM signal by the decoding module, driving the vehicle-mounted loudspeaker to realize voice playback, forming a downlink of remote shouting.

[0081] Further, the port isolation mechanism of the audio stream and the video stream avoids the media type mixed packet problem and supports the network layer to implement differentiated QoS policy according to the service type, such as audio stream priority forwarding. During the channel activation process, the unmanned vehicle end synchronously establishes the audio codec context, and caches the encoding parameters determined by the negotiation, such as sampling rate and quantization precision, to ensure the format consistency of the bidirectional audio transmission. It should be understood that the independent port allocation follows the definition of the RTP protocol on the payload type field, and the audio packet is identified by a fixed Payload Type, such as 0x08 corresponding to G.729, which is clearly distinguished from the video packet such as 0x12 corresponding to H.264, facilitating the receiving end to identify and process the stream type.

[0082] Through the embodiment, the independent port-based audio media stream transmission mechanism realizes the bidirectional separate transmission of remote voice instructions and environmental audio, guarantees the real-time performance and reliability of the shouting function, and provides an efficient voice interaction channel for remote control of the unmanned vehicle.

[0083] Embodiment Ten: To realize the systematization landing of the unmanned vehicle video backhaul control method in a complex network environment, the embodiment provides an unmanned vehicle video backhaul control system based on the SIP communication protocol, which realizes the collaborative work of signaling interaction, media processing, encoding control and network adaptation through modular design.

[0084] Specifically, a signaling interaction module is provided, which integrates a TLS encryption transmission channel to ensure the security of SIP signaling at the transmission layer. The core component, an SDP analysis unit, is responsible for identifying a=decode-only or a=encode-only attributes and analyzing the video capabilities of both parties during the session establishment phase: when a local a=decode-only attribute and an unmanned vehicle a=encode-only attribute are detected, configuration instructions for asymmetric media transmission are generated to drive the media channel module to perform differential initialization. The signaling interaction follows the IETF standard protocol, transmits SDP capability descriptions through extended header fields, establishes an encrypted signaling connection to negotiate video processing roles and audio codec formats, and provides parameter basis for subsequent media channel activation.

[0085] Further, a media channel module is provided, which performs channel activation logic based on the negotiation results output by the signaling interaction module: independent receive / send ports are allocated for video streams (the local receive port follows the RTP even number / RTCP odd number rule), and video decoding modules (local) or encoding modules (unmanned vehicle) are initialized; a bidirectional channel is established for audio streams, and codec instances are created according to the negotiated codec format (such as G.711 a-law) to ensure compression and unpacking processing of voice data. Media stream transmission uses RTP protocol encapsulation, and time stamp and sequence number fields are used to guarantee inter-frame timing, support low-delay characteristics of the UDP transmission layer, and meet the real-time requirements of remote control scenarios.

[0086] Further, an encoding control module is provided, and a quality monitoring unit analyzes the local video decoding quality in real time, such as frame decoding delay and picture blurriness. When network bandwidth fluctuations or decoding abnormalities are detected, a SIP info format I-frame request message containing decoding timestamp and frame number information is generated. In response to the request received by the unmanned vehicle end, the frame generation unit locates the target position on the encoding time axis, skips the B / P frame generation process and directly generates an I-frame, which is then returned to the local end after RTP encapsulation. This module implements dynamic code rate adjustment logic, reduces redundant data by transmitting key frames on demand, and adapts to efficient transmission requirements in low-bandwidth network environments.

[0087] Further, a network adaptation module is provided, which penetrates the ICE framework and STUN / TURN protocol to perform network address translation (NAT) penetration: the STUN acquires a public network reflection address, the TURN handles the scene that cannot be directly penetrated as a relay node, the ICE dynamically selects the host address, reflection address or relay address as the optimal transmission path to guarantee the cross-NAT reachability of signaling and media stream. The congestion control unit analyzes the RTCP feedback packet sent by the local end, contains the DLEP and PLI fields, extracts the network packet loss rate, transmission delay and other parameters, drives the unmanned vehicle end encoding module to adjust the resolution, frame rate or code rate, forms a "feedback-adjustment" closed loop to optimize the network transmission performance.

[0088] Further, each module realizes data interaction through a standardized interface, the SDP parameter output by the signaling interaction module drives the media channel module to initialize, the I frame request of the encoding control module triggers the transmission path optimization of the network adaptation module, and the congestion state of the network adaptation module is fed back to the encoding control module to adjust the encoding strategy. The control system of the embodiment realizes the automatic management of the whole process from signaling negotiation, media processing to network adaptation through functional decoupling and collaborative control, and provides stable and reliable system support for the remote video backhaul and control of unmanned vehicles.

[0089] Through the embodiment, the modular system architecture converts the technical features at the method level into functional entities that can be implemented, and through the collaborative work of each unit, effectively solves the reliability, real-time performance and resource optimization problems of media transmission in a complex network environment, and meets the engineering application requirements of the remote control scene of unmanned vehicles.

[0090] Although the present application is specifically described above with reference to the preferred embodiments thereof, it is to be understood that the present application is not limited to the above-described embodiments, and various modifications and changes can be made by those skilled in the art without departing from the spirit of the present application, and such modifications and changes shall fall within the scope of the appended claims and their equivalents.

Claims

1. A method for unmanned vehicle video backhaul control based on SIP communication protocol, characterized in that, The steps of the control method comprise: The local end sends a SIP connection request to the unmanned vehicle end by TLS encryption as a SIP client, the SIP connection request carries an extended video capability attribute a=decode-only in the SDP, and the unmanned vehicle end carries an extended video capability attribute a=encode-only in the SDP when responding, so as to explicitly indicate that the local end is a video-only decoding end and the unmanned vehicle end is a video-only encoding end, and to negotiate audio bidirectional communication capability, and to establish an encrypted signaling connection; After the local end analyzes the SDP response of the unmanned vehicle end, the local end activates a video receiving channel according to the a=encode-only attribute, and synchronously opens an audio sending / receiving channel, the unmanned vehicle end keeps a video encoding sending state and activates an audio receiving channel based on the negotiation result, and forms a bidirectional communication link of "vehicle end video upload-local end audio download"; During video decoding, the local end sends an I-frame request to the unmanned vehicle end through a SIP info message, the unmanned vehicle end generates and returns a key frame after triggering a video encoding module upon receiving the I-frame request, and reduces a video transmission code rate to adapt to a low-bandwidth network; The local end and the unmanned vehicle end register to a SIP server, realize network penetration by using an ICE framework combined with STUN / TURN protocols, and dynamically adjust a video transmission code rate by feeding back network packet loss state and packet arrival time through an RTCP packet; The video capability attributes a=decode-only or a=encode-only extended in the SDP are respectively self-defined negotiation fields, and are used to uniquely limit video processing roles of the local end and the unmanned vehicle end, and the TLS encryption covers the whole process of SIP signaling transmission; The activation mechanism of the video receiving channel is that when the local end detects the a=encode-only attribute in the SDP, an RTP receiving port is automatically configured for video stream analysis, and the audio sending / receiving channel supports G.711 or G.729 encoding / decoding formats for remote voice interaction; The execution process of feeding back network packet loss state and packet arrival time through the RTCP packet is that the local end records an RTP packet arrival timestamp and a sequence number as a receiving end, encapsulates an RTCP feedback packet containing DLEP and PLI fields, and sends the RTCP feedback packet to the unmanned vehicle end, and the unmanned vehicle end adjusts resolution, frame rate or code rate of video encoding based on feedback information of the RTCP feedback packet.

2. The unmanned vehicle video backhaul control method based on SIP communication protocol according to claim 1, characterized in that, The I-frame request includes current decoding timestamp and frame number information, the unmanned vehicle end encoding module locates an encoding position and generates a corresponding I-frame according to the I-frame request, reduces B / P frame redundancy by on-demand transmission of the I-frame, and sends the I-frame to the local end after encapsulation through RTP. 3.The unmanned vehicle video backhaul control method based on SIP communication protocol of claim 1, wherein, The network penetration includes NAT and firewall penetration of signaling and media stream, and the implementation is that public network NAT mapping addresses of both ends are obtained through the STUN protocol, a TURN server is used as a relay node to forward media stream that cannot be directly penetrated, and an ICE framework dynamically selects a host address, a reflection address or a relay address as an optimal transmission path.

4. The unmanned vehicle video backhaul control method based on SIP communication protocol according to claim 1, characterized in that, The control method further comprises a session update mechanism, when network bandwidth fluctuation exceeds a preset threshold or device power is lower than a critical value, the local end initiates a session update request through SIP signaling, reconfigures encoding format and encapsulation strategy through SDP negotiation, and performs parameter switching by a media transmission module. 5.The unmanned vehicle video backhaul control method based on SIP communication protocol of claim 1, wherein, The SIP signaling interaction complies with IETF standard protocols, and the video capability attributes a=decode-only or a=encode-only of the SDP extension are transmitted through SIP extension headers, forming a communication mode of "video one-way transmission-audio bidirectional interaction", which is different from a traditional bidirectional video session. 6.The unmanned vehicle video backhaul control method based on SIP communication protocol of claim 1, wherein, After the unmanned vehicle end activates the audio receiving channel, the audio media stream sent by the local end can be received, so that the remote voice shouting function is realized, and the audio media stream and the video stream are transmitted through different RTP ports.

7. A SIP-based unmanned vehicle video backhaul control system for implementing the SIP-based unmanned vehicle video backhaul control method according to any one of claims 1-6, characterized in that, The control system comprises: a signaling interaction module configured to transmit SIP signaling through TLS encryption, extend a=decode-only or a=encode-only attributes in SDP to negotiate video processing roles and audio capabilities, and establish an encrypted signaling connection; a media channel module configured to activate a local end video receiving channel and an audio bidirectional channel, an unmanned vehicle end video sending channel and an audio receiving channel according to an SDP negotiation result, and support RTP encapsulated media stream transmission; an encoding control module configured to generate an I frame request in SIP info format at the local end, generate and send an I frame after the unmanned vehicle end responds to the I frame request, and realize dynamic code rate adjustment; a network adaptation module configured to integrate ICE+STUN / TURN protocols to realize network penetration, analyze RTCP feedback packets to perform congestion control, and optimize network transmission performance. The signaling interaction module comprises an SDP analysis unit configured to identify a=decode-only / a=encode-only attributes; the encoding control module comprises a quality monitoring unit and a frame generation unit configured to trigger an I frame request and generate a key frame, respectively; and the network adaptation module comprises a penetration processing unit and a congestion control unit, which cooperatively process NAT penetration and dynamic bandwidth adjustment.

Images