Method and apparatus for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream

Abstract

This invention provides a method for converting GB28181 protocol RTP video streams into low-latency HTTP-PS video streams, applied to an HTTP-PS server, including the following steps: Step 1, receiving a streaming media playback HTTP connection request from a client; Step 2, parsing the target video source's device encoding (DEV1) and channel encoding (CHANNEL1) from the streaming media playback HTTP connection request, and creating an HTTP response object; Step 3, randomly obtaining an unused TCP port P1 from the local machine, and starting a Socket service S1 to listen on port P1; Step 4, initiating a stream retrieval request to a SIP server to request that it send the target video source's RTP video stream to the Socket service S1; Step 5, after receiving the RTP data after each PS data packet sent by the target video source, the Socket service S1 reassembles the RTP data into packets in real time to obtain PS data packets, and writes them into the HTTP response object so that the client can obtain PS packet data from the HTTP response object and display a frame of live video; Step 6, stopping the Socket service S1 when the HTTP connection is detected to be disconnected.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, specifically to a method and apparatus for converting GB28181 protocol RTP video streams into low-latency HTTP-PS video streams. Background Technology

[0002] In recent years, with increasing emphasis on public safety, video surveillance has become increasingly important in maintaining social order. However, compatibility and interoperability issues between different manufacturers make it difficult for systems to achieve effective information sharing and interconnection, thus hindering the realization of the original value of public safety monitoring.

[0003] The GB28181 protocol was developed to address this issue. It aims to provide a unified framework and standard that enables video surveillance equipment from different manufacturers and platforms to achieve interconnectivity and information sharing.

[0004] RTP (Real-time Transport Protocol) plays a crucial role in the GB28181 protocol. As part of GB28181, it is primarily responsible for the transmission of real-time audio and video data in PS format. It provides functions such as streaming, timestamps, and sequence numbers to ensure the real-time performance and reliability of audio and video data. The role of the RTP protocol in the GB28181 protocol is as follows: Figure 1 As shown.

[0005] ZLMediaKit is a streaming media service framework developed based on C++11. It supports multiple streaming media protocols and can convert between various protocols, such as RTSP, RTMP, HLS, and GB28181. It supports L3 streaming media live broadcast with end-to-end latency kept to within 1 second. It also supports on-demand streaming of some protocols. It is widely used in scenarios such as video stream conversion and preview.

[0006] However, it has the following defects in applications related to the GB28181 protocol:

[0007] 1. Direct output of HTTP-PS video streams (i.e., PS video streams with HTTP payloads) is not supported. It requires converting the PS stream with RTP payloads in the GB28181 protocol to other formats, introducing additional computational overhead and network latency. ZLMediaKit supports the following output formats: Figure 2 As shown;

[0008] 2. Automatic management of source video streams (also known as on-demand streaming) is not supported in GB28181 protocol scenarios. The source video stream must be manually cut off after playback is closed. ZLMediaKit supports on-demand streaming of RTSP, RTMP, HTTPFLV, and WSFLV formats, but this feature is not yet supported in GB28181 application scenarios (because GB28181 RTP video streams are acquired in a special passive manner and require SIP interaction). Its on-demand streaming is mainly implemented through the ` / index / api / addStreamProxy` interface. This interface currently only supports RTSP / RTMP / HLS / HTTP-TS / HTTP-FLV format video streams.

[0009] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0010] Based on this, it is necessary to provide a method and apparatus for converting RTP video streams in the GB28181 protocol into low-latency HTTP-PS video streams to address the aforementioned technical problems. This avoids video stream format conversion, reduces computational overhead and end-to-end latency, and enables on-demand streaming by automatically pulling / cutting the stream when it plays / stops. It can provide ready-to-use video stream services and reduce usage costs.

[0011] To achieve the above objectives, the first aspect of the present invention provides a method for converting GB28181 protocol RTP video streams into low-latency HTTP-PS video streams, applied to an HTTP-PS server, comprising the following steps:

[0012] Step 1: Receive the streaming media playback HTTP connection request from the client. The streaming media playback HTTP connection request includes at least a video stream identifier URL string. The video stream identifier URL string is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding.

[0013] Step 2: Parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and create an HTTP response object;

[0014] Step 3: Randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1;

[0015] Step 4: Send a stream request to the SIP server to request it to send the RTP video stream of the target video source to the Socket service S1.

[0016] Step 5: When Socket service S1 receives the RTP data after the first PS data packet is packetized from the target video source, it reassembles the RTP data into packets in real time to obtain the first PS data packet and writes it into the HTTP response object so that the client can obtain the first PS packet data from the HTTP response object and display the first frame of the live broadcast.

[0017] Step 5: After receiving the RTP data after each PS data packet sent by the target video source, the Socket service S1 reassembles the RTP data into packets in real time to obtain the PS data packet and writes it into the HTTP response object so that the client can obtain the PS packet data from the HTTP response object and display a frame of live video.

[0018] Step 6: When an HTTP connection is detected to be disconnected, stop the Socket service S1.

[0019] To achieve the above objectives, a second aspect of the present invention provides an apparatus for converting GB28181 protocol RTP video streams into low-latency HTTP-PS video streams, comprising:

[0020] The HTTP connection request acquisition module is used to receive streaming media playback HTTP connection requests. The streaming media playback HTTP connection request includes at least a video stream identifier URL string, which is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding.

[0021] The HTTP connection request parsing module is used to parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and to create an HTTP response object;

[0022] The port enablement module is used to randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1.

[0023] The stream retrieval request sending module is used to initiate a stream retrieval request to the SIP server, requesting it to send the RTP video stream of the target video source to the Socket service S1;

[0024] When the data receiving module receives the RTP data after the i-th PS data packet is packetized by the target video source through the Socket service S1, it reassembles the RTP data into packets in real time to obtain the i-th PS data packet and writes it into the HTTP response object so that the client can obtain the i-th PS packet data from the HTTP response object and display the corresponding live screen.

[0025] The connection disconnect module is used to stop the Socket service S1 when an HTTP connection disconnection is detected.

[0026] To achieve the above objectives, a third aspect of the present invention provides a conversion system for GB28181 protocol RTP video streams, comprising:

[0027] The target video source is used to generate an RTP video stream using the GB28181 protocol;

[0028] An HTTP-PS server, wherein the HTTP-PS server is configured to perform the steps of the method according to any one of claims 1 to 6;

[0029] A network switch is used to enable communication between the target video source and the HTTP-PS service.

[0030] To achieve the above objectives, a fourth aspect of the present invention provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements the steps of the method described in the first aspect.

[0031] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0032] The beneficial effects of this invention are as follows:

[0033] This invention enables the conversion of RTP video streams into low-latency HTTP-PS video streams based on the HTTP protocol, avoiding video stream format conversion and reducing computational overhead and end-to-end latency.

[0034] This invention achieves on-demand streaming by automatically pulling / cutting the stream when the video stream is playing / stopping, providing an instant video streaming service and reducing usage costs. Attached Figure Description

[0035] Figure 1 This is a typical communication structure for GB28181 devices.

[0036] Figure 2 It is a live streaming format supported by ZLMediaKit.

[0037] Figure 3 It is a typical video keyframe PS packet structure.

[0038] Figure 4 This is a flowchart illustrating the method described in Embodiment 1 of the present invention.

[0039] Figure 5This is a principle block diagram of a verification embodiment of the present invention.

[0040] Figure 6 This is a diagram showing the effect of step 1 in the verification embodiment of the present invention.

[0041] Figure 7 This is a diagram showing the effect of step 2 in the verification embodiment of the present invention.

[0042] Figure 8 This is a diagram showing the effect of step 3 in the verification embodiment of the present invention.

[0043] Figure 9 This is a schematic diagram of the experimental results of Example 1 of the present invention.

[0044] Figure 10 This is a result diagram of the initial use of VLC player to directly play ZlMedia video stream in the verification embodiment 2 of the present invention.

[0045] Figure 11 This is a screenshot showing the result of playing the ZlMedia video stream using the VLC player after manually calling the stream retrieval interface in Verification Embodiment 2 of the present invention.

[0046] Figure 12 This is a diagram showing the result of the VLC player playing an HTTP-PS video stream in Embodiment 2 of the present invention.

[0047] Figure 13 This is a schematic diagram of the device described in Embodiment 2 of the present invention.

[0048] Figure 14 This is a flowchart illustrating step 1 of the system described in Embodiment 3 of the present invention.

[0049] Figure 15 This is a flowchart illustrating step 3 of the system described in Embodiment 3 of the present invention.

[0050] Figure 16 This is a flowchart illustrating step 4 of the system described in Embodiment 3 of the present invention.

[0051] Figure 17 This is a flowchart illustrating step 5 of the system described in Embodiment 3 of the present invention.

[0052] Figure 18 This is a flowchart illustrating step 6 of the system described in Embodiment 3 of the present invention.

[0053] Figure 19 This is a flowchart illustrating step 7 of the system described in Embodiment 3 of the present invention.

[0054] Figure 20 This is a flowchart illustrating step 9 of the system described in Embodiment 3 of the present invention.

[0055] Figure 21 This is a schematic diagram of the structure of the computer device described in Embodiment 4 of the present invention. Detailed Implementation

[0056] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0057] To facilitate understanding, the interactive parties and / or terms and / or custom terms involved in this invention will first be explained in conjunction with the technical solution of this invention:

[0058] PS is short for MPEG-PS, a container for multiplexing digital audio and video, etc. It was originally designed for reliable and stable storage media such as optical discs.

[0059] The GB28181 protocol uses RTP-based PS (Personal Signal Processing) encapsulation for audio and video data. This encapsulates audio and video streams into PS packets, which are then packaged into RTP packets as payloads. During PS encapsulation, each video frame is encapsulated into a single PS packet. Each keyframe's PS packet contains a System Header and a Program Stream Map (PSM). The System Header and PSM are placed after the PS packet header and before the first PES (Program Stream Map) packet. Specifically... Figure 3 As shown.

[0060] The existing solution has two technical problems, as follows:

[0061] Issue 1: The HTTP-PS format is not supported, resulting in unnecessary video stream format conversion, which increases computational overhead and end-to-end latency.

[0062] Question 2: It does not support automatic management of source video streams (i.e., on-demand streaming) in scenarios with the GB28181 protocol, requiring additional maintenance of video stream opening and closing, which increases the cost of use.

[0063] To address this issue, this invention implements a low-latency HTTP-PS video stream conversion for RTP video streams based on the HTTP protocol, avoiding video stream format conversion and reducing computational overhead and end-to-end latency.

[0064] This invention addresses problem 2 by automatically pulling / cutting the video stream when it plays / stops, enabling on-demand streaming and providing an instant video streaming service, thus reducing usage costs.

[0065] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0066] Example 1

[0067] This embodiment describes a method for converting GB28181 protocol RTP video streams to low-latency HTTP-PS video streams, applied to an HTTP-PS server, such as... Figure 4 As shown, it includes the following steps:

[0068] Step 1: Receive the HTTP connection request for streaming media playback from the client. The HTTP connection request for streaming media playback includes at least a video stream identifier URL string. The video stream identifier URL string is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding.

[0069] Preferably, the client is a web-based streaming media player, a mobile streaming media player, VLC player, FFMPEG client, etc.

[0070] Step 2: Parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and create an HTTP response object.

[0071] In some embodiments, parsing the device codec DEV1 and channel codec CHANNEL1 of the target video source from the streaming media playback HTTP connection request includes:

[0072] Obtain the video stream identifier URL string from the HTTP connection request for streaming media playback;

[0073] The regular expression matching function is called to match the video stream identifier URL string to obtain the device code DEV1 and channel code CHANNEL1 of the target video source.

[0074] Step 3: Randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1.

[0075] Step 3 includes the following specific steps:

[0076] Step 3.1: Randomly obtain a port p within the local TCP / 20000-30000 port range;

[0077] Step 3.2: Attempt to bind the port. If the port binding attempt is successful, it proves that port p is available. Set P1 equal to p and proceed to step 3.3. If the port binding attempt fails, it proves that port p is unavailable. Return to step 3.1.

[0078] Step 3.3: Start the TCP Socket service S1 to listen on port P1.

[0079] Step 4: Send a stream request to the SIP server to request that it send the RTP video stream of the target video source to the Socket service S1.

[0080] Step 4 includes the following specific steps:

[0081] Step 4.1: Based on its own IP address, its own port P1, the device code DEV1 and channel code CHANNEL1 of the target video source, the stream request is sent to the SIP server. The stream request includes at least the parameters DEV1, CHANNEL1, IP address, and P1, so that the SIP server can receive the stream request, determine the target video source based on DEV1 and CHANNEL1, and initiate a stream request operation to the target video source, instructing the target video source to send the video stream to the HTTP-PS server corresponding to (IP address, P1) in TCP mode.

[0082] Step 4.2: Receive the TCP connection request initiated by the target video source, establish CONN1 with the target video source, so that the target video source can send PS data packets to the Socket service S1 through CONN1.

[0083] Step 5: After receiving the RTP data after each PS data packet sent by the target video source, the Socket service S1 reassembles the RTP data into packets in real time to obtain the PS data packet and writes it into the HTTP response object so that the client can obtain the PS packet data from the HTTP response object and display a frame of live video.

[0084] Specifically, the steps in step 5 are as follows:

[0085] Socket service S1 receives and buffers the RTP data sent by the target video source one by one. The RTP data includes the RTP packets corresponding to the i-th PS data packet: RTP packet 1, RTP packet 2, RTP packet 3... RTP packet m, which are written to the CONN1 object one by one by the target video source, where i is an integer.

[0086] For each RTP packet received, check if its packet end flag is 1. If the packet end flag is not 1, store the RTP packet in the buffer and continue to receive the next RTP packet. If the packet end flag is 1, assemble all the RTP packets in the buffer to obtain the i-th PS data packet.

[0087] Write the i-th PS data packet to the HTTP response object so that the client can obtain the corresponding PS data packet from the HTTP response object for display.

[0088] Step 6: When an HTTP connection is detected to be disconnected, stop the Socket service S1.

[0089] The beneficial effects of this invention compared to existing solutions are: (1) It realizes the conversion of PS video streams carried by RTP in GB28181 protocol to HTTP-PS video streams, which avoids video format conversion and reduces end-to-end latency compared to existing solutions.

[0090] (2) Automatic on-demand streaming is achieved, which reduces the complexity of interface control compared with the existing solution and lowers the cost of using GB28181 video stream.

[0091] Furthermore, this embodiment provides verification experiments to demonstrate the beneficial effects achieved in the above two aspects.

[0092] 1. Hardware Environment

[0093] like Figure 5 As shown, this is a Windows workstation with an i7-10700 processor, 16GB of memory, Windows 10 Home operating system, a 1000M wired network, and an IP address of 192.168.1.164.

[0094] The Ubuntu workstation has a Ryzen 7-3700X processor, 32GB of RAM, runs Ubuntu 24.04 Server, and is connected to a 1000M wired network with the IP address 192.168.1.122.

[0095] 2. Software Environment

[0096] The Windows workstation uses TimeSync for real-time timestamps, HappyTimeSoft's GB28181 Device binary software for virtual GB28181, version 3.0.21 for VLC, and the official pre-compiled binary version of ffplay based on ffmpeg 7.0 for ffplay.

[0097] The Ubuntu workstation uses wvp-GB28181-pro for GB28181 signaling service, ZlMedia streaming media service uses the pre-built Docker image from ZlMedia, and HTTP-PS service is developed using this embodiment and executed as a Docker container.

[0098] 3. Experimental Environment Setup

[0099] Step 1: Start the GB28181 signaling service (wvp-GB28181-pro service) on the Ubuntu workstation, start the ZlMedia streaming media service, and associate the ZlMedia service with the wvp-GB28181-pro service. After successful association between wvp-GB28181-pro and ZlMedia, as shown... Figure 6 As shown.

[0100] Start the HTTP-PS service (the implementation of this method) and associate it with the video stream retrieval interface of the wvp-GB28181-pro service.

[0101] HTTP-PS uses environment variable configuration to achieve on-demand streaming. When starting the HTTP-PS service, the value of the environment variable INVITE_API is set to: http: / / 192.168.1.122:18080 / api / play / startSVAC. This variable is used to automatically initiate a streaming call to the signaling server when streaming is on demand.

[0102] Step 2: Launch the TimeSync window program on the Windows workstation. After launching, it will look like this. Figure 7 As shown.

[0103] Step 3: Configure the virtual GB28181 device program (GB28181 Device) to capture the window from Step 2 as a video source, and register it with the wvp-GB28181-pro service started in Step 1.

[0104] After successful registration, the list of WVP-GB28181-Pro service devices is as follows: Figure 8 As shown.

[0105] Step 4: Obtain the video stream URLs corresponding to the Zlmedia service and the HTTP-PS service, respectively. See the table below:

[0106] streaming service types Video stream URL ZlMedia http: / / 192.168.1.122:40080 / rtp / 41000000112008000164_41000000112008000164.live.mp4 HTTP-PS (this example) http: / / 192.168.1.122:8019 / gb28181 / 41000000112008000164 / 41000000112008000164.ps

[0107] Experiment 1: End-to-End Delay Experiment

[0108] End-to-end latency can be represented by the difference between the time displayed by the real-time timestamp program TimeSync and the real-time playback of the image in ffplay's unbuffered mode. Figure 9 The mid-to-end latency is 341 milliseconds.

[0109] It is understandable that end-to-end latency assessments of streaming services based on a single measurement are unreliable. To more accurately represent end-to-end latency, the following formula is used:

[0110]

[0111] in Representing the The value of the end-to-end delay is measured.

[0112] The results of the end-to-end latency experiment conducted using ffplay on a Windows workstation are as follows:

[0113]

[0114] Data shows that, under current experimental conditions, this solution reduces the average end-to-end latency by 22 milliseconds compared to ZlMedia, which is significantly better than existing solutions.

[0115] Experiment 2: On-Demand Automatic Flow Harvesting Experiment

[0116] To verify the superiority of the method described in this embodiment in terms of on-demand automatic streaming, the number of additional interface calls required by ZlMedia and HTTP-PS (based on this method) when playing video streams was compared using VLC player on a Windows workstation.

[0117] Play ZlMedia video streams directly using VLC player, such as... Figure 10 As shown.

[0118] After manually calling the stream fetching interface, play the ZlMedia video stream using VLC player, such as... Figure 11 As shown, normal playback of its video stream still relies on additional interface calls.

[0119] Playing HTTP-PS video streams directly using VLC player, such as... Figure 12 As shown, it enables on-demand streaming, making it ready to use immediately.

[0120] The experimental results are summarized below:

[0121] streaming service types Player Additional API calls Automatic stream fetching ZlMedia vlc 1 Not supported HTTP-PS (this solution) vlc 0 support

[0122] Experimental results show that this method achieves automatic on-demand streaming, reduces complex interface control compared to existing solutions, supports direct playback without additional interface calls, and lowers the cost of using GB28181 video streams.

[0123] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0124] Example 2

[0125] Based on the same inventive concept, this application also provides an apparatus for implementing the GB28181 protocol RTP video stream to low-latency HTTP-PS video stream conversion method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method. Therefore, the specific limitations in one or more apparatus embodiments provided below can be found in Embodiment 1 or Embodiment 2 regarding the limitations of the GB28181 protocol RTP video stream to low-latency HTTP-PS video stream conversion method, and will not be repeated here.

[0126] Specifically, such as Figure 13 As shown, the apparatus for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream includes:

[0127] The HTTP connection request acquisition module is used to receive streaming media playback HTTP connection requests. The streaming media playback HTTP connection request includes at least a video stream identifier URL string, which is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding.

[0128] The HTTP connection request parsing module is used to parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and to create an HTTP response object;

[0129] The port enablement module is used to randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1.

[0130] The stream retrieval request sending module is used to initiate a stream retrieval request to the SIP server, requesting it to send the RTP video stream of the target video source to the Socket service S1;

[0131] When the data receiving module receives the RTP data after the i-th PS data packet is packetized by the target video source through the Socket service S1, it reassembles the RTP data into packets in real time to obtain the i-th PS data packet and writes it into the HTTP response object so that the client can obtain the i-th PS packet data from the HTTP response object and display the corresponding live screen.

[0132] The HTTP connection disconnect module is used to stop the Socket service S1 when an HTTP connection disconnection is detected.

[0133] Example 3

[0134] This embodiment provides a GB28181 protocol RTP video stream conversion system, including:

[0135] The target video source is used to generate an RTP video stream using the GB28181 protocol;

[0136] An HTTP-PS server, wherein the HTTP-PS server is configured to perform the steps of the method according to any one of claims 1 to 6;

[0137] A network switch is used to enable communication between the target video source and the HTTP-PS service.

[0138] like Figure 14-20 As shown, the conversion system is implemented according to the following steps:

[0139] Step 1: The client constructs a video stream identifier URL string based on the device encoding (DEV1) and channel encoding (CHANNEL1) rules of the target video source.

[0140] The specific steps for step 1 are as follows:

[0141] Step 1.1: The client obtains the IP address and port of the HTTP-PS server;

[0142] Step 1.2: The client obtains the device encoding (DEV1) and channel encoding (CHANNEL1) of the target video source;

[0143] Step 1.3: The client calls the URL concatenation function to construct the video stream identifier URL string using the HTTP-PS server's IP address, HTTP-PS server port, device encoding obtained in Step 1.1, and channel encoding.

[0144] The client in step 1 is usually a web-based streaming media player, a mobile streaming media player, VLC player, FFMPEG client, etc.

[0145] Step 2: The client initiates a streaming media playback HTTP connection request to the HTTP-PS server. The streaming media playback HTTP connection request includes at least a video stream identifier URL string.

[0146] Step 3: The HTTP-PS server obtains the video stream identifier URL string from the HTTP request, parses the target video source device codec DEV1 and channel codec CHANNEL1 from the video stream identifier URL string, and then creates an HTTP response object.

[0147] The specific steps for step 3 are as follows:

[0148] Step 3.1: The HTTP-PS server receives the HTTP request initiated by the client;

[0149] Step 3.2: HTTP-PS retrieves the video stream playback URL string from the HTTP request;

[0150] Step 3.3: The HTTP-PS server calls the regular expression matching function to pass the video stream playback URL string obtained in step 3.2 to the corresponding string matching function to parse out the target video source device encoding DEV1 and channel encoding CHANNEL1.

[0151] Step 4: The HTTP-PS server randomly obtains an unused TCP port P1 from the local machine and starts the Socket service S1 to listen on port P1.

[0152] Step 4 is as follows:

[0153] Step 4.1: The HTTP-PS server randomly obtains a port p within the local TCP / 20000-30000 port range;

[0154] Step 4.2: Attempt to bind to the port. If the binding attempt is successful, it proves that port p is available. Set P1 equal to p and proceed to step 4.3. If the binding attempt fails, it proves that port p is unavailable. Proceed to step 4.1.

[0155] Step 4.3: Start the TCP Socket service S1 to listen on port P1.

[0156] Step 5: The HTTP-PS server sends a stream request to the SIP server, requesting it to send the RTP video stream of the target video source (DEV1 device encoding and CHANNEL1 channel encoding) to the Socket service S1. The target video source initiates a TCP connection CONN1 to S1.

[0157] Step 5 specifically includes:

[0158] Step 5.1: The HTTP-PS server initiates a stream retrieval request to the SIP server based on the target video source device code DEV1, channel code CHANNEL1, IP address obtained in step 3.2, and port P1 obtained in step 4.3, and passes the following key parameters: DEV1, CHANNEL1, IP address, P1.

[0159] Step 5.2: The SIP server receives the request and retrieves the parameters DEV1, CHANNEL1, IP, and P1.

[0160] Step 5.3: The SIP server initiates a stream retrieval operation to the target video source, instructing it to send the video stream in TCP mode to the following address: (IP, P1);

[0161] Step 5.4: The target video source initiates a TCP connection request to (IP, P1);

[0162] Step 5.5: A connection CONN1 is successfully established between the target video source and S1 (established via step 4.3).

[0163] Step 6: The target video source sends the RTP data after the first PS data packet is packetized to S1 through CONN1. The Socket service S1 in the HTTP-PS server receives the RTP data sent by the target device. The HTTP-PS server reassembles the RTP video stream into packets in real time, obtains the first PS data packet, and then immediately writes it into the HTTP response object. The client obtains the first PS packet data from the HTTP response object and displays the first frame of the live broadcast.

[0164] Step 6 is as follows:

[0165] Step 6.1: The target video source writes the RTP packets corresponding to the first PS data packet to the CONN1 object one by one: RTP packet 1, RTP packet 2, RTP packet 3... RTP packet m;

[0166] Step 6.2, the HTTP-PS server's Socket service S1 receives the RTP data sent by the target video source one by one: RTP packet 1, RTP packet 2, RTP packet 3... RTP packet m;

[0167] Step 6.3: After receiving the RTP packet m, the HTTP-PS server finds that its packet end marker is 1, and reassembles all the RTP packets in the buffer to obtain the first PS data packet.

[0168] Step 6.4: The HTTP-PS server writes the first PS packet to the HTTP response object;

[0169] Step 6.5: The client obtains the first PS packet data from the HTTP response and displays the first frame of the live broadcast.

[0170] Step 7: The target video source sends the RTP data after the next PS data packet is packetized to S1 through CONN1. The Socket service S1 in the HTTP-PS server receives the new RTP data sent by the target device. The HTTP-PS server reassembles the RTP video stream into packets in real time, obtains the next PS data packet, and then immediately writes it into the HTTP response object. The client obtains the next PS packet data from the HTTP response object and displays the next frame of the live broadcast.

[0171] Step 7 is as follows:

[0172] Step 7.1: The target video source writes the RTP packets corresponding to the k-th PS data packet to the CONN1 object one by one: RTP packet k, RTP packet k+1, RTP packet k+2...RTP packet k+n;

[0173] Step 7.2, the HTTP-PS server's Socket service S1 receives the RTP data sent by the target video source one by one: RTP packet k, RTP packet k+1, RTP packet k+2...RTP packet k+n;

[0174] Step 7.3: After the HTTP-PS server receives the RTP packet k+n, it finds that its packet end marker is 1, and reassembles all the RTP packets in the buffer to obtain the kth PS data packet;

[0175] Step 7.4: The HTTP-PS server writes the k-th PS data packet to the HTTP response object;

[0176] Step 7.5: The client obtains the k-th PS packet data from the HTTP response and displays the k-th frame of the live broadcast.

[0177] Step 8: By repeating step 7, the client continues to watch the screen for N seconds.

[0178] Step 9: The client closes the viewing window, disconnects the HTTP connection maintained with the HTTP-PS server, and the HTTP-PS server stops receiving RTP video streams sent by the target video source.

[0179] Step 9 is as follows:

[0180] Step 9.1 The client closes the playback window;

[0181] Step 9.2 The client disconnects the HTTP connection maintained with the HTTP-PS server;

[0182] Step 9.3 The HTTP-PS server detects that the HTTP connection has been closed;

[0183] Step 9.4 The HTTP-PS server stops the TCP Socket service S1 and stops listening on port P1;

[0184] Step 9.5 The CONN1 connection (established via step 5.5) is disconnected, and the reception of RTP video streams sent by the target video source is stopped.

[0185] Example 4

[0186] This embodiment provides a computer device, which may be a terminal, and its internal structure diagram may be as follows. Figure 21 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for converting GB28181 protocol RTP video streams to low-latency HTTP-PS video streams. The display unit of the computer device forms a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0187] Those skilled in the art will understand that Figure 21The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] Example 6

[0189] Based on the above embodiments, this embodiment provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: Step 1, receiving a streaming media playback HTTP connection request from a client, wherein the streaming media playback HTTP connection request includes at least a video stream identifier URL string, wherein the video stream identifier URL string is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding;

[0190] Step 2: Parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and create an HTTP response object;

[0191] Step 3: Randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1;

[0192] Step 4: Send a stream request to the SIP server to request it to send the RTP video stream of the target video source to the Socket service S1.

[0193] Step 5: After receiving the RTP data after each PS data packet sent by the target video source, the Socket service S1 reassembles the RTP data into packets in real time to obtain the PS data packet and writes it into the HTTP response object so that the client can obtain the PS packet data from the HTTP response object and display a frame of live video.

[0194] Step 6: When an HTTP connection is detected to be disconnected, stop the Socket service S1.

[0195] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0196] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A method for converting GB28181 protocol RTP video streams into low-latency HTTP-PS video streams, applied to an HTTP-PS server, characterized in that... Includes the following steps: Step 1: Receive the streaming media playback HTTP connection request from the client. The streaming media playback HTTP connection request includes at least a video stream identifier URL string. The video stream identifier URL string is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding. Step 2: Parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and create an HTTP response object; Step 3: Randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1; Step 4: Send a stream request to the SIP server to request it to send the RTP video stream of the target video source to the Socket service S1. Step 5: After receiving the RTP data after each PS data packet sent by the target video source, the Socket service S1 reassembles the RTP data into packets in real time to obtain the PS data packet and writes it into the HTTP response object so that the client can obtain the PS packet data from the HTTP response object and display a frame of live video. Step 6: When an HTTP connection is detected to be disconnected, stop the Socket service S1.

2. The method for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream according to claim 1, characterized in that: In step 2, the device codec (DEV1) and channel codec (CHANNEL1) of the target video source are parsed from the HTTP connection request for streaming media playback, including: Obtain the video stream identifier URL string from the HTTP connection request for streaming media playback; The regular expression matching function is called to match the video stream identifier URL string to obtain the device code DEV1 and channel code CHANNEL1 of the target video source.

3. The method for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream according to claim 1 or 2, characterized in that: Step 3 includes the following specific steps: Step 3.1: Randomly obtain a port p within the local TCP / 20000-30000 port range; Step 3.2: Attempt to bind the port. If the port binding attempt is successful, it proves that port p is available. Set P1 equal to p and proceed to step 3.

3. If the port binding attempt fails, it proves that port p is unavailable. Return to step 3.

1. Step 3.3: Start the TCP Socket service S1 to listen on port P1.

4. The method for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream according to claim 3, characterized in that, Step 4 includes the following specific steps: Step 4.1: Based on its own IP address, its own port P1, the device code DEV1 and channel code CHANNEL1 of the target video source, the stream request is sent to the SIP server. The stream request includes at least the parameters DEV1, CHANNEL1, IP address, and P1, so that the SIP server can receive the stream request, determine the target video source based on DEV1 and CHANNEL1, and initiate a stream request operation to the target video source, instructing the target video source to send the video stream to the HTTP-PS server corresponding to (IP address, P1) in TCP mode. Step 4.2: Receive the TCP connection request initiated by the target video source, establish CONN1 with the target video source, so that the target video source can send PS data packets to the Socket service S1 through CONN1.

5. The method for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream according to claim 1, characterized in that: In steps 5 and 6, the Socket service S1 receives and caches the RTP data sent by the target video source one by one. The RTP data includes the RTP packets corresponding to the i-th PS data packet: RTP packet 1, RTP packet 2, RTP packet 3... RTP packet m, which are written to the CONN1 object one by one by the target video source. For each RTP packet received, check if its packet end flag is 1. If the packet end flag is not 1, store the RTP packet in the buffer and continue to receive the next RTP packet. If the packet end flag is 1, assemble all the RTP packets in the buffer to obtain the i-th PS data packet. Write the i-th PS data packet to the HTTP response object so that the client can obtain the corresponding PS data packet from the HTTP response object for display.

6. The method for converting GB28181 protocol RTP video stream to low-latency HTTP-PS video stream according to claim 1, step 7 specifically includes: Step 7.1: When the HTTP connection with the client is detected to be disconnected, stop the TCP Socket service S1 and stop listening on port P1; wherein, the HTTP connection is disconnected by the client after closing the playback window; Step 7.2: Disconnect CONN1 and stop receiving RTP video streams sent by the target video source.

7. An apparatus for converting GB28181 protocol RTP video streams to low-latency HTTP-PS video streams, characterized in that, include: The HTTP connection request acquisition module is used to receive streaming media playback HTTP connection requests. The streaming media playback HTTP connection request includes at least a video stream identifier URL string, which is formed by the client concatenating the IP address of the HTTP-PS server, the port of the HTTP-PS server, the device encoding, and the channel encoding. The HTTP connection request parsing module is used to parse the device codec DEV1 and channel codec CHANNEL1 of the target video source from the HTTP connection request for streaming media playback, and to create an HTTP response object; The port enablement module is used to randomly obtain an unused TCP port P1 from the local machine and start the Socket service S1 to listen on port P1. The stream retrieval request sending module is used to initiate a stream retrieval request to the SIP server, requesting it to send the RTP video stream of the target video source to the Socket service S1; When the data receiving module receives the RTP data after the i-th PS data packet is packetized by the target video source through the Socket service S1, it reassembles the RTP data into packets in real time to obtain the i-th PS data packet and writes it into the HTTP response object so that the client can obtain the i-th PS packet data from the HTTP response object and display the corresponding live screen. The connection disconnect module is used to stop the Socket service S1 when an HTTP connection disconnection is detected.

8. A conversion system for GB28181 protocol RTP video streams, characterized in that, include: The target video source is used to generate an RTP video stream using the GB28181 protocol; An HTTP-PS server, wherein the HTTP-PS server is configured to perform the steps of the method according to any one of claims 1 to 6; A network switch is used to enable communication between the target video source and the HTTP-PS service.

9. A computer device, characterized in that: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images