A method and system for playing RTSP video stream data

Abstract

The application provides an RTSP video stream data playing method and system, the method comprises the following steps: selecting a target RTSP video source and completing matching of a decoder output resolution to a large lamp physical resolution; establishing a TCP control connection with an RTSP server to perform a standard handshake, analyzing a Session identifier and a server RTP / RTCP port; creating a UDP socket to bind a random port, configuring a receiving buffer and a timeout parameter, and starting an independent receiving thread; detecting packet loss and disorder of FU-A fragments based on sequence numbers, and recombining the FU-A fragments to complete H.264 data; extracting a luminance component to generate a gray image after decoding, and pushing the gray image to a driving layer after correcting a direction by vertical flipping; monitoring a stream timeout, sending a TEARDOWN and releasing resources after interruption. The application realizes real-time and stable projection of a remote video stream on a vehicle-mounted pixel headlamp, and significantly improves content update flexibility and playing robustness in a weak network environment.

Description

Technical Field

[0001] This invention belongs to the technical field of in-vehicle video streaming decoding, specifically relating to a method and system for playing RTSP video stream data. Background Technology

[0002] With the development of intelligent and connected vehicles, in-vehicle Ethernet, which has the characteristics of high bandwidth, low latency and high reliability, has become the mainstream communication carrier for the electronic and electrical architecture of intelligent connected vehicles. At the same time, pixel headlights based on MicroLED chip solutions are also widely used in various in-vehicle lighting scenarios.

[0003] The traditional implementation of MicroLED pixel headlights has several inherent drawbacks: First, it lacks flexibility in content updates, requiring light and shadow materials to be stored locally in the headlights beforehand, making it impossible to switch dynamic content in real time and limiting interactive functions; second, it consumes high transmission bandwidth, as high-definition raw pixel data can crowd out bandwidth resources for other in-vehicle services; third, it is difficult to adapt the image format, as external video streams require multiple steps such as decoding, format conversion, and resolution matching, and traditional solutions lack standardized processes; fourth, the in-vehicle link lacks robustness, lacking protection mechanisms against interference and link failures during driving, which can easily lead to safety hazards such as abnormal headlight illumination.

[0004] At the same time, the vehicle-mounted Ethernet carries multiple services such as optical and visual material transmission, traditional control, streaming media, and debugging and calibration. Different services have different requirements for transmission latency, jitter, and packet loss rate: control services have high requirements for latency and reliability and need deterministic transmission; streaming media services have a high tolerance for latency but are sensitive to jitter and packet loss, and transmission abnormalities will lead to abnormal projection images; debugging and calibration services have no requirements for latency but need to ensure data integrity.

[0005] Given the limited bandwidth, hardware resources, and main control computing power of the vehicle, the industry urgently needs a dedicated transmission and image processing mechanism adapted to MicroLED pixel headlights in order to achieve differentiated scheduling and transmission for different services and ensure the quality of streaming media projection. However, existing mainstream solutions still have shortcomings: First, most of them use proprietary protocols from manufacturers, which are difficult to standardize and have high adaptation costs. If a local push streaming architecture is adopted, it will significantly increase the main control computing load and the power consumption of the whole machine, which violates the lightweight and low power consumption design goal. Second, the general RTSP transmission mechanism is only designed for general monitoring LANs and is not optimized for the complex in-vehicle environment. It cannot solve the problem of jitter and packet loss, which can easily lead to abnormalities such as afterimages, tearing, and flickering in the projected image. Third, there is a lack of fault tolerance logic for the corresponding external push streaming and in-vehicle pull streaming scenarios. If the network fluctuates and disconnects, the projection will be interrupted directly, affecting the debugging and demonstration effects.

[0006] In summary, the industry urgently needs methods and systems to stably adapt and output RTSP network video streams to MicroLED pixel headlights, solving multiple technical problems of traditional solutions and existing streaming media solutions. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes an RTSP video stream data playback method and system, aiming to solve the technical issues of insufficient flexibility in updating content displayed by existing pixel headlights, high complexity in adapting network video streams, and poor playback stability in vehicle-mounted weak network environments.

[0008] In a first aspect, the present invention provides the following technical solution: a method for playing RTSP video stream data, comprising: Select the target RTSP video source according to the external SOME / IP control command or user configuration, complete the initialization operation of the MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of the pixel headlight; Create a TCP control socket based on the preset RTSP server address, port and stream name to complete the establishment of a control connection with the RTSP server; The RTSP standard handshake process is completed by sending OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence. Among them, the SETUP request carries the local RTP / RTCP port information and parses the Session identifier and the server's RTP and RTCP ports from the server response message. The local port allocation adopts the method of binding port 0, and the actual allocated RTP port is obtained through getsockname. The RTCP port is set to the RTP port number plus 1. Create a UDP socket and bind it to a random system port. Configure the receive buffer and receive timeout parameters for the socket. Complete the RTP receive channel setup and start an independent data receive thread. The thread continuously receives RTP data packets, parses the sequence number, marker bit and payload type in the packet header, processes the payload according to the NALU type, and reassembles it to obtain a complete H.264 data unit with AnnexB start code. The reconstructed H.264 data unit is sent to the decoder to obtain the original video frame. After being converted to RGB format, the R and B channel components are cleared, and only the Y luminance component is retained to generate a grayscale image. The grayscale image is vertically flipped to complete the orientation correction to adapt to the two up and down scanning output directions of the pixel headlight, and the image to be displayed is generated. The corrected image to be displayed is sent to the pixel headlight driver layer, which then controls the brightness output of the corresponding LED based on the pixel grayscale value. The system records the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined to be a stream interruption. After a delay, the receiving thread is closed, a TEARDOWN request is sent to the RTSP server, and TCP sockets, UDP sockets, and fragment buffer resources are released.

[0009] Compared to existing technologies, the advantages of this invention are as follows: This invention achieves real-time transmission of remote video streams to in-vehicle MicroLED pixel headlights through a complete RTSP / RTP standard protocol link, eliminating reliance on local static images and animation materials, significantly improving the flexibility of headlight display content updates, and supporting dynamic updates of various display content such as welcome animations, ADB light carpets, and road projections; H.264 encoding is used to compress transmitted video data, effectively reducing the bandwidth overhead of in-vehicle Ethernet, making it suitable for application scenarios with limited in-vehicle network resources and multiple devices sharing bandwidth; To address data out-of-order and packet loss issues in weak in-vehicle network environments, a FU-A fragmentation and reassembly and sequence number verification mechanism is designed, which can filter abnormal fragmented data and avoid mosaic effects during playback. It significantly improves the smoothness and robustness of video playback, addressing issues such as screen tearing and stuttering. Built-in grayscale conversion and vertical image flip correction logic allow the output image to directly match the hardware characteristics of headlights—monochrome illumination and fixed scanning direction—eliminating the need for additional hardware image processing chips and reducing hardware adaptation costs. A stream timeout detection and standardized resource reclamation process are designed to proactively release sockets, caches, and other resources when the network is interrupted, preventing abnormal display issues such as headlight image freezing and screen tearing, thus eliminating abnormal images from interfering with the driver's vision and ensuring nighttime driving safety. Simultaneously, unified management of sockets, caches, decoders, and other resources allows for complete resource reclamation when the video stream ends or is interrupted, preventing memory leaks and port occupancy failures during long-term operation of embedded devices.

[0010] Preferably, the steps of sequentially sending OPTIONS, DESCRIBE, SETUP, and PLAY requests to complete the RTSP standard handshake process include: The DESCRIBE request carries an Accept:application / sdp header. The response message body returned by the RTSP server contains SDP description information, which includes media type, transmission protocol, codec format, video resolution, and frame rate parameters. The Session field in the response header is parsed to obtain the session identifier string. If the Session field contains a semicolon and timeout parameters, only the part before the semicolon is taken as the session identifier. The SETUP request carries a Transport field, the format of which is Transport:RTP / AVP / UDP;unicast;client_port=<local RTP port>-<local RTCP port>; the SETUP response returned by the RTSP server carries a server_port=<server-side RTP port>-<server-side RTCP port> field, from which the server-side RTP port and the server-side RTCP port are parsed. The PLAY request carries the Session identifier and the Range:npt=0.000- field, indicating that playback starts from the beginning of the media.

[0011] Preferably, the steps of creating a UDP socket and binding it to a random system port, and configuring the receive buffer and receive timeout parameters for the socket include: Call the socket function to create a UDP datagram socket of type AF_INET as a local RTP socket; The setsockopt function is called to set the receive buffer size of the local UDP socket to 2MB and the receive timeout to 1 second; The bind function is called to bind the local UDP socket to the local address INADDR_ANY and port 0, and the operating system allocates a random port as the local RTP port; Call the getsockname function to obtain the actual bound local RTP port, and set the local RTCP port to the local RTP port plus 1.

[0012] Preferably, the step of processing the payload according to the NALU type and reassembling it to obtain a complete H.264 data unit with AnnexB start code includes: Determine if the version number of the RTP data packet is 2; otherwise, discard the RTP data packet. Calculate the actual load start position based on the CSRC Count field and Extension flag in the RTP header, and read the NALU type from the actual load start position. If the NALU type is a single NALU packet, then AnnexB start codes are added to the H.264 payload data to obtain a complete H.264 data unit; if there are previously incomplete FU-A fragment caches, then the FU-A fragment caches are cleared, the cache length is set to zero, and the completion flag is set. If the NALU type is FU-A fragment, upon receiving a fragment with the Start flag set to 1, a fragment buffer with a capacity of 256KB is initialized, the original NALU type in the FU Header is recorded, and the payload after the Start fragment is stored in the fragment buffer. Upon receiving an intermediate fragment with both the Start and End flags set to 0, the current payload is appended to the fragment buffer. Upon receiving a fragment with the End flag set to 1, the current payload is appended to the fragment buffer, and then the complete NALU header is recovered based on the original NALU type and the first three bits of the original NALU header. The complete NALU header is combined with the data in the fragment buffer, and an AnnexB start code is added to obtain the complete H.264 data unit. During the appending process in the fragment buffer, if the length of the appended data reaches or exceeds the maximum capacity of the fragment buffer, the fragment buffer is reset and the current fragment is discarded. For FU-A fragmented packets, packet loss and out-of-order anomalies are detected based on sequence numbers. Abnormal fragments are directly discarded and the corresponding cache is cleared.

[0013] Preferably, the step of detecting packet loss and out-of-order anomalies based on sequence numbers, directly discarding abnormal fragments and clearing the corresponding cache includes: Maintain the previously received RTP sequence number and the sequence number validity flag; If the sequence number validity flag is true, then calculate the expected next RTP sequence number to be received. The expected next RTP sequence number to be received is the previous sequence number plus 1 and modulo 65536. If the sequence number of the current RTP packet is not equal to the expected next RTP sequence number, then packet loss or out-of-order delivery is determined, the current fragment buffer is discarded, the buffer length is set to zero, the completion flag is set, the NALU type is cleared, and the sequence number validity flag is cleared.

[0014] Preferably, the step of converting to RGB format, clearing the R and B channel components, retaining only the Y luminance component to generate a grayscale image, and performing vertical flipping on the grayscale image to complete orientation correction includes: Convert the decoded NV21 format raw video frames into RGB24 data; Read the brightness value of the Y plane in the original video frame, and map the brightness value to the three channels of the grayscale image after black bit cropping. The black bit cropping is to subtract a preset offset of 20 from the brightness value and limit it to the range of 0 to 255. The R and B channels in the RGB24 data are cleared to zero, and only the luminance component is retained as the pixel value of the grayscale image; A vertical flip operation is performed on the grayscale image, swapping the pixel data of the top row with the bottom row, and the second-top row with the second-bottom row, until the entire image is flipped vertically to adapt to the scanning direction of the MicroLED pixel headlight from top to bottom or from bottom to top.

[0015] Preferably, the steps of recording the real-time data packet reception timestamp, determining a stream interruption if no data is received beyond a preset timeout threshold, closing the receiving thread after a delay, sending a TEARDOWN request to the RTSP server, and releasing the TCP socket, UDP socket, and fragmented buffer resources include: After each successful reception of an RTP data packet, the system time function is called to obtain the current time, and the current time is updated to the last time the data was received. Periodically compare the time difference between the current system time and the last received data time; If the time difference exceeds the preset timeout threshold of 2 seconds, then after a preset exit delay of 2 seconds, a global exit flag is set to terminate the RTP data receiving thread; during the operation of the RTP data receiving thread, if a stop playback command is received, then the global exit flag is set. After the global exit flag is set, a TEARDOWN request carrying the Session identifier is sent to the RTSP server, the shutdown function is called to terminate the bidirectional data transmission of the TCP control socket, and the close function is called to release the TCP control socket and the local UDP socket, and the fragment buffer is released.

[0016] Secondly, the present invention provides the following technical solution: an RTSP video stream data playback system, the system comprising: The initialization configuration module is used to select the target RTSP video source according to external SOME / IP control instructions or user configuration, complete the initialization operation of the MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of the pixel headlight. The socket creation module is used to create TCP control sockets based on the preset RTSP server address, port and stream name, and complete the establishment of a control connection with the RTSP server. The RTSP client module is used to send OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence to complete the standard RTSP handshake process. Among them, the SETUP request carries local RTP / RTCP port information and parses the Session identifier and server RTP and RTCP ports from the server response message. The local port allocation adopts the method of binding to port 0, and the actual allocated RTP port is obtained through getsockname. The RTCP port is set to the RTP port number plus 1. The RTP receive channel module is used to create UDP sockets and bind them to random system ports, configure receive buffers and receive timeout parameters for the sockets, complete the RTP receive channel setup, and start an independent data receive thread. The frame reassembly and parsing module is used to receive RTP data packets in a loop within a thread, parse the sequence number, marker bit and payload type in the packet header, process the payload according to the NALU type, and reassemble it to obtain a complete H.264 data unit with AnnexB start code. The display adaptation and correction module is used to send the reconstructed complete H.264 data unit into the decoder to decode it into the original video frame. After converting it into RGB format, the R and B channel components are cleared, and only the Y luminance component is retained to generate a grayscale image. The grayscale image is vertically flipped to complete the orientation correction in order to adapt to the two up and down scanning output directions of the pixel headlight and generate the image to be displayed. The pixel headlight driver module is used to send the calibrated image to be displayed to the pixel headlight driver layer, which then controls the brightness output of the corresponding LED based on the pixel grayscale value. The stream monitoring and resource reclamation module is used to record the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined that the stream is interrupted. After a delay, the receiving thread is closed, and a TEARDOWN request is sent to the RTSP server to release TCP sockets, UDP sockets and fragmented buffer resources.

[0017] Thirdly, the present invention provides the following technical solution: a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the RTSP video stream data playback method as described above.

[0018] Fourthly, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the RTSP video stream data playback method as described above. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the RTSP video stream data playback method provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the vehicle-mounted hardware deployment for the RTSP video stream data playback method provided in Embodiment 1 of the present invention; Figure 3 This is a timing diagram of the RTSP standard handshake interaction provided in Embodiment 1 of the present invention; Figure 4 This is a flowchart of the RTP / RTCP UDP channel creation and port negotiation provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the RTP data packet format and H.264 payload encapsulation structure provided in Embodiment 1 of the present invention; Figure 6 This is a flowchart of a single NALU packet processing method provided in Embodiment 1 of the present invention; Figure 7 This is a flowchart of the FU-A packet reassembly and packet loss / out-of-order detection process provided in Embodiment 1 of the present invention; Figure 8 This is a flowchart of H.264 decoding and luminance domain conversion processing provided in Embodiment 1 of the present invention; Figure 9 This is a flowchart of the orientation correction and MicroLED pixel headlight driving display provided in Embodiment 1 of the present invention; Figure 10 This is a flowchart of RTSP video stream timeout detection and resource reclamation provided in Embodiment 1 of the present invention; Figure 11 This is a structural block diagram of the RTSP video stream data playback system provided in Embodiment 2 of the present invention; Figure 12 This is a schematic diagram of the hardware structure of a computer provided for another embodiment of the present invention.

[0021] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0023] In this embodiment of the invention, unless otherwise specified, RTSP (Real-Time Streaming Protocol) is a network application layer protocol used to control media session establishment, playback, pause, and stop operations between a streaming media server and a client; RTP (Real-time Transport Protocol) is used to carry media data; RTCP (RTP Control Protocol) works in conjunction with RTP to transmit session control information; SDP (Session Description Protocol) is a format for describing multimedia sessions, used to negotiate parameters such as media type, transport protocol, and encoding / decoding format; NALU (Network Abstraction Layer Unit) is a basic network transmission unit defined in the H.264 video coding standard; FU-A (Fragmentation Unit A) is an RTP packetization mode defined in the H.264 standard; SOME / IP (Scalable Service-Oriented Middleware over IP) is a network application layer protocol used to control media session establishment, playback, pause, and stop operations between a streaming media server and a client; RTP (Real-time Transport Protocol) is used to carry media data; RTCP (Relative TCP Control Protocol) works in conjunction with RTP to transmit session control information; SDP (Session Description Protocol) is a format for describing multimedia sessions, used to negotiate parameters such as media type, transport protocol, and encoding / decoding format; NALU (Network Abstraction Layer Unit) is a basic network transmission unit defined in the H.264 video coding standard; FU-A (Fragmentation Unit A) is an RTP packetization mode defined in the H.264 standard; and SOME / IP (Scalable Service-Oriented Middleware over IP) is a network application layer protocol used to control media session establishment, playback, pause, and stop operations between a streaming media server and a client; RTP (Real-time Transport Protocol) is ... IP (IP-based scalable service-oriented middleware) is an in-vehicle Ethernet communication protocol; NV21 is a YUV color encoding format, which includes a Y luminance plane and an alternating VU chrominance plane; RGB24 is an RGB color encoding format, where each pixel is represented by 24 bits, with each of the R, G, and B channels occupying 8 bits; PWM (Pulse Width Modulation) is a technique that controls average current by adjusting the pulse width.

[0024] Example 1 In Embodiment 1 of the present invention, as Figure 1 As shown, an RTSP video stream data playback method is provided. The method is deployed inside the vehicle and connected to the vehicle network and MicroLED pixel headlight array. It is used to receive the remote RTSP video stream and project the video content to the ground or target area in front of the vehicle in real time, so as to realize intelligent lighting functions such as welcome animation, ground marking, and AFS / ADB light carpet display. This invention employs a modular, layered architecture, comprising, from top to bottom, an application control layer, a streaming media processing layer, an image adaptation layer, and a hardware driver layer. The application control layer is responsible for responding to external control commands, such as SOME / IP control messages from the vehicle domain controller, in-vehicle infotainment system, or user mobile terminal, to determine parameters such as the RTSP video source to be played, playback start and end times, and display mode. The streaming media processing layer is responsible for establishing a connection with a remote RTSP server, receiving RTP media data packets, and recovering the complete H.264 video stream. The image adaptation layer is responsible for converting the decoded video frames into a brightness domain image suitable for MicroLED pixel headlight display and performing necessary geometric corrections. The hardware driver layer is responsible for pushing the final image data to the MicroLED pixel headlight driver chip, controlling each LED to emit light according to its pixel value.

[0025] like Figure 2 As shown, in a vehicle hardware deployment scenario, the system of this embodiment can be integrated into the headlight controller or body domain controller. The headlight controller is connected to the vehicle gateway via an onboard Ethernet switch, and the onboard gateway further communicates with external networks via a 4G / 5G mobile communication module, a Wi-Fi module, or a V2X module. A remote RTSP video source server is deployed in the cloud, on a roadside unit, or on a user's mobile terminal, and pushes H.264 encoded video streams to the vehicle via the standard RTSP / RTP protocol. After receiving the video stream, the headlight controller sequentially performs decoding, brightness domain conversion, and direction correction, and finally outputs the image data to the MicroLED pixel headlight array located at the vehicle's headlight position. The MicroLED pixel headlight array consists of a large number of micron-sized light-emitting diodes arranged in a matrix, and each LED can be independently controlled to turn on and off, thereby forming a high-resolution dynamic light pattern on the ground in front of the vehicle or in the target area ahead.

[0026] This embodiment introduces the RTSP / RTP network streaming media protocol into the vehicle lighting system, breaking through the limitation of traditional pixel headlights that only support locally preset content. This allows the vehicle to acquire and play remote video content in real time, significantly improving the interactivity and personalization capabilities of the pixel headlights. Simultaneously, the use of H.264 compression encoding for video data transmission greatly reduces the bandwidth consumption of the vehicle network compared to directly transmitting raw image data, adapting to application scenarios where vehicle network bandwidth is limited.

[0027] A method for playing RTSP video stream data, the method comprising: S1. Select the target RTSP video source according to the external SOME / IP control command or user configuration, complete the initialization operation of the MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of the pixel headlight. Specifically, in this embodiment of the invention, the target RTSP video source is first selected based on external SOME / IP control commands or user configuration. SOME / IP is a service-oriented middleware protocol commonly used in in-vehicle Ethernet communication. External control commands can come from the vehicle domain controller, the in-vehicle infotainment system, or the user's mobile terminal. Simultaneously, the initialization operations of the MicroLED pixel headlight driver layer and the H.264 decoder are completed, and the decoder output resolution is matched to the physical resolution of the pixel headlight. In this embodiment, the physical resolution of the MicroLED pixel headlight is 320×80, which can be adjusted according to hardware configuration in practical applications, such as 256×64, 512×128, etc.

[0028] S2. Create a TCP control socket based on the preset RTSP server address, port and stream name to complete the establishment of a control connection with the RTSP server; Specifically, a TCP control socket is created based on the preset RTSP server address, port, and stream name to establish a control connection with the RTSP server. TCP (Transmission Control Protocol) is a connection-oriented, reliable transmission protocol. In this embodiment, RTSP control signaling is carried over TCP to ensure reliability. Configuration information can be dynamically set by the application control layer according to external control instructions, or it can be pre-stored in the system memory. The RTSP client module calls the Socket API (Socket Application Programming Interface) provided by the operating system to create a TCP stream socket, sets the address family, port number, and IP address in the server address structure, and then calls the connection function to establish a TCP control connection with the RTSP server.

[0029] S3. Send OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence to complete the standard RTSP handshake process. Among them, the SETUP request carries the local RTP / RTCP port information. The Session identifier (used to identify an RTSP session and carried in all subsequent requests to maintain the session state) and the server's RTP and RTCP ports are parsed from the server response message. The local port allocation adopts the method of binding port 0. The actual allocated RTP port is obtained through getsockname, and the RTCP port is set to the RTP port number plus 1. like Figure 3 , Figure 4 As shown, the steps of sequentially sending OPTIONS, DESCRIBE, SETUP, and PLAY requests to complete the RTSP standard handshake process include: S31. The DESCRIBE request carries an Accept:application / sdp header. The response message body returned by the RTSP server contains SDP description information (Session Description Protocol, a text-based protocol used to describe the media type, transmission protocol, codec format, resolution, frame rate, and other parameters of a multimedia session). The SDP description information includes media type, transmission protocol, codec format, video resolution, and frame rate parameters. The Session field in the response header is parsed to obtain the session identifier string. If the Session field contains a semicolon and timeout parameters, only the part before the semicolon is extracted as the session identifier. S32. The SETUP request carries a Transport field (RTSP transport parameter header, used to negotiate the transport protocol and port). The format of the Transport field is Transport:RTP / AVP / UDP;unicast;client_port=<local RTP port>-<local RTCP port>. The SETUP response returned by the RTSP server carries a server_port=<server-side RTP port>-<server-side RTCP port> field. The server-side RTP port and the server-side RTCP port are parsed from the field. S33. The PLAY request carries the Session identifier and the Range:npt=0.000- field, indicating that playback starts from the beginning position of the media.

[0030] Among them, OPTIONS, DESCRIBE, SETUP, and PLAY are all RTSP request methods, used to query the list of methods supported by the server, obtain the SDP description information of the media session, negotiate the transport protocol and port and establish a media transport channel, and start media streaming playback, respectively.

[0031] Specifically, the stages of the RTSP standard handshake process are as follows: During the OPTIONS phase, the RTSP client module constructs an OPTIONS request message, with the request line format being OPTIONSrtsp: / / <server_ip> :<server_port> / <stream_name> In RTSP / 1.0, the message header contains an incrementing CSeq sequence number (Command Sequence, an incrementing sequence number in the RTSP protocol that identifies the correspondence between requests and responses) and a User-Agent identifier (used in the RTSP request header to identify the client type and version). The CSeq is an incrementing sequence number in the RTSP protocol used to identify the correspondence between requests and responses. The RTSP server returns a 200 OK response. The Public field in the response header (an RTSP response header that lists the methods supported by the server) lists the methods supported by the server, such as OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, etc. The RTSP client module parses this response to confirm the available subsequent operations. During the DESCRIBE phase, the RTSP client module constructs a DESCRIBE request message, including `Accept: application / sdp` in the request header, indicating its desire to receive SDP session description information. The response message body returned by the RTSP server contains the SDP description, which describes media session parameters such as media type, transport protocol, codec format, video resolution, and frame rate. The RTSP client module parses the Session field from the response header to obtain the session identifier string; if the Session field contains a semicolon and timeout parameters, only the portion before the semicolon is used as the session identifier. This session identifier will be used in subsequent SETUP, PLAY, and TEARDOWN requests to maintain the same RTSP session state. During the SETUP phase, the RTSP client module constructs a SETUP request message. The request header includes a Transport field in the format: Transport:RTP / AVP / UDP;unicast;client_port=<local RTP port>-<local RTCP port>, indicating that media data is transmitted using RTP over UDP unicast. Local port allocation uses a binding to port 0, obtaining the actual allocated RTP port via getsockname, and setting the RTCP port to the RTP port number plus 1. The response message returned by the RTSP server includes a Transport field, where server_port=<server-side RTP port>-<server-side RTCP port> specifies the port used by the server to send RTP data and receive RTCP data. The RTSP client module parses this field and saves the server-side RTP port to the server address structure so that the subsequent RTP receiving module can receive data from the correct port. Here, `getsockname` is a function in the Socket API used to obtain the local protocol address (i.e., local IP address and local port number) currently bound to the socket. In this embodiment, since the port number specified when binding using the `bind` function is 0 (randomly assigned by the operating system), the actual bound port number cannot be known before calling `bind`. Therefore, after calling `bind`, the RTSP client module calls the `getsockname` function to obtain the local port number actually assigned by the operating system and uses this port number as the local RTP port; simultaneously, it sets the local RTCP port to this RTP port number plus 1 to satisfy the negotiation requirements for the client's RTP / RTCP port pair in the Transport field of the RTSP SETUP request.

[0032] During the PLAY phase, the RTSP client module constructs a PLAY request message, with the request header carrying a Session field and a Range:npt=0.000- field, where npt (Normal Play Time) indicates the starting position for playback. After the RTSP server returns a 200 OK response, it begins sending RTP packets to the client through the server-side RTP port.

[0033] S4. Create a UDP socket and bind it to a random system port. Configure the receive buffer and receive timeout parameters for the socket. Complete the RTP receive channel setup and start an independent data receive thread. Step S4 includes: S41. Call the socket function to create a UDP datagram socket of type AF_INET (Address Family Internet, IPv4 address family, specifying the use of IPv4 protocol in the socket function) as a local RTP socket; S42. Call the setsockopt function to set the receive buffer size of the local UDP Socket to 2MB and the receive timeout to 1 second; S43. Call the bind function to bind the local UDP socket to the local address INADDR_ANY (a wildcard address, indicating binding to all available network interfaces on the local machine, rather than binding to a specific IP address) and port 0, and the operating system will allocate a random port as the local RTP port; S44. Call the getsockname function to obtain the actual bound local RTP port, and set the local RTCP port to the local RTP port plus 1.

[0034] Specifically, the RTSP client module calls `socket(AF_INET, SOCK_DGRAM, 0)` to create a UDP (User Datagram Protocol) datagram socket as a local RTP socket. UDP is a connectionless transport protocol; in this embodiment, RTP is carried over UDP to reduce transmission latency. To increase the receive buffer capacity to cope with network jitter, the `setsockopt` function is called to set the socket receive buffer size to 2MB. Simultaneously, to support stream timeout detection, the socket receive timeout is set to 1 second; that is, if no RTP packets are received within 1 second, the `recvfrom` call will return a timeout error. Subsequently, the socket is bound to the local address `INADDR_ANY` and port 0, with the operating system automatically allocating an available random port as the local RTP port. After successful binding, `getsockname` is called to obtain the actual bound local address and port number, and the local RTCP port is set to the local RTP port plus 1.

[0035] The `socket` function is the basic function in the Socket API used to create a socket descriptor. When calling this function, you need to specify the address family, socket type, and protocol type. The `setsockopt` function is a function in the Socket API used to set socket options and adjust the behavior parameters of the socket. The `bind` function is a function in the Socket API used to bind a socket to a local IP address and a local port number.

[0036] S5. Receive RTP data packets in a loop within the thread, parse the sequence number, marker bit and payload type in the packet header, process the payload according to the NALU type, and reassemble it to obtain a complete H.264 data unit with AnnexB start code. like Figure 5 , Figure 6 , Figure 7 As shown, specifically, an RTP packet consists of two parts: the RTP fixed header and the payload. The RTP fixed header contains at least 12 bytes, in the following order: Version (2 bits), Padding (1 bit), Extension (1 bit), CSRCCount (4 bits), Marker (1 bit), Payload Type (7 bits), Sequence Number (16 bits), Timestamp (32 bits), and SSRC (32 bits). Version is the version number field; the current RTP protocol version number is 2. Marker is a marker bit used to indicate the end of a data frame. Payload Type is the payload type, used to identify the media encoding format carried by the RTP payload. Sequence Number is the sequence number field used to identify the transmission order of RTP packets; it increments by 1 with each transmitted RTP packet and can be used by the receiver to detect packet loss and out-of-order delivery. Timestamp is the timestamp field used to identify the timestamp of the same RTP packet frame. SSRC (Synchronization Source Identifier) ​​is used to uniquely identify the source of the RTP data stream. CSRC (Contributing Source Identifier) ​​is used to identify the various original sources in the mixer's synthesized data. CSRC Count indicates the number of CSRC identifiers, and the Extension bit indicates whether an RTP extension header is present; if a CSRC or extension header is present, the actual load starting position needs to be offset accordingly. After stripping the RTP header, the resulting payload is either H.264 NALU data or H.264 fragment data. The H.264 NALU header is one byte long, with the first three bits being the forbid bits and reference level, and the last five bits indicating the NALU type. Common NALU types include 1 (non-IDR fragment, i.e., coded stripe of non-instantaneous decoding refresh frames), 5 (IDR fragment, i.e., coded stripe of instantaneous decoding refresh frames), 7 (SPS, Sequence Parameter Set), 8 (PPS, Picture Parameter Set), and 28 (FU-A fragment), etc. The SPS and PPS contain the key parameters required for H.264 decoding, while the IDR frame is the key frame for decoding.

[0037] The step of processing the payload according to the NALU type and reassembling it to obtain a complete H.264 data unit with AnnexB start code includes: S51. Determine whether the version number of the RTP data packet is 2; otherwise, discard the RTP data packet. Calculate the actual load start position based on the CSRC Count field (number of contributor source identifiers, indicating the length of the CSRC list) and the Extension flag (extension header flag, indicating whether there is an extension header after the RTP header) in the RTP header, and read the NALU type from the actual load start position. S52. If the NALU type is a single NALU packet, add AnnexB start codes (the bitstream format defined in Annex B of the H.264 standard, using start codes 0x00 0x00 0x00 0x01 or 0x00 0x00 0x01 to identify the start position of each NALU) to the H.264 payload data to obtain a complete H.264 data unit; if there are previously incomplete FU-A fragmented caches, clear the FU-A fragmented caches, set the cache length to zero, and set the completion flag; Specifically, when the received RTP packet payload is a single NALU packet (i.e., the NALU type is not equal to 28), the RTP parsing and H.264 reassembly module directly adds AnnexB start codes to the payload. The AnnexB start code is 4 bytes (0x00 0x000x00 0x01), a stream segmentation marker defined in the H.264 standard, used to identify the start position of a NALU. The data unit after adding the start code is a complete H.264 NALU, which can be directly sent to the H.264 decoding module. If, when a single NALU packet is received, there are still incomplete FU-A fragment data in the fragment buffer, the fragment buffer should be cleared, the buffer length set to zero, and the completion flag set to avoid incorrect concatenation of old fragment data with new single NALU data.

[0038] S53. If the NALU type is FU-A fragment, when a fragment with the Start flag set to 1 is received, a fragment buffer with a capacity of 256KB is initialized, the original NALU type in the FU Header is recorded, and the payload after the Start fragment is stored in the fragment buffer; when an intermediate fragment with both the Start and End flags set to 0 is received, the current payload is appended to the fragment buffer; when a fragment with the End flag set to 1 is received, the current payload is appended to the fragment buffer, and then the complete NALU header is restored based on the original NALU type and the first three bits of the original NALU header. The complete NALU header is combined with the data in the fragment buffer, and AnnexB start code is added to obtain the complete H.264 data unit; wherein, during the appending process of the fragment buffer, if the length of the appended data reaches or exceeds the maximum capacity of the fragment buffer, the fragment buffer is reset and the current fragment is discarded; S54. For FU-A fragmented packets, packet loss and out-of-order anomalies are detected based on the sequence number. Abnormal fragments are directly discarded and the corresponding cache is cleared.

[0039] Step S54 includes: S541. Maintain the previously received RTP sequence number and the sequence number validity flag; S542. If the sequence number validity flag is true, calculate the expected next RTP sequence number to be received. The expected next RTP sequence number to be received is the previous sequence number plus 1 and modulo 65536. S543. If the sequence number of the current RTP data packet is not equal to the expected next RTP sequence number, then packet loss or out-of-order delivery is determined, the current fragment buffer is discarded, the buffer length is set to zero, the completion flag is set, the NALU type is cleared, and the sequence number validity flag is cleared.

[0040] Specifically, when the received RTP packet payload is a FU-A fragment (i.e., NALU type equal to 28), the RTP parsing and H.264 reassembly module first performs fragment continuity detection. Specifically, the module maintains a variable for the previously received RTP sequence number and a sequence number validity flag. If the sequence number validity flag is true, it calculates the expected next RTP sequence number, which is the previous sequence number plus 1 modulo 65536. The RTP sequence number is a 16-bit unsigned integer, ranging from 0 to 65535, and restarts counting from 0 after overflow. If the sequence number of the current RTP packet is not equal to the expected sequence number, packet loss or out-of-order delivery is determined. In this case, the current fragment buffer is discarded, the buffer length is set to zero, the completion flag is set, the NALU type is cleared, the sequence number validity flag is cleared, and the module waits for the arrival of the next Start fragment.

[0041] If the current RTP packet sequence number matches the expected sequence number, the previous sequence number variable is updated, and the FU Header parsing continues. The FU Header is the second byte after the NALU header, where the highest bit is the Start flag, the second highest bit is the End flag, and the lower 5 bits are the original NALU type. Based on the different flags in the FU Header, fragmentation processing falls into the following three cases: In the first case, the Start flag is 1, indicating that the current fragment is the starting fragment of the same NALU. In this case, the RTP parsing and H.264 reassembly module first checks if the fragment cache has been allocated. If not, it initializes a fragment cache with a maximum capacity of 256KB; then, it sets the cache length to zero, clears the completion flag, sets the fragment sequence counter to 1, and records the original NALU type in the FUHeader; finally, it appends the payload data after the FU Header to the fragment cache. If the payload length is zero or exceeds the maximum cache capacity after appending, the cache is reset and the current fragment is discarded.

[0042] In the second case, both the Start and End flags are 0, indicating that the current fragment is an intermediate fragment of the same NALU. In this case, the module checks the completion flag of the fragment cache. If the completion flag is true, it means that the Start fragment has not yet been received or the previous fragment has been discarded, and it returns directly; otherwise, the fragment sequence counter is incremented, and the current load is appended to the fragment cache. Similarly, if the appended load exceeds the maximum cache capacity, the cache is reset.

[0043] In the third case, the End flag is set to 1, indicating that the current fragment is the end fragment of the same NALU. In this case, the module checks if the fragment buffer is in an incomplete state. If so, it increments the fragment sequence counter and appends the current payload to the fragment buffer. Then, the module reconstructs the complete NALU header based on the original NALU type and the first three bits of the original NALU header. It combines the complete NALU header with the data in the fragment buffer and adds the AnnexB start code to the beginning of the combined data to obtain a complete H.264 data unit. After outputting the complete H.264 data unit, the fragment buffer length is set to zero, and the completion flag is set to prepare for receiving the next NALU fragment.

[0044] S6. The reconstructed complete H.264 data unit is sent to the decoder to decode and obtain the original video frame. After converting it to RGB format, the R and B channel components are cleared, and only the Y luminance component is retained to generate a grayscale image. The grayscale image is vertically flipped to complete the orientation correction to adapt to the two up and down scanning output directions of the pixel headlight and generate the image to be displayed. like Figure 8 , Figure 9 As shown, specifically, step S6 includes: S61. Convert the decoded NV21 format raw video frames into RGB24 data; S62. Read the brightness value of the Y plane in the original video frame, and map the brightness value to the three channels of the grayscale image after black-level cropping. The black-level cropping is to subtract a preset offset of 20 from the brightness value and limit it to the range of 0 to 255. S63. Clear the R and B channels in the RGB24 data to zero, and retain only the luminance component as the pixel value of the grayscale image; S64. Perform a vertical flip operation on the grayscale image, swapping the pixel data of the top row with the bottom row, and the second-top row with the second-bottom row, until the entire image is flipped vertically to adapt to the scanning direction of the MicroLED pixel headlight from top to bottom or from bottom to top.

[0045] Specifically, the complete H.264 data unit output by the RTP parsing and H.264 reassembly module is received and sent to the AWStreamPlayer decoder (the specific decoder framework name used in this embodiment, whose internal implementation follows the standard H.264 decoding specification and supports asynchronous decoding and callback output) via the sendFrameToThread interface (an interface function provided by the AWStreamPlayer decoder for sending H.264 data units to the decoding thread for decoding). During initialization, the AWStreamPlayer decoder sets decoding parameters according to the physical resolution of the MicroLED pixel headlight, for example, setting the output width to 320, the output height to 80, and the codec type to CODEC_H264. After the decoder is created, an independent decoding thread is started to asynchronously decode the input H.264 data unit, and the original video frame is output through a registered callback function after decoding is complete. In this embodiment, the original video frame format output by the decoder is NV21, containing a Y luminance plane and an alternating VU chrominance plane.

[0046] The decoding callback function retrieves the raw video frame in NV21 format. First, it checks if the image width and height in the callback parameters match the preset resolution of the MicroLED pixel headlight; if not, the frame data is discarded. After confirming the resolution match, the raw NV21 video frame is converted into intermediate data in RGB24 format. Specifically, for each pixel position, the R, G, and B components are calculated according to the luminance value in the Y-plane and the chrominance value in the VU-plane, following the YUV to RGB conversion relationship.

[0047] Since the light output of MicroLED pixel headlights is primarily modulated based on luminance information, color information is not essential for automotive headlight projection applications. Therefore, the display adaptation module further performs a luminance domain conversion operation. It reads the luminance values ​​from the Y-plane of the original NV21 format video frame, performs black-level cropping on the luminance values, and maps them to the three channels of the grayscale image. Black-level cropping involves subtracting a preset offset of 20 from the luminance value and limiting it to the range of 0 to 255 to eliminate noise interference in low-brightness areas. Simultaneously, the R and B channels of the RGB24 data are cleared to zero, retaining only the luminance component as the pixel value of the grayscale image.

[0048] Meanwhile, since the scanning order of video frames in memory may differ from the scanning direction of the physical array of MicroLED pixel headlights, a direction correction operation needs to be performed on the grayscale image. The direction correction operation is a vertical flip, which involves swapping the pixel data of the top row with the bottom row, the second-top row with the second-bottom row, and so on, until the entire image is flipped vertically. The vertically flipped image data correctly corresponds to the scanning sequence of the MicroLED pixel headlights from top to bottom or bottom to top, ensuring that the direction of the projected light pattern is consistent with the video source content.

[0049] S7. Send the corrected image to be displayed to the pixel headlight driver layer, and the driver layer controls the brightness output of the corresponding LED beads according to the pixel grayscale value. like Figure 10 As shown, specifically, after direction correction is completed, grayscale image data with a resolution of 320×80 is pushed to the MicroLED pixel headlight driver module. This module communicates with the MicroLED driver chip through BMPStreamerPlayer (the specific name of the driver data push framework used in this embodiment, used to push the image data to be displayed to the MicroLED driver chip according to a predetermined timing and protocol format), pushing the image data to be displayed to the MicroLED driver chip according to a predetermined timing and protocol format. The brightness value of each pixel corresponds to the driving current or PWM (Pulse Width Modulation) duty cycle of one LED, and the driver chip controls the luminous intensity of each LED according to the driving command. Since each LED in the MicroLED pixel headlight array can be independently addressed and controlled, a high-resolution, high-contrast dynamic light pattern can be formed on the ground or target area in front of the vehicle.

[0050] S8. Record the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined to be a stream interruption. After a delay, close the receiving thread and send a TEARDOWN request (RTSP request method, used to terminate the session and release server-side resources) to the RTSP server, releasing TCP sockets, UDP sockets and fragmented buffer resources.

[0051] Specifically, step S8 includes: S81. After each successful reception of an RTP data packet, the system time function is called to obtain the current time, and the current time is updated to the last data reception time. S82. Periodically compare the time difference between the current system time and the last received data time; S83. If the time difference exceeds the preset timeout threshold of 2 seconds, then after a preset exit delay time of 2 seconds, set a global exit flag and terminate the RTP data receiving thread; during the operation of the RTP data receiving thread, if a stop playback command is received, then set the global exit flag. S84. After the global exit flag is set, a TEARDOWN request carrying the Session identifier is sent to the RTSP server, and the shutdown function (used to close the socket's send or receive channel; in this embodiment, it is used to terminate the bidirectional data transmission of the TCP socket) is called to terminate the bidirectional data transmission of the TCP control socket. Then, the close function is called to release the TCP control socket and the local UDP socket, and the fragment buffer is released.

[0052] Specifically, after the RTSP handshake is successful and the system enters the PLAY state, the RTP receiving module starts the RTP data receiving thread. This thread repeatedly calls the `recvfrom` function (used to receive datagrams from the UDP socket; in this embodiment, the RTP receiving thread repeatedly calls this function to receive RTP packets) to receive RTP packets from the local RTP socket. Each time an RTP packet is successfully received, the thread calls a system function to obtain the current time and updates it to the last received data time. The stream monitoring and resource reclamation module continuously monitors the time difference between the last received data time and the current system time. When this time difference exceeds a preset timeout threshold of 2 seconds, it is determined that the current RTSP video stream has been interrupted, possibly due to network failure, the RTSP server stopping streaming, or a wireless signal interruption.

[0053] Upon detecting a stream interruption, the stream monitoring and resource reclamation module delays the preset exit time by 2 seconds before setting the global exit flag and ending the RTP data receiving thread. The purpose of setting this delayed exit is to provide the system with a buffer period to receive RTP packets that may be delayed due to network jitter, while also giving upper-layer applications an opportunity to handle stream interruption events.

[0054] After the delay period ends, the global exit flag is set, and the RTP data receiving thread exits the receive loop and enters the resource reclamation phase. During the resource reclamation phase, the RTSP client module first sends a TEARDOWN request to the RTSP server. TEARDOWN is a request method in the RTSP protocol used to terminate a session and release server-side resources. The request header carries a Session field to identify the RTSP session that needs to be released. Upon receiving the TEARDOWN request, the RTSP server stops sending RTP packets to the client and releases the server-side resources associated with that session.

[0055] Then, the RTSP TCP control socket is closed. During closure, the `shutdown` function is first called to terminate bidirectional data transmission, followed by the `close` function to release the socket descriptor, ensuring that data in the buffer is correctly flushed and preventing the connection from being in a half-open state. Simultaneously, the local RTP UDP socket is closed by directly calling the `close` function (used to release the socket descriptor and close the socket) to release the socket descriptor and related resources. The flow monitoring and resource reclamation module further releases the H.264 fragment buffer, including releasing the dynamic memory occupied by the fragment buffer, setting the buffer length and maximum length to zero, and setting the completion flag.

[0056] During the execution of the RTP data receiving thread, various error situations may occur. For EINTR errors (receive operation interrupted by a signal), the thread continues to receive data in a loop; for EAGAIN or EWOULDBLOCK errors (receive buffer temporarily empty), the thread also continues to loop, while the stream monitoring and resource reclamation module determines whether a stream interruption has occurred based on the last received data time. For other fatal errors, the thread immediately sets an exit flag and enters the resource reclamation phase.

[0057] In addition, when the application control layer actively requests to stop playback, such as when the user switches the display mode or turns off the vehicle headlight projection function, the system calls the stop interface to set a global exit flag and closes the RTSP TCP control socket and RTP UDP socket. The RTP data receiving thread also enters the resource reclamation phase after detecting the exit flag.

[0058] The RTSP video stream data playback method provided in Embodiment 1 of this invention achieves real-time transmission of remote video streams to in-vehicle MicroLED pixel headlights through a complete RTSP / RTP standard protocol link. This eliminates the reliance on local static images and animation materials, significantly improving the flexibility of headlight display content updates and supporting dynamic updates of various display content such as welcome animations, ADB light carpets, and road projections. H.264 encoding is used to compress the transmitted video data, effectively reducing the bandwidth overhead of the in-vehicle Ethernet and adapting to application scenarios with limited in-vehicle network resources and shared bandwidth among multiple devices. To address data out-of-order and packet loss issues in weak in-vehicle network environments, a FU-A fragment reassembly and sequence number verification mechanism is designed to filter abnormal fragment data and prevent errors during playback. It eliminates issues such as pixelation, screen tearing, and stuttering, significantly improving the smoothness and robustness of video playback. Built-in grayscale conversion and vertical image flip correction logic allow the output image to directly match the hardware characteristics of headlights—monochrome illumination and fixed scanning direction—eliminating the need for additional hardware image processing chips and reducing hardware adaptation costs. A stream timeout detection and standardized resource reclamation process are also included, proactively releasing sockets, caches, and other resources during network interruptions to prevent headlight image freezing, screen tearing, and other abnormal display issues, thus preventing abnormal images from interfering with driver visibility and ensuring nighttime driving safety. Furthermore, unified management of sockets, caches, decoders, and other resources allows for complete resource reclamation upon video stream termination or interruption, preventing memory leaks and port occupancy issues during long-term operation of embedded devices.

[0059] Example 2 like Figure 11 As shown, in Embodiment 2 of the present invention, an RTSP video stream data playback system is provided, the system comprising: Initialization configuration module 1 is used to select the target RTSP video source according to external SOME / IP control instructions or user configuration, complete the initialization operation of MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of pixel headlight; Socket creation module 2 is used to create TCP control sockets based on the preset RTSP server address, port and stream name, and complete the establishment of control connection with the RTSP server; RTSP client module 3 is used to send OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence to complete the standard RTSP handshake process. Among them, the SETUP request carries local RTP / RTCP port information and parses the Session identifier and server RTP and RTCP ports from the server response message. The local port allocation adopts the method of binding port 0, and the actual allocated RTP port is obtained through getsockname. The RTCP port is set to the RTP port number plus 1. RTP receiving channel module 4 is used to create UDP sockets and bind them to random system ports, configure the receiving buffer and receiving timeout parameters for the sockets, complete the RTP receiving channel setup, and start an independent data receiving thread. The frame reassembly and parsing module 5 is used to receive RTP data packets in a loop within a thread, parse the sequence number, marker bit and payload type in the packet header, process the payload according to the NALU type, and reassemble it to obtain a complete H.264 data unit with AnnexB start code. The display adaptation and correction module 6 is used to send the reconstructed complete H.264 data unit into the decoder to decode the original video frame, convert it to RGB format, clear the R and B channel components, retain only the Y luminance component to generate a grayscale image, perform vertical flipping on the grayscale image to complete the orientation correction, so as to adapt to the two up and down scanning output directions of the pixel headlight and generate the image to be displayed. The pixel headlight driver module 7 is used to send the corrected image to be displayed to the pixel headlight driver layer, and the driver layer controls the brightness output of the corresponding lamp beads according to the pixel grayscale value. The stream monitoring resource reclamation module 8 is used to record the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined that the stream is interrupted. After a delay, the receiving thread is closed, and a TEARDOWN request is sent to the RTSP server to release TCP sockets, UDP sockets and fragmented buffer resources.

[0060] The RTSP client module 3 is used for: The DESCRIBE request carries an Accept:application / sdp header. The response message body returned by the RTSP server contains SDP description information, which includes media type, transmission protocol, codec format, video resolution, and frame rate parameters. The Session field in the response header is parsed to obtain the session identifier string. If the Session field contains a semicolon and timeout parameters, only the part before the semicolon is taken as the session identifier. The SETUP request carries a Transport field, the format of which is Transport:RTP / AVP / UDP;unicast;client_port=<local RTP port>-<local RTCP port>; the SETUP response returned by the RTSP server carries a server_port=<server-side RTP port>-<server-side RTCP port> field, from which the server-side RTP port and the server-side RTCP port are parsed. The PLAY request carries the Session identifier and the Range:npt=0.000- field, indicating that playback starts from the beginning of the media.

[0061] The RTP receiving channel module 4 is used for: Call the socket function to create a UDP datagram socket of type AF_INET as a local RTP socket; The setsockopt function is called to set the receive buffer size of the local UDP socket to 2MB and the receive timeout to 1 second; The bind function is called to bind the local UDP socket to the local address INADDR_ANY and port 0, and the operating system allocates a random port as the local RTP port; Call the getsockname function to obtain the actual bound local RTP port, and set the local RTCP port to the local RTP port plus 1.

[0062] The frame reassembly and parsing module 5 is used for: Determine if the version number of the RTP data packet is 2; otherwise, discard the RTP data packet. Calculate the actual load start position based on the CSRC Count field and Extension flag in the RTP header, and read the NALU type from the actual load start position. If the NALU type is a single NALU packet, then AnnexB start codes are added to the H.264 payload data to obtain a complete H.264 data unit; if there are previously incomplete FU-A fragment caches, then the FU-A fragment caches are cleared, the cache length is set to zero, and the completion flag is set. If the NALU type is FU-A fragment, upon receiving a fragment with the Start flag set to 1, a fragment buffer with a capacity of 256KB is initialized, the original NALU type in the FU Header is recorded, and the payload after the Start fragment is stored in the fragment buffer. Upon receiving an intermediate fragment with both the Start and End flags set to 0, the current payload is appended to the fragment buffer. Upon receiving a fragment with the End flag set to 1, the current payload is appended to the fragment buffer, and then the complete NALU header is recovered based on the original NALU type and the first three bits of the original NALU header. The complete NALU header is combined with the data in the fragment buffer, and an AnnexB start code is added to obtain the complete H.264 data unit. During the appending process in the fragment buffer, if the length of the appended data reaches or exceeds the maximum capacity of the fragment buffer, the fragment buffer is reset and the current fragment is discarded. For FU-A fragmented packets, packet loss and out-of-order anomalies are detected based on sequence numbers. Abnormal fragments are directly discarded and the corresponding cache is cleared.

[0063] The frame reassembly and parsing module 5 is further configured to: Maintain the previously received RTP sequence number and the sequence number validity flag; If the sequence number validity flag is true, then calculate the expected next RTP sequence number to be received. The expected next RTP sequence number to be received is the previous sequence number plus 1 and modulo 65536. If the sequence number of the current RTP packet is not equal to the expected next RTP sequence number, then packet loss or out-of-order delivery is determined, the current fragment buffer is discarded, the buffer length is set to zero, the completion flag is set, the NALU type is cleared, and the sequence number validity flag is cleared.

[0064] The display adaptation correction module 6 is used for: Convert the decoded NV21 format raw video frames into RGB24 data; Read the brightness value of the Y plane in the original video frame, and map the brightness value to the three channels of the grayscale image after black bit cropping. The black bit cropping is to subtract a preset offset of 20 from the brightness value and limit it to the range of 0 to 255. The R and B channels in the RGB24 data are cleared to zero, and only the luminance component is retained as the pixel value of the grayscale image; A vertical flip operation is performed on the grayscale image, swapping the pixel data of the top row with the bottom row, and the second-top row with the second-bottom row, until the entire image is flipped vertically to adapt to the scanning direction of the MicroLED pixel headlight from top to bottom or from bottom to top.

[0065] The flow monitoring resource reclamation module 8 is used for: After each successful reception of an RTP data packet, the system time function is called to obtain the current time, and the current time is updated to the last time the data was received. Periodically compare the time difference between the current system time and the last received data time; If the time difference exceeds the preset timeout threshold of 2 seconds, then after a preset exit delay of 2 seconds, a global exit flag is set to terminate the RTP data receiving thread; during the operation of the RTP data receiving thread, if a stop playback command is received, then the global exit flag is set. After the global exit flag is set, a TEARDOWN request carrying the Session identifier is sent to the RTSP server, the shutdown function is called to terminate the bidirectional data transmission of the TCP control socket, and the close function is called to release the TCP control socket and the local UDP socket, and the fragment buffer is released.

[0066] In other embodiments of the present invention, the present invention provides the following technical solution: a computer, including a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101, wherein the processor 101 executes the computer program to implement the RTSP video stream data playback method as described above.

[0067] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0068] The memory 102 may include a large-capacity memory for data or instructions. For example, and not limitingly, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include removable or non-removable (or fixed) media. Where appropriate, the memory 102 may be internal or external to a data processing device. In a particular embodiment, the memory 102 is non-volatile memory. In a particular embodiment, the memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0069] The memory 102 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101.

[0070] The processor 101 implements the above-described RTSP video stream data playback method by reading and executing computer program instructions stored in the memory 102.

[0071] In some embodiments, the computer may further include a communication interface 103 and a bus 100. For example, Figure 12 As shown, the processor 101, memory 102, and communication interface 103 are connected through bus 100 and communicate with each other.

[0072] The communication interface 103 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 103 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0073] Bus 100 includes hardware, software, or both, that couples components of a computer device together. Bus 100 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although specific buses are described and illustrated in the embodiments of the present invention, the present invention is contemplated by any suitable bus or interconnect.

[0074] The computer can execute the RTSP video stream data playback method of the present invention based on the RTSP video stream data playback system, thereby realizing RTSP video stream data playback.

[0075] In some further embodiments of the present invention, in conjunction with the above-described RTSP video stream data playback method, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described RTSP video stream data playback method.

[0076] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0077] More specific examples of readable media (a non-exhaustive list) include: electrical connections (electronic devices) with one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0078] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for playing RTSP video stream data, characterized in that, include: Select the target RTSP video source according to the external SOME / IP control command or user configuration, complete the initialization operation of the MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of the pixel headlight; Create a TCP control socket based on the preset RTSP server address, port and stream name to complete the establishment of a control connection with the RTSP server; The RTSP standard handshake process is completed by sending OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence. Among them, the SETUP request carries the local RTP / RTCP port information and parses the Session identifier and the server's RTP and RTCP ports from the server response message. The local port allocation adopts the method of binding port 0, and the actual allocated RTP port is obtained through getsockname. The RTCP port is set to the RTP port number plus 1. Create a UDP socket and bind it to a random system port. Configure the receive buffer and receive timeout parameters for the socket. Complete the RTP receive channel setup and start an independent data receive thread. The thread continuously receives RTP data packets, parses the sequence number, marker bit and payload type in the packet header, processes the payload according to the NALU type, and reassembles it to obtain a complete H.264 data unit with AnnexB start code. The reconstructed H.264 data unit is sent to the decoder to obtain the original video frame. After being converted to RGB format, the R and B channel components are cleared, and only the Y luminance component is retained to generate a grayscale image. The grayscale image is vertically flipped to complete the orientation correction to adapt to the two up and down scanning output directions of the pixel headlight, and the image to be displayed is generated. The corrected image to be displayed is sent to the pixel headlight driver layer, which then controls the brightness output of the corresponding LED based on the pixel grayscale value. The system records the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined to be a stream interruption. After a delay, the receiving thread is closed, a TEARDOWN request is sent to the RTSP server, and TCP sockets, UDP sockets, and fragment buffer resources are released.

2. The RTSP video stream data playback method according to claim 1, characterized in that, The steps for completing the RTSP standard handshake process by sequentially sending OPTIONS, DESCRIBE, SETUP, and PLAY requests include: The DESCRIBE request carries an Accept:application / sdp header. The response message body returned by the RTSP server contains SDP description information, which includes media type, transmission protocol, codec format, video resolution, and frame rate parameters. The Session field in the response header is parsed to obtain the session identifier string. If the Session field contains a semicolon and timeout parameters, only the part before the semicolon is taken as the session identifier. The SETUP request carries a Transport field, the format of which is Transport:RTP / AVP / UDP;unicast;client_port=<local RTP port>-<local RTCP port>; the SETUP response returned by the RTSP server carries a server_port=<server-side RTP port>-<server-side RTCP port> field, from which the server-side RTP port and the server-side RTCP port are parsed. The PLAY request carries the Session identifier and the Range:npt=0.000- field, indicating that playback starts from the beginning of the media.

3. The RTSP video stream data playback method according to claim 1, characterized in that, The steps of creating a UDP socket and binding it to a random system port, and configuring the receive buffer and receive timeout parameters for the socket include: Call the socket function to create a UDP datagram socket of type AF_INET as a local RTP socket; The setsockopt function is called to set the receive buffer size of the local UDP socket to 2MB and the receive timeout to 1 second; The bind function is called to bind the local UDP socket to the local address INADDR_ANY and port 0, and the operating system allocates a random port as the local RTP port; Call the getsockname function to obtain the actual bound local RTP port, and set the local RTCP port to the local RTP port plus 1.

4. The RTSP video stream data playback method according to claim 1, characterized in that, The step of processing the payload according to the NALU type and reassembling it to obtain a complete H.264 data unit with AnnexB start code includes: Determine if the version number of the RTP data packet is 2; otherwise, discard the RTP data packet. Calculate the actual load start position based on the CSRC Count field and Extension flag in the RTP header, and read the NALU type from the actual load start position. If the NALU type is a single NALU packet, then AnnexB start codes are added to the H.264 payload data to obtain a complete H.264 data unit; if there are previously incomplete FU-A fragment caches, then the FU-A fragment caches are cleared, the cache length is set to zero, and the completion flag is set. If the NALU type is FU-A fragment, upon receiving a fragment with the Start flag set to 1, a fragment buffer with a capacity of 256KB is initialized, the original NALU type in the FU Header is recorded, and the payload after the Start fragment is stored in the fragment buffer. Upon receiving an intermediate fragment with both the Start and End flags set to 0, the current payload is appended to the fragment buffer. Upon receiving a fragment with the End flag set to 1, the current payload is appended to the fragment buffer, and then the complete NALU header is recovered based on the original NALU type and the first three bits of the original NALU header. The complete NALU header is combined with the data in the fragment buffer, and an AnnexB start code is added to obtain the complete H.264 data unit. During the appending process in the fragment buffer, if the length of the appended data reaches or exceeds the maximum capacity of the fragment buffer, the fragment buffer is reset and the current fragment is discarded. For FU-A fragmented packets, packet loss and out-of-order anomalies are detected based on sequence numbers. Abnormal fragments are directly discarded and the corresponding cache is cleared.

5. The RTSP video stream data playback method according to claim 4, characterized in that, The steps of detecting packet loss and out-of-order anomalies based on sequence numbers, directly discarding abnormal fragments and clearing the corresponding cache include: Maintain the previously received RTP sequence number and the sequence number validity flag; If the sequence number validity flag is true, then calculate the expected next RTP sequence number to be received. The expected next RTP sequence number to be received is the previous sequence number plus 1 and modulo 65536. If the sequence number of the current RTP packet is not equal to the expected next RTP sequence number, then packet loss or out-of-order delivery is determined, the current fragment buffer is discarded, the buffer length is set to zero, the completion flag is set, the NALU type is cleared, and the sequence number validity flag is cleared.

6. The RTSP video stream data playback method according to claim 1, characterized in that, The steps of converting to RGB format, clearing the R and B channel components, retaining only the Y luminance component to generate a grayscale image, and performing vertical flipping on the grayscale image to complete orientation correction include: Convert the decoded NV21 format raw video frames into RGB24 data; Read the brightness value of the Y plane in the original video frame, and map the brightness value to the three channels of the grayscale image after black bit cropping. The black bit cropping is to subtract a preset offset of 20 from the brightness value and limit it to the range of 0 to 255. The R and B channels in the RGB24 data are cleared to zero, and only the luminance component is retained as the pixel value of the grayscale image; A vertical flip operation is performed on the grayscale image, swapping the pixel data of the top row with the bottom row, and the second-top row with the second-bottom row, until the entire image is flipped vertically to adapt to the scanning direction of the MicroLED pixel headlight from top to bottom or from bottom to top.

7. The RTSP video stream data playback method according to claim 1, characterized in that, The steps of recording the real-time data packet reception timestamp, determining a stream interruption if no data is received beyond a preset timeout threshold, closing the receiving thread after a delay, sending a TEARDOWN request to the RTSP server, and releasing TCP sockets, UDP sockets, and fragmented buffer resources include: After each successful reception of an RTP data packet, the system time function is called to obtain the current time, and the current time is updated to the last time the data was received. Periodically compare the time difference between the current system time and the last received data time; If the time difference exceeds the preset timeout threshold of 2 seconds, then after a preset exit delay of 2 seconds, a global exit flag is set to terminate the RTP data receiving thread; during the operation of the RTP data receiving thread, if a stop playback command is received, then the global exit flag is set. After the global exit flag is set, a TEARDOWN request carrying the Session identifier is sent to the RTSP server, the shutdown function is called to terminate the bidirectional data transmission of the TCP control socket, and the close function is called to release the TCP control socket and the local UDP socket, and the fragment buffer is released.

8. An RTSP video stream data playback system, characterized in that, The system includes: The initialization configuration module is used to select the target RTSP video source according to external SOME / IP control instructions or user configuration, complete the initialization operation of the MicroLED pixel headlight driver layer and H.264 decoder, and match the decoder output resolution to the physical resolution of the pixel headlight. The socket creation module is used to create TCP control sockets based on the preset RTSP server address, port and stream name, and complete the establishment of a control connection with the RTSP server. The RTSP client module is used to send OPTIONS, DESCRIBE, SETUP, and PLAY requests in sequence to complete the standard RTSP handshake process. Among them, the SETUP request carries local RTP / RTCP port information and parses the Session identifier and server RTP and RTCP ports from the server response message. The local port allocation adopts the method of binding to port 0, and the actual allocated RTP port is obtained through getsockname. The RTCP port is set to the RTP port number plus 1. The RTP receive channel module is used to create UDP sockets and bind them to random system ports, configure receive buffers and receive timeout parameters for the sockets, complete the RTP receive channel setup, and start an independent data receive thread. The frame reassembly and parsing module is used to receive RTP data packets in a loop within a thread, parse the sequence number, marker bit and payload type in the packet header, process the payload according to the NALU type, and reassemble it to obtain a complete H.264 data unit with AnnexB start code. The display adaptation and correction module is used to send the reconstructed complete H.264 data unit into the decoder to decode it into the original video frame. After converting it into RGB format, the R and B channel components are cleared, and only the Y luminance component is retained to generate a grayscale image. The grayscale image is vertically flipped to complete the orientation correction in order to adapt to the two up and down scanning output directions of the pixel headlight and generate the image to be displayed. The pixel headlight driver module is used to send the calibrated image to be displayed to the pixel headlight driver layer, which then controls the brightness output of the corresponding LED based on the pixel grayscale value. The stream monitoring and resource reclamation module is used to record the data packet reception timestamp in real time. If no data is received within the preset timeout threshold, it is determined that the stream is interrupted. After a delay, the receiving thread is closed, and a TEARDOWN request is sent to the RTSP server to release TCP sockets, UDP sockets and fragmented buffer resources.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the RTSP video stream data playback method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the RTSP video stream data playback method as described in any one of claims 1 to 7.

Images