A method for real-time video playing under QT environment

Abstract

The application discloses a real-time video playing method in QT environment, which comprises the following steps: S1, creating a compiling environment; S2, making a transmission service rule; S3, initializing ffmpeg stream pulling and decoding resources; S4, creating a stream pulling main thread; S5, decoding a data packet object obtained by stream pulling; S6, converting a decoded image frame; S7, obtaining the converted image frame and displaying in real time; and S8, releasing decoding channel resources. The application adopts a soft decoding method, effectively reduces decoding time, simultaneously adopts an RTSP video transmission protocol to obtain a video stream, effectively reduces playing delay, and guarantees video playing fluency.

Description

Technical Field

[0001] This invention belongs to the field of video decoding technology in cross-platform systems. In application scenarios where the system is uncertain, it designs and implements a real-time video playback method compatible with multiple system types in a QT environment, taking into account the decoding characteristics of FFmpeg and combining it with a QT module to design an image display scheme. The invention addresses situations where the system has multiple uncertain types by first using the FFmpeg software decoding library to pull the network stream to RTSP, effectively acquiring the video stream. Simultaneously, it sets settings based on the characteristics of RTSP streams, unifying numbering, and optimizing transmission services. Then, it decodes the stream based on its encoding characteristics and displays the decoded image, thus completing the decoding, playback, and display process. Background Technology

[0002] Qt is a cross-platform graphical user interface application development framework for building high-performance desktop, mobile, and web applications. It can also be used to develop non-GUI programs, such as console tools and servers. Qt is an object-oriented framework that, using special code generation extensions and macros, is easily extensible and allows for true component programming.

[0003] Ffmpeg is an open-source computer program used to record, convert, and stream digital audio and video. It provides a complete solution for recording, converting, and streaming audio and video. It includes highly advanced audio / video codec libraries.

[0004] Real-Time Streaming Protocol (RTSP) is an application layer protocol in the TCP / IP protocol suite. This protocol defines how one-to-many applications can efficiently transmit multimedia data over an IP network.

[0005] In projects with unclear requirements, choosing a cross-platform tool for development can save development costs. Combining Qt and ffmpeg solves multi-platform compatibility issues, facilitates easier module porting, and utilizes the RTSP video transmission protocol to address real-time video playback across platforms. Furthermore, considering the characteristics of maritime transmission, video streams are optimized and categorized to effectively utilize bandwidth for transmission service optimization. Summary of the Invention

[0006] (I) Purpose of the Invention

[0007] The purpose of this invention is to provide a real-time video playback method in a QT environment based on a decoding method combining QT and ffmpeg, thereby solving the problem of real-time video decoding and playback across multiple systems.

[0008] (II) Technical Solution

[0009] To solve the above-mentioned technical problems, the present invention provides a

[0010] (III) Beneficial Effects

[0011] The real-time video playback method provided by the above technical solution in the QT environment can solve the video decoding and transmission problems under multiple systems. This method has been algorithmically verified and experimentally tested. The results show that this solution can effectively reduce decoding time by using a software decoding method to solve the image decoding problem, and by using the RTSP video transmission protocol to obtain the video stream, it effectively reduces playback latency and ensures smooth video playback. Detailed Implementation

[0012] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to examples.

[0013] The real-time video playback method in the QT environment of this embodiment includes the following steps:

[0014] S1: Creating the compilation environment

[0015] S11: Obtain the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, and libswscale.lib from ffmpeg (the file extensions of dynamic library files vary depending on the system, so adjust accordingly) and copy them to the working directory lib.

[0016] S12: Copy the ffmpeg include header file to the include directory of the project;

[0017] S13: Add the environment variable FFMPEGLIBPATH to the system and set this environment variable to the lib directory under the project directory.

[0018] S14: Add an external library to the Qt project file. Add the library file from step 1, using the path indicated in step 3.

[0019] S2: Formulate transmission service rules

[0020] To address the video transmission issues between maritime communication devices, transmission service rules were established to effectively improve transmission efficiency. The specific steps are as follows:

[0021] S21: The streaming application registers with the communication system;

[0022] S22: The streaming application compresses the raw video;

[0023] S23: The streaming application uses ffmpeg for RTSP streaming;

[0024] S24: The streaming application sets the TOS field in the RTSP stream message;

[0025] S25: The communication system prioritizes the transmission of RTSP streams received from the network according to the level of the TOS field, ensuring the transmission of high-priority tasks.

[0026] S3: Initialize ffmpeg streaming and decoding resources

[0027] Create a new thread class `decode` in the QT project directory. This class inherits from `QThread`. The specific steps for implementing the streaming initialization resource `InitVideo` in this class are as follows:

[0028] S301: Use the library function avformat_network_init to initialize the network;

[0029] S302: Use the library function avformat_alloc_context to create a streaming context object decoder_avfc;

[0030] S303: Check if the context object decoder_avfc is empty. If it is empty, exit the process; otherwise, proceed to the next step.

[0031] S304: Use the library function av_log_set_level to enable logging functionality;

[0032] S305: Use the library function av_dict_set to set the avdic parameter of rstp;

[0033] S306: Use the library function avformat_open_input to open the preset RTSP streaming URL;

[0034] S307: Determine whether the result of step S306 is successful. If it is unsuccessful, record the error message and exit the process; otherwise, proceed to the next step.

[0035] S308: Check if the parameter avdic is empty. If not, use the library function av_dict_free to release the object's resources; otherwise, proceed to the next step.

[0036] S309: Use the library function avformat_find_stream_info to obtain the video stream. If the acquisition fails, use the library function avformat_close_input to close the context object decoder_avfc and release resources. Otherwise, proceed to the next step.

[0037] The specific steps for implementing decoding and initializing resources in this class are as follows:

[0038] S311: Use the library function av_packet_alloc to create a packet object pkt. If creation is successful, proceed to the next step; otherwise, exit the process.

[0039] S312: Use the library function avcodec_find_decoder to create a decoder codec. If creation is successful, proceed to the next step; otherwise, exit the process.

[0040] S313: Use the library function av_parser_init to create a packet analysis object parser. If creation is successful, proceed to the next step; otherwise, exit the process.

[0041] S314: Use the library function avcodec_alloc_context3 to create a decoding environment context object. If creation is successful, proceed to the next step; otherwise, exit the process.

[0042] S315: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process; otherwise, proceed to the next step.

[0043] S316: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process; otherwise, proceed to the next step.

[0044] S4: Create the main thread for pulling the stream

[0045] S41: Perform main loop for current pulling;

[0046] S42: Check if the parameter setting flag is successful. If successful, proceed to the next step; otherwise, delay for 10µs.

[0047] S43: Determine whether the InitVideo stream initialization step in process 2 was successful. If it was, exit the connection establishment loop and continue to the next step; otherwise, continue the connection operation.

[0048] S44: Enter the video stream acquisition loop;

[0049] S45: Create a streaming data packet;

[0050] S46: Initialize packet using the library function av_init_packet;

[0051] S47: Set the packet's data pointer to a null pointer;

[0052] S48: Set the packet's data size to 0;

[0053] S49: Check if the streaming context decode_avfc is empty. If it is, return failure and exit the process; otherwise, proceed to the next step.

[0054] S410: Use the library function av_read_frame to retrieve the streaming video data packets opened by decode_avfc. If opening fails, release the packet object; otherwise, store the data in the packet and proceed to the next step.

[0055] S411: Determine if the stream index number of the packet object is the macro-defined video stream vdieo_Stream. If it is, proceed to the next decoding process and release the packet object; otherwise, release the packet object directly.

[0056] S5: Decode the data packet object obtained from the pull stream.

[0057] The specific steps are as follows:

[0058] S51: Use the library function avcode_send_packet to send the video stream data packet object (packet) obtained in process 3 to the decoder. If the sending fails, report an error and exit the process; otherwise, proceed to the next step.

[0059] S52: Enter the loop to acquire decoded image frames;

[0060] S53: Determine whether the result ret of the acquired image frame is greater than or equal to zero. If it is, proceed to the next step; otherwise, exit the loop process of acquiring image frames.

[0061] S54: Use the library function avcodec_receive_frame to get the video frame where the context object dec_ctx is located. If the acquisition is successful, it is stored in the image frame object frame and the next step is performed. If it fails, an error message is printed and the process is exited.

[0062] S55: Use the system function fflush to force the output of the decoder's image frames;

[0063] S6: Convert the format of the decoded image frames.

[0064] The decoded image is in YUV format, which cannot be directly displayed in the QT interface. It needs to be converted to RGB format. The specific steps are as follows:

[0065] S61: Use the library function av_frame_alloc to create an RGB image frame (rgbFrame);

[0066] S62: Use the library function avpicture_get_size to calculate the data size numBytes required to convert to rgb24 format;

[0067] S63: Use the library function av_malloc to dynamically allocate an array rgbBuffer to store RGB image data. The size is numBytes*sizeof(uint8_t) (that is, one byte multiplied by the required data size, converted to bits).

[0068] S64: Use the library function avpicture_fill to index the array of RGB image frames rgbFrame into rgbBuffer;

[0069] S65: Check if the conversion context object image_convert_ctx is empty. If not, use the library function sws_freeContext to release the object; otherwise, proceed to the next step.

[0070] S66: Use the library function sws_getContext to create a conversion context pair image_convert_ctx;

[0071] S67: Determine if the conversion context pair image_convert_ctx is empty. If it is, exit the process; otherwise, proceed to the next step.

[0072] S68: Use the library function sws_scale to convert the image frame and store the data in rgbFrame. Store the conversion result in result.

[0073] S69: Determine if the conversion result is successful. If it is, proceed to the next step; otherwise, exit the process.

[0074] S610: Sends the converted image frame rgbBuffer to the interface;

[0075] S7: Acquire and display the converted image frames in real time.

[0076] The specific steps are as follows:

[0077] S71: Check if the image frame object pointer is null. If it is, exit the process; otherwise, proceed to the next step.

[0078] S72: Create an image display object, which is of type QImage;

[0079] S73: Create a pixel map object pix, which is of type QPixmap;

[0080] S74: Get the pointer to the display image object of the QLabel class, image_display;

[0081] S75: Use the system function setPixmap to set the image object pointer image_display to display the image. The parameter is the return object of the pix function executed fromImage.

[0082] S8: Release decoding channel resources

[0083] The specific steps are as follows:

[0084] S81: Use the library function av_parser_close to close the packet analysis object parser;

[0085] S82: Use the library function avcodec_free_context to release the decoding context object;

[0086] S83: Use the library function avformat_close_input to release the streaming context object decoder_avfc;

[0087] S84: Use the library function av_packet_free to release the packet object;

[0088] S85: Use the library function av_frame_free to release the image frame object;

[0089] S86: Use the library function av_free to release the converted RGB array rgbBuffer.

[0090] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for real-time video playback in a QT environment, characterized in that, include: S1: Create the compilation environment; S2: Formulate transmission service rules; S3: Initialize ffmpeg streaming and decoding resources; S4: Create the main thread for pulling the stream; S5: Decode the data packet object obtained from the streaming process; S6: Convert the format of the decoded image frames; S7: Acquire the converted image frames and display them in real time; S8: Release the decoding channel resources; Step S1 includes the following sub-steps: S11: Obtain the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, and libswscale.lib from ffmpeg and copy them to the working directory lib. S12: Copy the ffmpeg include header file to the include directory of the project; S13: Add the environment variable FFMPEGLIBPATH to the system and set this environment variable to the lib directory under the project directory; S14: Add an external library to the Qt project file. Add the library file from step S11, using the path indicated in step S13. Step S2 includes the following processes: S21: The streaming application registers with the communication system; S22: The streaming application compresses the raw video; S23: The streaming application uses ffmpeg for RTSP streaming; S24: The streaming application sets the TOS field in the RTSP stream message; S25: The communication system prioritizes the transmission of RTSP streams received from the network according to the level of the TOS field, ensuring the transmission of high-priority tasks. In step S3, a new thread class named decode is created in the QT project directory. This class inherits from QThread. In step S3, the steps for implementing the streaming initialization resource InitVideo in the thread class decode are as follows: S301: Use the library function avformat_network_init to initialize the network; S302: Use the library function avformat_alloc_context to create a streaming context object decoder_avfc; S303: Check if the context object decoder_avfc is empty. If it is empty, exit the process; otherwise, proceed to the next step. S304: Use the library function av_log_set_level to enable logging functionality; S305: Use the library function av_dict_set to set the avdic parameter of rstp; S306: Use the library function avformat_open_input to open the preset RTSP streaming URL; S307: Determine whether the result of step S306 is successful. If it is unsuccessful, record the error message and exit the process; otherwise, proceed to the next step. S308: Check if the parameter avdic is empty. If not, use the library function av_dict_free to release the object's resources; otherwise, proceed to the next step. S309: Use the library function avformat_find_stream_info to obtain the video stream. If the acquisition fails, use the library function avformat_close_input to close the context object decoder_avfc and release resources. Otherwise, proceed to the next step. In step S3, the steps for initializing decoding resources in the thread class decode are as follows: S311: Use the library function av_packet_alloc to create a packet object pkt. If creation is successful, proceed to the next step; otherwise, exit the process. S312: Use the library function avcodec_find_decoder to create a decoder codec. If creation is successful, proceed to the next step; otherwise, exit the process. S313: Use the library function av_parser_init to create a packet analysis object parser. If creation is successful, proceed to the next step; otherwise, exit the process. S314: Use the library function avcodec_alloc_context3 to create a decoding environment context object. If creation is successful, proceed to the next step; otherwise, exit the process. S315: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process; otherwise, proceed to the next step. S316: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process; otherwise, proceed to the next step. Step S4 includes the following processes: S41: Perform main loop for current pulling; S42: Check if the parameter setting flag is successful. If successful, proceed to the next step; otherwise, delay for 10µs. S43: Determine whether the streaming initialization step InitVideo in process S42 was successful. If it was, exit the connection establishment loop and continue to the next step; otherwise, continue the connection operation. S44: Enter the video stream acquisition loop; S45: Create a streaming data packet; S46: Initialize packet using the library function av_init_packet; S47: Set the packet's data pointer to a null pointer; S48: Set the packet's data size to 0; S49: Check if the streaming context decode_avfc is empty. If it is, return failure and exit the process; otherwise, proceed to the next step. S410: Use the library function av_read_frame to retrieve the streaming video data packets opened by decode_avfc. If opening fails, release the packet object; otherwise, store the data in the packet and proceed to the next step. S411: Determine if the stream index of the packet object is the macro-defined video stream vdieo_Stream. If it is, proceed to the next decoding process and release the packet object; otherwise, release the packet object directly.

2. The real-time video playback method in a QT environment as described in claim 1, characterized in that, Step S5 Includes the following processes: S51: Use the library function avcode_send_packet to send the video stream data packet object (packet) obtained in process S3 to the decoder. If the sending fails, an error is reported and the process is exited; otherwise, proceed to the next step. S52: Enter the loop to acquire decoded image frames; S53: Determine whether the result ret of the acquired image frame is greater than or equal to zero. If it is, proceed to the next step; otherwise, exit the loop process of acquiring image frames. S54: Use the library function avcodec_receive_frame to get the video frame where the context object dec_ctx is located. If the acquisition is successful, it is stored in the image frame object frame and the next step is performed. If it fails, an error message is printed and the process is exited. S55: Use the system function fflush to force the decoder to output image frames.

3. The real-time video playback method in a QT environment as described in claim 2, characterized in that, Step S6 Includes the following processes: S61: Use the library function av_frame_alloc to create an RGB image frame (rgbFrame); S62: Use the library function avpicture_get_size to calculate the data size numBytes required to convert to rgb24 format; S63: Use the library function av_malloc to dynamically allocate an array rgbBuffer to store RGB image data, with a size of numBytes*sizeof(uint8_t); S64: Use the library function avpicture_fill to index the array of RGB image frames rgbFrame into rgbBuffer; S65: Check if the conversion context object image_convert_ctx is empty. If not, use the library function sws_freeContext to release the object; otherwise, proceed to the next step. S66: Use the library function sws_getContext to create a conversion context pair image_convert_ctx; S67: Determine if the conversion context pair image_convert_ctx is empty. If it is, exit the process; otherwise, proceed to the next step. S68: Use the library function sws_scale to convert the image frame and store the data in rgbFrame. Store the conversion result in result. S69: Determine if the conversion result is successful. If it is, proceed to the next step; otherwise, exit the process. S610: Send the converted image frame rgbBuffer to the interface.

4. The real-time video playback method in a QT environment as described in claim 3, characterized in that, Step S7 includes the following processes: S71: Check if the image frame object pointer is null. If it is, exit the process; otherwise, proceed to the next step. S72: Create an image display object, which is of type QImage; S73: Create a pixel map object pix, which is of type QPixmap; S74: Get the pointer to the display image object of the QLabel class, image_display; S75: Use the system function setPixmap to set the image object pointer image_display to display the image. The parameter is the return object of the pix function executed fromImage. Step S8 includes the following processes: S81: Use the library function av_parser_close to close the packet analysis object parser; S82: Use the library function avcodec_free_context to release the decoding context object; S83: Use the library function avformat_close_input to release the streaming context object decoder_avfc; S84: Use the library function av_packet_free to release the packet object; S85: Use the library function av_frame_free to release the image frame object; S86: Use the library function av_free to release the converted RGB array rgbBuffer.