Video real-time playing method in QT environment

Abstract

The invention discloses a real-time video playing method in a QT environment. The method comprises the following steps: S1, creating a compiling environment; s2, formulating a transmission service rule; s3, an ffmpeg pull stream and a decoding resource are initialized; s4, creating a pull stream main thread; s5, decoding the data packet object obtained by stream pulling; s6, carrying out format conversion on the decoded image frame; s7, obtaining the image frame after format conversion and displaying the image frame in real time; and S8, releasing decoding channel resources. The soft decoding method is adopted, the decoding time is effectively shortened, the RTSP video transmission protocol is adopted to obtain the video stream, the playing delay is effectively shortened, and the video playing fluency is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of video decoding technology under a cross-platform system. In an application scenario where the system is uncertain, the ffmpeg decoding characteristics are combined with the QT module to design a display image solution, and a video real-time playback method in a QT environment compatible with multiple system types is designed and implemented. In the case of a system with multiple types of uncertainty, the present invention first uses the FFMPEG soft decoding library to perform rtsp streaming on the network code stream to effectively obtain the video stream, and at the same time, settings are made according to the characteristics of the rtsp stream, unified numbering, and optimized transmission services. Decoding is performed according to the coding characteristics of the code stream, and the decoded image is displayed, thereby completing the decoding, playback and display process. Background Art

[0002] Qt is a cross-platform graphical user interface application development framework that can build high-performance desktop, mobile and web application frameworks. It can also be used to develop non-GUI programs, such as console tools and servers. Qt is an object-oriented framework that uses special code generation extensions and some macros. Qt is easy to extend and allows real component programming.

[0003] Ffmpeg is an open source computer program that can be used to record, convert digital audio and video, and convert them into streams. It provides a complete solution for recording, converting and streaming audio and video. It contains a very advanced audio / video codec library.

[0004] Real-time Transport Protocol (RTSP) is an application layer protocol in the TCP / IP protocol system. This protocol defines how one-to-many applications can effectively transmit multimedia data through IP networks.

[0005] In projects with unclear requirements, choosing a cross-platform tool for development can save development costs. The combination of Qt and ffmpeg can solve the problem of multi-platform compatibility, facilitate the transplantation of module content, and use the RTSP video transmission protocol to solve the problem of real-time video playback across platforms. At the same time, according to the characteristics of sea transmission, the video stream is optimized and graded, and the bandwidth is effectively used to optimize the transmission service. Summary of the invention

[0006] (I) Purpose of the invention

[0007] The purpose of the present invention is to provide a video real-time playback method under a QT environment based on a decoding method combining QT and ffmpeg, so as to solve the problem of real-time playback of video decoding under multiple systems.

[0008] (II) Technical solution

[0009] In order to solve the above technical problems, the present invention provides a

[0010] (III) Beneficial effects

[0011] The real-time video playback method in the QT environment provided by the above technical solution can solve the video decoding and transmission problems in multiple systems. The method has been verified by the algorithm and experimentally tested. The results show that the solution can solve the image decoding problem by using a soft decoding method to effectively reduce the decoding time. At the same time, the RTSP video transmission protocol is used to obtain the video stream, which effectively reduces the playback delay and ensures the smoothness of video playback. DETAILED DESCRIPTION

[0012] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with embodiments.

[0013] The method for real-time video playback under the QT environment of this embodiment includes the following steps:

[0014] S1: Create a compilation environment

[0015] S11: Get the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, libswscale.lib in ffmpeg (dynamic library files of different systems have different suffixes, adjust according to the specific situation) and copy them to the working directory lib.

[0016] S12: Copy the included header file of ffmpeg to the include of the project directory;

[0017] S13: Add the environment variable FFMPEGLIBPATH to the system and set the environment variable to the lib in the project directory.

[0018] S14: Add an external library to the Qt project file, add the library file in step 1, and use the path indicated in step 3.

[0019] S2: Formulate transmission service rules

[0020] In order to solve the problem of video transmission between maritime communication equipment, transmission service rules are formulated to effectively improve transmission efficiency. The specific steps are as follows:

[0021] S21: The streaming application is registered in the communication system;

[0022] S22: The streaming application compresses the original video;

[0023] S23: The streaming application uses ffmpeg to perform rtsp streaming;

[0024] S24: The streaming application sets the tos field in the rtsp stream message;

[0025] S25: The communication system performs priority transmission according to the level of the tos field based on the rtsp stream received from the network, ensuring the transmission task with a high level;

[0026] S3: Initialize ffmpeg streaming and decoding resources

[0027] Create a new thread class decode in the QT project directory. This class inherits QThread. The specific steps to implement the stream initialization resource InitVideo in this class are as follows:

[0028] S301: Use library function avformat_network_init to initialize the network;

[0029] S302: Use the library function avformat_alloc_context to create a stream pull context object decoder_avfc;

[0030] S303: Determine whether the context object decoder_avfc is empty, if it is empty, exit the process, otherwise proceed to the next step;

[0031] S304: Use the library function av_log_set_level to enable the log function;

[0032] S305: Use the library function av_dict_set to set the parameter avdic of rstp;

[0033] S306: Use the library function avformat_open_input to open the preset rtsp streaming address url;

[0034] S307: Determine whether the result of step S306 is successful. If not, record the error message and exit the process. Otherwise, proceed to the next step.

[0035] S308: Determine whether the parameter avdic is empty. If not, use the library function av_dict_free to release the object resources. Otherwise, proceed to the next step.

[0036] S309: Use the library function avformat_find_stream_info to obtain the video stream. If the acquisition fails, use the library function avformat_close_input to close the context object decoder_avfc to release resources. Otherwise, proceed to the next step.

[0037] The specific steps to implement decoding initialization resources in this class are as follows:

[0038] S311: Use the library function av_packet_alloc to create a data packet object pkt. If the creation is successful, proceed to the next step, otherwise exit the process;

[0039] S312: Use the library function avcodec_find_decoder to create a decoder codec. If the creation is successful, proceed to the next step, otherwise exit the process;

[0040] S313: Use the library function av_parser_init to create a data packet analysis object parser. If the creation is successful, proceed to the next step, otherwise exit the process;

[0041] S314: Use library function avcodec_alloc_context3 to create a decoding environment context object. If the creation is successful, proceed to the next step, otherwise exit the process;

[0042] S315: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process, otherwise proceed to the next step;

[0043] S316: Use library function av_frame_alloc to create image frame object frame, if it fails, exit the process, otherwise proceed to the next step;

[0044] S4: Create the main stream pulling thread

[0045] S41: Perform the main flow pulling cycle;

[0046] S42: Check whether the parameter setting flag is successful, if successful, proceed to the next step, otherwise delay 10us;

[0047] S43: Determine whether the video stream initialization step InitVideo of process 2 is successfully operated. If so, exit the connection establishment loop and proceed to the next step. Otherwise, continue the connection operation.

[0048] S44: Entering a loop for acquiring video stream;

[0049] S45: create a stream pull data packet;

[0050] S46: Initialize pakcet using library function av_init_packet;

[0051] S47: Set the packet data pointer to a null pointer;

[0052] S48: Set the packet data size to 0;

[0053] S49: Determine whether the stream pulling context decode_avfc is empty, if so, return failure and exit the process, otherwise proceed to the next step;

[0054] S410: Use the library function av_read_frame to obtain the streaming video data packet opened by decode_avfc. If the opening fails, release the data packet object. Otherwise, store the data in the packet and proceed to the next step.

[0055] S411: Determine whether the stream index number of the data packet object packet is the macro-defined video stream vdieo_Stream, if yes, proceed to the next decoding process and release the data packet object packet, otherwise directly release the data packet object packet;

[0056] S5: Decode the data packet object obtained by pulling the stream

[0057] The specific steps are as follows:

[0058] S51: Use the library function avcode_send_packet to send the video stream data packet object packet obtained in process 3 to the decoder. If the sending fails, an error is reported and the process is exited, otherwise proceed to the next step;

[0059] S52: Entering the process of looping to obtain decoded image frames;

[0060] S53: determining whether the result ret of acquiring the image frame is greater than or equal to zero, if so, proceeding to another step, otherwise exiting the loop process of acquiring the image frame;

[0061] S54: using the library function avcodec_receive_frame to obtain the video frame where the context object dec_ctx is located, if the acquisition is successful, it is stored in the image frame object frame and the next step is performed, if it fails, an error message is printed and the process is exited;

[0062] S55: Use the system function fflush to force the decoder to output the image frame;

[0063] S6: Convert the decoded image frame into a new format

[0064] The decoded image is in YUV format and cannot be directly displayed in the QT interface. The YUV format needs to be converted into RGB format. The specific steps are as follows:

[0065] S61: Use library function av_frame_alloc to create rgb image frame rgbFrame;

[0066] S62: Use the library function avpicture_get_size to calculate the data size numBytes required for conversion into rgb24 format;

[0067] S63: Use the library function av_malloc to dynamically apply for an array rgbBuffer for storing RGB image data, the size of which is numBytes*sizeof(uint8_t) (i.e., one byte multiplied by the required data size, converted into bits);

[0068] S64: Use the library function avpicture_fill to index the array of rgb image frame rgbFrame into rgbBuffer;

[0069] S65: Determine whether the conversion context object image_convert_ctx is empty, if not, use the library function sws_freeContext to release the object, otherwise proceed to the next step;

[0070] S66: Use the library function sws_getContext to create the conversion context pair image_convert_ctx;

[0071] S67: Determine whether the context pair image_convert_ctx for format conversion is empty, if so, exit the process, otherwise proceed to the next step;

[0072] S68: Use the library function sws_scale to convert the image frame frame, and store the data in rgbFrame, and store the conversion result in result;

[0073] S69: Determine whether the conversion result is successful, if yes, proceed to the next step, otherwise exit the process;

[0074] S610: Send the converted image frame rgbBuffer to the interface;

[0075] S7: Obtain the converted image frame and display it in real time

[0076] The specific steps are as follows:

[0077] S71: Determine whether the image frame object pointer is null, if so, exit the process, otherwise proceed to the next step;

[0078] S72: Create an image display object image, the object type is QImage;

[0079] S73: Create a pixel map object pix, the object type is QPixmap;

[0080] S74: Get the display image object pointer image_display of the QLabel class;

[0081] S75: Use the system function setPixmap to set the image object pointer image_display to display the image, and the parameter content is the return object of the pix execution fromImage function;

[0082] S8: Release the decoding channel resources

[0083] The specific steps are as follows:

[0084] S81: Use the library function av_parser_close to close the data packet analysis object parser;

[0085] S82: Use the library function avcodec_free_context to release the decoding context object context;

[0086] S83: Use the library function avformat_close_input to release the stream pulling context object decoder_avfc;

[0087] S84: Use the library function av_packet_free to release the data packet object packet;

[0088] S85: Use the library function av_frame_free to release the image frame object frame;

[0089] S86: Use the library function av_free to release the converted rgb array rgbBuffer.

[0090] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for real-time video playback in a QT environment, characterized in that: include: S1: Create a compilation environment; S2: Formulate transmission service rules; S3: Initialize ffmpeg streaming and decoding resources; S4: Create the main stream pulling thread; S5: Decode the data packet object obtained by pulling the stream; S6: Convert the format of the decoded image frame; S7: Obtain the image frame after the format conversion and display it in real time; S8: Release the decoding channel resources.

2. The method for real-time video playback in a QT environment as claimed in claim 1, characterized in that: Step S1 includes the following sub-steps: S11: Get the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, libswscale.lib in ffmpeg and copy them to the working directory lib; S12: Copy the included header file of ffmpeg to the include of the project directory; S13: Add the environment variable FFMPEGLIBPATH to the system and set the environment variable to the lib in the project directory; S14: Add an external library to the qt project file, add the library file in step S11, and select the path indicated by step S13.

3. The method for real-time video playback in a QT environment as claimed in claim 2, characterized in that: Step S2 includes the following process: S21: The streaming application is registered in the communication system; S22: The streaming application compresses the original video; S23: The streaming application uses ffmpeg to perform rtsp streaming; S24: The streaming application sets the tos field in the rtsp stream message; S25: The communication system performs priority transmission according to the level of the TOS field based on the RTSP stream received from the network to ensure high-level transmission tasks.

4. The method for real-time video playback in a QT environment as claimed in claim 3, characterized in that: In step S3, a new thread class decode is created in the QT project directory, and the class inherits QThread.

5. The method for real-time video playback in a QT environment as claimed in claim 4, characterized in that: In step S3, the steps of implementing the stream initialization resource InitVideo in the thread class decode are as follows: S301: Use library function avformat_network_init to initialize the network; S302: Use the library function avformat_alloc_context to create a stream pull context object decoder_avfc; S303: Determine whether the context object decoder_avfc is empty, if it is empty, exit the process, otherwise proceed to the next step; S304: Use the library function av_log_set_level to enable the log function; S305: Use the library function av_dict_set to set the parameter avdic of rstp; S306: Use the library function avformat_open_input to open the preset rtsp streaming address url; S307: Determine whether the result of step S306 is successful. If not, record the error message and exit the process. Otherwise, proceed to the next step. S308: Determine whether the parameter avdic is empty. If not, use the library function av_dict_free to release the object resources. Otherwise, proceed to the next step. S309: Use the library function avformat_find_stream_info to obtain the video stream. If the acquisition fails, use the library function avformat_close_input to close the context object decoder_avfc to release resources. Otherwise, proceed to the next step.

6. The method for real-time video playback in a QT environment as claimed in claim 5, characterized in that: In step S3, the steps of implementing decoding initialization resources in the thread class decode are as follows: S311: Use the library function av_packet_alloc to create a data packet object pkt. If the creation is successful, proceed to the next step, otherwise exit the process; S312: Use the library function avcodec_find_decoder to create a decoder codec. If the creation is successful, proceed to the next step, otherwise exit the process; S313: Use the library function av_parser_init to create a data packet analysis object parser. If the creation is successful, proceed to the next step, otherwise exit the process; S314: Use library function avcodec_alloc_context3 to create a decoding environment context object. If the creation is successful, proceed to the next step, otherwise exit the process; S315: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process, otherwise proceed to the next step; S316: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process, otherwise proceed to the next step.

7. The method for real-time video playback in a QT environment as claimed in claim 6, characterized in that: Step S4 includes the following process: S41: Perform the main flow pulling cycle; S42: Check whether the parameter setting flag is successful, if successful, proceed to the next step, otherwise delay 10us; S43: Determine whether the video stream initialization step InitVideo of process S42 is successfully operated. If yes, exit the connection establishment loop and proceed to the next step. Otherwise, continue the connection operation. S44: Entering a loop for acquiring video stream; S45: create a stream pull data packet; S46: Initialize pakcet using library function av_init_packet; S47: Set the packet data pointer to a null pointer; S48: Set the packet data size to 0; S49: Determine whether the stream pulling context decode_avfc is empty, if so, return failure and exit the process, otherwise proceed to the next step; S410: Use the library function av_read_frame to obtain the streaming video data packet opened by decode_avfc. If the opening fails, release the data packet object. Otherwise, store the data in the packet and proceed to the next step. S411: Determine whether the stream index number of the data packet object packet is the macro-defined video stream vdieo_Stream, if yes, proceed to the next decoding process and release the data packet object packet, otherwise directly release the data packet object packet.

8. The method for real-time video playback in a QT environment as claimed in claim 7, characterized in that: Step S5 The process includes: S51: Use the library function avcode_send_packet to send the video stream data packet object packet obtained in process S3 to the decoder. If the sending fails, an error is reported and the process is exited, otherwise proceed to the next step; S52: Entering the process of looping to obtain decoded image frames; S53: determining whether the result ret of acquiring the image frame is greater than or equal to zero, if so, proceeding to another step, otherwise exiting the loop process of acquiring the image frame; S54: using the library function avcodec_receive_frame to obtain the video frame where the context object dec_ctx is located, if the acquisition is successful, it is stored in the image frame object frame and the next step is performed, if it fails, an error message is printed and the process is exited; S55: Use the system function fflush to force the decoder to output the image frame.

9. The method for real-time video playback in a QT environment as claimed in claim 8, characterized in that: Step S6 The process includes: S61: Use library function av_frame_alloc to create rgb image frame rgbFrame; S62: Use the library function avpicture_get_size to calculate the data size numBytes required for conversion into rgb24 format; S63: Use the library function av_malloc to dynamically apply for the array rgbBuffer for storing rgb image data, the size of which is numBytes*sizeof(uint8_t); S64: Use the library function avpicture_fill to index the array of rgb image frame rgbFrame into rgbBuffer; S65: Determine whether the conversion context object image_convert_ctx is empty, if not, use the library function sws_freeContext to release the object, otherwise proceed to the next step; S66: Use the library function sws_getContext to create the conversion context pair image_convert_ctx; S67: Determine whether the context pair image_convert_ctx for format conversion is empty, if so, exit the process, otherwise proceed to the next step; S68: Use the library function sws_scale to convert the image frame frame, and store the data in rgbFrame, and store the conversion result in result; S69: Determine whether the conversion result is successful, if yes, proceed to the next step, otherwise exit the process; S610: Send the converted image frame rgbBuffer to the interface.

10. The method for real-time video playback in a QT environment as claimed in claim 9, characterized in that: Step S7 includes the following process: S71: Determine whether the image frame object pointer is null, if so, exit the process, otherwise proceed to the next step; S72: Create an image display object image, the object type is QImage; S73: Create a pixel map object pix, the object type is QPixmap; S74: Get the display image object pointer image_display of the QLabel class; S75: Use the system function setPixmap to set the image object pointer image_display to display the image, and the parameter content is the return object of the pix execution fromImage function; Step S8 includes the following process: S81: Use the library function av_parser_close to close the data packet analysis object parser; S82: Use the library function avcodec_free_context to release the decoding context object context; S83: Use the library function avformat_close_input to release the stream pulling context object decoder_avfc; S84: Use the library function av_packet_free to release the data packet object packet; S85: Use the library function av_frame_free to release the image frame object frame; S86: Use the library function av_free to release the converted rgb array rgbBuffer.