Decoding method based on video distribution service

Abstract

The application discloses a decoding method based on a video distribution service, comprising the following steps: S1, formulating a video distribution service rule; S2, creating a compiling environment; S3, initializing decoding resources; S4, acquiring video data of the video distribution service; S5, decoding the video data; S6, carrying out format conversion on the decoded image frames; S7, acquiring the image frames after the format conversion and displaying in real time; and S8, releasing the decoding channel resources. The application adopts a soft decoding method, effectively reduces the decoding time, simultaneously adopts a video distribution rule to acquire a video stream, effectively reduces a playing delay, and guarantees the fluency of video playing.

Description

Technical Field

[0001] This invention belongs to the field of video decoding technology in cross-platform systems. In application scenarios where the system is uncertain, it designs and implements a video decoding scheme compatible with multiple system types by considering the decoding characteristics of ffmpeg and combining it with a QT module to design an image display scheme. Specifically, this invention addresses situations where the system is of multiple types and uncertain, and the application environment involves cross-network communication between land and sea. First, a video distribution service is used to transmit the remote network stream, effectively acquiring the video stream. Then, the stream is decoded based on its encoding characteristics, and the decoded image is displayed, thus completing the decoding, playback, and display process. Background Technology

[0002] Qt is a cross-platform graphical user interface application development framework for building high-performance desktop, mobile, and web applications. It can also be used to develop non-GUI programs, such as console tools and servers. Qt is an object-oriented framework that, using special code generation extensions and macros, is easily extensible and allows for true component programming.

[0003] Ffmpeg is an open-source computer program used to record, convert, and stream digital audio and video. It provides a complete solution for recording, converting, and streaming audio and video. It includes highly advanced audio / video codec libraries.

[0004] In projects with unclear requirements, choosing a cross-platform tool for development can save development costs. Combining Qt and ffmpeg can solve multi-platform compatibility issues and facilitate easier module porting. In maritime and land-based communication environments, where cross-network video transmission is required, establishing a video service can effectively solve the problem of efficient video stream transmission. Summary of the Invention

[0005] (I) Purpose of the Invention

[0006] The purpose of this invention is to solve the problem of real-time playback of video decoding under multiple systems by adopting a decoding method based on a combination of QT and ffmpeg, and to provide a decoding method based on a video distribution service.

[0007] (II) Technical Solution

[0008] To address the aforementioned technical problems, this invention provides a decoding method based on a video distribution service, comprising the following steps:

[0009] S1: Formulate rules for video distribution services;

[0010] S2: Create the compilation environment;

[0011] S3: Initialize decoding resources;

[0012] S4: Obtain video data from the video distribution service;

[0013] S5: Decodes video data;

[0014] S6: Convert the format of the decoded image frames;

[0015] S7: Acquire the converted image frames and display them in real time;

[0016] S8: Release the decoding channel resources.

[0017] Step S1, which involves defining the video distribution service rules, includes:

[0018] S11: Application Registration

[0019] Marine or land-based equipment sends a registration request to the communication system service center. After successful registration, file and message transmission is optimized.

[0020] S12: Registration Message Format

[0021] It consists of three fields: application system identifier, application system entity number, and sender IP.

[0022] S13: Registration Feedback

[0023] When the transmission service software receives the application registration message, it sends it to the application system;

[0024] S14: Message format for registration feedback

[0025] It consists of three fields: application system identifier, application system entity number, and feedback result;

[0026] S15: Application Logout

[0027] S16: Cancellation message format

[0028] It consists of three fields: application system identifier, application system entity number, and sender IP.

[0029] S17: Message Transmission

[0030] Message-based applications provide service transmission requests and carry message service data at the same time;

[0031] S18: Message Format

[0032] It is divided into the following fields: Business Unified Identifier, Message Transmission Type, Sender's Application System Identifier, Sender's Application Entity Identifier, Number of Receivers, Receiver Information, Business Information Type, Reliability, Priority, Timeout, Data Size, and Data Content;

[0033] S19: Receive message format, same as send message format.

[0034] In step S2, the process of creating the compilation environment is as follows:

[0035] S21: Obtain the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, and libswscale.lib from ffmpeg and copy them to the working directory lib.

[0036] S22: Copy the ffmpeg include header file to the include directory of the project;

[0037] S23: Add the system environment variable FFMPEGLIBPATH and set this environment variable to the lib directory under the project directory;

[0038] S24: Add an external library to the Qt project file. Add the library file from step S21, using the path indicated in step S23.

[0039] In step S3, a new thread class `decode` is created in the Qt project directory. The steps for decoding and initializing resources in the `decode` thread class are as follows:

[0040] S31: Use the library function av_packet_alloc to create a packet object pkt. If creation is successful, proceed to the next step; otherwise, exit the process.

[0041] S32: Use the library function avcodec_find_decoder to create a decoder codec. If creation is successful, proceed to the next step; otherwise, exit the process.

[0042] S33: Use the library function av_parser_init to create a packet analysis object parser. If creation is successful, proceed to the next step; otherwise, exit the process.

[0043] S34: Use the library function avcodec_alloc_context3 to create a decoding environment context object. If the creation is successful, proceed to the next step; otherwise, exit the process.

[0044] S35: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process; otherwise, proceed to the next step.

[0045] S36: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process; otherwise, proceed to the next step.

[0046] In step S4, the process of obtaining video data from the video distribution service is as follows:

[0047] S41: Create a new network receiving thread, netrecv;

[0048] S42: Extract the message content of the video transmission according to the protocol message identifier;

[0049] S43: Lock the semaphore;

[0050] S44: Store the video data in the data queue;

[0051] S45: Unlock the signal;

[0052] S46: Create a thread to acquire video data;

[0053] S47: Entering the process of reading data from the loop queue;

[0054] S48: Check if the data queue is empty. If it is, delay for 10us; otherwise, proceed to the next step.

[0055] S49: Lock the semaphore;

[0056] S410: Check if the length of the queue is less than the maximum data length. If it is, set the length len to be retrieved this time to the queue length; otherwise, set the length len to the maximum length.

[0057] S411: Take data of length len from the queue in sequence and store it in the array inbuf;

[0058] S412: Unlock the semaphore;

[0059] S413: Set the data packet length data_size to len;

[0060] S414: Assign the pointer to the array inbuf to data;

[0061] S415: The process of analyzing data packets;

[0062] S416: Determine if data_size is greater than 0. If it is, proceed to the next step; otherwise, exit the analysis process.

[0063] S417: Use the library function av_parser_parser2 to analyze the data and convert it into a data packet. The length of the converted data packet is stored in ret. If the conversion is successful, proceed to the next step; otherwise, print an error message and exit the process.

[0064] S418: Move the array data pointer by the length ret, and subtract the length ret from the array length data_size;

[0065] S419: Determine if the size of the converted data packet is greater than zero. If it is, proceed to the next decoding process; otherwise, jump to step S415.

[0066] In step S5, the process of decoding the video data is as follows:

[0067] S51: Use the library function avcode_send_packet to send the video stream data packet object (packet) obtained in process S3 to the decoder. If the sending fails, an error is reported and the process is exited; otherwise, proceed to the next step.

[0068] S52: Enter the loop to acquire decoded image frames;

[0069] S53: Determine whether the result ret of the acquired image frame is greater than or equal to zero. If it is, proceed to the next step; otherwise, exit the loop process of acquiring image frames.

[0070] S54: Use the library function avcodec_receive_frame to get the video frame where the context object dec_ctx is located. If the acquisition is successful, it is stored in the image frame object frame and the next step is performed. If it fails, an error message is printed and the process is exited.

[0071] S55: Use the system function fflush to force the decoder to output image frames.

[0072] In step S6, the process of format conversion of the decoded image frame is as follows:

[0073] S61: Use the library function av_frame_alloc to create an RGB image frame (rgbFrame);

[0074] S62: Use the library function avpicture_get_size to calculate the data size numBytes required to convert to rgb24 format;

[0075] S63: Use the library function av_malloc to dynamically allocate an array rgbBuffer to store RGB image data, with a size of numBytes*sizeof(uint8_t);

[0076] S64: Use the library function avpicture_fill to index the array of RGB image frames rgbFrame into rgbBuffer;

[0077] S65: Check if the conversion context object image_convert_ctx is empty. If not, use the library function sws_freeContext to release the object; otherwise, proceed to the next step.

[0078] S66: Use the library function sws_getContext to create a conversion context pair image_convert_ctx;

[0079] S67: Determine if the conversion context pair image_convert_ctx is empty. If it is, exit the process; otherwise, proceed to the next step.

[0080] S68: Use the library function sws_scale to convert the image frame and store the data in rgbFrame. Store the conversion result in result.

[0081] S69: Determine if the conversion result is successful. If it is, proceed to the next step; otherwise, exit the process.

[0082] S610: Send the converted image frame rgbBuffer to the interface.

[0083] The steps to implement step S7 are as follows:

[0084] S71: Check if the image frame object pointer is null. If it is, exit the process; otherwise, proceed to the next step.

[0085] S72: Create an image display object, which is of type QImage;

[0086] S73: Create a pixel map object pix, which is of type QPixmap;

[0087] S74: Get the pointer to the display image object of the QLabel class, image_display;

[0088] S75: Use the system function setPixmap to set the image object pointer image_display to display the image. The parameter is the return object of the pix from the image function.

[0089] The steps for implementing step S8 are as follows:

[0090] S81: Use the library function av_parser_close to close the packet analysis object parser;

[0091] S82: Use the library function avcodec_free_context to release the decoding context object;

[0092] S83: Use the library function av_packet_free to release the packet object;

[0093] S84: Use the library function av_frame_free to release the image frame object;

[0094] S85: Use the library function av_free to release the converted RGB array rgbBuffer.

[0095] (III) Beneficial Effects

[0096] The decoding method based on video distribution services provided by the above technical solution can solve the video decoding and transmission problems in multiple systems. This method has been algorithmically verified and experimentally tested. Results show that this solution can effectively reduce decoding time by using a software decoding method to solve image decoding problems, and simultaneously reduce playback latency by using video distribution rules to obtain the video stream, ensuring smooth video playback. Detailed Implementation

[0097] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to examples.

[0098] The decoding method based on video distribution service in this embodiment includes the following steps:

[0099] S1: Formulate rules for video distribution services

[0100] S11: Application Registration. Marine or land-based equipment sends a registration request to the communication system service center. This request is sent when the application system boots up or when a new software module is launched. The application registers with the transmission service through this interface. After successful registration, optimization of file and message transmission and other services can be performed.

[0101] S12: Registration message format. It consists of four fields: application system identifier, application system entity number, and sender IP.

[0102] S13: Registration Feedback. The transmission service software sends this to the application system upon receiving the application registration message;

[0103] S14: Message format for registration feedback. It consists of three fields: application system identifier, application system entity number, and feedback result.

[0104] S15: Application Deregistration. Applications deregister via this interface.

[0105] S16: Cancellation message format. It consists of three fields: application system identifier, system entity number, and sender IP.

[0106] S17: Message Transmission. Message-based applications provide service transmission requests while carrying message service data.

[0107] S18: Message Format. It consists of the following fields: Service Uniform Identifier (representing a message task), Message Transmission Type (point-to-point, point-to-many), Sender's Application System Identifier, Sender's Application Entity Identifier, Number of Receivers, Receiver Information (Receiver Application 1 System Flag, Receiver Application 1 Entity Identifier... Receiver Application N System Flag, Receiver Application N Entity Identifier), Service Information Type, Reliability, Priority, Timeout, Data Size, and Data Content.

[0108] S19: Receive message format, same as send message format.

[0109] S2: Creating the compilation environment

[0110] S21: First, obtain the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, and libswscale.lib from ffmpeg (the file extensions of dynamic library files vary depending on the system, so adjust accordingly) and copy them to the working directory lib.

[0111] S22: Copy the ffmpeg include header file to the include directory of the project;

[0112] S23: Add the environment variable FFMPEGLIBPATH to the system and set this environment variable to the lib directory under the project directory.

[0113] S24: Add an external library to the Qt project file. Add the library file from step S21, using the path indicated in step S23.

[0114] S3: Initialize decoding resources

[0115] Create a new thread class named decode in the QT project directory. This class inherits from QThread.

[0116] The specific steps for implementing decoding and initializing resources in this class are as follows:

[0117] S31: Use the library function av_packet_alloc to create a packet object pkt. If creation is successful, proceed to the next step; otherwise, exit the process.

[0118] S32: Use the library function avcodec_find_decoder to create a decoder codec. If creation is successful, proceed to the next step; otherwise, exit the process.

[0119] S33: Use the library function av_parser_init to create a packet analysis object parser. If creation is successful, proceed to the next step; otherwise, exit the process.

[0120] S34: Use the library function avcodec_alloc_context3 to create a decoding environment context object. If the creation is successful, proceed to the next step; otherwise, exit the process.

[0121] S35: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process; otherwise, proceed to the next step.

[0122] S36: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process; otherwise, proceed to the next step.

[0123] S4: Obtain video data from the video distribution service

[0124] S41: Create a new network receiving thread, netrecv;

[0125] S42: Extract the message content of the video transmission according to the protocol message identifier;

[0126] S43: Lock the semaphore;

[0127] S44: Store the video data in the data queue;

[0128] S45: Unlock the signal;

[0129] S46: Create a thread to acquire video data;

[0130] S47: Entering the process of reading data from the loop queue;

[0131] S48: Check if the data queue is empty. If it is, delay for 10us; otherwise, proceed to the next step.

[0132] S49: Lock the semaphore;

[0133] S410: Check if the length of the queue is less than the maximum data length. If it is, set the length len to be retrieved this time to the queue length; otherwise, set the length len to the maximum length.

[0134] S411: Take data of length len from the queue in sequence and store it in the array inbuf;

[0135] S412: Unlock the semaphore;

[0136] S413: Set the data packet length data_size to len;

[0137] S414: Assign the pointer to the array inbuf to data;

[0138] S415: The process of analyzing data packets;

[0139] S416: Determine if data_size is greater than 0. If it is, proceed to the next step; otherwise, exit the analysis process.

[0140] S417: Use the library function av_parser_parser2 to analyze the data and convert it into a data packet. The length of the converted data packet is stored in ret. If the conversion is successful, proceed to the next step; otherwise, print an error message and exit the process.

[0141] S418: Move the array data pointer by the length ret, and subtract the length ret from the array length data_size;

[0142] S419: Determine if the size of the converted data packet is greater than zero. If it is, proceed to the next decoding process; otherwise, jump to step S415.

[0143] S5: Decode video data

[0144] The specific steps are as follows:

[0145] S51: Use the library function avcode_send_packet to send the video stream data packet object (packet) obtained in process 3 to the decoder. If the sending fails, report an error and exit the process; otherwise, proceed to the next step.

[0146] S52: Enter the loop to acquire decoded image frames;

[0147] S53: Determine whether the result ret of the acquired image frame is greater than or equal to zero. If it is, proceed to the next step; otherwise, exit the loop process of acquiring image frames.

[0148] S54: Use the library function avcodec_receive_frame to get the video frame where the context object dec_ctx is located. If the acquisition is successful, it is stored in the image frame object frame and the next step is performed. If it fails, an error message is printed and the process is exited.

[0149] S55: Use the system function fflush to force the output of the decoder's image frames;

[0150] S6: Convert the format of the decoded image frames.

[0151] The decoded image is in YUV format, which cannot be directly displayed in the QT interface. It needs to be converted to RGB format. The specific steps are as follows:

[0152] S61: Use the library function av_frame_alloc to create an RGB image frame (rgbFrame);

[0153] S62: Use the library function avpicture_get_size to calculate the data size numBytes required to convert to rgb24 format;

[0154] S63: Use the library function av_malloc to dynamically allocate an array rgbBuffer to store RGB image data. The size is numBytes*sizeof(uint8_t) (that is, one byte multiplied by the required data size, converted to bits).

[0155] S64: Use the library function avpicture_fill to index the array of RGB image frames rgbFrame into rgbBuffer;

[0156] S65: Check if the conversion context object image_convert_ctx is empty. If not, use the library function sws_freeContext to release the object; otherwise, proceed to the next step.

[0157] S66: Use the library function sws_getContext to create a conversion context pair image_convert_ctx;

[0158] S67: Determine if the conversion context pair image_convert_ctx is empty. If it is, exit the process; otherwise, proceed to the next step.

[0159] S68: Use the library function sws_scale to convert the image frame and store the data in rgbFrame. Store the conversion result in result.

[0160] S69: Determine if the conversion result is successful. If it is, proceed to the next step; otherwise, exit the process.

[0161] S610: Sends the converted image frame rgbBuffer to the interface;

[0162] S7: Acquire and display the converted image frames in real time.

[0163] The specific steps are as follows:

[0164] S71: Determine if the image frame object pointer is null. If it is, exit the process; otherwise, proceed to the next step.

[0165] S72: Create an image display object, which is of type QImage;

[0166] S73: Create a pixel map object pix, which is of type QPixmap;

[0167] S74: Get the pointer to the display image object of the QLabel class, image_display;

[0168] S75: Use the system function setPixmap to set the image object pointer image_display to display the image. The parameter is the return object of the pix function executed fromImage.

[0169] S8: Release decoding channel resources

[0170] The specific steps are as follows:

[0171] S81: Use the library function av_parser_close to close the packet analysis object parser;

[0172] S82: Use the library function avcodec_free_context to release the decoding context object;

[0173] S83: Use the library function av_packet_free to release the packet object;

[0174] S84: Use the library function av_frame_free to release the image frame object;

[0175] S85: Use the library function av_free to release the converted RGB array rgbBuffer.

[0176] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A decoding method based on a video distribution service, characterized in that, Includes the following steps: S1: Formulate rules for video distribution services; S2: Create the compilation environment; S3: Initialize decoding resources; S4: Obtain video data from the video distribution service; S5: Decode the video data; S6: Convert the format of the decoded image frames; S7: Acquire the converted image frames and display them in real time; S8: Release the decoding channel resources; Step S1, which involves defining the video distribution service rules, includes: S11: Application Registration Marine or land-based equipment sends a registration request to the communication system service center. After successful registration, file and message transmission is optimized. S12: Registration Message Format It consists of three fields: application system identifier, application system entity number, and sender IP. S13: Registration Feedback When the transmission service software receives the application registration message, it sends it to the application system; S14: Message format for registration feedback It consists of three fields: application system identifier, application system entity number, and feedback result; S15: Application Logout S16: Cancellation message format It consists of three fields: application system identifier, application system entity number, and sender IP. S17: Message Transmission Message-based applications provide service transmission requests and carry message service data at the same time; S18: Message Format It is divided into the following fields: Business Unified Identifier, Message Transmission Type, Sender's Application System Identifier, Sender's Application Entity Identifier, Number of Receivers, Receiver Information, Business Information Type, Reliability, Priority, Timeout, Data Size, and Data Content; S19: Receive message format, same as send message format; In step S2, the process of creating the compilation environment is as follows: S21: Obtain the dynamic library files libacodec.lib, libavformat.lib, libavutil.lib, and libswscale.lib from ffmpeg and copy them to the working directory lib. S22: Copy the ffmpeg include header file to the include directory of the project; S23: Add the system environment variable FFMPEGLIBPATH and set this environment variable to the lib directory under the project directory; S24: Add an external library to the Qt project file. Add the library file from step S21, using the path indicated in step S23. In step S3, a new thread class `decode` is created in the Qt project directory. The steps for decoding and initializing resources in the `decode` thread class are as follows: S31: Use the library function av_packet_alloc to create a packet object pkt. If creation is successful, proceed to the next step; otherwise, exit the process. S32: Use the library function avcodec_find_decoder to create a decoder codec. If creation is successful, proceed to the next step; otherwise, exit the process. S33: Use the library function av_parser_init to create a packet analysis object parser. If creation is successful, proceed to the next step; otherwise, exit the process. S34: Use the library function avcodec_alloc_context3 to create a decoding environment context object. If the creation is successful, proceed to the next step; otherwise, exit the process. S35: Use the library function avcodec_oepn2 to open the decoder. If it fails, exit the process; otherwise, proceed to the next step. S36: Use the library function av_frame_alloc to create an image frame object frame. If it fails, exit the process; otherwise, proceed to the next step. In step S4, the process of obtaining video data from the video distribution service is as follows: S41: Create a new network receiving thread, netrecv; S42: Extract the message content of the video transmission according to the protocol message identifier; S43: Lock the semaphore; S44: Store the video data in the data queue; S45: Unlock the signal; S46: Create a thread to acquire video data; S47: Entering the process of reading data from the loop queue; S48: Check if the data queue is empty. If it is, delay for 10us; otherwise, proceed to the next step. S49: Lock the semaphore; S410: Check if the length of the queue is less than the maximum data length. If it is, set the length len to be retrieved this time to the queue length; otherwise, set the length len to the maximum length. S411: Take data of length len from the queue in sequence and store it in the array inbuf; S412: Unlock the semaphore; S413: Set the data packet length data_size to len; S414: Assign the pointer to the array inbuf to data; S415: The process of analyzing data packets; S416: Determine if data_size is greater than 0. If it is, proceed to the next step; otherwise, exit the analysis process. S417: Use the library function av_parser_parser2 to analyze the data and convert it into a data packet. The length of the converted data packet is stored in ret. If the conversion is successful, proceed to the next step; otherwise, print an error message and exit the process. S418: Move the array data pointer by the length ret, and subtract the length ret from the array length data_size; S419: Determine if the size of the converted data packet is greater than zero. If it is, proceed to the next decoding process; otherwise, jump to step S415. In step S5, the process of decoding the video data is as follows: S51: Use the library function avcode_send_packet to send the video stream data packet object (packet) obtained in process S3 to the decoder. If the sending fails, an error is reported and the process is exited; otherwise, proceed to the next step. S52: Enter the loop to acquire decoded image frames; S53: Determine whether the result ret of the acquired image frame is greater than or equal to zero. If it is, proceed to the next step; otherwise, exit the loop process of acquiring image frames. S54: Use the library function avcodec_receive_frame to get the video frame where the context object dec_ctx is located. If the acquisition is successful, it is stored in the image frame object frame and the next step is performed. If it fails, an error message is printed and the process is exited. S55: Use the system function fflush to force the output of the decoder's image frames; In step S6, the process of format conversion of the decoded image frame is as follows: S61: Use the library function av_frame_alloc to create an RGB image frame (rgbFrame); S62: Use the library function avpicture_get_size to calculate the data size numBytes required to convert to rgb24 format; S63: Use the library function av_malloc to dynamically allocate an array rgbBuffer to store RGB image data, with a size of numBytes*sizeof(uint8_t); S64: Use the library function avpicture_fill to index the array of RGB image frames rgbFrame into rgbBuffer; S65: Check if the conversion context object image_convert_ctx is empty. If not, use the library function sws_freeContext to release the object; otherwise, proceed to the next step. S66: Use the library function sws_getContext to create a conversion context pair image_convert_ctx; S67: Determine if the conversion context pair image_convert_ctx is empty. If it is, exit the process; otherwise, proceed to the next step. S68: Use the library function sws_scale to convert the image frame and store the data in rgbFrame. Store the conversion result in result. S69: Determine if the conversion result is successful. If it is, proceed to the next step; otherwise, exit the process. S610: Sends the converted image frame rgbBuffer to the interface; The steps to implement step S7 are as follows: S71: Check if the image frame object pointer is null. If it is, exit the process; otherwise, proceed to the next step. S72: Create an image display object, which is of type QImage; S73: Create a pixel map object pix, which is of type QPixmap; S74: Get the pointer to the display image object of the QLabel class, image_display; S75: Use the system function setPixmap to set the image object pointer image_display to display the image. The parameter is the return object of the pix function executed fromImage. The steps for implementing step S8 are as follows: S81: Use the library function av_parser_close to close the packet analysis object parser; S82: Use the library function avcodec_free_context to release the decoding context object; S83: Use the library function av_packet_free to release the packet object; S84: Use the library function av_frame_free to release the image frame object; S85: Use the library function av_free to release the converted RGB array rgbBuffer.