A software-defined general-purpose real-time display method for optoelectronic video streams of aircraft

Abstract

This invention discloses a software-defined, general-purpose real-time display method for aircraft photoelectric video streams. It reads HEVC-encoded video files captured by the aircraft, parses the HEVC-encoded video files to obtain the number of preceding IDR frames for all P-frames and the word length and position information of each frame; decodes the HEVC-encoded video files to obtain the decoded image; and outputs and displays the decoded image. During the display process, based on the parsed information of the number of preceding IDR frames for all P-frames and the word length and position information of each frame, video playback and progress control are achieved. This invention effectively solves the important problem that when aircraft-mounted camera devices use multi-core encoding to store video files in HEVC encoding format, the video display state cannot be freely controlled, making it difficult to meet the needs of users for analyzing and monitoring high-value targets in flight.

Description

Technical Field

[0001] This invention relates to the field of telemetry, tracking, and command (TT&C) communication, and in particular to a software-defined method for real-time display of general-purpose aircraft optoelectronic video streams. Background Technology

[0002] HEVC has significant advantages over traditional methods in terms of encoding performance. It can transmit video images of the same or higher resolution at a lower bit rate, which is beneficial for the real-time transmission of images acquired by aircraft and the optimization of storage resource configuration. Therefore, it is increasingly being used in real-time monitoring systems for aircraft.

[0003] To ensure the real-time transmission and processing of video data, aircraft-mounted camera devices typically employ multi-core encoding. Instead of directly encapsulating the acquired bitstream into higher-level formats, they store it in HEVC encoding. However, existing players cannot directly extract valid video duration information from HEVC files. This results in a lack of free control over the video display status during playback, making it difficult to meet the user's needs for analyzing and monitoring high-value targets in flight. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a software-defined general-purpose real-time display method for aircraft photoelectric video streams. This method is used to display and play HEVC encoded videos acquired by photoelectric detection devices under flight conditions, solves the playback failure problem of existing players, realizes free control of the real-time playback and display status of HEVC video files, and ensures that designers can effectively monitor and analyze the flight status of high-value targets at different times.

[0005] The technical solution of this invention is: a software-defined, universal method for real-time display of optoelectronic video streams in aircraft, comprising:

[0006] S1. Read the HEVC encoded video files captured by the aircraft;

[0007] S2. Parsing HEVC encoded video files:

[0008] Parameter initialization: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0;

[0009] For each frame of an HEVC-encoded video file, perform the following judgments and execute the corresponding sub-steps until the last frame;

[0010] Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1:

[0011] (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends.

[0012] Decision 2: If the current frame is an IDR frame, proceed to sub-step 2:

[0013] (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (2.2) Increment the NALU count by 1. (2.3) Set the IDR flag to true. (2.4) Sub-step 2 is completed, and the current frame judgment ends.

[0014] Decision 3: If the current frame is a P-frame, proceed to sub-step 3:

[0015] Determine if the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment.

[0016] S3. Decode the HEVC encoded video file to obtain the decoded image;

[0017] S4. Output and display the decoded image. During the display process, based on the position and word length information of each frame recorded after parsing in S2, as well as the total number of recorded frames, the video playback and progress control are realized.

[0018] Furthermore, FFmpeg is used to decode the HEVC encoded video file to obtain an image in YUV422 format.

[0019] Furthermore, the decoded YUV422 format image is converted into the YUY2 format supported by the DirectDraw architecture for output display.

[0020] Furthermore, the display frame rate is set to 25 frames per second, and the display interface size can be adjusted according to needs.

[0021] The present invention also provides a computer program product that, when executed by a processor, implements the steps of the method as described above.

[0022] The present invention also provides an airborne computing terminal, including a terminal body, an operating system, and a controller. The airborne computing terminal is deployed on a launch vehicle. The controller includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described above, and plays and displays HEVC encoded video files acquired by the airborne photoelectric detection equipment under flight conditions, and controls the progress.

[0023] This invention also provides a software-defined general-purpose aircraft optoelectronic video stream real-time display system, comprising:

[0024] The data interface module is used to read HEVC-encoded video files captured by the aircraft.

[0025] The parsing module includes:

[0026] Initialization Unit: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0;

[0027] The loop parsing unit is used to perform the following judgments and execute corresponding sub-steps frame by frame on the image frame in the HEVC encoded video file until the last frame;

[0028] Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1:

[0029] (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends.

[0030] Decision 2: If the current frame is an IDR frame, proceed to sub-step 2:

[0031] (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (2.2) Increment the NALU count by 1. (2.3) Set the IDR flag to true. (2.4) Sub-step 2 is completed, and the current frame judgment ends.

[0032] Decision 3: If the current frame is a P-frame, proceed to sub-step 3:

[0033] Determine if the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment.

[0034] The decoding module is used to decode HEVC encoded video files to obtain the decoded image;

[0035] The image display module is used to output and display the decoded image;

[0036] The display control module is used to control the normal playback and progress of the video during the display process based on the position and word length information of each frame recorded by the parsing module and the total number of frames recorded.

[0037] Furthermore, in the decoding module, FFmpeg is used to decode HEVC encoded video files to obtain YUV422 format images.

[0038] Furthermore, in the image display module, the decoded YUV422 format image is converted into the YUY2 format supported by the DirectDraw architecture for output display.

[0039] Furthermore, in the image display module, the display frame rate is set to 25 frames / s, and the size of the display interface can be adjusted as needed.

[0040] The advantages of this invention compared to the prior art are:

[0041] (1) This invention effectively solves the important problem that when an aircraft-mounted camera device uses multi-core encoding to store video files in HEVC encoding format, it is impossible to freely control the video display status, making it difficult to meet the needs of users to analyze and monitor high-value targets in flight. The method proposed in this invention can accurately calculate information such as the number of keyframes, word length, and position, and the playback progress is flexibly adjustable, with free control over the display status.

[0042] (2) Based on the photoelectric video stream information, the present invention can accurately calculate the duration of the video stream and realize the direct decoding and playback of HEVC encoded files. This solves the problem that existing players need to perform secondary encapsulation of HEVC files before they can be played. The present invention does not require multiple encapsulation and is suitable for the direct and efficient processing of HEVC photoelectric video information on rockets.

[0043] (3) This invention is based on the FFmpeg open-source architecture design, realizes the software-defined gating function, has a general development architecture, is independently controllable in design, and is easy to develop. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method of the present invention;

[0045] Figure 2 This is a schematic diagram of the system composition of the present invention. Detailed Implementation

[0046] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] This invention proposes a software-defined, general-purpose real-time display method for optoelectronic video streams in aircraft. The process can be referred to as follows. Figure 1 Specifically, it includes:

[0048] Step 1: Read the HEVC encoded video file captured by the aircraft (consisting of a series of P frames, IDR frames, SPS frames, and VPS frames).

[0049] Step 2: Parse the HEVC encoded video file:

[0050] Parameter initialization: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0;

[0051] For each frame of an HEVC-encoded video file, perform the following judgments and execute the corresponding sub-steps until the last frame;

[0052] Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1:

[0053] (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends.

[0054] Decision 2: If the current frame is an IDR frame, proceed to sub-step 2:

[0055] (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (2.2) Increment the NALU count by 1. (2.3) Set the IDR flag to true. (2.4) Sub-step 2 is completed, and the current frame judgment ends.

[0056] Decision 3: If the current frame is a P-frame, proceed to sub-step 3:

[0057] Determine whether the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame (i.e., the IDR frame) of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment.

[0058] Step 3: Decode the HEVC encoded video file using FFmpeg to obtain a YUV422 format image.

[0059] Step 4: Convert the decoded YUV422 format image into the YUY2 format supported by the DirectDraw architecture for output display. Set the display frame rate to 25 frames / s, and the display interface size can be adjusted as needed. During the display process, the video playback and progress control are realized based on the position and word length information of each frame recorded in S2 and the total number of recorded frames (i.e., the number of IDR frames before all recorded P frames).

[0060] The present invention also provides a decoding display system, the module composition of which is as follows: Figure 2 As shown, it includes:

[0061] The data interface module is used to read HEVC-encoded video files captured by the aircraft.

[0062] The parsing module includes:

[0063] Initialization Unit: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0;

[0064] The loop parsing unit is used to perform the following judgments and corresponding operations on each frame of the image frame in the HEVC encoded video file until the last frame;

[0065] Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1:

[0066] (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends.

[0067] Decision 2: If the current frame is an IDR frame, proceed to sub-step 2:

[0068] (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (2.2) Increment the NALU count by 1. (2.3) Set the IDR flag to true. (2.4) Sub-step 2 is completed, and the current frame judgment ends.

[0069] Decision 3: If the current frame is a P-frame, proceed to sub-step 3:

[0070] Determine if the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment.

[0071] The decoding module is used to decode HEVC encoded video files to obtain the decoded image;

[0072] The image display module is used to output and display the decoded image;

[0073] The display control module is used to control the playback and progress of the video based on the position and word length information of each frame recorded by the parsing module and the total number of frames recorded during the display process.

[0074] The present invention also relates to a computer program product that, when executed by a processor, implements the steps of the aforementioned method.

[0075] The present invention also relates to an airborne computing terminal, including a terminal body, an operating system, and a controller. The airborne computing terminal is deployed on a launch vehicle. The controller includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned method and plays and displays HEVC encoded video files acquired by the airborne photoelectric detection equipment under flight conditions, and controls the progress.

[0076] It is understood that this invention has been described through embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of this invention. Furthermore, under the teachings of this invention, these features and embodiments can be modified to adapt to specific circumstances without departing from the spirit and scope of this invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are protected by this invention.

[0077] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A software-defined, general-purpose real-time display method for optoelectronic video streams of aircraft, characterized in that, include: S1. Read the HEVC encoded video files captured by the aircraft; S2. Parsing HEVC encoded video files: Parameter initialization: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0; For each frame of an HEVC-encoded video file, perform the following judgments and execute the corresponding sub-steps until the last frame; Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1: (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends. Decision 2: If the current frame is an IDR frame, proceed to sub-step 2: (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1; otherwise, keep the IDR frame count unchanged. (2.2) Increment the NALU count by 1; (2.3) Set the IDR flag to true; (2.4) Sub-step 2 is completed, and the current frame judgment ends; Decision 3: If the current frame is a P-frame, proceed to sub-step 3: Determine whether the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment. S3. Decode the HEVC encoded video file to obtain the decoded image; S4. Output and display the decoded image. During the display process, based on the position and word length information of each frame recorded after parsing in S2, as well as the total number of recorded frames, the video playback and progress control are realized.

2. The real-time display method for optoelectronic video streams of general-purpose aircraft oriented towards software definition according to claim 1, characterized in that: Use FFmpeg to decode HEVC encoded video files to obtain YUV422 format images.

3. The real-time display method for optoelectronic video streams of general-purpose aircraft oriented towards software definition according to claim 2, characterized in that: The decoded YUV422 format image is converted into the YUY2 format supported by the DirectDraw architecture for output display.

4. The real-time display method for optoelectronic video streams of general-purpose aircraft oriented towards software definition according to claim 1, characterized in that: The display frame rate is set to 25 frames per second, and the display interface size can be adjusted as needed.

5. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-4.

6. An airborne computing terminal, comprising a terminal body, an operating system, and a controller, characterized in that: The airborne computing terminal is deployed on the launch vehicle. The controller includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-4, and plays and displays the HEVC encoded video file acquired by the airborne photoelectric detection equipment under flight conditions and controls the progress.

7. A software-defined general-purpose aircraft optoelectronic video stream real-time display system, characterized in that, include: The data interface module is used to read HEVC-encoded video files captured by the aircraft. The parsing module includes: Initialization Unit: Set the IDR flag to false, the IDR frame count to 0, and the NALU count to 0; The loop parsing unit is used to perform the following judgments and execute corresponding sub-steps frame by frame on the image frame in the HEVC encoded video file until the last frame; Decision 1: If the current frame is a VPS frame or an SPS frame, execute sub-step 1: (1.1) Determine if the current IDR flag is false. If so, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (1.2) Increment the NALU count by 1. (1.3) Sub-step 1 is completed, and the current frame judgment ends. Decision 2: If the current frame is an IDR frame, proceed to sub-step 2: (2.1) Determine if the current IDR flag is false. If it is, increment the IDR frame count by 1. Otherwise, the IDR frame count remains unchanged. (2.2) Increment the NALU count by 1. (2.3) Set the IDR flag to true. (2.4) Sub-step 2 is completed, and the current frame judgment ends. Decision 3: If the current frame is a P-frame, proceed to sub-step 3: Determine whether the current IDR frame count is equal to the NALU count and the IDR flag is true. Otherwise, end the current frame judgment. If yes: (3.1) Set the IDR frame count and NALU count to zero, set the IDR flag to false, and record the position and word length information of the previous frame of this P frame; (3.2) Sub-step 3 is completed, and end the current frame judgment. The decoding module is used to decode HEVC encoded video files to obtain the decoded image; The image display module is used to output and display the decoded image; The display control module is used to control the normal playback and progress of the video during the display process based on the position and word length information of each frame recorded by the parsing module and the total number of frames recorded.

8. The software-defined general-purpose aircraft optoelectronic video stream real-time display system according to claim 7, characterized in that: In the decoding module, FFmpeg is used to decode HEVC encoded video files to obtain YUV422 format images.

9. The software-defined general-purpose aircraft optoelectronic video stream real-time display system according to claim 7, characterized in that: In the image display module, the decoded YUV422 format image is converted into the YUY2 format supported by the DirectDraw architecture for output display.

10. The software-defined general-purpose aircraft optoelectronic video stream real-time display system according to claim 7, characterized in that: In the image display module, the display frame rate is set to 25 frames / s, and the size of the display interface can be adjusted as needed.

Images