Method, device, storage medium and system for video OSD adaptive image resolution

Abstract

The present application relates to a video OSD adaptive image resolution method, device, storage medium and system, the method comprises the following steps: A. unpacking the accessed video, obtaining the video bare stream and the encoded compressed metadata; B. video decoding and metadata analysis, obtaining the video frame and the video resolution after video decoding, and storing the video frame into the cache queue; C. according to the obtained video resolution and the original image resolution meeting the current OSD display standard, calculating the coordinate and size adjustment of the analyzed metadata, so that it is adapted to the obtained video resolution when displayed; D. obtaining the saved video frame from the cache queue, and drawing the metadata information onto the image according to the calculation result of step C; E. rendering the image superimposed with the metadata information to the screen. The present application can adapt to various different resolution videos, so that the OSD information can be normally displayed on the screen, and the increased time consumption can be negligible.

Description

TECHNICAL FIELD

[0001] The present application relates to a video image processing method and device, in particular to a video OSD adaptive image resolution method and device, storage medium and system. BACKGROUND

[0002] In recent years, unmanned aerial vehicles (UAVs) have been widely used in many fields. Currently, a UAV system generally includes a UAV, a ground station, and a video transmission (video transmission, VT) and data transmission (data transmission, DT) link responsible for ground-air transmission. The UAV is loaded with a VT and DT transmitter, and the ground station is installed with a VT and DT receiver for data interaction. Thus, UAV data can be transmitted to the ground for display and processing, and the ground station can also control the UAV according to specific circumstances. In the actual use of the UAV, on-screen display (OSD) technology is a very important link. OSD originally means a screen menu type adjustment method, and later extends to a screen display technology for superimposed display of characters, graphics and images on a terminal. OSD in the UAV system generally refers to the text, images and the like superimposed in the video, which superimposes the information of the UAV (onboard information, payload information and target information, etc., hereinafter referred to as metadata) on the video transmitted back. OSD can provide a friendly man-machine interface for users, and enable users to obtain a lot of additional information, which is the reference standard for users to control the UAV.

[0003] For OSD display in the UAV system, the commonly used technical solution is to transmit the video and metadata to the ground station through the link, the ground station software receives the data, and then analyzes the video and metadata. After the metadata and the video frame are synchronized, the metadata is drawn to the video picture, and then rendered to the display together. The simple structure of the whole system is shown in FIG. 1. Figure 1

[0004] Figure 1 ​In the prior art, the video image resolution provided by the camera currently carried by the unmanned aerial vehicle is generally 1080P, and when the ground station draws the metadata information onto the video image by software, the display form of the OSD depends on the UI design of each merchant, but the area where the OSD information is drawn and the position coordinates where various information is drawn are determined according to the current video image resolution. With the change of market demand, the gradual improvement of image transmission technology, the increase of data bandwidth, and the improvement of camera technology, there are more than one camera on the unmanned aerial vehicle, there may be an infrared camera and an ultra-high-definition camera, there may be multiple video stream switching display, the video image resolution may be greater than 1080P or less than 1080P, and the OSD drawing module for determining the OSD information display coordinates according to the image resolution is no longer universal. According to the needs of different users, the video image resolution received by the ground station software may be dozens of kinds, if the OSD information drawing coordinates are still determined according to a fixed resolution, when displayed on the screen, the OSD information display coordinates may be offset, some information may be missing, and the display may be very messy, which is obviously unacceptable to users; if the display coordinates of the OSD information are determined for various resolutions respectively, the workload of the program developers is obviously too large, and this is not a long-term solution. How to design an OSD drawing module to make the OSD information drawing adapt to video images of different resolutions and be normally displayed is a major technical problem. SUMMARY

[0005] The present application provides a video OSD adaptive image resolution method and device to solve how to normally display the OSD information in the expected way through a universal OSD drawing method on video images of different resolutions.

[0006] The video OSD adaptive image resolution method of the present application comprises the following steps:

[0007] A. The video is unpacked to obtain the video raw stream and the encoded and compressed metadata;

[0008] B. The video is decoded and the metadata is parsed, the video frame and the video resolution are obtained after the video is decoded, and the video frame is stored in the cache queue;

[0009] C. According to the obtained video resolution and the original image resolution meeting the current OSD display standard, the coordinates and size of the parsed metadata are adjusted and calculated to adapt to the obtained video resolution when displayed;

[0010] D. The saved video frame is obtained from the cache queue, and the metadata information is drawn onto the image according to the calculation result of step C;

[0011] E. rendering the image superimposed with the metadata information to a screen.

[0012] The original image resolution satisfying the current OSD display standard refers to the existing OSD display standard capable of meeting the current screen requirement, that is, based on the commonly used image resolution, each field in the OSD information has met the display requirement and can be normally superimposed on the video picture for display.

[0013] According to the new resolution obtained from the video, the coordinates and size of each OSD information are calculated respectively, so that the self-adaptation to various video resolutions is realized, and the OSD information in different resolution videos can be correctly superimposed in the video picture for display. The average time consumption of metadata analysis and coordinate and size calculation is far less than the average time consumption of video decoding per frame. For example, the time consumption of the i5-9400 model CPU released in 2019 is in the order of microseconds, so the increased delay can be ignored, and the memory of the video frame queue in the cache will not continuously increase.

[0014] Further, the calculation in step C is to obtain the new field coordinates by multiplying the ratio of the field coordinates of the current OSD display standard to the original current image resolution by the obtained video resolution.

[0015] The new field coordinates are calculated based on the field coordinates of the current OSD display standard and the original current image resolution, so that the new field display coordinates can be more accurate.

[0016] Specifically, the calculation method of the new field coordinates is as follows:

[0017]

[0018] wherein x new is the x coordinate value of the new field adapted to the new image resolution, y new is the y coordinate value of the new field adapted to the new image resolution, x origin is the field x coordinate of the current OSD display standard, width origin is the pixel width of the original current image resolution, width new is the pixel width of the obtained video resolution, y origin is the field y coordinate of the current OSD display standard, height origin is the pixel height of the original current image resolution, and height new is the pixel height of the obtained video resolution.

[0019] Further, the calculation of step C is to obtain the ratio of the video resolution and the original current image resolution, and then multiply the original current field size to obtain the new field size.

[0020] The new field size is calculated based on the original current image resolution and the original current field size, so that the new field size can be displayed more accurately.

[0021] Specifically, the calculation method of the new field size is as follows:

[0022]

[0023] wide new long new width new width origin wide origin height new height origin long origin height

[0024] Preferably, in step B, the video decoding and metadata parsing are performed synchronously. The synchronous performance of the video decoding and metadata parsing can further improve the data processing efficiency.

[0025] The application also provides a device for the above method, which comprises a video receiving device for receiving incoming video, an output end of the video receiving device being connected to a video unpacking module, the video unpacking module being connected to a metadata parsing module and a video decoding module respectively, the metadata parsing module and the video decoding module being connected to an OSD calculation module together, and the OSD calculation module being connected to an OSD information rendering module to render the calculated new field coordinates and field size to the screen.

[0026] The video received by the receiving device is unpacked by the unpacking module to obtain encoded metadata and a video raw stream. The encoded metadata is input into the metadata analysis module, and the video raw stream is input into the video decoding module. The video resolution is obtained through video decoding, and the video resolution and the decoded metadata output by the metadata analysis module are input into the OSD calculation module to calculate new field coordinates and field size. Then, the OSD information is rendered onto the screen by the OSD information drawing module according to the calculation result.

[0027] Further, the video decoding module comprises a decoder and a video frame buffer unit connected in sequence, wherein the input end of the decoder is connected with the video unpacking module, the video resolution output end of the decoder is connected with the OSD calculation module, the video frame output end of the decoder is connected with the video frame buffer unit, and the output end of the video frame buffer unit is connected with the OSD information drawing module.

[0028] The application further provides a data storage medium, wherein a computer program for executing the above method is stored in the data storage medium.

[0029] The application further provides a data processing system, which comprises a data processor, the above device, and the above data storage medium, wherein the data processor is signal-connected with the data storage medium, and the data processor is signal-connected with each module or unit in the device. The data processor can be one or multiple according to actual conditions.

[0030] The application has the following advantages:

[0031] 1. The application can adapt to video images with different resolutions, and can display various OSD information on the screen in an expected manner.

[0032] 2. The application almost does not increase redundant calculation cost, and the increased time consumption is in the order of microseconds, which can be ignored.

[0033] 3. The application is simple and has small development workload. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is an OSD display structure diagram in the prior art unmanned aerial vehicle system.

[0035] Figure 2 It is a timing diagram of the method for adapting the video OSD to the image resolution.

[0036] Figure 3 It is a block diagram of the device for adapting the OSD to the image resolution. DETAILED DESCRIPTION

[0037] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0038] Embodiment 1:

[0039] As shown in the method for adapting image resolution of video OSD of the present application, comprising the steps of: Figure 2

[0040] A. unpacking the accessed video to obtain video raw stream and encoded and compressed metadata;

[0041] B. decoding the video and parsing the metadata, obtaining video frames and video resolution after decoding the video, and storing the video frames in a cache queue;

[0042] C. according to the obtained video resolution and the original image resolution meeting the current OSD display standard, performing coordinate and size adjustment calculation on the parsed metadata, so that it is adapted to the obtained video resolution when displayed;

[0043] D. obtaining the saved video frames from the cache queue, and drawing the metadata information onto the image according to the calculation result of step C;

[0044] E. rendering the image superimposed with the metadata information to the screen.

[0045] The original image resolution meeting the current OSD display standard refers to the existing OSD display standard that can meet the current screen requirements, that is, based on the commonly used image resolution (for example: 1920x1080, other resolutions are also available), each field in the OSD information has met the display requirements and can be normally superimposed on the video picture for display. The video decoding and metadata parsing in step B are performed synchronously to further improve the data processing efficiency.

[0046] The video raw stream refers to a video transmission mode without error code resistance, which is simply a video stream that has only been encoded and has not been encapsulated, for example, H264 raw stream.

[0047] ​Video frame refers to the smallest unit of image in the animation, which is equivalent to each frame of film on the lens. A frame is a still picture, and continuous frames form animation, such as movie video.

[0048] The present application calculates the coordinates and size of each OSD information according to the new resolution of the obtained video, so as to realize the self-adaptation of various video resolutions, and make the OSD information in different resolution videos can be correctly superimposed and displayed in the video picture. The average time consumption of metadata analysis and coordinate and size calculation is far less than the average time consumption of video decoding per frame. Taking the CPU of i5-9400 model released in 2019 as an example, the time consumption is in the order of microseconds, so the increased delay can be ignored, and the memory of the video frame queue will not continue to grow.

[0049] Embodiment 2:

[0050] On the basis of embodiment 1, the calculation in step C is to obtain the new field coordinates (i.e. the coordinates of the field to be displayed) by multiplying the field coordinates of the current OSD display standard by the ratio of the original current image resolution, and then multiplying the obtained video resolution. And,

[0051] The new field size (i.e. the size of the field to be displayed) is obtained by multiplying the original current field size by the ratio of the obtained video resolution and the original current image resolution.

[0052] Let the original current image resolution (i.e. pixel resolution) be width origin *height origin (wide*high), the original current field coordinates be (x origin ,y origin ), and the original current font size be wide origin *long origin (wide*high). The obtained video resolution is width new *height new (wide*high), the new field coordinates are (x new ,y new ), and the new field size is wide new *long new .

[0053] The calculation method of the new field coordinates is:

[0054]

[0055] Wherein, x new is the x coordinate value of the new field adapted to the new image resolution, y newx origin width origin width new width origin height origin height new height

[0056] The new field size is calculated as follows:

[0057]

[0058] wide new width new width new width origin wide origin height new height origin long origin height

[0059] Thus, the OSD information display coordinates and field size adapted to the accepted video resolution are calculated, so that they can be correctly superimposed on the screen.

[0060] Embodiment 3:

[0061] The application also provides a device for video OSD self-adapting image resolution, as shown in Figure 3 The device comprises a video receiving device for receiving incoming video, the output end of the video receiving device is connected to a video unpacking module, the video unpacking module is respectively connected to a metadata analysis module and a video decoding module, the metadata analysis module and the video decoding module are jointly connected to an OSD calculation module, and the new field coordinates and field size calculated by the OSD calculation module are rendered to the screen through an OSD information drawing module.

[0062] The video received by the receiving device is unpacked by the unpacking module to obtain encoded metadata and a video raw stream.

[0063] The video decoding module comprises a decoder and a video frame buffer unit connected in series, wherein the input end of the decoder is connected with the video unpacking module, the video resolution output end of the decoder is connected with the OSD calculation module, the video frame output end of the decoder is connected with the video frame buffer unit, and the output end of the video frame buffer unit is connected with the OSD information drawing module.

[0064] Embodiment 4

[0065] The application further provides a data storage medium, which stores a computer program for executing the method of Embodiment 1 and / or Embodiment 2.

[0066] Embodiment 5

[0067] The application further provides a data processing system, which comprises a data processor, the device of Embodiment 3, and the data storage medium of Embodiment 4, wherein the data processor is signal-connected with the data storage medium, and the data processor is signal-connected with each module or unit in the device.

[0068] The above embodiments only express the specific implementation of the application, and the description is more specific and detailed, but it cannot be understood as the limitation of the protection scope of the application. It should be pointed out that, for ordinary skilled in the art, without departing from the technical scheme concept of the application, related deformations and improvements can be made, which belong to the protection scope of the application.

Claims

1. A method for video OSD adaptive image resolution, The method comprises the following steps: A. unpacking the accessed video to obtain a video raw stream and encoded metadata; B. decoding the video and parsing the metadata, obtaining video frames and video resolution after decoding the video, and storing the video frames in a cache queue, wherein the decoding of the video and the parsing of the metadata are performed synchronously; C. performing coordinate and size adjustment calculation on the parsed metadata according to the obtained video resolution and the original image resolution of the OSD display standard, so that the metadata is adapted to the obtained video resolution and the original image resolution of the OSD display standard during display, and based on the original image resolution of the OSD display standard, each field in the OSD information has met the display requirement and can be normally superimposed on the video picture for display; D. obtaining the saved video frames from the cache queue, and drawing the metadata information onto the image according to the calculation result of step C; E. rendering the image superimposed with the metadata information to a screen; The coordinate calculation of step C is obtained by multiplying the field coordinate of the current OSD display standard by the ratio of the obtained image resolution of the OSD display standard to the original video resolution. The size calculation of step C is obtained by multiplying the size of the field of the current OSD display standard by the ratio of the obtained image resolution of the OSD display standard to the original video resolution.

2. The method for video OSD adaptive image resolution according to claim 1, wherein, The calculation method of the field coordinate during display is: ， ， wherein x new is the x coordinate value of the field at the time of display, y new is the y coordinate value of the field at the time of display, x origin is the field x coordinate of the original OSD display standard, width origin is the pixel width of the image resolution of the original OSD display standard, width new is the pixel width of the obtained video resolution, y origin is the field y coordinate of the original OSD display standard, height origin is the pixel height of the image resolution of the original OSD display standard, height new is the pixel height of the obtained video resolution.

3. The method for video OSD adaptive image resolution according to claim 1, wherein, The calculation method of the field size during display is: ， ， wherein, wide new is the width of the field when displayed, long new is the height of the field when displayed, width new is the pixel width of the video resolution obtained, width origin is the pixel width of the image resolution of the original OSD display standard, wide origin is the field width of the original OSD display standard, height new is the pixel height of the video resolution obtained, height origin is the pixel height of the image resolution of the original OSD display standard, long origin is the field height of the original OSD display standard.

4. Apparatus for video OSD adaptive image resolution for use in a method as claimed in any one of claims 1 to 3, characterized in that, The method comprises the following steps: The method comprises the following steps:

5. The apparatus for video OSD adaptive image resolution according to claim 4, wherein, The video decoding module comprises a decoder and a video frame buffer unit connected thereto, wherein the input end of the decoder is connected to the video unpacking module, the video resolution output end of the decoder is connected to the OSD calculation module, the video frame output end of the decoder is connected to the video frame buffer unit, and the output end of the video frame buffer unit is connected to the OSD information drawing module.

6. A data storage medium characterized by, The data storage medium stores a computer program for executing the method of any one of claims 1 to 3.

7. A data processing system, characterized by a data processor, and the apparatus of claim 4 or 5, and the data storage medium of claim 6, the data processor being in signal connection with the data storage medium, and the data processor being in signal connection with each module or unit in the apparatus.

Images