A bypass thumbnail video method, device and related medium for low-latency splicing

Abstract

The present invention discloses a bypass thumbnail video method, device and related medium for low-latency splicing. The method includes: scheduling a graphics processor to start image decoding processing and detecting an image stream; activating a flag bit according to the image stream, and copying the complete image of the graphics processor by using the flag bit to obtain a bypass acquisition image; performing thumbnail on the bypass acquisition image by using an image reduction algorithm to obtain a thumbnail image; sending the thumbnail image to the graphics processor for multi-layer merging to obtain and output a spliced image; wherein, the multi-layer merging includes the complete image and the thumbnail image. The present invention copies the complete image by using the flag bit, bypasses to generate a thumbnail image, and the generated multiple video thumbnail pictures are updated in real time. Thus, the processing delay of the thumbnail image is reduced, and at the same time, the thumbnail image has timeliness.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a bypass thumbnail video method, device and related medium for low-latency stitching. Background Art

[0002] Thumbnailing is an important means for video preview or image preview. By shrinking the picture according to certain conditions, it ensures that users can quickly obtain key information, and at the same time relatively reduces the system memory occupancy.

[0003] In the prior art, there are mainly two solutions for video thumbnail processing technology: The first is for the processed video file. Extract pictures, specific segments or multiple key-frame synthesis segments from the video file, and generate thumbnails through decoding or other operations, such as video covers or thumbnail animations, etc.; this solution mainly processes the video file. The second is for real-time video. Extract one or more frames from the video stream or the processed video picture, and after certain processing, generate a thumbnail image of this frame of the picture.

[0004] However, both of these two solutions currently have certain limitations. It is required that the video content is known, that is, it has been encoded and exists in the form of a video file; or only one video can be processed at a time. Extracting a certain frame from the video stream as a thumbnail picture cannot reflect the changes of the video in real time, resulting in the lack of timeliness. In addition, to generate a thumbnail, it is necessary to process the real-time video signal, extract the video content from it, and then output the processed picture, which increases the main video processing time to a certain extent in the process. Summary of the Invention

[0005] Embodiments of the present invention provide a bypass thumbnail video method, device and related medium for low-latency stitching, aiming to solve the problems of high processing delay and lack of timeliness for thumbnails in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a bypass thumbnail video method for low-latency stitching, including:

[0007] Schedule the graphics processor to start image decoding processing and detect the image stream;

[0008] According to the activation flag bit of the image stream, use the flag bit to copy the complete image of the graphics processor to obtain a bypass acquisition image;

[0009] Use an image shrinking algorithm to thumbnail the bypass acquisition image to obtain a thumbnail image;

[0010] Send the thumbnail image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the thumbnail image.

[0011] In a second aspect, an embodiment of the present invention provides a bypass thumbnail video device for low-latency splicing, including:

[0012] An image detection unit, configured to schedule a graphics processor to start image decoding processing and detect an image stream;

[0013] An image copying unit, configured to activate a flag bit according to the image stream, and copy a complete image of the graphics processor by using the flag bit to obtain a bypass-acquired image;

[0014] An image thumbnail unit, configured to perform thumbnail on the bypass-acquired image by using an image reduction algorithm to obtain a thumbnail image;

[0015] An image merging unit, configured to send the thumbnail image to the graphics processor for multi-layer merging to obtain and output a spliced image; wherein, the multi-layer merging includes the complete image and the thumbnail image.

[0016] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the low-latency splicing bypass thumbnail video method of the first aspect is implemented.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the low-latency splicing bypass thumbnail video method of the first aspect is implemented.

[0018] An embodiment of the present invention provides a low-latency splicing bypass thumbnail video method, the method including: scheduling a graphics processor to start image decoding processing and detect an image stream; activating a flag bit according to the image stream, and copying a complete image of the graphics processor by using the flag bit to obtain a bypass-acquired image; performing thumbnail on the bypass-acquired image by using an image reduction algorithm to obtain a thumbnail image; sending the thumbnail image to the graphics processor for multi-layer merging to obtain and output a spliced image; wherein, the multi-layer merging includes the complete image and the thumbnail image. The present invention copies the complete image by using the flag bit, bypasses to generate a thumbnail image, and the generated multiple video thumbnail pictures are updated in real time. Thus, the processing delay of the thumbnail image is reduced, and at the same time, the thumbnail image has timeliness.

[0019] An embodiment of the present invention further provides a low-latency splicing bypass thumbnail video device, a computer device, and a storage medium, which also have the above beneficial effects. Description of the Drawings

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic flowchart of a low-latency splicing bypass thumbnail video method provided by an embodiment of the present invention;

[0022] Figure 2 It is another schematic flowchart of a low-latency splicing bypass thumbnail video method provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic block diagram of a low-latency splicing bypass thumbnail video device provided by an embodiment of the present invention. Detailed implementation manners

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0025] It should be understood that when used in this specification and the appended claims, the terms "comprises" and "comprising" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0026] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0027] It should be further understood that the term "and / or" used in this specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] Please refer to the following Figure 1 , Figure 1 It is a schematic flowchart of a low-latency splicing bypass thumbnail video method provided by an embodiment of the present invention, specifically including: steps S101 to S104.

[0029] S101. The scheduling graphics processor starts image decoding processing and detects the image stream.

[0030] S102. Activate the flag bit according to the image stream, and use the flag bit to copy the complete image of the graphics processor to obtain a bypass acquired image.

[0031] S103. Thumbnail the bypass acquired image using an image scaling algorithm to obtain a thumbnail image.

[0032] S104. Send the thumbnail image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the thumbnail image.

[0033] In step S101, the central processing unit (CPU) starts to schedule the graphics processing unit (GPU) to perform image decoding processing, and then detects the image stream.

[0034] In one embodiment, step S101 includes:

[0035] Detect the image stream of the video input device; use the complete frame as a unit to trigger the graphics processor to perform image decoding processing.

[0036] In this embodiment, the central processing unit detects the image stream of the video input device and uses a complete frame as a unit to trigger the graphics processor to perform image decoding processing.

[0037] In step S102, the flag bit is the "valid image" flag bit. According to the flag bit, the complete image in the graphics processor is copied to the memory space of the central processing unit. This process only takes 3 ms and has a very low latency.

[0038] In one embodiment, step S102 includes:

[0039] Determine whether the complete frame is detected; if not, re-detect; if so, activate the flag bit and use the flag bit to copy the complete image of the graphics processor to obtain a bypass acquired image.

[0040] In this embodiment, when the complete frame is detected, it indicates that a complete image is input. At this time, only the flag bit needs to be activated, and the complete image of the graphics processor is copied using the flag bit to obtain a bypass acquired image. When the complete frame is not detected, it indicates that no complete image is input, and only re-detection is required until the complete frame is detected.

[0041] In step S103, the central processing unit uses an image reduction algorithm to reduce the bypass-acquired image to obtain a reduced image. It should be noted that there are various ways of the image reduction algorithm, and it can be specifically selected according to actual needs.

[0042] In one embodiment, step S103 includes:

[0043] Use the bilinear interpolation algorithm to reduce the bypass-acquired image to obtain the reduced image.

[0044] Preferably, perform image matrix sampling on the bypass-acquired image to obtain the reduced image.

[0045] In this embodiment, the bilinear interpolation algorithm can be used to reduce the bypass-acquired image. The bilinear interpolation algorithm is also known as bilinear interpolation; mathematically, bilinear interpolation is the linear interpolation extension of an interpolation function with two variables, and its core idea is to perform linear interpolation in two directions respectively. As an interpolation algorithm in numerical analysis, bilinear interpolation is widely used in signal processing, digital images, and video processing.

[0046] Preferably, since the image processor spends a lot of time in processing images, the central processing unit will always wait for the image processor to finish processing. At this time, the central processing unit is in an idle waiting state (with a duration of 26 ms). At this time, the waiting time of the central processing unit can be used to perform the reduction operation; perform image matrix sampling on the bypass-acquired image in a sampling reduction manner of 1:2 and 1:4 to quickly (with a duration of 10 ms) obtain the reduced images that are 1 / 2 and 1 / 4 of the complete image. It should be noted that the sampling reduction ratio can have different settings, and specific adaptive adjustments can be made according to the actual situation.

[0047] In step S104, after the central processing unit processes to obtain the reduced image, it sends the reduced image to the image processor, and the image processor performs merging of multiple layers to obtain the stitched image; that is, the reduced image is merged onto the complete image to achieve simultaneous display of the complete image and the reduced image (with a duration of less than 1 ms). Specifically, while the image processor performs image processing, the central processing unit simultaneously completes the reduction of the image, making full use of the waiting time of the central processing unit, and the occupied time is extremely short (within 4 ms).

[0048] Combined Figure 3 shown, Figure 3 is a schematic block diagram of a low-latency stitched bypass-reduced video device provided by an embodiment of the present invention. The low-latency stitched bypass-reduced video device 300 includes:

[0049] An image detection unit 301, configured to schedule a graphics processor to start image decoding processing and detect an image stream;

[0050] An image copying unit 302, configured to activate a flag bit according to the image stream, and copy a complete image of the graphics processor by using the flag bit to obtain a bypass-acquired image;

[0051] An image thumbnail unit 303, configured to perform thumbnail processing on the bypass-acquired image by using an image reduction algorithm to obtain a thumbnail image;

[0052] An image merging unit 304, configured to send the thumbnail image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the thumbnail image.

[0053] In this embodiment, first, the image detection unit 301 schedules the graphics processor to start image decoding processing and detects an image stream; the image copying unit 302 activates a flag bit according to the image stream, and copies a complete image of the graphics processor by using the flag bit to obtain a bypass-acquired image; the image thumbnail unit 303 performs thumbnail processing on the bypass-acquired image by using an image reduction algorithm to obtain a thumbnail image; finally, the image merging unit 304 sends the thumbnail image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the thumbnail image.

[0054] In one embodiment, the image detection unit includes:

[0055] A detection unit, configured to detect an image stream of a video input device;

[0056] A decoding unit, configured to trigger the graphics processor to perform image decoding processing by using a complete frame as a unit.

[0057] In one embodiment, the image copying unit includes:

[0058] A judgment unit, configured to judge whether a complete frame is detected; if not, re-detect; if so, activate the flag bit, and copy a complete image of the graphics processor by using the flag bit to obtain a bypass-acquired image.

[0059] In one embodiment, the image thumbnail unit includes:

[0060] A sampling unit, configured to perform image matrix sampling on the bypass-acquired image to obtain the thumbnail image.

[0061] A thumbnail unit, configured to perform thumbnail processing on the bypass-acquired image by using a bilinear interpolation algorithm to obtain the thumbnail image.

[0062] Since the embodiments in the apparatus part correspond to those in the method part, for the descriptions of the embodiments in the apparatus part, please refer to the descriptions of the embodiments in the method part, which will not be elaborated here temporarily.

[0063] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the steps provided in the above embodiments can be implemented. The storage medium may include: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0064] The embodiments of the present invention also provide a computer device, which may include a memory and a processor. When the processor calls the computer program stored in the memory, the steps provided in the above embodiments can be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0065] The embodiments in the specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method part. It should be noted that for those of ordinary skill in the art in the technical field of the present application, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0066] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A bypass thumbnail video method for low-latency splicing, characterized in that Including: Schedule the graphics processor to start image decoding processing and detect the image stream; Activate the flag bit according to the image stream, and use the flag bit to copy the complete image of the graphics processor to obtain a bypass-acquired image; Use an image reduction algorithm to reduce the bypass-acquired image to obtain a reduced image; Send the reduced image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the reduced image; The scheduling of the graphics processor to start image decoding processing and detect the image stream includes: detecting the image stream of the video input device; using a complete frame as a unit to trigger the graphics processor to perform image decoding processing; The activating the flag bit according to the image stream and using the flag bit to copy the complete image of the graphics processor to obtain a bypass-acquired image includes: determining whether the complete frame is detected; if not, re-detect; if so, activate the flag bit and use the flag bit to copy the complete image of the graphics processor to obtain a bypass-acquired image; The using an image reduction algorithm to reduce the bypass-acquired image to obtain a reduced image includes: performing image matrix sampling on the bypass-acquired image in a sampling reduction manner of 1:2 or 1:4 to obtain the reduced image.

2. A bypass thumbnail video device for low-latency splicing, characterized in that, Including: An image detection unit for scheduling the graphics processor to start image decoding processing and detect the image stream; An image copying unit for activating a flag bit according to the image stream and using the flag bit to copy the complete image of the graphics processor to obtain a bypass-acquired image; An image reduction unit for using an image reduction algorithm to reduce the bypass-acquired image to obtain a reduced image; An image merging unit for sending the reduced image to the graphics processor for multi-layer merging to obtain and output a stitched image; wherein, the multi-layer merging includes the complete image and the reduced image; The image detection unit includes: a detection unit for detecting the image stream of the video input device; a decoding unit for using a complete frame as a unit to trigger the graphics processor to perform image decoding processing; The image copying unit includes: a determination unit for determining whether the complete frame is detected; if not, re-detect; if so, activate the flag bit and use the flag bit to copy the complete image of the graphics processor to obtain a bypass-acquired image; The image reduction unit includes: a sampling unit for performing image matrix sampling on the bypass-acquired image in a sampling reduction manner of 1:2 or 1:4 to obtain the reduced image.

3. A computer device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, it implements the low-latency stitching bypass reduction video method as claimed in claim 1.

4. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, it implements the low-latency stitching bypass reduction video method as claimed in claim 1.

Images