A method for implementing multi-layer video editing in nonlinear editing
By dynamically uploading graphics card memory during non-linear editing, the bottleneck problem of multi-layer editing of high-quality video footage is solved, enabling efficient multi-layer video editing and compositing, and improving editing smoothness and output speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 北京广播电视台
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-05
AI Technical Summary
With high-quality video footage, GPU upload bandwidth becomes a bottleneck for multi-layer video editing. Existing technologies solve multi-layer video editing by degrading the quality of the footage, but this results in a rough display and makes it impossible to achieve high-quality multi-layer video compositing.
By dynamically uploading valid data to the graphics card memory during non-linear editing, different uploading strategies are adopted according to the size and format of the video material, including scaling and line extraction, to reduce the amount of data, directly synthesize the output file, and ensure image quality.
While ensuring image quality, the amount of uploaded data was reduced, improving the real-time performance and number of computation layers in multi-layer video editing, reducing CPU usage and video memory bandwidth, and enhancing the smoothness of multi-layer video editing and the packaging speed of output files.
Smart Images
Figure CN122160539A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for implementing multi-layer video editing in non-linear editing, an electrical processing method for computer networks, and a method for editing video data in a network. Background Technology
[0002] In non-linear video editing, users often want to see multiple layers of video visuals simultaneously on a single screen for stunning visual effects (artistic and informational) or for multi-camera editing (for reference and selection). For low-quality footage (e.g., 1920×1080 pixels, 8 bits), GPU upload bandwidth is sufficient to support editing of dozens of layers. However, for high-quality footage (e.g., 3840×2160 pixels, 10 bits, or 7680×4320 pixels, 10 bits), GPU upload bandwidth can become a critical bottleneck. The more layers of data being edited, the more pronounced the bottleneck becomes. Existing solutions involve using a de-optimization editing mode to handle multi-layer video editing, i.e., reducing the display resolution and the size of the input video. However, this results in a coarse-grained video and prevents video compositing output files. A key challenge is how to reduce the amount of uploaded image data while maintaining image texture quality, thus overcoming the bottleneck of multi-layer video editing. Summary of the Invention
[0003] To overcome the problems of existing technologies, this invention proposes a method for achieving multi-layer video editing in non-linear editing. The method described here, under full-quality editing conditions, dynamically uploads effective data based on the video's texture size, achieving real-time multi-layer editing and directly synthesizing the output file. This significantly reduces the amount of uploaded data and achieves a breakthrough in multi-layer editing while preserving image quality.
[0004] The objective of this invention is achieved as follows: a method for implementing multi-layer video editing in non-linear editing, the steps of which are as follows:
[0005] Step 1, Receive Texture Task: Receive texture storyboard materials, which are divided into video materials, subtitle materials, special effects materials, and audio materials, including individual or mixed materials;
[0006] Step 2, Calculate the texture range: Calculate the position range of each texture image on the screen according to the task requirements, including the width and height of the texture image;
[0007] Step 3, Distinguish color formats: Read the video image parameter information to distinguish whether the image format is a single-pixel color format or a complex pixel color format;
[0008] Step 4, Dynamically upload graphics card memory: Different upload strategies are adopted according to different parameter information. The strategies include two main categories: those for single-pixel color formats and those for complex pixel color formats.
[0009] For color formats where the image format is a single pixel:
[0010] a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory;
[0011] b: When the width and height of the video file are both larger than the width and height of the screen texture, if the decoder supports scaling decoding, it will directly decode the data that approximates the required width and height and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image, and upload it to the graphics card memory; if the decoder can only decode video images of one resolution, it will first decode at this resolution and then use the CPU to scale the image.
[0012] c: When the width of the video file is less than the width of the screen texture but the height is greater than the height of the screen texture, if the decoder supports scaling decoding, it directly decodes the data with the width unchanged and the height scaled, and uploads the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the height direction, and upload it to the graphics card memory.
[0013] d: When the height of the video file is less than the height of the screen texture and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the width direction, and upload it to the graphics card memory.
[0014] For image formats with complex pixel color formats:
[0015] a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory;
[0016] b: If the width and height of the video file are both greater than the width and height of the screen texture, if the decoder supports scaling decoding, it will directly decode the data of approximately the required width and height and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will decode the original image; if the image height is greater than or equal to twice the height of the screen texture, the image data will be copied to the graphics card memory in a line-by-line manner, otherwise the original image size data will be uploaded directly to the graphics card memory.
[0017] c: If the width of the video file is smaller than the width of the screen texture, and the height of the video file is greater than the height of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the width unchanged and the height scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will decode the original image. If the image height is greater than or equal to twice the height of the screen texture, the image data will be copied to the graphics card memory in a line-by-line manner; otherwise, the original image size data will be uploaded directly to the graphics card memory.
[0018] d: When the height of the video file is less than the height of the screen texture, and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will directly decode the original image and upload the image directly to the graphics card memory.
[0019] Step 5, texturing operation: The video image is retrieved from the graphics card memory, texturing is performed, and the result is calculated and rendered to create a composite output file.
[0020] The advantages and beneficial effects of this invention are as follows: This invention utilizes the video of the texture to be textured before the texture is applied. When the video is displayed on the screen, it is a scaled-down window. Therefore, while ensuring a certain display quality, it is not necessary to upload the entire high-definition screen. Instead, the original data is quickly scaled according to the size of the screen display, and the scaled-down image is uploaded to the GPU for texture application. This can significantly increase the number of computational layers, thereby minimizing CPU usage and bandwidth usage for uploading to GPU memory, and increasing the number of real-time layers. Attached Figure Description
[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0022] Figure 1 This is a flowchart of the method described in an embodiment of the present invention. Detailed Implementation
[0023] Example:
[0024] This embodiment describes a method for implementing multi-layer video editing in non-linear editing, such as... Figure 1As shown. In non-linear editing, the decoder directly decodes the image into system memory, then uploads it to the graphics card's video memory, where it is calculated and rendered. Taking 2D picture-in-picture as an example, multiple videos are displayed on the screen, often as scaled-down images pasted onto the target screen. If each video's frame size is 4K (3840×2160), the amount of data uploaded is enormous. Therefore, this embodiment utilizes the principle that when a video frame is displayed on the screen, it is a scaled-down window. Thus, while ensuring a certain display quality, it is not necessary to upload the entire 4K frame. Instead, the original data is quickly scaled according to the size of the corresponding frame displayed on the screen, and the scaled-down image is uploaded to the GPU for texturing. This significantly increases the number of computational layers. This embodiment can be applied to 4K high-definition and other high-definition editing formats, achieving the same fast upload effect. The specific steps of the method described in this embodiment are as follows:
[0025] Step 1, Receive Texture Task: Receive texture storyboard materials, which are divided into video materials, subtitle materials, special effects materials, and audio materials, including individual or mixed materials;
[0026] Since the bottleneck in non-linear editing is mainly in video editing, this embodiment focuses on editing video footage. Video footage contains various image information, including color format, image size, etc.
[0027] Step 2, Calculate the texture range: Calculate the position range of each texture image on the screen according to the task requirements, including the width and height of the texture image.
[0028] The texture can be placed at various locations such as the bottom, top, or middle of the overall image. Its size on the overall screen is determined according to the task requirements. Since the width and height of the texture directly affect the subsequent strategy selection, this step is crucial.
[0029] Step 3, distinguish color format: Read the video image parameter information to distinguish whether the image format is a single-pixel color format (e.g., RGB) or a complex pixel color format (e.g., v210).
[0030] Step 4, Dynamically upload to graphics card memory: Different upload strategies are adopted according to different parameter information, and the video organized by the upload strategy is uploaded to the graphics card memory. The strategies include two main categories: for single-pixel color formats and for complex pixel color formats.
[0031] For single-pixel color formats: Single-pixel color formats fall into four categories: video file width and height are both less than or equal to the screen texture's width and height; video file width and height are both greater than the screen texture's width and height; video file width is less than the screen texture's width and height is greater than the screen texture's height; and video file height is less than the screen texture's height, while video file width is greater than the screen texture's width. The solutions for these four categories are as follows:
[0032] a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory;
[0033] b: When the width and height of the video file are both larger than the width and height of the screen texture, if the decoder supports scaling decoding, it directly decodes the data to approximate (the decoded width and height must both be greater than or equal to the width and height of the screen texture) the required width and height, and uploads the image directly to the graphics card's video memory. If the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image (using simple or complex scaling depending on the CPU's capabilities), and upload it to the graphics card's video memory. If the decoder can only decode video images of one resolution, it first decodes at that resolution and then uses the CPU to scale the image.
[0034] Some decoders only support decoding at the original file size. For example, if the original compressed image size is 3840x2160, some decoders can only decode at 3840x2160. However, some decoders support decoding at different sizes such as 3840x216, 1920x1080, and 960x540. If the decoder only supports 3840x2160 decoding, then the decoder should decode the image to 3840x2160, and then the CPU should be used to reduce the image size before uploading the image to the graphics card's video memory.
[0035] c: When the width of the video file is less than the width of the screen texture but the height is greater than the height of the screen texture, if the decoder supports scaling decoding, it directly decodes the data with the width unchanged and the height scaled, and uploads the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the height direction (simplified or complex scaling depending on the CPU capability), and upload it to the graphics card memory.
[0036] d: When the height of the video file is less than the height of the screen texture, and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the width direction (simplified or complex scaling depending on the CPU capability), and upload it to the graphics card memory.
[0037] For image formats with complex pixel color formats (such as v210): Complex pixel color formats are also divided into four cases, which are addressed as follows:
[0038] a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory.
[0039] b: If the width and height of the video file are both greater than the width and height of the screen texture (the image size in the video file, for example, 3840x2160), if the decoder supports scaling decoding, it will directly decode the data to approximate (the decoded width and height must be greater than or equal to the width and height of the screen texture) the required width and height, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will decode the original image; if the image height is greater than or equal to twice the height of the screen texture, the image data will be copied to the graphics card memory in a row-by-row manner (taking values every N rows, where N is the image height divided by the screen texture height), otherwise the original image size data will be directly uploaded to the graphics card memory.
[0040] Some decoders only support decoding at the original file size. For example, if the original compressed image size is 3840x2160, some decoders can only decode at 3840x2160, while others support decoding at different sizes such as 3840x216, 1920x1080, and 960x540. Furthermore, the decoded image is in V210 or other formats (this format consists of multiple horizontal pixels grouped together). If the decoder only supports 3840x2160 decoding, then the decoder will decode a 3840x2160 image. If the image height is greater than or equal to twice the screen texture height, the image data is copied to the graphics card's video memory in a row-by-row manner (taking values every N rows, where N is the image height divided by the screen texture height). Otherwise, the original image size data is directly uploaded to the graphics card's video memory.
[0041] c: If the width of the video file is less than (the image size in the video file, for example, 3840x2160; if the screen texture width is 4000, then the width of the video file 3840 is less than the texture width) screen texture width, and the height of the video file is greater than (the image size in the video file, for example, 3840x2160; if the screen texture height is 1000, then the height of the video file 2160 is greater than the screen texture height 1000) screen texture height, if the decoder supports scaling decoding, it directly decodes the data with the width unchanged and the height scaled, and uploads the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder decodes the original image, and if the image height is greater than or equal to twice the screen texture height, the image data is copied to the graphics card memory in a row-by-row manner (taking values every N rows, where N is the image height divided by the screen texture height); otherwise, the original image size data is directly uploaded to the graphics card memory.
[0042] d: When the height of the video file is less than the height of the screen texture, and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will directly decode the original image and upload the image directly to the graphics card memory.
[0043] Step 5, texturing operation: The video image is retrieved from the graphics card memory, texturing is performed, and the result is calculated and rendered to create a composite output file.
[0044] After the image to be textured is transmitted to the graphics card in the manner described above, the texture is applied inside the graphics card. Since this embodiment uses the effective data volume upload mode, the upload bandwidth is greatly reduced, the upload time is effectively improved, the graphics card texture calculation workload is reduced, making multi-layer real-time editing smoother and the speed of packaging and outputting files is significantly improved.
[0045] Finally, it should be noted that the above is only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention (such as the form of the editing system, the form of the decoder, the order of steps, etc.) without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for implementing multi-layer video editing in non-linear editing, characterized in that, The steps of the method shown are as follows: Step 1, Receive Texture Task: Receive texture storyboard materials, which are divided into video materials, subtitle materials, special effects materials, and audio materials, including individual or mixed materials; Step 2, Calculate the texture range: Calculate the position range of each texture image on the screen according to the task requirements, including the width and height of the texture image; Step 3, Distinguish color formats: Read the video image parameter information to distinguish whether the image format is a single-pixel color format or a complex pixel color format; Step 4, Dynamically upload graphics card memory: Different upload strategies are adopted according to different parameter information. The strategies include two main categories: for image formats with single-pixel color formats and for image formats with complex pixel color formats. For color formats where the image format is a single pixel: a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory; b: When the width and height of the video file are both larger than the width and height of the screen texture, if the decoder supports scaling decoding, that is, directly decode the data that is approximately the required width and height, and upload the image directly to the graphics card memory; If the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image, and upload it to the graphics card memory. If the decoder can only decode video images of one resolution, it should first decode at that resolution and then use the CPU to scale the image. c: When the width of the video file is less than the width of the screen texture but the height is greater than the height of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the width unchanged and the height scaled, and upload the image directly to the graphics card memory; If the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the height direction, and upload it to the graphics card memory. d: When the height of the video file is less than the height of the screen texture and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, that is, directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; If the decoder does not support scaling decoding, the decoder needs to decode the original image, scale the image in the width direction, and upload it to the graphics card memory. For image formats with complex pixel color formats: a: If the width and height of the video file are both less than or equal to the width and height of the screen texture, then the file will be fully decoded and the image will be directly uploaded to the graphics card memory; b: If the width and height of the video file are both larger than the width and height of the screen texture, if the decoder supports scaling decoding, that is, directly decode the data that is approximately the required width and height, and upload the image directly to the graphics card memory; If the decoder does not support scaling decoding, the decoder decodes the original image; if the image height is greater than or equal to twice the screen texture height, the image data is copied to the graphics card memory in a line-by-line manner, otherwise the original image size data is directly uploaded to the graphics card memory. c: If the width of the video file is smaller than the width of the screen texture, and the height of the video file is greater than the height of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the width unchanged and the height scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will decode the original image. If the image height is greater than or equal to twice the height of the screen texture, the image data will be copied to the graphics card memory in a line-by-line manner; otherwise, the original image size data will be uploaded directly to the graphics card memory. d: When the height of the video file is less than the height of the screen texture, and the width of the video file is greater than the width of the screen texture, if the decoder supports scaling decoding, it will directly decode the data with the height unchanged and the width scaled, and upload the image directly to the graphics card memory; if the decoder does not support scaling decoding, the decoder will directly decode the original image and upload the image directly to the graphics card memory. Step 5, texturing operation: The video image is retrieved from the graphics card memory, texturing is performed, and the result is calculated and rendered to create a composite output file.