Video image synthesizing and editing method and system

Abstract

The invention discloses a video image synthesizing and editing method and system, and the method comprises the following steps: obtaining a first material which is an image containing an alpha channel or a video with a video frame containing an alpha channel; obtaining a second material, wherein the second material is an image or a video; detecting a transparent area of a first material according to the alpha channel; adjusting the size of the second material according to the information of the transparent area; and taking the second material of which the size is adjusted as an embedding material, embedding the embedding material into the transparent area of the first material, and storing the synthesized video. The system can automatically adjust various parameters in the synthesis and editing process, reduces tedious operation of manual adjustment, improves the intelligent level of processing, achieves efficient synthesis of multiple videos, improves the efficiency and quality of video and image manufacturing, and has high applicability.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a video image synthesis and editing method and system. Background Art

[0002] Multimedia video synthesis technology has been widely used in different industries. Current video synthesis technology usually simply synthesizes two different videos, and cannot flexibly control the position and time of the synthesized or embedded video, and the synthesized video shows an unrealistic and rigid splicing effect. For example, if you rely on professional video editing software (such as Adobe Premiere, Final Cut Pro, etc.), manual operation is required to adjust the position, size and transparency of the video. Manual operation is not only time-consuming and laborious, but also requires high professional skills of the operator, and it is difficult to achieve large-scale and fast video processing. Some video editing software provides partially automated functions, such as automatic editing and filter application, but these functions still require users to perform more manual adjustments and parameter settings.

[0003] The Chinese invention patent application with publication number CN110830736A discloses a method, electronic device and computer-readable storage medium for video synthesis. The method includes: obtaining a first video frame from a first video and including a first area to be embedded; obtaining a first frame to be embedded from a second video to be embedded in the first video; obtaining a first mask-processed frame of a mask-processed video; and based on fusion configuration information specific to the first mask-processed frame, the first frame to be embedded is fused to the first video frame in the first area to be embedded by using the first mask-processed frame as a mask. This solution solves the problem of controlling the time and position of the video embedded after synthesis, and solves the fusion edge processing by introducing a transparent channel mask video solution. However, in large-scale automated processing, there are technical problems such as low processing efficiency and insufficient applicability. Summary of the invention

[0004] The present invention provides a video image synthesis and editing method and system, aiming to solve the technical problems of low processing efficiency and insufficient applicability when processing video image synthesis in large quantities automatically.

[0005] To solve the above technical problems, the synthesis editing method proposed by the present invention comprises the following steps:

[0006] Acquire a first material, where the first material is an image containing an alpha channel, or a video containing an alpha channel in a video frame;

[0007] Acquire a second material, where the second material is an image or a video;

[0008] Detecting a transparent area of ​​the first material according to the alpha channel;

[0009] Adjusting the size of the second material, the adjustment being based on the information of the transparent area;

[0010] The resized second material is used as an embedded material, and the embedded material is embedded into the transparent area of ​​the first material, and the synthesized video is saved.

[0011] Preferably, when the first material is an image containing an alpha channel, the transparent area detection method is:

[0012] Separate the alpha channel of the first material, identify and record pixel positions with values ​​of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and the y-coordinate in all records, and calculate the transparent area according to the extracted four extreme values.

[0013] Preferably, when the first material is a video containing an alpha channel in the video frame, the transparent area detection method is:

[0014] Read the video file and extract the first frame of the video, separate the alpha channel of the first frame of the video, identify and record the pixels with a value of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and y-coordinate in all records, and calculate the transparent area based on the four extracted extreme values.

[0015] Preferably, the method for adjusting the size of the second material is specifically:

[0016] The target size data is set according to the size data of the transparent area in the first material, the size data of the second material is obtained, and the size of the second material is adjusted according to the ratio of the size data of the second material and the target size data.

[0017] Preferably, the method for adjusting the size of the second material is specifically:

[0018] Calculate and compare the aspect ratios of the transparent area in the first material and the second material respectively. If the aspect ratios are different, reacquire the second material until the acquired second material has the same aspect ratio as the transparent area. Set target size data according to the size data of the transparent area in the first material, acquire the size data of the second material, and adjust the size of the second material according to the ratio of the size data of the second material to the target size data.

[0019] Preferably, the target size is the same as the size of the transparent area in the first material.

[0020] Preferably, the target size is smaller than the size of the transparent area in the first material, and after the embedded material is embedded into the first material, transition pixels are reserved between the periphery of the embedded material and the first material.

[0021] The embedding material is embedded into the transparent area of ​​the first material, and the specific method is as follows:

[0022] Read the first material and the embedded material, if the embedded material is an image, set the duration of the image clip to the duration of the first material, if the embedded material is a video, adjust the duration of the embedded material by time alignment;

[0023] The position of the embedded material in the first material is adjusted according to the position of the transparent area, and the video and the image are synthesized.

[0024] Preferably, the time alignment uses one or more of the following methods:

[0025] Speed ​​up or slow down the playback speed of one of the videos so that the two videos have the same length;

[0026] Loop playback: the short video will be played in a loop until the long video is finished;

[0027] Add a still picture. After the short video ends, add a still picture or a black screen to keep the long video playing.

[0028] Modify the frame rate to adjust the duration of either video, or adjust the frame rate of both videos simultaneously to make them the same duration.

[0029] Accordingly, the present invention also provides a video image synthesis and editing system, wherein the system implements the above synthesis and editing method, including:

[0030] A material reading module, used to obtain the first material and the second material;

[0031] A transparent area detection module, detecting a transparent area of ​​the first material according to an alpha channel;

[0032] A material processing module, adjusting the size of the second material according to the information of the transparent area and the information of the second material, and generating an embedded material;

[0033] The synthesis and editing module uses the resized second material as an embedded material, embeds the embedded material into the transparent area of ​​the first material, and saves the synthesized video. The synthesis and editing module also edits the synthesized video according to set parameters.

[0034] Compared with the prior art, the present invention has the following technical effects:

[0035] 1. The synthetic editing method proposed in the present invention supports batch processing of multiple video and image files by automatically identifying transparent areas, can process a large amount of multimedia content at the same time, can achieve efficient synthesis of multiple videos, and improve the efficiency of video production; and improves the applicability of the synthetic editing method through dynamic adjustment of materials.

[0036] 2. The synthetic editing method proposed in the present invention can accurately identify the boundaries of transparent areas by automatically detecting transparent areas in images and videos, thereby ensuring seamless fusion of videos and images and improving the quality of video synthesis.

[0037] 3. The synthesis editing method proposed in the present invention adjusts the position and size of the video or image according to the transparent area information and the synthesis object information, so that it can be naturally embedded in the image or other videos, thereby improving the naturalness and aesthetics of the synthesis effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic flow chart of the synthesis editing method of the present invention;

[0039] Figure 2 is a schematic diagram of a rectangular transparent channel according to an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of an irregular transparent channel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments of the present application and with reference to the accompanying drawings.

[0042] Embodiment 1

[0043] This embodiment is a video image synthesis editing method. Figure 1 As shown, the steps include steps one to five:

[0044] Step 1, obtain the first material, which is an image containing an alpha channel, or a video containing an alpha channel in a video frame. In this step, to ensure that the image or video frame needs to be loaded in an appropriate format so that the transparency information therein can be accessed. Common formats include RGBA (red, green, blue and transparency) or LA (brightness and transparency) with a transparent channel. In one embodiment of the present invention, Python's PIL (Pillow) library can be used to open an image file or video frame to check the image mode, and check whether the image.mode attribute contains a transparent channel. If the image mode is RGBA or LA, it indicates that the image has transparency information.

[0045] Step 2: Obtain a second material, which is an image or video. In this embodiment, the second material is an image or video that is embedded in a set area of ​​the first material and played synchronously with the first material.

[0046] Step three, detect the transparent area of ​​the first material according to the alpha channel. For the first material obtained in step one, if the image contains transparency information (Alpha channel / alpha channel / α channel), the next step is to separate this channel. The alpha channel is a grayscale image, in which the value of each pixel represents its transparency, 0 means completely transparent, and 255 means completely opaque. Using the extracted alpha channel, the boundary of the transparent area is determined by analyzing the position of the transparent pixels. The value of the transparent pixel is 0, and the value of the non-transparent pixel is greater than 0. In one embodiment of the present invention, after the image returns the alpha channel, it is converted into a NumPy array through np.array(), and the minimum and maximum coordinates of these two-dimensional arrays, that is, the boundary of the transparent area, are calculated.

[0047] When the first material is an image containing an alpha channel, the transparent area detection method is:

[0048] Separate the alpha channel of the first material, identify and record pixel positions with values ​​of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and the y-coordinate in all records, and calculate the transparent area according to the extracted four extreme values.

[0049] When the first material is a video containing an alpha channel in the video frame, the transparent area detection method is:

[0050] Read the video file and extract the first frame of the video, separate the alpha channel of the first frame of the video, identify and record the pixels with a value of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and y-coordinate in all records, and calculate the transparent area based on the four extracted extreme values.

[0051] like Figure 2 As shown in the figure, it is a schematic diagram of a rectangular transparent channel in this embodiment. In this figure, the area with a value of 0 in the transparent channel forms a complete rectangle. The outermost rectangle is the first material loaded, in which the blank rectangle ABCD is the rectangular transparent area in the first material. If the lower left corner of the first material is taken as the coordinate origin, then for the transparent area, point A has the minimum coordinates min_x and min_y on the x-axis and y-axis; correspondingly, the maximum coordinates max_x and max_y on the x-axis and y-axis exist at point C; taking points A and C as the diagonal points of the rectangle, the rectangle formed is the transparent area in this embodiment.

[0052] like Figure 3As shown in FIG. 1 , it is a schematic diagram of an irregular transparent channel in this embodiment. In this figure, the shape of the area with a value of 0 in the transparent channel is a figure surrounded by the irregular curve shown in the figure. If the lower left corner of the first material is taken as the origin of the coordinates, min_x is located at point E, min_y is located at point H, max_x is located at point G, and max_y is located at point F, the four extreme points can determine that the coordinates (min_x, min_y) are located at point A, and the coordinates (max_x, max_y) are located at point C. Point A and point C are taken as the diagonal points of the rectangle, and the rectangle formed is the transparent area in this embodiment.

[0053] Step 4: adjust the size of the second material based on the information of the transparent area.

[0054] In one embodiment of the present invention, the resizing operation is performed without considering the aspect ratio parameter of the operation object. If the aspect ratio of the operation object is different, the second material will be automatically stretched when the size of the second material is adjusted, but the style after stretching will be problematic. It is up to the user to decide whether it will affect the video effect after editing. The resizing method of the second material is specifically as follows:

[0055] The target size data is set according to the size data of the transparent area in the first material, the size data of the second material is obtained, and the size of the second material is adjusted according to the ratio of the size data of the second material and the target size data.

[0056] In another embodiment of the present invention, considering the difference in display effects caused by stretching or reducing the second material to different aspect ratios, the aspect ratio parameter of the operation object is considered when performing the resizing operation. If the aspect ratio of the operation object is different, the second material is re-acquired until the aspect ratio of the second material is consistent with that of the transparent area. The resizing method of the second material is specifically as follows:

[0057] Calculate and compare the aspect ratios of the transparent area in the first material and the second material respectively. If the aspect ratios are different, reacquire the second material until the acquired second material has the same aspect ratio as the transparent area. Set target size data according to the size data of the transparent area in the first material, acquire the size data of the second material, and adjust the size of the second material according to the ratio of the size data of the second material to the target size data.

[0058] According to the embedding requirements, if the embedded material needs to be embedded into the transparent area, and the embedded material and the transparent area completely overlap after embedding, the second material size parameter can be set to the same parameter as the size of the transparent area. In this way, there is no border between the embedded material and the first material, showing a completely fitting effect.

[0059] In other embodiments of the present invention, the target size is smaller than the size of the transparent area in the first material. After the embedded material is embedded into the first material, transition pixels are reserved between the periphery of the embedded material and the first material. Compared with the size parameters of the transparent area in the first material, the width and height of the target size are reduced by 10px respectively. In this way, after the embedded material is embedded into the transparent area of ​​the first material in a center-aligned manner, there are 5px transition pixels around the embedded material and the first material. The transition pixels can be set as a decorative display or a transitional display effect as needed. The size of the transition area can be set as needed.

[0060] Step 5: Use the resized second material as the embedded material, embed the embedded material into the transparent area of ​​the first material, and save the synthesized video.

[0061] The embedding material is embedded into the transparent area of ​​the first material, and the specific method is as follows:

[0062] Read the first material and the embedded material, if the embedded material is an image, set the duration of the image clip to the duration of the first material, if the embedded material is a video, adjust the duration of the embedded material by time alignment;

[0063] Adjust the position of the embedded material in the first material according to the position of the transparent area, and synthesize the video and image. For the synthesis of video clips, use the CompositeVideoClip() method to synthesize the video clips and image clips into one material. In this step, the set_position() method can be used to specify the position of the image. For example, the parameters ("left", "top") place the embedded material in the upper left corner, while setting the parameters ("center", "center") places the embedded material in the center. The two parameters in the set_position() method can also be set to the coordinate values ​​of the corresponding position of the transparent area in the first material as needed.

[0064] Specifically, the time alignment uses one or more of the following methods:

[0065] Speed ​​up or slow down the playback speed of one of the videos so that the two videos have the same length;

[0066] Loop playback: the short video will be played in a loop until the long video is finished;

[0067] Add a still picture. After the short video ends, add a still picture or a black screen to keep the long video playing.

[0068] Modify the frame rate to adjust the duration of either video, or adjust the frame rate of both videos simultaneously to make them the same duration.

[0069] Embodiment 2

[0070] This embodiment is a video image synthesis and editing system, which implements the synthesis and editing method as described in the first embodiment, including:

[0071] A material reading module, used to obtain the first material and the second material;

[0072] A transparent area detection module, detecting a transparent area of ​​the first material according to an alpha channel;

[0073] A material processing module, adjusting the size of the second material according to the information of the transparent area and the information of the second material, and generating an embedded material;

[0074] The synthesis and editing module uses the resized second material as an embedded material, embeds the embedded material into the transparent area of ​​the first material, and saves the synthesized video. The synthesis and editing module also edits the synthesized video according to set parameters.

[0075] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, which all fall within the protection scope of the present invention.

Claims

1. A video image synthesis and editing method, characterized in that: The following steps are involved: Acquire a first material, where the first material is an image containing an alpha channel, or a video containing an alpha channel in a video frame; Acquire a second material, where the second material is an image or a video; Detecting a transparent area of ​​the first material according to the alpha channel; Adjusting the size of the second material, the adjustment being based on the information of the transparent area; The resized second material is used as an embedded material, and the embedded material is embedded into the transparent area of ​​the first material, and the synthesized video is saved.

2. A video image synthesis and editing method according to claim 1, characterized in that: When the first material is an image containing an alpha channel, the transparent area detection method is: Separate the alpha channel of the first material, identify and record pixel positions with values ​​of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and the y-coordinate in all records, and calculate the transparent area according to the extracted four extreme values.

3. A video image synthesis and editing method according to claim 1, characterized in that: When the first material is a video containing an alpha channel in the video frame, the transparent area detection method is: Read the video file and extract the first frame of the video, separate the alpha channel of the first frame of the video, identify and record the pixels with a value of 0 in the alpha channel, extract the minimum and maximum values ​​of the x-coordinate and y-coordinate in all records, and calculate the transparent area based on the four extracted extreme values.

4. The video image synthesis and editing method according to claim 1, characterized in that: The method for adjusting the size of the second material is specifically as follows: The target size data is set according to the size data of the transparent area in the first material, the size data of the second material is obtained, and the size of the second material is adjusted according to the ratio of the size data of the second material and the target size data.

5. The video image synthesis and editing method according to claim 1, characterized in that: The method for adjusting the size of the second material is specifically as follows: Calculate and compare the aspect ratios of the transparent area in the first material and the second material respectively. If the aspect ratios are different, reacquire the second material until the acquired second material has the same aspect ratio as the transparent area. Set target size data according to the size data of the transparent area in the first material, acquire the size data of the second material, and adjust the size of the second material according to the ratio of the size data of the second material to the target size data.

6. A video image synthesis and editing method according to claim 4 or 5, characterized in that: The target size is the same as the size of the transparent area in the first material.

7. A video image synthesis and editing method according to claim 4 or 5, characterized in that: The target size is smaller than the size of the transparent area in the first material. After the embedded material is embedded into the first material, transition pixels are reserved between the periphery of the embedded material and the first material.

8. The video image synthesis and editing method according to claim 1, characterized in that: The embedding material is embedded into the transparent area of ​​the first material, and the specific method is as follows: Read the first material and the embedded material, if the embedded material is an image, set the duration of the image clip to the duration of the first material, if the embedded material is a video, adjust the duration of the embedded material by time alignment; The position of the embedded material in the first material is adjusted according to the position of the transparent area, and the video and the image are synthesized.

9. The video image synthesis and editing method according to claim 1, characterized in that: The time alignment uses one or more of the following methods: Speed ​​up or slow down the playback speed of one of the videos so that the two videos have the same length; Loop playback: the short video will be played in a loop until the long video is finished; Add a still picture. After the short video ends, add a still picture or a black screen to keep the long video playing. Modify the frame rate to adjust the duration of either video, or adjust the frame rate of both videos simultaneously to make them the same duration.

10. A video image synthesis and editing system, characterized in that: The system implements the synthetic editing method according to any one of claims 1 to 9, comprising: A material reading module, used to obtain the first material and the second material; A transparent area detection module, detecting a transparent area of ​​the first material according to an alpha channel; A material processing module, adjusting the size of the second material according to the information of the transparent area and the information of the second material, and generating an embedded material; The synthesis and editing module uses the resized second material as an embedded material, embeds the embedded material into the transparent area of ​​the first material, and saves the synthesized video. The synthesis and editing module also edits the synthesized video according to set parameters.

Images