Template-based video synthesis method, device, equipment and storage medium

Abstract

The present invention relates to the field of video synthesis, and discloses a template-based video synthesis method, apparatus, device, and storage medium. The method comprises: receiving a video synthesis instruction, reading a target video corresponding to the video synthesis instruction, and matching a video rendering tag from a video database based on the target video; rendering the target video and the video rendering tag according to a preset rendering algorithm to obtain a rendered video; extracting key frames of the rendered video to obtain a verification frame set; verifying the verification frame set according to a preset rendering verification algorithm to obtain a verification result; when the verification result is qualified, analyzing the video template corresponding to the video rendering tag to obtain a hit template name; and adjusting the video rendering tag matching the target video according to the hit template name.

Description

Technical Field

[0001] The present invention relates to the field of video synthesis, and in particular to a template-based video synthesis method, device, equipment and storage medium. Background Art

[0002] The user at the front end of the terminal selects the live streaming editing service on an app or website, enters the live broadcast link in the live streaming editing function, and completes the stream. The live streaming editing function slices the live stream into segments of a specific duration. The live streaming is then pushed to the live streaming editing robot for analysis, cropping, template matching, and final video synthesis. The MongoDB database storage medium at the front end of the terminal stores live streaming service information tags. Users can initiate multiple live streaming editing services simultaneously.

[0003] The live streaming editing robot is a supporting service used in some live video editing applications, connecting the audio and video rendering application with the end-user frontend. After uploading a video template through the robot's backend, it analyzes and crops it based on rules, including animation, text, sound, and video elements. The resulting video is then exported and distributed to the end-user frontend. The live streaming editing robot requires a large selection of video templates. After receiving streaming information from the end-user frontend, the live streaming editing robot submits the sliced ​​data to an MQ queue as a task, generating a production and consumption queue record. The consumer service then calls the audio and video rendering application to complete the loop. A live streaming editing service's previous stream pull may generate multiple task records on the MQ queue. Each task corresponds to a video result that is ultimately returned to the live streaming editing service frontend. Each task retains a unique identifier in the live streaming editing robot's MongoDB database. Similarly, the video result exported by the rendering application also retains a unique video identifier and task tag in the rendering application's MongoDB database.

[0004] A video template is a set of data json files, cover images, and sample videos that anchor the baseline video material content before audio and video synthesis, calibrate the format, quantity, and location of the replacement files uploaded by the user, and are used by the underlying rendering program. Ultimately, the video template will be used as the base, and the user-uploaded materials will be replaced with the corresponding layer materials in the template, and then given to the rendering program to complete the export of the final video.

[0005] Testers need to manually upload a large number of templates to the backend of the live broadcast editing robot business. When the number is large, the template upload rate will be insufficient in the short term, resulting in the video pushed to the live broadcast editing business by the terminal front end at that time not being well matched with as many new video templates as possible. It also makes the hit rate distorted when compared with the video templates prepared in advance in the real online environment. In addition, manually uploaded video templates cannot be compared one by one to see if there are already the same video templates in the library, which can easily lead to uploading multiple templates of the same type, affecting the template matching rate calculation.

[0006] Therefore, during the editing and rendering process, there are technical problems such as inconvenient template management and insufficient template and video matching hit rate. A technology is needed to solve the current problems. Summary of the Invention

[0007] The main purpose of the present invention is to solve the technical problems of inconvenient template management and insufficient template-video matching hit rate during the editing and rendering process.

[0008] A first aspect of the present invention provides a template-based video synthesis method, the template-based video synthesis method comprising:

[0009] receiving a video synthesis instruction, reading a target video corresponding to the video synthesis instruction, and matching a video rendering tag from a video database based on the target video;

[0010] Rendering the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video;

[0011] Extracting key frames of the rendered video to obtain a verification frame set;

[0012] Performing verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result;

[0013] When the verification result is qualified, analyzing the video template corresponding to the video rendering mark to obtain the hit template name;

[0014] According to the hit template name, the video rendering mark matched by the target video is adjusted.

[0015] Optionally, in a first implementation of the first aspect of the present invention, before receiving the video synthesis instruction and reading the target video corresponding to the video synthesis instruction, the method further includes:

[0016] Receive the uploaded video template and read the sample video in the video template;

[0017] Calculate the md5 hash value of the sample video and the dHash value of each frame of the sample video;

[0018] Determine whether the md5 hash value exists in the preset md5 hash record table;

[0019] If it does not exist in the md5 hash record table, then calculate the Hamming distance between the dHash value of each frame of the sample video and the dHash value of the cover sample frame of the preset existing template to obtain a similar value set;

[0020] Determine whether there is a similarity value exceeding a preset similarity threshold in the similarity value set;

[0021] If there is no similarity value exceeding the preset similarity threshold, determining whether the data structure object file of the video template is inconsistent with the data structure object file of the preset existing template;

[0022] If the data structure object files are not consistent with the preset existing templates, the video template is confirmed as an existing template.

[0023] Optionally, in a second implementation of the first aspect of the present invention, rendering the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video includes:

[0024] Transmitting the target video and the video rendering mark to a preset rendering server so as to perform rendering processing on the target video and the video rendering mark;

[0025] Determine whether the current network status is smooth;

[0026] If not, retransmitting the target video and the video rendering mark to the preset rendering server;

[0027] If so, determining whether the target video is in a rendering state;

[0028] If it is in the rendering state, it will monitor the rendering success identifier. When the rendering success identifier is obtained, it will obtain the video URL address.

[0029] The rendered video is obtained from the video URL address.

[0030] Optionally, in a third implementation of the first aspect of the present invention, extracting key frames of the rendered video to obtain a verification frame set includes:

[0031] Based on a preset computer vision library, the first frame, the middle frame, and the last frame of the rendered video are extracted to obtain a verification frame set.

[0032] Optionally, in a fourth implementation of the first aspect of the present invention, performing verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result includes:

[0033] Based on a preset image processing database, determining whether the verification frame set has a rendering failure frame;

[0034] If there is no rendering failure frame, determining whether all the verification frames in the verification frame set meet the preset image quality standard;

[0035] If all the verification frames meet the preset image quality standards, a qualified verification result is generated.

[0036] Optionally, in a fifth implementation of the first aspect of the present invention, determining whether all verification frames in the verification frame set meet a preset image quality standard includes:

[0037] Analyzing the video frame rate, resolution, and image size of the check frames in the check frame set based on a preset computer vision library;

[0038] Determine whether the video frame rate is greater than a preset frame rate threshold, determine whether the resolution is greater than a preset resolution threshold, and determine whether the image size meets a preset limited size;

[0039] When the video frame rate is greater than a preset frame rate threshold, the resolution is greater than a preset resolution threshold, and the image size meets a preset limited size, the check frame is determined as a qualified check frame.

[0040] Optionally, in a sixth implementation of the first aspect of the present invention, adjusting the video rendering mark matched by the target video according to the hit template name includes:

[0041] Modify the hit distribution value of the preset existing template according to the hit template name;

[0042] Calculating a hit distribution rate of a preset existing template based on the hit distribution value;

[0043] An existing template with a maximum hit rate in the hit distribution rate is searched, a rendering mark corresponding to the existing template with the maximum hit rate is extracted, and the rendering mark is determined as a video rendering mark matched by the target video.

[0044] A second aspect of the present invention provides a template-based video synthesis device, the template-based video synthesis device comprising:

[0045] A matching module is configured to receive a video synthesis instruction, read a target video corresponding to the video synthesis instruction, and match a video rendering tag from a video database based on the target video;

[0046] A rendering module, configured to render the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video;

[0047] An extraction module, configured to extract key frames of the rendered video to obtain a verification frame set;

[0048] A verification module, configured to perform verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result;

[0049] An analysis module, configured to analyze the video template corresponding to the video rendering mark to obtain a hit template name when the verification result is qualified;

[0050] An adjustment module is used to adjust the video rendering mark matched by the target video according to the hit template name.

[0051] The third aspect of the present invention provides a template-based video synthesis device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the template-based video synthesis device executes the above-mentioned template-based video synthesis method.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned template-based video synthesis method.

[0053] In an embodiment of the present invention, the name of the hit template is analyzed by performing a reverse check on the synthesized rendered video, and the matching relationship between the target video and the template is automatically adjusted in reverse, so that the rendering mark matched from the video database each time will be adaptively modified, thereby achieving the effect of facilitating template management and improving the matching hit rate between the template and the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of an embodiment of a template-based video synthesis method according to an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of an embodiment of a template-based video synthesis device according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of another embodiment of a template-based video synthesis device according to an embodiment of the present invention;

[0057] Figure 4 Schematic diagram of an embodiment of a template-based video synthesis device in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The embodiments of the present invention provide a template-based video synthesis method, apparatus, device and storage medium.

[0059] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0060] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the template-based video synthesis method in the embodiment of the present invention includes:

[0061] 101. Receive a video synthesis instruction, read a target video corresponding to the video synthesis instruction, and match a video rendering tag from a video database based on the target video;

[0062] In this embodiment, a web UI automation environment is built, and Chrome, Firefox, Edge system, and various browser drivers such as Chromedriver are installed. The browser and browser driver need to have an accurate corresponding version relationship (Chromedriver controls the browser through Chrome's automatic proxy framework) to make web UI automation possible; the Python language environment is installed and configured, and then the selenium, win32gui, win32con, opencv-contrib-python, pymongo, xlutils, pytest, and tkinter libraries are installed through Python's built-in command pip, and the browser driver is placed in the Python compiler directory.

[0063] Based on python+selenium+win32gui and win32con, this solution provides stable import capabilities for video templates after batch import and analysis and filtering.

[0064] a. Encapsulate the Chrome, Firefox, and Edge browser drivers, detect whether the current operating system can run the correct browser version, and then start the browser. Otherwise, an error message will be displayed. A Tkinter-based pop-up window will prompt that the browser version is missing.

[0065] b. Enable Selenium's positioning capabilities and set the timeout to 60 seconds instead of the default 30 seconds. Set the polling frequency from the default 0.5 to 1. If the program throws a timeout error or a page non-interactive error, re-execute the program to ensure fault tolerance and robustness.

[0066] c. Check whether the template data size and file format are standard. If the size and file format exceed the standard setting, the next step will not be performed.

[0067] d. Check the template size. Based on a 3G network speed of 100 Kb / s, set time.sleep to the total template upload time + 10 seconds.

[0068] Based on Python and pymongo, tag data is collected for summary calculation. The live broadcast tag in the current streaming task is queried from the video database MongoDB database using the live broadcast business tag information from the live broadcast editing front-end. The live broadcast tag obtained from the live broadcast editing robot is queried from the MongoDB database for the triggered task tag, and tasks with a failed status when submitted to the rendering program for consumption are filtered out.

[0069] Furthermore, before step 101, you can also perform the following steps:

[0070] 1011. Receive the uploaded video template and read the sample video in the video template;

[0071] 1012. Calculate the md5 hash value of the sample video and the dHash value of each frame of the sample video;

[0072] 1013. Determine whether the md5 hash value exists in the preset md5 hash record table;

[0073] 1014. If the dHash value does not exist in the md5 hash record table, then calculate the Hamming distance between the dHash value of each frame of the sample video and the dHash value of the cover sample frame of the preset existing template to obtain a similar value set;

[0074] 1015. Determine whether there is a similarity value exceeding a preset similarity threshold in the similarity value set;

[0075] 1016. If there is no similarity value exceeding the preset similarity threshold, determining whether the data structure object file of the video template is inconsistent with the data structure object file of the preset existing template;

[0076] 1017. If the data structure object files are not consistent with the preset existing templates, the video template is confirmed as the existing template.

[0077] In steps 1011-1017, in order to prevent a large number of video templates from having the same content when being uploaded and to reduce the impact of the same template on the calculation of the template hit rate, the analysis and filtering link will conduct a detailed comparison of the uploaded sample video template materials to determine whether any materials have been uploaded. Every time a video template is successfully uploaded in the background of the live editing robot business, the similarity data information of the 3 sub-materials of this video template will be judged. Subsequent uploaded video templates will be compared with the recorded material information according to the sub-material type. If any material is judged to be the same, it will be judged as an uploaded video template and will not be uploaded again. The md5 value data of the sample video is written in a local excel file using xlwt and read out using xlrd during the comparison process. The execution process can be as follows:

[0078] a. Step 1: Get the MD5 hash value and each frame difference hash pHash record of the newly uploaded video template sample in the video template, and record the MD5 and template name in the local Excel file;

[0079] b. Determine whether the MD5 value of the newly uploaded video template sample is the same as the MD5 value of each sample video in the record;

[0080] C. If they are the same, it is determined that the video template already exists and will not be uploaded again, and no subsequent d~j will be performed;

[0081] d. If it does not exist, continue to determine the sample video and the recorded sample video frame dHash Hamming distance similarity comparison, the similarity exceeds the set value of 0.9, it is determined to be a duplicate and will not be uploaded, and no further e~j will be performed. The sample video frame that is determined to be duplicated will be returned to the result directory for record, which can facilitate the second manual verification of the program robustness in the future;

[0082] e. Record the difference hash dHash of the cover image uploaded in the video template;

[0083] f. Determine the dHash value of the newly uploaded cover image and the difference hash dHash value of the cover image of each template recorded by Hamming distance similarity comparison;

[0084] g. If the similarity exceeds the set value of 0.9, the video template is determined to already exist and will not be uploaded again;

[0085] h. Use the json module to compare the content of the json file uploaded in the video template (JSON file is a data structure object file);

[0086] i. Determine whether the content of the newly uploaded JSON file is consistent with the JSON file content of each template in the record;

[0087] j. If they are the same, the video template is determined to exist and will not be uploaded again;

[0088] k. If they are different, the uploaded video template will be confirmed as an existing template.

[0089] 102. Render the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video;

[0090] In this embodiment, the rendering process can be performed independently by itself, or data can be sent to an external server, the rendering process of the external server can be monitored, and then the rendered video can be obtained and entered into the next step.

[0091] Furthermore, at step 102 , the following steps may be performed:

[0092] 1021. Transmit the target video and the video rendering mark to a preset rendering server so as to render the target video and the video rendering mark;

[0093] 1022. Determine whether the current network status is smooth;

[0094] 1023. If the connection is not smooth, retransmit the target video and the video rendering mark to the preset rendering server;

[0095] 1024. If so, determine whether the target video is in a rendering state;

[0096] 1025. If the rendering state is in progress, monitor the rendering success identifier. When the rendering success identifier is obtained, obtain the video URL address.

[0097] 1026. Obtain the rendered video from the video URL address.

[0098] In steps 1021-1026, because the video rendering result is time-consuming and depends on the network and status, it is necessary to exclude the task error mark obtained due to network reasons and the status itself. If the rendering program returns to the live editing robot video rendering result and is in the rendering state, continue to poll this task status mark. This solution also provides a network monitoring status query during the rendering process based on the subprocess library. The current network status is queried in real time during the rendering state. When the network status code is not 200, the network status will be continuously detected. When the network status is 200, the query task mark is executed. The relationship between the live broadcast mark and the successful task mark that can finally synthesize the result video is obtained. Further, this solution will determine whether the video quality meets the preset requirements, read the video URL address returned by the rendering task, query the video template information mark reported in this task with the queried task mark, and obtain the relationship between the task and the template information mark. The obtained live broadcast mark, task mark, and template information mark relationship can obtain the relationship between the live broadcast mark and the actually used video template information mark, and then obtain all the video template information marks actually used in this live editing business.

[0099] 103. Extract key frames of the rendered video to obtain a verification frame set;

[0100] In this embodiment, image frames at fixed positions of the rendered video are extracted as key frames to obtain a verification frame set.

[0101] Furthermore, at step 103 , the following steps may be performed:

[0102] 1031. Based on a preset computer vision library, extract the first frame, the middle frame, and the last frame of the rendered video to obtain a verification frame set.

[0103] In this embodiment, a computer vision library is implemented based on the CV2 library, which is mainly used to extract key frames. This solution extracts the first frame, the middle frame, and the last frame, and determines whether the three frames of the video are black based on the PIL library. If the screen is black, the video rendering task fails and is filtered out.

[0104] 104. Perform verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result;

[0105] In this embodiment, the PIL library is used to determine whether the video frame 3 is black. If it is black, the video rendering task fails and is filtered out. Then, the CV2 library is used to verify the video frame rate, resolution, and size information. If it does not meet the preset requirements, the video rendering task is filtered out.

[0106] Furthermore, at step 104 , the following steps may be performed:

[0107] 1041. Based on a preset image processing database, determine whether the verification frame set contains any rendering failure frames;

[0108] 1042. If there is no rendering failure frame, determine whether all the verification frames in the verification frame set meet the preset image quality standard;

[0109] 1043. If all the verification frames meet the preset image quality standard, a qualified verification result is generated.

[0110] In steps 1041-1043, the PIL library, an image processing library, analyzes the first, middle, and last frames for any black screen images. If no black screen images are present, the CV2 library's video frame rate, resolution, and size information are used to verify compliance with the specified settings. Only when both checks pass are considered acceptable is the verified frame set considered qualified, generating a qualified verification result.

[0111] Furthermore, at 1042 , the following steps may be performed:

[0112] 10421. Analyze the video frame rate, resolution, and image size of the verification frames in the verification frame set based on the preset computer vision library;

[0113] 10422. Determine whether the video frame rate is greater than a preset frame rate threshold, determine whether the resolution is greater than a preset resolution threshold, and determine whether the image size meets the preset limited size;

[0114] 10423. When the video frame rate is greater than a preset frame rate threshold, the resolution is greater than a preset resolution threshold, and the image size meets the preset limited size, the check frame is determined as a qualified check frame.

[0115] In steps 10421-10423, based on the preset CV2 library, the video frame rate, resolution, and image size of each verification frame in the verification frame set are extracted. Each verification frame must meet the requirements that the video frame rate is greater than the preset frame rate threshold, the resolution is greater than the preset resolution threshold, and the image size meets the preset limited size. Only then can the entire verification frame set be determined as a qualified verification frame.

[0116] 105. When the verification result is qualified, analyze the video template corresponding to the video rendering mark to obtain the hit template name;

[0117] In this embodiment, the video template information tag reported in the video rendering tag task is queried to obtain the relationship between the task and the template information tag. Based on the relationship between the live broadcast tag, task tag, and template information tag, the relationship between the live broadcast tag and the actual video template information tag can be obtained. In turn, all video template information tags actually used in this live broadcast editing service can be obtained. All template names are queried to obtain the matching template name.

[0118] 106. Adjust the video rendering mark of the target video matching according to the hit template name.

[0119] In this embodiment, the number of times each template appears in the history is compared with the total number of templates to obtain the matching rate of each template, and the video rendering mark of the target video matching is adjusted according to the matching rate to achieve a more accurate matching solution.

[0120] Furthermore, at step 106 , the following steps may be performed:

[0121] 1061. Modify the hit distribution value of the preset existing template according to the hit template name;

[0122] 1062. Calculate the hit distribution rate of the preset existing template based on the hit distribution value;

[0123] 1063. Query the existing template with the maximum hit rate in the hit distribution rate, extract the rendering mark corresponding to the existing template with the maximum hit rate, and determine the rendering mark as the video rendering mark matching the target video.

[0124] In steps 1061-1063, the hit template is named A. The template hit value of A is increased by 1 in the statistical distribution of the existing template to obtain a new hit distribution value. The hit distribution rate of each template is calculated based on the hit distribution value. That is, the hit number of each template is divided by the total number to obtain the hit distribution rate.

[0125] Finally, the existing template with the largest probability in the hit distribution rate is determined as the matching template, and the tag data of the existing template is determined as the matching video rendering tag of the target video.

[0126] In an embodiment of the present invention, the name of the hit template is analyzed by performing a reverse check on the synthesized rendered video, and the matching relationship between the target video and the template is automatically adjusted in reverse, so that the rendering mark matched from the video database each time will be adaptively modified, thereby achieving the effect of facilitating template management and improving the matching hit rate between the template and the video.

[0127] The above describes the video synthesis method based on templates in the embodiment of the present invention. The following describes the video synthesis device based on templates in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a template-based video synthesis device includes:

[0128] The matching module 201 is configured to receive a video synthesis instruction, read a target video corresponding to the video synthesis instruction, and match a video rendering tag from a video database based on the target video;

[0129] A rendering module 202 is configured to render the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video;

[0130] An extraction module 203 is used to extract key frames of the rendered video to obtain a verification frame set;

[0131] The verification module 204 is used to perform verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result;

[0132] An analysis module 205 is configured to analyze the video template corresponding to the video rendering mark to obtain a hit template name when the verification result is qualified;

[0133] The adjustment module 206 is configured to adjust the video rendering mark matched by the target video according to the hit template name.

[0134] In an embodiment of the present invention, the name of the hit template is analyzed by performing a reverse check on the synthesized rendered video, and the matching relationship between the target video and the template is automatically adjusted in reverse, so that the rendering mark matched by the video database each time is adaptively modified, thereby achieving the effect of facilitating template management and improving the matching hit rate between the template and the video.

[0135] See also Figure 3 Another embodiment of the template-based video synthesis device in the embodiment of the present invention includes:

[0136] The matching module 201 is configured to receive a video synthesis instruction, read a target video corresponding to the video synthesis instruction, and match a video rendering tag from a video database based on the target video;

[0137] A rendering module 202 is configured to render the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video;

[0138] An extraction module 203 is used to extract key frames of the rendered video to obtain a verification frame set;

[0139] The verification module 204 is used to perform verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result;

[0140] An analysis module 205 is configured to analyze the video template corresponding to the video rendering mark to obtain a hit template name when the verification result is qualified;

[0141] The adjustment module 206 is configured to adjust the video rendering mark matched by the target video according to the hit template name.

[0142] The template-based video synthesis device further includes an upload module 207, which is specifically configured to:

[0143] Receive the uploaded video template and read the sample video in the video template;

[0144] Calculate the md5 hash value of the sample video and the dHash value of each frame of the sample video;

[0145] Determine whether the md5 hash value exists in the preset md5 hash record table;

[0146] If it does not exist in the md5 hash record table, then calculate the Hamming distance between the dHash value of each frame of the sample video and the dHash value of the cover sample frame of the preset existing template to obtain a similar value set;

[0147] Determine whether there is a similarity value exceeding a preset similarity threshold in the similarity value set;

[0148] If there is no similarity value exceeding the preset similarity threshold, determining whether the data structure object file of the video template is inconsistent with the data structure object file of the preset existing template;

[0149] If the data structure object files are not consistent with the preset existing templates, the video template is confirmed as an existing template.

[0150] The rendering module 202 is specifically configured to:

[0151] Transmitting the target video and the video rendering mark to a preset rendering server so as to perform rendering processing on the target video and the video rendering mark;

[0152] Determine whether the current network status is smooth;

[0153] If not, retransmitting the target video and the video rendering mark to the preset rendering server;

[0154] If so, determining whether the target video is in a rendering state;

[0155] If it is in the rendering state, it will monitor the rendering success identifier. When the rendering success identifier is obtained, it will obtain the video URL address.

[0156] The rendered video is obtained from the video URL address.

[0157] The extraction module 203 is specifically configured to:

[0158] Based on a preset computer vision library, the first frame, the middle frame, and the last frame of the rendered video are extracted to obtain a verification frame set.

[0159] The verification module 204 is specifically configured to:

[0160] Based on a preset image processing database, determining whether the verification frame set has a rendering failure frame;

[0161] If there is no rendering failure frame, determining whether all the verification frames in the verification frame set meet the preset image quality standard;

[0162] If all the verification frames meet the preset image quality standards, a qualified verification result is generated.

[0163] The verification module 204 may also be specifically configured to:

[0164] Analyzing the video frame rate, resolution, and image size of the check frames in the check frame set based on a preset computer vision library;

[0165] Determine whether the video frame rate is greater than a preset frame rate threshold, determine whether the resolution is greater than a preset resolution threshold, and determine whether the image size meets a preset limited size;

[0166] When the video frame rate is greater than a preset frame rate threshold, the resolution is greater than a preset resolution threshold, and the image size meets a preset limited size, the check frame is determined as a qualified check frame.

[0167] The adjustment module 206 is specifically configured to:

[0168] Modify the hit distribution value of the preset existing template according to the hit template name;

[0169] Calculating a hit distribution rate of a preset existing template based on the hit distribution value;

[0170] An existing template with a maximum hit rate in the hit distribution rate is searched, a rendering mark corresponding to the existing template with the maximum hit rate is extracted, and the rendering mark is determined as a video rendering mark matched by the target video.

[0171] In an embodiment of the present invention, the name of the hit template is analyzed by performing a reverse check on the synthesized rendered video, and the matching relationship between the target video and the template is automatically adjusted in reverse, so that the rendering mark matched from the video database each time will be adaptively modified, thereby achieving the effect of facilitating template management and improving the matching hit rate between the template and the video.

[0172] above Figure 2 and Figure 3 The template-based video synthesis apparatus in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The template-based video synthesis device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0173] Figure 4 1 is a schematic structural diagram of a template-based video synthesis device provided by an embodiment of the present invention. The template-based video synthesis device 400 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 410 (for example, one or more processors) and a memory 420, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 433 or data 432. The memory 420 and the storage medium 430 may be temporary storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the template-based video synthesis device 400. Furthermore, the processor 410 may be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the template-based video synthesis device 400.

[0174] The template-based video synthesis device 400 may further include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input and output interfaces 460, and / or one or more operating systems 431, such as Windows Server, Mac OS X, Unix, Linux, Free BSD, etc. It will be appreciated by those skilled in the art that Figure 4 The illustrated structure of the template-based video synthesis device does not limit the template-based video synthesis device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0175] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the steps of the template-based video synthesis method.

[0176] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0178] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A template-based video synthesis method, characterized in that: Including steps: receiving a video synthesis instruction, reading a target video corresponding to the video synthesis instruction, and matching a video rendering tag from a video database based on the target video; Rendering the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video; Extracting key frames of the rendered video to obtain a verification frame set; Performing verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result; When the verification result is qualified, analyzing the video template corresponding to the video rendering mark to obtain the hit template name; Adjusting the video rendering mark of the target video match according to the hit template name; The step of adjusting the video rendering mark matched by the target video according to the hit template name includes: Modify the hit distribution value of the preset existing template according to the hit template name; Calculating a hit distribution rate of a preset existing template based on the hit distribution value; A preset existing template with a maximum hit rate in the hit distribution rate is queried, a rendering mark corresponding to the preset existing template with the maximum hit rate is extracted, and the target video is re-rendered according to the rendering mark to generate a re-rendered target video corresponding to the rendering mark.

2. The template-based video synthesis method according to claim 1, characterized in that: Before receiving the video synthesis instruction and reading the target video corresponding to the video synthesis instruction, the method further includes: Receive the uploaded video template and read the sample video in the video template; Calculate the md5 hash value of the sample video and the dHash value of each frame of the sample video; Determine whether the md5 hash value exists in the preset md5 hash record table; If it does not exist in the preset md5 hash record table, then calculate the Hamming distance between the dHash value of each frame of the sample video and the dHash value of the cover sample frame of the preset existing template to obtain a similarity value set; Determine whether there is a similarity value exceeding a preset similarity threshold in the similarity value set; If there is no similarity value exceeding the preset similarity threshold, determining whether the data structure object file of the video template is inconsistent with the data structure object file of the preset existing template; If the data structure object files are not consistent with the preset existing templates, the video template is confirmed as an existing template.

3. The template-based video synthesis method according to claim 1, characterized in that: The rendering process of the target video and the video rendering mark according to the preset rendering algorithm to obtain the rendered video includes: Transmitting the target video and the video rendering mark to a preset rendering server so as to perform rendering processing on the target video and the video rendering mark; Determine whether the current network status is smooth; If not, retransmitting the target video and the video rendering mark to the preset rendering server; If so, determining whether the target video is in a rendering state; If it is in the rendering state, it will monitor the rendering success identifier. When the rendering success identifier is obtained, it will obtain the video URL address. The rendered video is obtained from the video URL address.

4. The template-based video synthesis method according to claim 1, characterized in that: Extracting the key frames of the rendered video to obtain a verification frame set includes: Based on a preset computer vision library, the first frame, the middle frame, and the last frame of the rendered video are extracted to obtain a verification frame set.

5. The template-based video synthesis method according to claim 1, characterized in that: The verification process is performed on the verification frame set according to the preset rendering verification algorithm to obtain the verification result, which includes: Based on a preset image processing database, determining whether there is a rendering failure frame in the verification frame set; If there is no rendering failure frame, determining whether all the verification frames in the verification frame set meet the preset image quality standard; If all the verification frames meet the preset image quality standards, a qualified verification result is generated.

6. The template-based video synthesis method according to claim 5, characterized in that: Determining whether all the check frames in the check frame set meet the preset image quality standard includes: Analyzing the video frame rate, resolution, and image size of the check frames in the check frame set based on a preset computer vision library; Determine whether the video frame rate is greater than a preset frame rate threshold, determine whether the resolution is greater than a preset resolution threshold, and determine whether the image size meets a preset limited size; When the video frame rate is greater than a preset frame rate threshold, the resolution is greater than a preset resolution threshold, and the image size meets a preset limited size, the check frame is determined as a qualified check frame.

7. A video synthesis device based on template, characterized in that: The template-based video synthesis device includes: A matching module is configured to receive a video synthesis instruction, read a target video corresponding to the video synthesis instruction, and match a video rendering tag from a video database based on the target video; A rendering module, configured to render the target video and the video rendering mark according to a preset rendering algorithm to obtain a rendered video; An extraction module, configured to extract key frames of the rendered video to obtain a verification frame set; A verification module, configured to perform verification processing on the verification frame set according to a preset rendering verification algorithm to obtain a verification result; An analysis module, configured to analyze the video template corresponding to the video rendering mark to obtain a hit template name when the verification result is qualified; An adjustment module, configured to adjust a video rendering mark matched by the target video according to the hit template name; The adjustment module is specifically used for: Modify the hit distribution value of the preset existing template according to the hit template name; Calculating a hit distribution rate of a preset existing template based on the hit distribution value; A preset existing template with a maximum hit rate in the hit distribution rate is queried, a rendering mark corresponding to the preset existing template with the maximum hit rate is extracted, and the target video is re-rendered according to the rendering mark to generate a re-rendered target video corresponding to the rendering mark.

8. A video synthesis device based on a template, characterized in that: The template-based video synthesis device includes: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the template-based video synthesis device to execute the template-based video synthesis method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the template-based video synthesis method according to any one of claims 1 to 6 is implemented.

Images