A short video clip effect evaluation method based on multi-modal fusion

Abstract

The application discloses a short video clip effect evaluation method based on multi-modal fusion, and relates to the technical field of video clip. The application realizes real-time collection of short video materials uploaded by a user, determines corresponding specification types for the short video materials, searches a corresponding popular material library according to the specification types to check whether there is a popular material matched with the short video materials, directly calls the popular material to input into an intelligent video clip model for clip if the popular material exists, effectively meets the actual demand of the user for quickly creating high-quality popular short videos, reduces the operation load of the intelligent video clip model, and greatly improves the clip efficiency of the user, and adjusts and modifies various specification parameter values of the short video materials through preset unified processing steps, so that the short video materials are adapted to the standard format of the intelligent video clip model, and then the whole intelligent clip process is successfully completed.

Description

Technical Field

[0001] This invention relates to the field of video editing technology, specifically to a method for evaluating the editing effect of short videos based on multimodal fusion. Background Technology

[0002] With the rapid development of the short video industry, the threshold for short video creation has continued to decrease, and various short video editing tools and related technologies have emerged. The evaluation of short video editing effects has gradually become a key link in improving the quality of creation and optimizing user experience. At present, there are a variety of short video editing models in existing technologies. These models integrate the dual core functions of editing and effect evaluation. They can not only perform intelligent editing processing on the input video materials, but also conduct effect evaluation on the finished video after editing. By analyzing single or a few dimensions of indicators such as the smoothness of the video, the rationality of the rhythm, and the completeness of the content, they can give preliminary editing effect evaluation results, provide users with simple optimization references, and thus meet the basic needs of short video creation and evaluation to a certain extent. From the perspective of the user group of short video editing, the main users of short video editing are ordinary creators, novice bloggers, and self-media enthusiasts. Most of these users do not have professional video editing skills and material processing capabilities. Their core needs are to quickly and conveniently produce short video content that meets the platform's dissemination requirements, rather than investing a lot of time in professional material polishing and editing optimization. Due to their own abilities and creative needs, the editing materials used by these users mostly come from publicly available film and television works, variety show clips, and trending short videos on the Internet. The material acquisition channels are diverse, covering various methods such as platform downloads, screen recordings, and sharing by others. This directly leads to the dispersion and irregularity of material sources, resulting in inconsistent specifications and parameters of various materials. In practical use, users need to input short video materials into the short video editing model for processing. However, due to the diverse ways ordinary users obtain materials, even the same film and television materials will have significantly different specifications when different users submit them to the editing model. Some materials also have problems such as insufficient clarity and compression distortion. At the same time, short video creation has a strong trend-driven nature. When a certain video scene, film clip, or topic becomes a hot topic in a short period of time, a large number of users will choose to edit and create based on it. This means that short video editing materials have significant short-term reusability. Efficient material reuse can not only greatly improve the user's editing efficiency, but also effectively reduce the computational load of the model. However, existing technologies do not pay attention to the popularity attribute of materials and have not established a material collection and reuse system based on video popularity. They cannot actively collect and standardize the specifications of currently popular video materials. Therefore, when multiple users submit the same or similar popular materials one after another, the editing model still needs to repeatedly adjust the specifications and edit the original materials submitted by each user. It cannot directly call the collected and optimized materials for replacement. This not only greatly increases the computational load of the model and reduces the editing efficiency of users, but also causes the video effects edited by different users based on the same popular materials to be inconsistent due to repeated processing of unoptimized original materials. As a result, the evaluation results of the editing effect fluctuate greatly, and ultimately cannot meet the actual needs of users to quickly create high-quality popular short videos. To address the above problems, this invention proposes a solution. Summary of the Invention

[0003] The purpose of this invention is to provide a method for evaluating the editing effect of short videos based on multimodal fusion, in order to solve the problems mentioned in the background art.

[0004] This invention provides a method for evaluating the editing effect of short videos based on multimodal fusion, comprising the following steps: S1: Receive short video materials uploaded by users in real time, extract several specification parameters of the short video materials, and determine the specification type of the short video materials based on them; S2: Based on the specification type of the short video material, search the popular material library of the specification type to see if there is a popular material that matches the short video material. Based on the search results, select one from the reused material and the standard material of the short video material as the editing object of the short video material. The reused material refers to the popular material that matches the short video material, and the standard material refers to the short video material after performing the preset unification processing steps. S3: Input the editing object of the short video material into the trained intelligent video editing model, obtain the edited video and editing effect score of the editing object output by the intelligent video editing model, and display it to the user for preview; S4: Obtain the editing effect score of the editing object, obtain the specification correction data and processing time of the short video material of the editing object under all corresponding processing methods by performing the corresponding unified processing steps, generate the editing record data of the editing object and store it. S5: At each analysis cycle, the editing record data of all editing objects stored in the analysis cycle is analyzed, and based on the analysis results, all popular materials to be added to the popular material library of each specification type in the analysis cycle are determined and added to the corresponding popular material library.

[0005] Furthermore, in step S1, the administrator has preset several specification types, and each specification type has a preset matching range corresponding to several specification parameters. For the short video material, based on all the extracted specification parameters of the short video material, the matching range corresponding to each specification parameter is determined one by one. Then, based on the belonging of the matching range corresponding to each specification parameter, the specification type corresponding to the short video material is finally determined.

[0006] Furthermore, in step S2, each specification type has a number of popular materials stored in its corresponding popular material library, and each popular material has a priority and a number of reuses; there are three priority settings, namely first priority, second priority and third priority.

[0007] Furthermore, in step S2, the specific content of performing the preset standardization processing steps on the short video material is as follows: S21: Based on the specification type of the short video material, obtain the execution strategy pre-configured by the administrator for the specification type. The execution strategy includes several processing methods and the adaptation weights corresponding to each processing method. The adaptation weights are preset by the administrator based on the processing effect and resource consumption of the corresponding processing method. The larger the value of the adaptation weight, the earlier the corresponding processing scheme is used in the execution order. After the short video material is processed using any processing method, the values ​​of several specification parameters of the short video material will change. S22: Extract all processing methods contained in the execution strategy. Based on the adaptation weight of each extracted processing method in the execution strategy, label all extracted processing methods as C1, C2, ..., Cc in descending order of their adaptation weight values, where c ≥ 1; S23: Use processing method C1 to perform specification processing on the short video material to obtain intermediate processing data of the short video material under processing method C1, and at the same time obtain the specification correction data and processing time of the short video material under processing method C1. The specification correction data includes the specification correction amount of all specification parameters in the short video material, wherein the specification correction amount of any specification parameter is the absolute value of the difference between the value of the specification parameter of the short video material before and after specification processing using processing method C1. S24: Use processing method C2 to perform specification processing on the intermediate processing data of the short video material under processing method C1, to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing method C2, and so on to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing methods C3, C4, ..., Cc. S25: The intermediate processing data of the short video material under processing method Cc is labeled as the specification material of the short video material.

[0008] Furthermore, in S4, editing record data is only generated when the editing object is a standard clip; if the editing object is a popular clip, its editing record data is not generated.

[0009] Furthermore, the following content is analyzed regarding the clip record data of all clip objects stored during the analysis period: S51: Traverse the editing record data of all editing objects stored in the analysis period, obtain the specification type of all editing objects and remove duplicates, and mark all remaining specification types after deduplication as D1, D2, ..., Dd, where d≥1; S52: Mark the clip record data of all clip objects with specification type D1 in the clip record data of all clip objects stored in the analysis period as E1, E2, ..., Ee, respectively, e≥1; S53: Based on the clip objects corresponding to the clip record data E1, E2, ..., Ee, select several clip objects and add them to an empty set to generate a set of clip association sets; S54: Based on the method of S53, filter all remaining clip objects that have not yet been added to any clip association set among the clip objects corresponding to the clip record data E1, E2, ..., Ee, and add several clip objects that meet the conditions to the empty set to obtain multiple clip association sets; All the group clip association sets obtained based on the clip objects corresponding to the clip record data E1, E2, ..., Ee are labeled as G1, G2, ..., Gg, respectively, where 1≤g≤e; S55: Based on the time decay characteristics of short video hotspots and combined with the duration of the analysis period, the analysis period is divided into several sub-periods of equal duration for heat monitoring; the three sub-periods of heat monitoring are marked as F1, F2, and F3 in chronological order. S56: Calculate and obtain the clip association heat value J1 of clip association combination G1 according to the preset calculation rules: S57: Calculate and obtain the overall optimization completion degree N1 of the clip association group G1 according to the preset calculation rules; S58: Calculate the comprehensive entry score Q1 of the editing association group G1 using the formula Q1=J1×λ1+N1×λ2. Compare Q1 and Q. If Q1≥Q, it is determined that the editing association group G1 meets the conditions for being added to the popular material library. Otherwise, it is determined that the editing association group G1 does not meet the conditions for being added to the popular material library and will not be included in the update scope of this popular material library. Q is the preset entry screening threshold, and λ1 and λ2 are the preset first and second proportion coefficients, respectively. S59: Determine whether the editing association groups G2, G3, ..., Gg meet the conditions for entering the popular material library according to S54 to S58, and re-mark all editing association groups that meet the conditions for entering the popular material library as R1, R2, ..., Rr, 1≤r≤g; S510: Select a clip object from the clip association group R1 according to the preset filtering rules and use it as a popular material in the popular material library with specification type D1 in the clip association group R1; S511: Following S510, select popular materials from the editing association groups R2, R3, ..., Rr as the popular material library with specification type D1 for editing association groups R2, R3, ..., Rr, and add them to the popular material library; S512: Determine all popular materials in the popular material library with specification types D2, D3, ..., Dd in sequence according to S51 to S511, and add them to the corresponding popular material library.

[0010] Furthermore, the feature is that, in S56, the calculation rule for obtaining the clip association heat value J1 of the clip association combination G1 is as follows: S561: Using the formula Calculate the heat trend coefficient I1 of the clip association combination G1, where H1, H2, and H3 are the total number of clip objects in the heat monitoring sub-cycles F1, F2, and F3, respectively, at the time when the clip record data is generated within the clip association combination G1. S562: Combine the preset time weights of each heat monitoring sub-cycle to calculate the editing association heat value J1 of the editing association combination G1. The calculation formula is J1=(ɑ1×H1+ɑ2×H2+ɑ3×H3)×(1+I1), where ɑ1, ɑ2, and ɑ3 are the preset time weights corresponding to the heat monitoring sub-cycles F1, F2, and F3, respectively. They are preset by the management personnel according to the time decay characteristics of short video hotspots, and satisfy ɑ1+ɑ2+ɑ3=1.

[0011] Furthermore, in S57, the calculation rules for the comprehensive optimization completion degree N1 of the clip association group G1 are as follows: S571: Label all specifications and parameters pre-selected and configured by the management personnel as K1, K2, ..., Kk, where k≥1; S572: Using the formula Calculate the optimization completion degree M1 of the unified processing steps performed by the clip association group G1 based on the specification parameter K1. In the formula, Lmax and Lmin are the maximum and minimum values ​​of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1, respectively. L is the average value of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1. σ is a preset minimum positive value to avoid the denominator being 0. S573: Calculate and obtain the optimization completion rates M2, M3, ..., Mk of the unified processing steps for the clip association group G1 based on specification parameters K2, K3, ..., Kk, according to S572. S574: Utilize formula Calculate the overall optimization completion degree N1 of the editing association group G1. In the formula, βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk, which are set by the administrator according to the degree of influence of each specification parameter on the editing effect. The sum of the proportion weights of all specification parameters is 1.

[0012] Furthermore, in S510, the filtering rules for selecting a clip object as a popular material in the popular material library with specification type D1 within the clip association group R1 are as follows: SS11: Label all clip objects contained in clip association group R1 as T1, T2, ..., Tt, where t≥1; SS12: Calculate the optimization completion degree V1 of clip object T1 based on the clip record data of T1 and the unified processing steps performed on the clip object T1 according to the specification parameter K1. The calculation formula is as follows: In the formula, Umax and Umin are the maximum and minimum values ​​of the specification correction amount of specification parameter K1 in the editing record data of editing object T1, and U is the average value of the specification correction amount of specification parameter K1 in the editing record data of editing object T1. Similarly, the optimization completion degree V2, V3, ..., Vk of editing object T1 based on specification parameters K2, K3, ..., Kk is calculated in turn. SS13: Utilizing Formulas Calculate the overall optimization completion degree W1 of the obtained editing object T1, where βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk respectively, and the sum of the proportion weights of all specification parameters is 1; SS14: Calculate and obtain the overall optimization completion of editing objects T2, T3, ..., Tt in sequence according to SS11 to SS13, and select the editing object corresponding to the highest overall optimization completion value as the popular material in the popular material library. Add the editing object to the popular material library. During the addition process, set the priority of the editing object to the first priority and the reuse count to 0.

[0013] Furthermore, after completing step S5, the following steps are also required: At each monitoring cycle, the priority of all popular materials in the popular material library of each specification type is dynamically updated. If, within the monitoring cycle, the frequency of changes in the reuse count of any popular material in any popular material library is lower than a preset threshold, then the priority of that popular material is downgraded. The specific adjustment rules for downgrading are as follows: the priority of popular materials with the first priority is downgraded to the second priority, the priority of popular materials with the second priority is downgraded to the third priority, and the priority of popular materials with the third priority remains unchanged and is not downgraded.

[0014] Compared with existing technologies, it has the following advantages: This invention collects user-uploaded short video materials in real time and determines their corresponding specifications. Based on the specifications, it searches the corresponding popular material library for matching popular materials. If a matching material is found, it is directly input into the intelligent video editing model for editing. This method not only ensures that the video effects edited by different users based on the same popular material are consistent, thus maintaining stable editing effect evaluation results and effectively meeting users' actual needs for quickly creating high-quality popular short videos, but also significantly reduces the computational load of the intelligent video editing model, greatly improving users' editing efficiency. If no matching popular material is found, a preset unified processing step is used to adjust and modify the various specification parameters of the short video material to adapt it to the standard format of the intelligent video editing model, thereby smoothly completing the entire intelligent editing process. This invention collects editing records of short video materials undergoing standardized processing steps and analyzes them periodically. During the analysis, based on the specification correction data and processing time of each short video material under different processing methods, popular materials are periodically selected for inclusion in the popular material library for each specification parameter. This method improves the reusability of popular materials based on popularity and allows for quantitative evaluation of material optimization effects based on specification correction data. It ensures that the popular materials selected for inclusion in the library are all high-quality materials with high reusability and have undergone standardized specification optimization processing, thus guaranteeing the quality consistency and specification compliance of popular materials from the source. At the same time, it ensures that the construction of the popular material library not only aligns with the trend of short video creation but also takes into account the technical rationality and efficiency of short video material processing. This invention assigns initial priority and reuse count to popular materials when adding them to a popular material library of corresponding specifications. Simultaneously, it dynamically adjusts the priority of each popular material based on its actual usage frequency according to a preset monitoring cycle. When performing popular material matching searches on user-uploaded short video materials, the system can match materials in descending priority order. This approach improves the matching efficiency of popular materials, allowing users to quickly retrieve currently highly reusable and popular high-quality materials, effectively shortening the matching time and improving the overall efficiency of the editing process. Furthermore, through dynamic priority control and targeted matching rules, high-priority, high-quality popular materials receive higher search exposure and matching probability. This ensures efficient reuse of popular materials and makes the resource allocation and calling logic within the material library more aligned with the trend of short video creation, preventing high-quality, popular materials from having their matching resources squeezed out by low-reusability materials. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 This application provides a method for evaluating the editing effect of short videos based on multimodal fusion, including the following steps: S1: Receive short video materials uploaded by users in real time, extract several specification parameters of the short video materials, and determine the specification type of the short video materials based on them. The specification parameters are pre-selected and configured by the administrator to characterize the attribute features of the short video materials. In this application, the specification parameters include, but are not limited to, video resolution, bitrate, frame rate, specification type, image clarity, and distortion rate. In step S1, the manager has preset several specification types, and each specification type has a preset matching range corresponding to several specification parameters. For the short video material, based on all the extracted specification parameters of the short video material, the matching range corresponding to each specification parameter is determined one by one. Then, based on the belonging of the matching range corresponding to each specification parameter, the specification type corresponding to the short video material is finally determined. In step S1, video resolution refers to the pixel arrangement size of the video image, that is, the product of the number of horizontal pixels and the number of vertical pixels. It is the core physical parameter that determines the clarity of the video image, and the unit is pixels, such as 1920×1080. Bitrate refers to the number of bits transmitted or stored in a video file per unit of time, and it determines the degree of compression and image quality of video data. Frame rate refers to the number of frames played per second in a video, which determines the smoothness of the video. Specification type refers to the encoding and packaging format of a video file, that is, the storage / transmission standard of video and audio data, which determines the compatibility, playability and compression efficiency of the video file; Image clarity refers to the clarity of details presented in a video image. It is an evaluation indicator that combines subjective perception and objective data. Its core reflects the detail reproduction and lack of blur in the video image. Objective judgment dimensions include basic clarity, detail clarity, and blur-free clarity. S2: Based on the specification type of the short video material, search the popular material library of the specification type to see if there is a popular material that matches the short video material. Based on the search results, select one from the reused material and the standard material of the short video material as the editing object of the short video material. The reused material refers to the popular material that matches the short video material, and the standard material refers to the short video material after performing the preset unification processing steps. In step S2, each specification type has a number of popular materials stored in its corresponding popular material library, and each popular material has a priority and number of reuses. This application sets three priorities: first priority, second priority and third priority. In step S2, retrieving from the popular material library of the specified type whether there is popular material matching the short video material specifically involves: Content features are extracted from the short video material, and all extracted content features are sequentially labeled as A1, A2, ..., Aa, where a is the total number of content features extracted from the short video material. The content features include, but are not limited to, image texture features, character features, scene element features, and audio features. Using formula The cosine similarity between the short video material and each popular material with the first priority in the popular material library is calculated, where V = (A1, A2, ..., Aa) represents the content feature vector corresponding to the short video material, B = (B1, B2, ..., Ba) represents the content feature vector corresponding to a popular material with the first priority in the popular material library, Ab refers to each content feature in the content feature vector V, and Bb refers to each content feature in the content feature vector B; Each time a cosine similarity is calculated, that is, the cosine similarity is compared with a preset similarity threshold; If the cosine similarity is greater than or equal to the preset similarity threshold, then the short video material is determined to match the corresponding popular material, the popular material is marked as a reused material of the short video material, the cosine similarity calculation process between the short video material and all other popular materials in the popular material library is terminated, and the reuse count of the popular material in the popular material library is incremented by 1; If the cosine similarity is less than the preset similarity threshold, then the next first-priority popular material in the popular material library is selected, and the cosine similarity calculation is continued until all first-priority popular materials have been calculated. If the cosine similarity between all first-priority popular materials in the popular material library and the short video material is less than a preset similarity threshold, then the cosine similarity between all second-priority and third-priority popular materials and the short video material is calculated sequentially according to priority. If the cosine similarity between all second-priority and third-priority popular materials and the short video material is less than a preset similarity threshold, then it is determined that the short video material does not match any of the popular materials in the corresponding popular material library. At this time, a preset standardization process is performed on the short video material to obtain the corresponding specification material. In step S2, the specific content of performing the preset standardization processing steps on the short video material is as follows: S21: Based on the specification type of the short video material, obtain the execution strategy pre-configured by the administrator for the specification type, wherein the execution strategy includes several processing methods and the adaptation weights corresponding to each processing method; In this application, the adaptation weight is preset by the administrator based on the processing effect of the corresponding processing method and the resource consumption. The larger the value of the adaptation weight, the earlier the corresponding processing scheme is used in the execution order. The processing method is used to perform standardization processing, quality optimization, format conversion and other processing on short video materials, including but not limited to resolution interpolation scaling, dynamic bitrate adjustment, frame rate synchronous frame interpolation, lossless format conversion, clarity enhancement, distortion repair, image denoising, color calibration and so on. It should be noted that after any processing method is used to process the specifications of the short video material, the values ​​of several specification parameters of the short video material will change. S22: Extract all processing methods contained in the execution strategy. Based on the adaptation weight of each extracted processing method in the execution strategy, label all extracted processing methods as C1, C2, ..., Cc in descending order of their adaptation weight values, where c ≥ 1; S23: Use processing method C1 to perform specification processing on the short video material to obtain intermediate processing data of the short video material under processing method C1, and at the same time obtain specification correction data of the short video material under processing method C1. The specification correction data includes the specification correction amount of all specification parameters in the short video material, wherein the specification correction amount of any specification parameter is the absolute value of the difference between the value of the specification parameter of the short video material before and after specification processing using processing method C1. The time interval from the start of using processing method C1 to the end of the time when the intermediate processing data is obtained is taken as the processing duration of the short video material under processing method C1; S24: Use processing method C2 to perform specification processing on the intermediate processing data of the short video material under processing method C1, to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing method C2, and so on to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing methods C3, C4, ..., Cc. S25: The intermediate processing data of the short video material under processing method Cc is labeled as the specification material of the short video material; S3: Input the editing object of the short video material into the trained intelligent video editing model, obtain the edited video and editing effect score of the editing object output by the intelligent video editing model, and display it to the user for preview; Intelligent video editing models extract the spatiotemporal features of the editing object, such as inter-frame differences, motion trajectories, and color distribution. Based on these features, they make editing decisions, including selecting editing points and controlling editing length, and evaluate the effects of the edited video, such as video smoothness and viewing experience. In this application, deep learning techniques, such as convolutional neural networks (CNN), are used to extract spatiotemporal features from the video; the video effect evaluation adopts a combination of subjective and objective evaluation methods to evaluate the effect of the edited video. S4: Obtain the editing effect score of the editing object, obtain the specification correction data and processing time of the short video material of the editing object under all corresponding processing methods by performing the unified processing steps, generate the editing record data of the editing object and store it. The editing record data is only generated when the editing object is a standard material, and no editing record data is generated when the editing object is a popular material. S5: At each analysis cycle, the editing record data of all editing objects stored in the analysis cycle is analyzed, and based on the analysis results, all popular materials to be added to the popular material library of each specification type in the analysis cycle are determined and added to the corresponding popular material library; The interval between analysis cycles is set by the administrators based on the short-term popularity characteristics of the short video materials; Step S5, the analysis content is as follows: S51: Traverse the editing record data of all editing objects stored in the analysis period, obtain the specification type of all editing objects and remove duplicates, and mark all remaining specification types after deduplication as D1, D2, ..., Dd, where d≥1; S52: Mark the clip record data of all clip objects with specification type D1 in the clip record data of all clip objects stored in the analysis period as E1, E2, ..., Ee, respectively, e≥1; S53: Based on the clip objects corresponding to the clip record data E1, E2, ..., Ee, select several clip objects and add them to an empty set to generate a set of clip associations. The specific process is as follows: Using the cosine similarity calculation method in step S2, first calculate the cosine similarity between the clip object corresponding to clip record data E1 and the clip objects corresponding to clip record data E2, E3, ..., Ee respectively; Each cosine similarity is compared with a preset similarity threshold. Clip objects with a cosine similarity greater than or equal to the preset similarity threshold are added to the same empty set to obtain a set of clip associations. Each clip object is only allowed to be added to the set once, and clip objects that have been added to the set will not participate in subsequent filtering again. The clip objects corresponding to the cosine similarity mentioned here are the two clip objects that participated in the cosine similarity calculation. S54: Following the method in S53, filter all remaining clip objects that have not yet been added to any clip association set among the clip objects corresponding to the clip record data E1, E2, ..., Ee. Add several clip objects that meet the conditions to an empty set to obtain multiple clip association sets; wherein, the cosine similarity between any two clip objects in any clip association set is greater than or equal to a preset similarity threshold. All the group clip association sets obtained based on the clip objects corresponding to the clip record data E1, E2, ..., Ee are labeled as G1, G2, ..., Gg, respectively, where 1≤g≤e; S55: Based on the time decay characteristics of short video hotspots and combined with the duration of the analysis period, the analysis period is divided into several equally long heat monitoring sub-periods; the three heat monitoring sub-periods are labeled as F1, F2, and F3 in chronological order, wherein the start time of heat monitoring sub-period F1 is the start time of the analysis period, and the end time of heat monitoring sub-period F3 is the end time of the analysis period; S56: Calculate and obtain the clip association heat value J1 of clip association combination G1 according to the preset calculation rules. The calculation rules are as follows: S561: Using the formula Calculate the heat trend coefficient I1 of the clip association combination G1, where H1, H2, and H3 are the total number of clip objects in the heat monitoring sub-cycles F1, F2, and F3, respectively, at the time when the clip record data is generated within the clip association combination G1. It should be noted that the popularity trend coefficient is artificially defined to characterize the growth or decline trend of the popularity of the material. I1∈[-1,2], I1>0 indicates that the popularity of the material is rising or stable, and I1≤0 indicates that the popularity of the material is declining. S562: The editing association popularity value J1 of the editing association combination G1 is calculated by combining the preset time weights of each popularity monitoring sub-cycle. The calculation formula is J1=(ɑ1×H1+ɑ2×H2+ɑ3×H3)×(1+I1), where ɑ1, ɑ2, and ɑ3 are the preset time weights corresponding to the popularity monitoring sub-cycles F1, F2, and F3, respectively. They are preset by the management personnel according to the time decay characteristics of short video hotspots, and satisfy ɑ1+ɑ2+ɑ3=1. In this application, ɑ1=0.5, ɑ2=0.3, and ɑ3=0.2. S57: Calculate the overall optimization completion rate N1 of the clip association group G1 according to the preset calculation rules. The calculation rules are as follows: S571: Label all specifications and parameters pre-selected and configured by the management personnel as K1, K2, ..., Kk, where k≥1; S572: Using the formula Calculate the optimization completion degree M1 of the unified processing steps performed by the clip association group G1 based on the specification parameter K1. In the formula, Lmax and Lmin are the maximum and minimum values ​​of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1, respectively. L is the average value of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1. σ is a preset minimum positive value to avoid the denominator being 0. M1∈[0,100], the higher the value of M1, the more thoroughly the specification parameter K1 is optimized; S573: Calculate and obtain the optimization completion rates M2, M3, ..., Mk of the unified processing steps for the clip association group G1 based on specification parameters K2, K3, ..., Kk, according to S572. S574: Utilize formula Calculate the overall optimization completion degree N1 of the editing association group G1. In the formula, βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk, which are set by the administrator according to the degree of influence of each specification parameter on the editing effect. The sum of the proportion weights of all specification parameters is 1. S58: Calculate the comprehensive entry score Q1 of the editing association group G1 using the formula Q1=J1×λ1+N1×λ2. Compare Q1 and Q. If Q1≥Q, it is determined that the editing association group G1 meets the conditions for being added to the popular material library. Otherwise, it is determined that the editing association group G1 does not meet the conditions for being added to the popular material library and will not be included in the update scope of this popular material library. Q is the preset entry screening threshold, λ1 and λ2 are the preset first and second proportion coefficients, respectively, and λ1+λ2=1. S59: Determine whether the editing association groups G2, G3, ..., Gg meet the conditions for entering the popular material library according to S54 to S58, and re-mark all editing association groups that meet the conditions for entering the popular material library as R1, R2, ..., Rr, 1≤r≤g; S510: Select a clip object from the clip association group R1 according to the preset filtering rules, and use it as a popular material in the popular material library of clip association group R1 with specification type D1. The filtering rules are as follows: SS11: Label all clip objects contained in clip association group R1 as T1, T2, ..., Tt, where t≥1; SS12: Calculate the optimization completion degree V1 of clip object T1 based on the clip record data of T1 and the unified processing steps performed on the clip object T1 according to the specification parameter K1. The calculation formula is as follows: In the formula, Umax and Umin are the maximum and minimum values ​​of the specification correction amount of specification parameter K1 in the editing record data of editing object T1, and U is the average value of the specification correction amount of specification parameter K1 in the editing record data of editing object T1. Similarly, the optimization completion degree V2, V3, ..., Vk of editing object T1 based on specification parameters K2, K3, ..., Kk is calculated in turn. SS13: Utilizing Formulas Calculate the overall optimization completion degree W1 of the editing object T1, where βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk, which are set by the administrator according to the degree of influence of each specification parameter on the editing effect. The sum of the proportion weights of all specification parameters is 1. SS14: Calculate and obtain the overall optimization completion of editing objects T2, T3, ..., Tt in sequence according to SS11 to SS13, and select the editing object corresponding to the largest overall optimization completion value as the popular material in the popular material library. Add the editing object to the popular material library. During the addition process, set the priority of the editing object to the first priority and the reuse count to 0. S511: Following S510, select popular materials from the editing association groups R2, R3, ..., Rr as the popular material library with specification type D1 for editing association groups R2, R3, ..., Rr, and add them to the popular material library; S512: Determine all popular materials in the popular material library with specification types D2, D3, ..., Dd in sequence according to S51 to S511, and add them to the corresponding popular material library; S6: At each monitoring cycle, the priority of all popular materials in the popular material library of each specification type is dynamically updated; if, within the monitoring cycle, the frequency of changes in the reuse count of any popular material in any popular material library is lower than the preset threshold, the priority of that popular material is downgraded. The specific adjustment rules are as follows: the priority of the first priority popular material is downgraded to the second priority, the priority of the second priority popular material is downgraded to the third priority, and the priority of the third priority popular material remains unchanged and is not downgraded.

[0018] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0019] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for evaluating the editing effect of short videos based on multimodal fusion, characterized in that, Includes the following steps: S1: Receive short video materials uploaded by users in real time, extract several specification parameters of the short video materials, and determine the specification type of the short video materials based on them; S2: Based on the specification type of the short video material, search the popular material library of the specification type to see if there is a popular material that matches the short video material. Based on the search results, select one from the reused material and the standard material of the short video material as the editing object of the short video material. The reused material refers to the popular material that matches the short video material, and the standard material refers to the short video material after performing the preset unification processing steps. S3: Input the editing object of the short video material into the trained intelligent video editing model, obtain the edited video and editing effect score of the editing object output by the intelligent video editing model, and display it to the user for preview; S4: Obtain the editing effect score of the editing object, obtain the specification correction data and processing time of the short video material of the editing object under all corresponding processing methods by performing the corresponding unified processing steps, generate the editing record data of the editing object and store it. S5: At each analysis cycle, the editing record data of all editing objects stored in the analysis cycle is analyzed, and based on the analysis results, all popular materials to be added to the popular material library of each specification type in the analysis cycle are determined and added to the corresponding popular material library.

2. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 1, characterized in that, In step S1, the administrator has preset several specification types, and each specification type has a preset matching range corresponding to several specification parameters. For the short video material, based on all the extracted specification parameters of the short video material, the matching range corresponding to each specification parameter is determined one by one. Then, based on the belonging of the matching range corresponding to each specification parameter, the specification type corresponding to the short video material is finally determined.

3. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 1, characterized in that, In step S2, each specification type has a number of popular materials stored in its corresponding popular material library, and each popular material has a priority and number of reuses. There are three priority settings: first priority, second priority and third priority.

4. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 1, characterized in that, In step S2, the specific content of performing the preset standardization processing steps on the short video material is as follows: S21: Based on the specification type of the short video material, obtain the execution strategy pre-configured by the administrator for the specification type. The execution strategy includes several processing methods and the adaptation weights corresponding to each processing method. The adaptation weights are preset by the administrator based on the processing effect and resource consumption of the corresponding processing method. The larger the value of the adaptation weight, the earlier the corresponding processing scheme is used in the execution order. After the short video material is processed using any processing method, the values ​​of several specification parameters of the short video material will change. S22: Extract all processing methods contained in the execution strategy. Based on the adaptation weight of each extracted processing method in the execution strategy, label all extracted processing methods as C1, C2, ..., Cc in descending order of their adaptation weight values, where c ≥ 1; S23: Use processing method C1 to perform specification processing on the short video material to obtain intermediate processing data of the short video material under processing method C1, and at the same time obtain the specification correction data and processing time of the short video material under processing method C1. The specification correction data includes the specification correction amount of all specification parameters in the short video material, wherein the specification correction amount of any specification parameter is the absolute value of the difference between the value of the specification parameter of the short video material before and after specification processing using processing method C1. S24: Use processing method C2 to perform specification processing on the intermediate processing data of the short video material under processing method C1, to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing method C2, and so on to obtain the intermediate processing data, specification correction data and processing time of the short video material under processing methods C3, C4, ..., Cc. S25: The intermediate processing data of the short video material under processing method Cc is labeled as the specification material of the short video material.

5. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 1, characterized in that, In S4, editing record data is only generated for standard footage, while editing record data is not generated for popular footage.

6. The method for evaluating the effect of short video editing based on multimodal fusion according to claim 4, characterized in that, The following is the content of the analysis of the clip record data of all clip objects stored during the analysis period: S51: Traverse the editing record data of all editing objects stored in the analysis period, obtain the specification type of all editing objects and remove duplicates, and mark all remaining specification types after deduplication as D1, D2, ..., Dd, where d≥1; S52: Mark the clip record data of all clip objects with specification type D1 in the clip record data of all clip objects stored in the analysis period as E1, E2, ..., Ee, respectively, e≥1; S53: Based on the clip objects corresponding to the clip record data E1, E2, ..., Ee, select several clip objects and add them to an empty set to generate a set of clip association sets; S54: Based on the method of S53, filter all remaining clip objects that have not yet been added to any clip association set among the clip objects corresponding to the clip record data E1, E2, ..., Ee, and add several clip objects that meet the conditions to the empty set to obtain multiple clip association sets; All the group clip association sets obtained based on the clip objects corresponding to the clip record data E1, E2, ..., Ee are labeled as G1, G2, ..., Gg, respectively, where 1≤g≤e; S55: Based on the time decay characteristics of short video hotspots and the duration of the analysis period, the analysis period is divided into several sub-periods of equal duration for heat monitoring; the three sub-periods of heat monitoring are marked as F1, F2, and F3 in chronological order. S56: Calculate and obtain the clip association heat value J1 of clip association combination G1 according to the preset calculation rules: S57: Calculate and obtain the overall optimization completion degree N1 of the clip association group G1 according to the preset calculation rules; S58: Calculate the comprehensive entry score Q1 of the editing association group G1 using the formula Q1=J1×λ1+N1×λ2. Compare Q1 and Q. If Q1≥Q, it is determined that the editing association group G1 meets the conditions for being added to the popular material library. Otherwise, it is determined that the editing association group G1 does not meet the conditions for being added to the popular material library and will not be included in the update scope of this popular material library. Q is the preset entry screening threshold, and λ1 and λ2 are the preset first and second proportion coefficients, respectively. S59: Determine whether the editing association groups G2, G3, ..., Gg meet the conditions for entering the popular material library according to S54 to S58, and re-mark all editing association groups that meet the conditions for entering the popular material library as R1, R2, ..., Rr, 1≤r≤g; S510: Select a clip object from the clip association group R1 according to the preset filtering rules and use it as a popular material in the popular material library with specification type D1 in the clip association group R1; S511: Following S510, select popular materials from the editing association groups R2, R3, ..., Rr as the popular material library with specification type D1 for editing association groups R2, R3, ..., Rr, and add them to the popular material library; S512: Determine all popular materials in the popular material library with specification types D2, D3, ..., Dd in sequence according to S51 to S511, and add them to the corresponding popular material library.

7. The method for evaluating the effect of short video editing based on multimodal fusion according to claim 6, characterized in that, S56, The specific calculation rules for the clip association heat value J1 of the clip association combination G1 are as follows: S561: Using the formula Calculate the heat trend coefficient I1 of the clip association combination G1, where H1, H2, and H3 are the total number of clip objects in the heat monitoring sub-cycles F1, F2, and F3, respectively, at the time when the clip record data is generated within the clip association combination G1. S562: Combine the preset time weights of each heat monitoring sub-cycle to calculate the editing association heat value J1 of the editing association combination G1. The calculation formula is J1=(ɑ1×H1+ɑ2×H2+ɑ3×H3)×(1+I1), where ɑ1, ɑ2, and ɑ3 are the preset time weights corresponding to the heat monitoring sub-cycles F1, F2, and F3, respectively. They are preset by the management personnel according to the time decay characteristics of short video hotspots, and satisfy ɑ1+ɑ2+ɑ3=1.

8. The method for evaluating the effect of short video editing based on multimodal fusion according to claim 6, characterized in that, S57, the calculation rules for the overall optimization completion degree N1 of the clip association group G1 are as follows: S571: Label all specifications and parameters pre-selected and configured by the management personnel as K1, K2, ..., Kk, where k≥1; S572: Using the formula Calculate the optimization completion degree M1 of the unified processing steps performed by the clip association group G1 based on the specification parameter K1. In the formula, Lmax and Lmin are the maximum and minimum values ​​of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1, respectively. L is the average value of the specification correction amount of the specification parameter K1 in the clip record data of all clip objects in the clip association group G1. σ is a preset minimum positive value to avoid the denominator being 0. S573: Calculate and obtain the optimization completion degree M2, M3, ..., Mk of the clip association group G1 based on the specification parameters K2, K3, ..., Kk in sequence according to S572; S574: Utilize formula Calculate the overall optimization completion degree N1 of the editing association group G1. In the formula, βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk, which are set by the administrator according to the degree of influence of each specification parameter on the editing effect. The sum of the proportion weights of all specification parameters is 1.

9. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 6, characterized in that, S510, the filtering rules for selecting a clip object as a popular material in the popular material library with specification type D1 and belonging to the clip association group R1 are as follows: SS11: Label all clip objects contained in clip association group R1 as T1, T2, ..., Tt, where t≥1; SS12: Calculate the optimization completion degree V1 of clip object T1 based on the clip record data of T1 and the unified processing steps performed on the clip object T1 according to the specification parameter K1. The calculation formula is as follows: In the formula, Umax and Umin are the maximum and minimum values ​​of the specification correction amount of specification parameter K1 in the editing record data of editing object T1, and U is the average value of the specification correction amount of specification parameter K1 in the editing record data of editing object T1. Similarly, the optimization completion degree V2, V3, ..., Vk of editing object T1 based on specification parameters K2, K3, ..., Kk is calculated in turn. SS13: Utilizing Formulas Calculate the overall optimization completion degree W1 of the obtained editing object T1, where βm refers to the preset proportion weight of specification parameters K1, K2, ..., Kk respectively, and the sum of the proportion weights of all specification parameters is 1; SS14: Calculate and obtain the overall optimization completion of editing objects T2, T3, ..., Tt in sequence according to SS11 to SS13, and select the editing object corresponding to the highest overall optimization completion value as the popular material in the popular material library. Add the editing object to the popular material library. During the addition process, set the priority of the editing object to the first priority and the reuse count to 0.

10. The method for evaluating the editing effect of short videos based on multimodal fusion according to claim 3, characterized in that, After completing step S5, the following steps are also required: At each monitoring cycle, the priority of all popular materials in the popular material library of each specification type is dynamically updated. If, within the monitoring cycle, the frequency of changes in the reuse count of any popular material in any popular material library is lower than a preset threshold, then the priority of that popular material is downgraded. The specific adjustment rules for downgrading are as follows: the priority of popular materials with the first priority is downgraded to the second priority, the priority of popular materials with the second priority is downgraded to the third priority, and the priority of popular materials with the third priority remains unchanged and is not downgraded.

Images