Short video low-repetition batch generation method and device based on matrix combination optimization and AIGC picture replacement and storage medium

Abstract

The application relates to a short video low-repetition batch generation method and device based on matrix combination optimization and AIGC picture replacement and a storage medium, which establishes a screenplay-material slot matrix comprising a screenplay dimension and a material slot dimension, reads candidate materials, and generates a plurality of candidate short video combinations in a screenplay order, calculates the material coincidence degree between versions to obtain a repetition rate score; when the repetition rate score is higher than a preset threshold, a target matrix unit that increases the repetition rate score is determined, an AIGC model is called to perform replacement generation on picture elements corresponding to the target matrix unit, the repetition rate score is recalculated after the replacement picture is backfilled, and a target version set with the lowest repetition rate score is output in a fixed number under the condition of meeting a preset constraint condition. The application can increase the number of output versions and reduce the repetition rate between versions on the basis of limited materials.

Description

Technical Field

[0001] This application relates to the field of short video generation technology, and in particular to a method, device and storage medium for low-repetition batch generation of short videos based on matrix combination optimization and AIGC screen replacement. Background Technology

[0002] In scenarios such as short video matrix accounts, product promotion, marketing campaigns, and information flow advertising, operators often want to generate as many versions as possible with limited materials, while minimizing the duplication rate between different versions to meet the platform's deduplication and distribution optimization requirements. Current technologies typically involve copying the master project and replacing a small number of visuals, text, and sound effects, or preparing several candidate materials for each storyboard and then sampling and combining them according to a fixed template or simple random method to generate multiple versions. When materials are insufficient, additional AIGC (AI Generic Generated Content) is called to generate individual scenes.

[0003] However, because this approach lacks a unified matrix model to describe the combination relationship between storyboard positions and candidate material slots, it can only rely on random sampling or manual experience to produce versions. It cannot obtain a stable number of versions under given constraints, and it cannot select the combination with the lowest repetition rate from a large number of candidates due to the lack of quantitative evaluation indicators for the degree of repetition between versions. Furthermore, when it is necessary to use AIGC to generate images to remedy the situation, since the AIGC generation process and the combination and deduplication are independent of each other and there is no linkage interface between the two with the repetition rate index as the link, AIGC cannot selectively replace the storyboard units that actually increase the repetition rate, and its generation results will not be backfilled and re-participated in the repetition rate calculation and combination selection. As a result, AIGC consumes computing power but cannot controllably reduce the overall repetition rate of the version set, and the entire generation process is difficult to trace and reproduce. Summary of the Invention

[0004] In order to increase the number of output versions and reduce the duplication rate between versions based on limited materials, this application provides a method, device and storage medium for low-duplication batch generation of short videos based on matrix combination optimization and AIGC screen replacement.

[0005] Firstly, this application provides a method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC image replacement, employing the following technical solution:

[0006] A method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC image replacement includes:

[0007] S1. Establish a storyboard-material slot matrix. The storyboard-material slot matrix includes N storyboard dimensions and M material slot dimensions. Each matrix unit stores one or more candidate materials.

[0008] S2. Read the candidate footage from the storyboard-feature slot matrix and generate multiple candidate short video combinations according to the storyboard order;

[0009] S3. Calculate the overlap of materials between versions of candidate short video combinations to obtain the duplication rate score for each candidate short video combination;

[0010] S4. In response to the repetition rate score being higher than a preset threshold, identify the target matrix unit that increases the repetition rate score, call the AIGC model to perform replacement generation on the screen element corresponding to the target matrix unit, and obtain the replacement screen.

[0011] S5. Replace the screen with the target matrix cell and recalculate the repetition rate score;

[0012] S6. Under the condition of satisfying the preset constraints, output a fixed number of K target versions with the lowest repetition rate scores from the candidate short video combination after recalculating the repetition rate score.

[0013] By adopting the above technical solution, the short video candidate material space is structurally represented by the storyboard-material slot matrix, so that the generation of candidate short video combinations has an addressable combination basis. On this basis, the degree of repetition between versions is quantified by the material overlap between versions and a repetition rate score is obtained, so that a unified quantitative standard is provided for the screening of a large number of candidate combinations. When the repetition rate score is higher than a preset threshold, the AIGC model is triggered by the repetition rate score to perform screen replacement on the target matrix unit that increases the repetition rate score. After the replacement screen is backfilled, the repetition rate score is recalculated. This makes the AIGC screen replacement and matrix combination deduplication form a closed loop linkage with the repetition rate score as the link, thereby increasing the number of output versions on the basis of limited materials and continuously reducing the overall repetition rate of the version set.

[0014] Optionally, S4 includes sub-steps S41-S43: S41. Determine the repetition rate contribution corresponding to each matrix unit. The repetition rate contribution is the expected decrease in the repetition rate score of the candidate short video combination after replacing the corresponding matrix unit. S42. Sort the matrix units from high to low according to their repetition rate contribution, and determine the preset number of matrix units at the top of the sorted list as target matrix units. S43. Call the AIGC model to perform replacement generation for the target matrix units in order of repetition rate contribution from high to low, until the number of replacements reaches the preset replacement budget.

[0015] By adopting the above technical solution, the expected decrease in the repetition rate score after replacement is used as the sorting criterion and the replacement budget is set, so that the limited number of AIGC replacements are concentrated on the matrix unit that contributes the most to reducing the repetition rate, thereby improving the deduplication benefit per unit of computing power.

[0016] Optionally, in S43, when performing replacement generation on the target matrix unit in sequence, the element type of the screen element corresponding to the target matrix unit is determined; in response to the element type being a main element, a local area replacement generation is performed on the screen element; in response to the element type being a background element, a whole frame replacement generation is performed on the screen where the screen element is located; wherein, the determination of the element type is also based on the hit risk of the preset constraint conditions, and the hit risk is the possibility that the preset constraint conditions will be violated after the corresponding screen element is replaced.

[0017] By adopting the above technical solution, partial replacement and whole-frame replacement are distinguished according to element type and constraint hit risk, so that the main subject-related elements are carefully replaced while preserving the theme and script logic, and background elements are fully replaced to maximize the weight reduction.

[0018] Optionally, main elements include at least one of partial main elements, clothing or decorative details, and partial textures or patterns; background elements include at least one of background, scene objects, lighting and color tone, and overall image style.

[0019] By adopting the above technical solution, the specific scope of the main element and the background element is clearly defined, so that the determination of the replacement granularity has an operable element classification basis.

[0020] Optionally, in S5, after the replacement screen is backfilled into the target matrix cell and the repetition rate score is recalculated, in response to the fact that the decrease in the repetition rate score for a consecutive preset number of rounds is lower than the preset decrease threshold, the replacement generation for the target matrix cell is stopped.

[0021] By adopting the above technical solution, the measured decrease in the repetition rate score after multiple rounds of backfilling and recalculation is used as the criterion for stopping the iteration, so that AIGC is no longer invalidated for target matrix units that can no longer be reduced in weight, thus controlling the consumption of iteration computing power.

[0022] Optionally, the overlap of materials between versions is determined based on at least one of the following: identical material identifiers, identical clip references, identical background elements, identical item elements, identical partial main elements, and identical style templates; the repetition rate score is obtained by summing the overlap of materials between versions of each candidate short video combination, combined with the weights corresponding to each scene dimension and the penalty weights corresponding to duplicate materials whose frequency of occurrence exceeds the preset frequency threshold.

[0023] By adopting the above technical solutions, the overlap is measured from multiple material dimensions and different storyboards are weighted, and highly reused materials are penalized, so that the repetition rate score can more accurately reflect the actual differences in visual composition between versions.

[0024] Optionally, in generating multiple candidate short video combinations according to the storyboard order, if the overlap between the current candidate short video combination and any generated candidate short video combination exceeds a preset pruning threshold, the candidate material to be added is removed. The preset pruning threshold and the overlap between the versions of the material are calculated using the same method.

[0025] By adopting the above technical solution, candidate materials that would result in high overlap are removed during the generation stage based on the same degree of overlap as the repetition rate, so that the candidate combinations entering the scoring stage already have a low repetition baseline, reducing the amount of calculation required for subsequent screening.

[0026] Optionally, the candidate materials stored in each matrix unit include at least one of the original video materials, cropped clip materials, and AIGC pre-generated materials, and each candidate material is associated with an attribute set, which includes at least one of the following: material identifier, theme tag, style tag, duration, background features, item features, subject features, and replaceable areas.

[0027] By adopting the above technical solution, a structured attribute set is established for each candidate material, so that the overlap calculation, constraint judgment and replacement area location all have a unified data foundation.

[0028] Optionally, the duplication rate score is formed by adding at least one of visual similarity, semantic similarity, or shot similarity to the material overlap between versions, thus creating a joint deduplication index.

[0029] By adopting the above technical solution, visual and semantic similarity is introduced, so that image-level similarity that cannot be identified by material identifiers alone is also taken into consideration for deduplication.

[0030] Optionally, version pairs can be coarsely screened based on the degree of overlap of the same material identifier or the same fragment reference. Only version pairs whose coarsely screened overlap exceeds a preset threshold are then calculated for visual similarity or semantic similarity.

[0031] By adopting the above technical solution, the computational overhead of visual and semantic computation is limited to a few highly overlapping version pairs, thereby reducing computational overhead while maintaining scoring accuracy.

[0032] Optionally, the preset threshold can be adaptively adjusted based on the richness of the current batch of materials; the lower the richness of the materials, the more lenient the preset threshold.

[0033] By adopting the above technical solution, the repetition rate threshold is matched with the material supply, thus avoiding frequent and ineffective triggering of AIGC replacement when materials are scarce.

[0034] Optionally, if the number of candidate materials for a certain matrix unit is one and still needs to be replaced, the AIGC model is called to pre-generate new candidate materials for that matrix unit and fill them in before participating in the combination.

[0035] By adopting the above technical solution, candidate materials are actively supplemented for matrix units that have no replacement candidates, so that the storyboard position can still participate in deduplication when the material is extremely scarce.

[0036] Optionally, weighted sampling can be performed using the reciprocal of the historical usage frequency of each candidate material as the sampling weight, with candidate materials having a higher probability of being selected due to their lower historical usage frequency.

[0037] By adopting the above technical solution, low-frequency materials are prioritized for inclusion in the combination, thereby reducing the probability of different versions reusing the same material from the source.

[0038] Optionally, the process of generating candidate short video combinations can be combined with at least one of dynamic pruning, local backtracking, or heuristic search, so that highly repetitive or unconstrained combination branches are eliminated during the generation process.

[0039] By adopting the above technical solution, low-quality branches are removed during the generation stage, making the candidate set more concise for subsequent screening.

[0040] Optionally, the storyboard-material slot matrix can be expanded into a multi-dimensional combination structure that further includes at least one of the style dimension, platform dimension, or language dimension.

[0041] By adopting the above technical solutions, the combination space is expanded from two dimensions to multiple dimensions, making style, platform, or language differences sources of version differentiation.

[0042] Optionally, platform rules can be used as one of the preset constraints to output low-duplication versions that satisfy the platform rules for different platforms.

[0043] By adopting the above technical solutions, the output version can be adapted to the deduplication and distribution rules of different platforms, thereby improving the usability of cross-platform deployment.

[0044] Optionally, at least one of the following can be incorporated as an additional storyboard dimension or combination dimension into the generation and selection of candidate short video combinations: cover, opening sequence, closing sequence, or audio, subtitle, or sticker style.

[0045] By adopting the above technical solutions, more variable elements can be included in the combination scope, further expanding the space for differentiation between versions.

[0046] Optionally, the fixed number K is automatically adjusted according to the scale of the candidate short video combination and the preset constraints, and the upper bound of K is determined by the number of versions that the candidate short video combination can stably satisfy under the preset threshold. The number of storyboard dimensions N and the number of material slot dimensions M are variable dimensions, and the number of candidate materials stored in each matrix unit is adjustable within a preset range.

[0047] By adopting the above technical solution, the output scale and matrix scale can be flexibly configured according to the material conditions, adapting to the batch generation needs of different sizes.

[0048] Optionally, calling the AIGC model to perform replacement generation can be triggered in response to at least one of the following: insufficient material quantity, style imbalance, or low quality score, in addition to the repetition rate score being higher than a preset threshold.

[0049] By adopting the above technical solution, the triggering conditions for AIGC replacement are expanded, enabling it to play a supplementary role in scenarios of material shortage or quality problems, in addition to the repetition rate.

[0050] Optionally, the preset constraints include at least one of the following: theme consistency constraint, style consistency constraint, and duration adaptation constraint.

[0051] By adopting the above technical solutions and clarifying the specific components of the constraints, the screening process can maintain consistency in the theme, style, and duration of the versions while reducing the duplication rate.

[0052] Optionally, each version in the output target version set is associated with a traceability record. The traceability record includes at least one of the following: version identifier, storyboard combination path, candidate material selection result, duplication rate score, AIGC replacement position, material identifiers before and after replacement, export parameters, and version generation time.

[0053] By adopting the above technical solution, a complete generation and replacement record is retained for each output version, making the version generation process reproducible, evaluable, and supporting subsequent iterative optimization.

[0054] Secondly, the computer device provided in this application adopts the following technical solution:

[0055] A computer device comprising:

[0056] One or more processors;

[0057] Memory;

[0058] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to:

[0059] Perform the above-described method for batch generation of short videos with low repetition.

[0060] Thirdly, this application provides a computer-readable storage medium that adopts the following technical solution:

[0061] A computer-readable storage medium storing an application program that can be loaded by a processor and execute the methods described above.

[0062] The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following:

[0063] The above describes a method for generating short videos with low repetition in batches.

[0064] In summary, this application includes at least one of the following beneficial technical effects:

[0065] 1. By using a storyboard-material slot matrix to structurally express the material space, measuring the degree of repetition by the overlap of materials between versions, and using the repetition rate score to trigger AIGC scene replacement, refilling and re-screening, the matrix combination deduplication and AIGC replacement form a closed loop linkage, thereby increasing the number of output versions and reducing the overall repetition rate of the version set on the basis of limited materials.

[0066] 2. By using the contribution of repetition rate to make targeted replacements and setting replacement budgets, distinguishing replacement granularity by element type and constraint hit risk, and determining the iteration stop by the continuous measured decrease amount in multiple rounds, the limited computing power is concentrated on the target matrix unit with the highest deduplication benefit, thereby controlling the AIGC call consumption while preserving the theme and script logic.

[0067] 3. By pruning with the same degree of overlap during the generation stage, prioritizing sampling of low-frequency materials, and superimposing two-level screening based on visual and semantic similarity during the scoring stage, the candidate combinations have a low repetition baseline before entering the replacement stage, further improving the overall deduplication efficiency and scoring accuracy. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the overall process of a method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC image replacement, according to one embodiment of this application.

[0069] Figure 2 This is a schematic diagram of the storyboard-material slot matrix according to one embodiment of this application.

[0070] Figure 3 This is a closed-loop flowchart of the targeted matrix unit directional replacement and backfill recalculation according to one embodiment of this application.

[0071] Figure 4This is a schematic diagram of a computer device according to this application. Detailed Implementation

[0072] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the scope of the application.

[0073] This application provides a method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC scene replacement. (Refer to...) Figure 1 This method organizes short videos into a storyboard-material slot matrix according to the storyboard structure, performs structured expression of candidate materials under each storyboard position, generates a repetition rate score by combining candidates and quantifying the overlap of materials between versions, and triggers AIGC screen replacement by repetition rate score when the repetition rate score exceeds a preset threshold. The replacement results are then filled back into the matrix and re-scored and filtered, thereby outputting multiple low-repetition short video versions based on limited materials.

[0074] Before detailing the embodiments of this application, some terms will be explained first.

[0075] Storyboard-material slot matrix, refer to Figure 2 It is a matrix built according to the storyboard structure of short videos. Its storyboard dimension corresponds to the storyboard position, and its material slot dimension corresponds to the material slot. Each matrix unit stores one or more candidate materials under that storyboard position.

[0076] A candidate short video combination is a version of a candidate video created by selecting one candidate material from each scene position in the matrix and stringing them together in the order of the scenes.

[0077] The overlap of materials between versions is a measure of the degree of overlap in the material composition between two candidate short video combinations.

[0078] The duplication rate score is a score that reflects the degree of duplication in the candidate short video combination, based on the sum of the overlap between the materials in different versions.

[0079] As an example, let's consider the batch generation of a short video featuring product promotion. This video contains seven scenes: an opening, a question introduction, product demonstration one, product demonstration two, highlighting selling points, comparison and explanation, and a closing conversion. Each scene has seven content slots, forming a 7×7 scene-content slot matrix. Each matrix unit contains 3 to 6 candidate content clips. The goal is to output K=20 versions with the lowest repetition rate based on this matrix. Subsequent examples will follow this scenario.

[0080] The method in this application includes steps S1 to S6, which respectively complete matrix establishment, candidate combination generation, repetition rate scoring, threshold replacement, backfilling recalculation, and low-repetition version output. The following provides a detailed description of each step in conjunction with the above scenario.

[0081] S1. Establish the storyboard-material slot matrix. The storyboard-material slot matrix includes N storyboard dimensions and M material slot dimensions. Each matrix unit stores one or more candidate materials. For example, in the above live-streaming sales scenario, N is 7 and M is 7. The 7 storyboard dimensions correspond to the 7 storyboard positions from the beginning to the end, and each matrix unit holds 3 to 6 candidate materials.

[0082] In some embodiments, the candidate materials stored in each matrix unit include at least one of the following: original video material, cropped clip material, and AIGC pre-generated material. For example, a matrix unit for an opening storyboard may simultaneously contain an original opening video, an opening clip cropped from a long video, and an AIGC pre-generated opening scene. Furthermore, each candidate material is associated with a set of attributes, including at least one of the following: material identifier, theme tag, style tag, duration, background features, object features, subject features, and replaceable regions. These attributes provide a unified data foundation for subsequent calculations of material overlap between versions, constraint judgments, and replacement region location.

[0083] In some embodiments, the storyboard-material slot matrix is ​​not limited to two dimensions, but is extended to a multi-dimensional combination structure that further includes at least one of the style dimension, platform dimension, or language dimension. For example, by adding a style dimension in addition to the two dimensions of storyboard and material slot, candidate materials can be attached to the same storyboard position according to different styles such as realism and cartoon, so that style differences can also become a source of version differentiation.

[0084] In some embodiments, at least one of the following can be incorporated as an additional storyboard dimension or combination dimension into the generation and screening of candidate short video combinations, in order to further expand the differentiation space between versions.

[0085] S2. Read the candidate footage from the storyboard-feature slot matrix and generate multiple candidate short video combinations according to the storyboard order. For example, select a candidate footage from a slot in the opening storyboard, select a candidate footage from a slot in the question-introduction storyboard, and so on, selecting one candidate footage from each of the 7 storyboard positions. Stringing these together in the storyboard order will form a candidate short video combination.

[0086] In some embodiments, during the process of generating multiple candidate short video combinations according to the storyboard sequence, in response to the candidate material to be added causing the overlap between the current candidate short video combination and any already generated candidate short video combination to exceed a preset pruning threshold, the candidate material to be added is pruned. The preset pruning threshold and the overlap between versions of the material are calculated using the same metric. This is because when multiple candidate materials are attached to each of the 7 storyboard positions, the total number of combinations will rapidly increase. For example, when 6 candidates are taken for each storyboard position, the total number of combinations can reach [amount missing]. (Approximately 280,000) entries would create a huge scoring burden if generated without constraints. For example, assuming a preset pruning threshold of 0.5, if a candidate short video combination being generated has a 0.6 overlap with an already generated version after selecting the first four storyboard clips, exceeding the threshold of 0.5, then no further candidate clips with increased overlap will be added to the combination, thus pruning the highly overlapping branch during the generation stage. Since the preset pruning threshold and the S3 calculation of the overlap between versions use the same caliber, the retained candidate short video combinations have a low repetition baseline before entering the S3 scoring, reducing the number of candidate pairs that S3 needs to score.

[0087] In some embodiments, weighted sampling is performed using the reciprocal of the historical usage frequency of each candidate material as the sampling weight. Candidate materials with lower historical usage frequency have a higher probability of being selected. For example, if a candidate material has a historical usage frequency of 10 and another candidate material has a historical usage frequency of 2, the latter has a sampling weight that is 5 times that of the former. This prioritizes less frequently reused materials in the combination, reducing the probability of different versions reusing the same material from the source.

[0088] In some embodiments, the process of generating candidate short video combinations is combined with at least one of dynamic pruning, local backtracking, or heuristic search, so that highly repetitive or unconstrained combination branches are eliminated during the generation process, instead of generating all videos first and then filtering them.

[0089] S3. Calculate the overlap of materials between versions of candidate short video combinations to obtain the repetition rate score for each candidate short video combination. For example, compare each of the generated candidate short video combinations pairwise, count the number of times they use the same materials in each shot, obtain the overlap of materials between versions, and then summarize them into the repetition rate score for each candidate short video combination.

[0090] In some embodiments, the overlap of materials between versions is determined based on at least one of the following: identical material identifiers, identical clip references, identical background elements, identical item elements, identical partial subject elements, and identical style templates. For example, if two candidate short video combinations use materials with the same material identifier in the product presentation segment and the same background element in the ending transition segment, then these two segments are counted as overlapping. The repetition rate score is obtained by combining the overlap of materials between versions of each candidate short video combination with the weights corresponding to each segment dimension and the penalty weights corresponding to repeated materials whose frequency of occurrence exceeds a preset frequency threshold. For example, higher weights are assigned to the more attention-grabbing product presentation segments 1 and 2, and lower weights are assigned to the opening segment; at the same time, penalty weights are applied to repeated materials whose frequency of occurrence exceeds a preset frequency threshold. For example, if a certain background material appears 15 times in the generated versions, exceeding the preset frequency threshold by 10 times, then additional penalties are applied to the versions using this background material, so that the summarized repetition rate score more accurately reflects the actual differences in visual composition between versions.

[0091] In some embodiments, the duplication rate score is formed by adding at least one of visual similarity, semantic similarity, or shot similarity to the overlap of materials between versions, thus creating a joint deduplication index. For example, although two versions do not use the same material identifier, their images are visually very similar. By adding visual similarity, this image-level similarity, which cannot be identified by material identifier alone, can also be included in the deduplication consideration.

[0092] In some embodiments, to reduce the overhead of full calculation of all version pairs by the joint deduplication metric, version pairs are first coarsely screened based on the overlap of the same material identifier or the same fragment reference. Only version pairs whose coarsely screened overlap exceeds a preset threshold are then calculated for visual or semantic similarity. For example, among 50 candidate version pairs, only a few version pairs with high coarsely screened overlap are included in the visual and semantic calculation, thereby reducing computational overhead while maintaining scoring accuracy.

[0093] S4. In response to a repetition rate score exceeding a preset threshold, identify the target matrix unit that increases the repetition rate score, and call the AIGC model to perform replacement generation on the screen elements corresponding to the target matrix unit to obtain the replacement screen. For example, if the preset threshold is 0.6, and the repetition rate score of a candidate short video combination is 0.72, which is higher than the preset threshold of 0.6, then identify the target matrix unit in the combination that increases the repetition rate score. For example, if the product display two-part shot and an already generated version share the same background matrix unit, call the AIGC model to perform replacement generation on the screen elements corresponding to the target matrix unit to obtain the replacement screen.

[0094] In some embodiments, refer to Figure 3 S4 includes sub-steps S41-S43.

[0095] S41. Determine the repetition rate contribution for each matrix unit. The repetition rate contribution is the expected decrease in the repetition rate score of the candidate short video combination after replacing the corresponding matrix unit. For example, for the aforementioned candidate short video combination with a repetition rate score of 0.72, it is estimated that after replacing the matrix units corresponding to the product display 2, selling point reinforcement, comparison explanation, opening, and closing conversion shots, the repetition rate score is expected to decrease to 0.58, 0.63, 0.66, 0.69, and 0.70, respectively. Then, the repetition rate contributions of each matrix unit are 0.14, 0.09, 0.06, 0.03, and 0.02, respectively.

[0096] S42. Sort each matrix unit according to its repetition rate contribution from high to low, and determine the first preset number of matrix units as target matrix units. For example, after sorting the matrix units according to their repetition rate contributions of 0.14, 0.09, 0.06, 0.03, and 0.02 from high to low, determine the first 5 matrix units as target matrix units.

[0097] S43. Based on the contribution of repetition rate from highest to lowest, the AIGC model is invoked sequentially to perform replacement generation for the target matrix units until the preset replacement budget is reached. For example, if the preset replacement budget is 3 times, the 3 target matrix units corresponding to Product Display II, Selling Point Enhancement, and Comparison Description are replaced sequentially based on contribution from highest to lowest. Replacement stops after the budget of 3 times is reached. The 2 target matrix units corresponding to the opening and closing conversions are no longer replaced because the budget is exhausted. This concentrates the limited AIGC replacement times on the matrix units that contribute the most to reducing the repetition rate, improving the deduplication benefit per unit of computing power.

[0098] The above describes one implementation of determining target matrix units, namely, sorting by repetition rate contribution and then performing targeted replacement. In other embodiments, target matrix units are determined in other ways. For example, instead of sorting by repetition rate contribution, all matrix units in the candidate short video combination that share the same material, background, or items as the generated version are identified as target matrix units, and the AIGC model is called to perform replacement generation for each of them. This alternative method also achieves the purpose of performing image replacement on matrix units that increase the repetition rate score, and belongs to the implementation of S4 determining and replacing target matrix units.

[0099] In some embodiments, when performing replacement generation sequentially on the target matrix units in S43, the element type of the screen element corresponding to the target matrix unit is determined; in response to the element type being a main element, a partial area replacement generation is performed on the screen element; in response to the element type being a background element, a whole frame replacement generation is performed on the screen containing the screen element; wherein, the determination of the element type is also based on the hit risk of preset constraints, the hit risk being the possibility that the corresponding screen element violates the preset constraints after being replaced. For example, in a product display two-part scene, the main product belongs to the main element category, and replacement generation is performed only on the local area where the product is located to preserve the theme and script logic of the scene; while the background wall of the scene belongs to the background element category, and whole frame replacement generation is performed on the screen containing it to maximize the reduction of redundancy. As another example, if there is a high probability that replacing a certain screen element will change the theme of the scene, i.e., the hit risk is high, it is classified as a main element and partial area replacement is used, thereby reducing the repetition rate while avoiding the replacement from destroying the theme consistency.

[0100] In some embodiments, the main elements include at least one of partial main elements, clothing or decorative details, and partial textures or patterns; the background elements include at least one of background, scene objects, lighting and color tone, and overall image style. For example, the clothing of the person giving the speech and partial patterns on the product are classified as main elements, while the background wall, the objects in the scene, and the overall color tone are classified as background elements, thus providing an operable basis for element classification to determine the replacement granularity.

[0101] In some embodiments, the preset threshold is adaptively adjusted based on the richness of the current batch of footage; the lower the richness of the footage, the more lenient the preset threshold. For example, when there are only an average of 3 candidate clips per storyboard position in a batch, the preset threshold is relaxed from 0.6 to 0.7 to avoid frequent triggering of AIGC replacement when footage is scarce; when there are sufficient candidate clips, the preset threshold is tightened to 0.5 to obtain a more stringent deduplication effect.

[0102] In some embodiments, the AIGC model is invoked to perform replacement generation in response to at least one of the following: insufficient material quantity, style imbalance, or low quality score, in addition to the repetition rate score being higher than a preset threshold.

[0103] In some embodiments, for matrix units with only one candidate clip that still needs to be replaced, the AIGC model is invoked to pre-generate new candidate clips and fill them in before participating in the combination. For example, when a matrix unit of a comparative illustration scene has only one candidate clip and no other clips are available for replacement, the AIGC model is invoked to pre-generate a new candidate clip for that scene position and fill it in, so that the scene position can still participate in deduplication when clips are scarce.

[0104] S5. Replace the replacement footage with the target matrix unit and recalculate the repetition rate score. For example, replace the product display two-part shot with the new footage generated and replace it with the corresponding matrix unit. This will update the candidate materials in that matrix unit with the replaced footage. Then, recalculate the repetition rate score for the candidate short video combination containing that matrix unit. The repetition rate score will decrease from 0.72 to 0.58.

[0105] In some embodiments, after the replacement image is backfilled into the target matrix cell and the repetition rate score is recalculated in S5, the replacement generation of the target matrix cell is stopped in response to the fact that the decrease in the repetition rate score for a consecutive preset number of rounds is lower than a preset decrease threshold. For example, if the preset decrease threshold is 0.02 and the consecutive preset number of rounds is 3, and the repetition rate score of a target matrix cell decreases by 0.05 after the first round of replacement backfilling, 0.015 in the second round, 0.008 in the third round, and 0.005 in the fourth round, then the decreases of 0.015, 0.008, and 0.005 for the three consecutive rounds from the second round onwards are all lower than 0.02, the replacement generation of the target matrix cell is stopped, thereby preventing the invalid call to the AIGC model for target matrix cells that are difficult to further reduce in weight, and controlling the iterative computing power consumption.

[0106] S6. Under the premise of satisfying preset constraints, output a fixed number of K target versions with the lowest repetition rate scores from the candidate short video combinations after recalculating the repetition rate scores. For example, for candidate short video combinations that have been replaced, backfilled, and re-scored, sort them from low to high repetition rate scores under the premise of satisfying preset constraints, and output the top K candidate short video combinations as the target version set. In this scenario, K is 20.

[0107] In some embodiments, the preset constraints include at least one of the following: theme consistency constraint, style consistency constraint, and duration adaptation constraint. For example, it is required that the seven shots of each candidate short video combination belong to the same theme of product promotion, maintain a consistent style, and have an overall duration within a preset duration range. Candidate short video combinations that do not meet the constraints are not included in the target version set.

[0108] In some embodiments, platform rules are used as one of the preset constraints, and low-duplication versions that satisfy the corresponding platform rules are output for different platforms. For example, a set of low-duplication versions is output for the deduplication and duration rules of one platform, and another set of low-duplication versions is output for the rules of another platform.

[0109] In some embodiments, the fixed quantity K is automatically adjusted based on the size of the candidate short video combination and preset constraints. The upper bound of K is determined by the number of versions of the candidate short video combination that can be stably satisfied under a preset threshold. The number of storyboard dimensions N and the number of material slot dimensions M are variable dimensions, and the number of candidate materials stored in each matrix unit is adjustable within a preset range. For example, K is automatically increased when there are more versions that satisfy the preset threshold, and decreased otherwise. The matrix size is also adjusted from 7×7 to other N×M sizes to adapt to batch generation needs of different volumes.

[0110] In some embodiments, each version in the output target version set is associated with a traceability record. The traceability record includes at least one of the following: version identifier, storyboard combination path, candidate material selection result, duplication rate score, AIGC replacement location, material identifiers before and after replacement, export parameters, and version generation time. In this way, each output version retains a complete generation and replacement record, enabling the version generation process to be reproduced, evaluated, and supporting subsequent iterative optimization.

[0111] This application also provides a computer device, as described in the embodiments. Figure 4 The system includes a memory and a processor. The memory stores an application program, and the processor executes the application program to implement the aforementioned method for batch generation of short videos with low repetition. This application also provides a computer-readable storage medium storing an application program, which, when executed by a processor, implements the aforementioned method for batch generation of short videos with low repetition.

[0112] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0113] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC image replacement, characterized in that, include: S1. Establish a storyboard-material slot matrix, which includes N storyboard dimensions and M material slot dimensions, with each matrix unit storing one or more candidate materials; S2. Read the candidate materials from the storyboard-material slot matrix and generate multiple candidate short video combinations according to the storyboard order; S3. Calculate the overlap of materials between versions among the candidate short video combinations to obtain the repetition rate score of each candidate short video combination; S4. In response to the repetition rate score being higher than a preset threshold, a target matrix unit that increases the repetition rate score is determined. The target matrix unit includes a matrix unit that shares the same material, background, or items with any of the generated candidate short video combinations. The AIGC model is called to perform replacement generation on the screen elements corresponding to the target matrix unit to obtain a replacement screen. S5. The replacement screen is backfilled into the target matrix unit, so that the candidate material of the target matrix unit is updated to the replacement screen, and the repetition rate score is recalculated for the candidate short video combination containing the target matrix unit; If the decrease in the repetition rate score for a consecutive preset number of rounds is lower than a preset decrease threshold, the replacement generation for the target matrix unit is stopped; if the recalculated repetition rate score is still higher than the preset threshold, the execution of step S4 is returned. S6. Under the condition of satisfying the preset constraints, output a fixed number of K target versions with the lowest repetition rate scores from the candidate short video combination after recalculating the repetition rate score; S4 includes the following sub-steps: S41. Determine the repetition rate contribution corresponding to each matrix unit, wherein the repetition rate contribution is the expected decrease in the repetition rate score of the candidate short video combination after replacing the corresponding matrix unit; S42. Sort each matrix unit according to its repetition rate contribution from high to low, and determine the first preset number of matrix units as the target matrix units; S43. The AIGC model is called sequentially to generate the replacement for the target matrix units in descending order of their repetition rate contribution, until the number of replacements reaches the preset replacement budget.

2. The method for batch generation of short videos with low repetition according to claim 1, characterized in that, In step S43, when the replacement generation is performed sequentially on the target matrix unit, the element type of the screen element corresponding to the target matrix unit is determined; in response to the element type being a main element, a local area replacement generation is performed on the screen element; in response to the element type being a background element, a whole frame replacement generation is performed on the screen where the screen element is located; wherein, the determination of the element type is also based on the hit risk of the preset constraint condition, the hit risk being the possibility that the corresponding screen element violates the preset constraint condition after being replaced.

3. The method for batch generation of short videos with low repetition according to claim 1, characterized in that, The overlap of materials between versions is determined based on at least one of the following: identical material identifiers, identical clip references, identical background elements, identical item elements, identical local main elements, and identical style templates; the repetition rate score is obtained by summing the overlap of materials between versions of each candidate short video combination, combined with the weights corresponding to each scene dimension and the penalty weights corresponding to repeated materials whose frequency of occurrence exceeds a preset frequency threshold.

4. The method for batch generation of short videos with low repetition according to claim 3, characterized in that, In the process of generating multiple candidate short video combinations according to the storyboard order, in response to the candidate material to be added causing the overlap between the current candidate short video combination and any generated candidate short video combination to exceed a preset pruning threshold, the candidate material to be added is pruned. The preset pruning threshold and the overlap between the versions of the material are calculated using the same caliber.

5. The method for batch generation of short videos with low repetition according to claim 1, characterized in that, The candidate materials stored in each matrix unit include at least one of original video materials, cropped clip materials, and AIGC pre-generated materials, and each candidate material is associated with an attribute set, which includes at least one of material identifier, theme tag, style tag, duration, background features, item features, subject features, and replaceable areas.

6. The method for batch generation of short videos with low repetition according to claim 1, characterized in that, The preset threshold is adaptively adjusted based on the richness of the materials in the current batch.

7. The method for batch generation of short videos with low repetition according to claim 1, characterized in that, The duplication rate score is based on the overlap of materials between versions, plus at least one of visual similarity, semantic similarity, or shot similarity to form a joint deduplication index; and firstly, the version pairs composed of two candidate short videos are coarsely screened based on the overlap of the same material identifier or the same segment reference, and only the version pairs whose overlap exceeds the preset threshold are then calculated for the visual similarity or the semantic similarity.

8. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores an application program, and the processor executes the application program to implement the method for low-repetition batch generation of short videos based on matrix combination optimization and AIGC screen replacement as described in any one of claims 1 to 7.

9. A computer-readable storage medium having an application program stored thereon, characterized in that, When the application is executed by the processor, it implements the short video low-repetition batch generation method based on matrix combination optimization and AIGC screen replacement as described in any one of claims 1 to 7.

Images