Short video marketing material intelligent clipping method based on lightweight large model

Abstract

The application discloses a short video marketing material intelligent clipping method based on a light-weight large model, and particularly relates to the technical field of short video image processing, and is used for solving the problems of lacking of film output frame retest and feedback writing mechanism in the existing short video marketing material clipping, leading to the problems of subject being cut, subtitle being overlaid, insufficient exposure of selling points and repeated occupation of materials. By establishing frame-level state records according to source material frame numbers, output frame retest is carried out after target short video generation, and according to feedback difference records, writing correction can be carried out on the cutting boundary, subtitle avoidance area, selling point binding state and material occupation locking interval to generate the next round of clipping permission state, so that the effect of continuous association of material judgment before clipping, actual film performance and subsequent clipping boundary is achieved, and the problems of subject being cut, subtitle being overlaid, insufficient exposure of selling points and repeated occupation of materials are reduced.

Description

Technical Field

[0001] This invention relates to the field of short video image processing technology, specifically to an intelligent editing method for short video marketing materials based on a lightweight large model. Background Technology

[0002] With the increasing demand for mass production of short video marketing content, existing intelligent editing methods for short video materials typically combine video image recognition, material tagging, script text analysis, or large-scale model semantic matching to screen candidate video materials and complete segment extraction, aspect ratio cropping, subtitle overlay, shot ordering, and transition splicing according to a final cut template to generate marketing short videos for different platforms. These methods can select candidate segments based on material content, script selling points, or template rules before editing and make preliminary arrangements for final cut specifications, subtitle positions, and segment order.

[0003] However, in actual mass production, the fact that a candidate segment is deemed usable before editing does not guarantee that it will meet the final requirements after the target short video is generated. This is especially true in scenarios involving vertical screen cropping, subtitle overlay, transitions, and multiple rounds of material reuse. Originally usable segments may result in issues such as cropped main areas, overlaid subtitles obscuring existing subtitles or main content, insufficient exposure of selling points, and repeated use of the same source material frame number range. Existing methods primarily focus on pre-editing material understanding, segment matching, and editing strategy generation, paying insufficient attention to the frame-by-frame performance of the target short video after output.

[0004] Therefore, existing intelligent editing methods for short video marketing materials suffer from a lack of continuous verification between pre-editing material judgment and the actual finished product performance. In particular, there is a lack of a mechanism to verify the output frame performance of the target short video against the source material frame number, cropping window, subtitle placement, selling point exposure, and material usage. This makes it difficult to constrain subsequent editing boundaries in a timely manner when problems arise after the finished product is produced, affecting the consistency and reusability of batch short video editing results. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent editing method for short video marketing materials based on a lightweight large model, in order to solve the problems mentioned in the background.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent editing of short video marketing materials based on a lightweight large model, comprising: S1. Read the marketing script's selling point nodes, target final frame constraints, candidate video materials, and historical material occupancy records. Establish a frame-level status record according to the source material's frame number. The frame-level status record includes the main body area, subtitle area, cropping safety area, and occupied segment interval. S2. Input the marketing script selling point nodes into the lightweight large model, output the initial binding relationship between the marketing script selling point nodes and the candidate segments, and generate a usable version of the finished product based on the frame-level state record. S3. Generate an editing strategy based on the available version of the finished film. The editing strategy includes segment truncation boundaries, cropping windows, subtitle landing points, shot ordering, transition positions, and expected finished film field groups. S4. According to the editing strategy, the candidate segments are extracted, cut, overlaid, sorted and transitioned to generate the target short video, and the output frame mapping record is generated simultaneously. S5. Perform output frame retesting on the target short video to obtain the main visible ratio, subtitle overlap ratio, selling point exposure duration and material occupation range, and compare it with the expected final video field group to generate feedback difference records. S6. Based on the feedback difference record, write back and correct the croppable boundary, subtitle avoidance area, selling point binding status and material occupation lock range, and generate the next round of editing permission status.

[0007] Furthermore, establishing frame-level status records based on the source material frame number includes: Lock the marketing script's key selling points version and the target final image frame constraint version according to the task number; Read and merge candidate video clips and historical clip usage records by clip number; Generate source material frame numbers according to the decoding order of candidate video materials; Using task number, material number, and source material frame number as the smallest unit of recording, the main area, subtitle area, cropping safe area, and occupied segment range are written into the frame-level status record.

[0008] Furthermore, the generation of initial binding relationships and shard-ready state versions includes: Merge consecutive frame-level state records of source material frame numbers within the same candidate video material into candidate segments; The lightweight large model forms an initial binding relationship between the marketing script's selling point nodes and the candidate segments based on the node text of the selling point nodes, the main area image within the candidate segments, and the text content of the subtitle area. Based on the main area, subtitle area, cropping safe area, and occupied segment range, the candidate segments pointed to by the initial binding relationship are checked to generate a usable version of the finished film.

[0009] Furthermore, the generation of editing strategies includes: Read the initial binding relationship that is available from the available state version of the complete piece; Determine the segment truncation boundaries based on the starting source material frame number, the ending source material frame number, and the suggested exposure time period; The cropping window is determined based on the cropping safety area and the target image frame constraints. The subtitle placement point is determined based on the allowed subtitle area, the prohibited occlusion area, and the subtitle area converted to the coordinates of the target final image according to the cropping window in the target final image constraints; The shot order is determined based on the node order and binding order; Transition positions are determined based on the boundaries between adjacent segments in the shot sequence; The expected final video field group is formed based on the segment capture boundaries, cropping window, subtitle landing point, shot order, and transition position.

[0010] Furthermore, the generation of the target short video and output frame mapping record includes: Read the source frame of the candidate segment by segment boundary; The target image is formed based on the cropping window and the target image size constraints. Write the node text of the selling point node in the marketing script according to the subtitle ending point; The target short video is created by sorting shots and transition positions; Output frame numbers are generated according to the order of the frames in the target short video, and output frame mapping records are generated simultaneously, including the target short video number, output frame number, node number, material number, source material frame number, cropping window, subtitle landing point, and editing strategy version.

[0011] Furthermore, the output frame retest includes: Read the target short video and output frame mapping records according to the target short video number, output frame number, and editing strategy version; Based on the output frame mapping record, trace the source material frame number and cropping window, and read the overlapping text area and the subtitle area converted to the coordinates of the target finished screen; Obtain the main body visibility ratio and subtitle overlap ratio according to the output frame number; Obtain the exposure duration of selling points by node number; Obtain the material's occupied range by material number.

[0012] Furthermore, the generation of feedback discrepancy records includes: Read the expected final video field group under the same editing strategy version; The main body visibility ratio, subtitle overlap ratio, selling point exposure duration, and material occupation range are compared with the expected final film field group respectively; Generate feedback difference records based on the target short video number, output frame number, node number, material number, source material frame number, and editing strategy version, and write the difference type in the feedback difference records.

[0013] Further write-back corrections include: Read the feedback difference record and the output frame mapping record; Based on the visible differences of the subject corresponding to the cut-off boundary, the actual visible range of the subject is converted from the coordinates of the target finished image back to the coordinates of the source material frame and then the croppable boundary is corrected. The subtitle avoidance area should be written based on the subtitle overlap difference; Update the selling point binding status based on the differences in selling point exposure; Write the material occupancy lock range according to the material occupancy range.

[0014] Furthermore, the next round of editing allows for the generation of states including: The accounting unit is the task number, node number, material number, source material frame number, and write-back version. Check the locked area occupied by the material, the croppable boundary, the subtitle avoidance area, and the binding status of the selling points in sequence; The next round of editing permission status will be determined in the following order: prohibited from use, pending cropping verification, pending subtitle verification, and pending binding verification. If none of the aforementioned states are triggered, it is determined to be allowed.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By establishing frame-level status records based on the source material frame number, including the main body area, subtitle area, cropping safety area, and occupied segment range, and after the target short video is generated, the output frame is retested based on the output frame mapping record. Then, based on the feedback difference record, the cropping boundary, subtitle avoidance area, selling point binding status, and material occupation lock range are written back to correct, generating the next round of editing allowable status. This achieves the effect of continuously linking the material judgment before editing, the actual finished product performance, and the next round of editing boundaries, avoiding the repeated problems of the main body being cropped, subtitles being covered, selling points not being exposed, and material being occupied after candidate segments are cut, cropped, superimposed, sorted, and transitioned.

[0016] 2. By initially binding the marketing script's selling point nodes with candidate segments, and generating a usable version of the finished product based on frame-level status records, the usable version of the finished product is then used to filter candidate segments that can enter the editing strategy. This achieves the effect of uniformly constraining the main body area, subtitle area, cropping safety area, and occupied segment range before the editing strategy is generated, reducing the number of unusable segments, segments awaiting verification, and occupied segments that directly enter the finished product generation process.

[0017] 3. By simultaneously generating output frame mapping records while generating the target short video, and associating the output frame number in the target short video with the node number, material number, source material frame number, cropping window, subtitle landing point, and editing strategy version, the final result can be traced frame by frame. This allows output frame retesting, feedback difference generation, and subsequent status write-back to locate specific material frames and specific editing fields, improving the verifiability and consistency of batch short video editing results. Attached Figure Description

[0018] Figure 1 A schematic diagram illustrating the overall process of intelligent editing methods for short video marketing materials; Figure 2 A schematic diagram for establishing frame-level state records; Figure 3 A schematic diagram is generated for the initial binding relationship and the available version of the finished product; Figure 4 Generate a diagram for the editing strategy; Figure 5 A schematic diagram illustrating the synchronous recording of frame mapping for the generation and output of the target short video. Figure 6 A schematic diagram for generating output frame retest and feedback difference records; Figure 7 A diagram illustrating the write-back and correction of feedback discrepancies. Figure 8 This diagram illustrates the allowed states for the next round of editing. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example: Combined with Appendix Figures 1-8 This embodiment provides a method for intelligent editing of short video marketing materials based on a lightweight large model, including: S1. Read the marketing script's selling point nodes, target final frame constraints, candidate video materials, and historical material occupancy records. Establish a frame-level status record based on the source material's frame number. The frame-level status record includes the main body area, subtitle area, cropping safety area, and occupied segment interval. Specifically, this is implemented as follows: The marketing script's selling point nodes are read from the planning and arrangement sheet of the script arrangement workstation when the production task for this batch of short video marketing materials is established. The planning and arrangement sheet uses the task number as the batch index and takes effect in the submission window after the script is confirmed. The marketing script's selling point nodes are organized item by item according to node number, node text, node order, and suggested exposure time. The node number is a fixed number that cannot be repeated within the same task. The node text is the text content confirmed by the script arrangement workstation. The node order is the arrangement position in the planning and arrangement sheet. The suggested exposure time is recorded in seconds and is only used as the task source identifier when establishing frame-level status records in this step. It is not used to generate editing strategies in this step.

[0021] When organizing, first merge the marketing script selling point nodes in the same batch according to the task number, and then remove empty nodes, duplicate node numbers and unconfirmed draft nodes. If the node number is missing, the node text is missing, the node order is missing, or the version of the planning and layout sheet cannot be traced, the marketing script selling point node will not be automatically filled in, but will be transferred to the pending supplementation. Confirmed nodes in the same batch will continue to be archived.

[0022] The target final frame aspect ratio constraint is read from the locked platform final frame template of the final frame specification configuration station. The target final frame aspect ratio constraint is recorded according to the final frame width, final frame height, orientation, allowed subtitle area, and prohibited occlusion area. The final frame width and height are recorded in pixels, and the orientation is recorded as one of landscape, portrait, or square screen. The boundary coordinates of the allowed subtitle area and the prohibited occlusion area are recorded with the upper left corner of the final frame as the origin. When the same task corresponds to multiple locked platform final frame templates, the target final frame aspect ratio constraint version is formed for each template, and the target final frame aspect ratio constraints of different platforms are not merged. The target final frame aspect ratio constraint is associated with the target short video frame rate, which is read from the same locked platform final frame template and recorded in frames per second. When the locked platform final frame template lacks the target short video frame rate, the template filing is stopped and it is transferred to the pending verification stage. It is not automatically supplemented from the source material frame rate of the candidate video material.

[0023] If the width, height, or orientation of the completed text is missing, the template creation process will be stopped and the template will be transferred to a pending review. If the subtitle area or the obscured area is missing, it is only allowed to be filled in from the locked default template of the same platform and orientation, and the default template number, default template version, and filling time will be recorded.

[0024] Candidate video materials are read from the material list and material file repository submitted by the material management workstation. Candidate video materials are confirmed by material number, material file identifier, source material duration, source material frame rate, source material width, source material height, and material authorization status. The source material duration is recorded in seconds, the source material frame rate is recorded in frames per second, the source material width and the source material height are recorded in pixels, and the material authorization status is read from the material entry approval record.

[0025] When organizing candidate video materials, first associate the material file with the material number, then check the material file summary, then read the source material duration, source material frame rate, source material width, and source material height, and finally establish the source material frame number according to the order of the decoded source material images; the first frame of the source material is numbered 0, and the subsequent source material frame numbers are incremented frame by frame according to the decoding order. If there is an undecodeable frame, the source material frame number is retained and an unreadable mark is recorded, and the subsequent source material frame numbers are not moved forward.

[0026] If the source material file is corrupted, the source material authorization status is invalid, the source material frame rate cannot be read, the source material width is missing, the source material height is missing, or the source material number is inconsistent with the source material file summary, the candidate video material will be transferred to pending verification without interrupting the archiving of other candidate video materials in the same batch.

[0027] Historical material usage records are read from the material usage ledger of submitted and effective finished films. Historical material usage records are recorded by task number, finished film number, material number, usage start frame number, usage end frame number, and usage version. Both the usage start frame number and usage end frame number include the endpoint. Records corresponding to withdrawn finished films, unsubmitted finished films, and uneffective finished films are not included in this step.

[0028] If no historical material usage record is found by material number, the occupied segment range is recorded as an empty range, and the search time and material usage ledger version are recorded.

[0029] When establishing frame-level status records based on source material frame numbers, first lock the marketing script selling point node version, then lock the target final image aspect ratio constraint version, then merge candidate video materials and historical material usage records, and finally write the frame-level status records with task number, material number, and source material frame number as the smallest accounting unit. The frame-level status records should at least include the main area, subtitle area, cropping safe area, and occupied segment range, and simultaneously record the marketing script selling point node version, target final image aspect ratio constraint version, candidate video material version, and historical material usage record version.

[0030] The main area records pixel coordinates with the top left corner of the source material frame image as the origin. The recording scope is the horizontal coordinate of the top left vertex, the vertical coordinate of the top left vertex, the horizontal coordinate of the bottom right vertex, and the vertical coordinate of the bottom right vertex. The main area comes from the main object annotation in the material library, the manual main object confirmation record, and the main object frame verification result in the source material frame. When all three exist and are inconsistent, the manual main object confirmation record shall prevail, and the replaced source shall be retained.

[0031] When a source material frame contains multiple main objects, the area that has been recorded and marked as the main object is selected; when no main object is marked, the complete object area with the largest coverage area in the center of the screen is selected; when the main area cannot form closed coordinates, the coordinates exceed the width and height boundaries of the source material, or the main object is occluded to the point that the boundary cannot be confirmed, the main area of ​​the source material frame is recorded as to be supplemented and will not be automatically supplemented.

[0032] The subtitle area records pixel coordinates with the top left corner of the source image frame as the origin, and the recording diameter is the same as the main body area; the subtitle area is the result of organizing the text overlay boxes of existing subtitles, corner marks, patch text and product description text in the source image frame.

[0033] When a source frame contains multiple text overlay boxes, each text overlay box is recorded separately, and adjacent text overlay boxes are not merged. If the boundary of a text overlay box is not closed, the text overlay box exceeds the boundary of the source frame, or it is not confirmed by manual review, the text overlay box is recorded as to be supplemented and is not automatically replaced by the subtitle area of ​​the adjacent frame.

[0034] The cropping safe zone is formed based on the target frame size constraints, the width of the source material, and the height of the source material. The area of ​​the image that can be retained is recorded according to the image coordinates of the source material frame. The cropping safe zone only records the safe boundaries that can participate in subsequent cropping under the target frame size constraints, and does not determine the actual cropping window in advance.

[0035] When the source image frame size and the target final image frame size constraint direction are consistent, the cropping safe zone covers the largest continuous area in the source image frame that satisfies the target final image frame ratio. If multiple largest continuous areas exist, the area covering the center of the main subject area is selected first. If multiple areas still exist, the area with the smaller horizontal coordinate of the top-left vertex is selected. If the horizontal coordinates are still the same, the area with the smaller vertical coordinate of the top-left vertex is selected. When the directions are inconsistent, the largest area within the source image frame that can cover the main subject area without exceeding the source image frame boundary is reserved based on the target final image frame size constraint. If the main subject area and the source image frame boundary cannot be covered simultaneously, the cropping safe zone of that source image frame is marked as pending verification.

[0036] The occupied segment range is filled back to the corresponding source material frame number range by the material number from the historical material occupation record. When there are multiple historical material occupation records for the same candidate video material, they are retained separately in frame number order, and records with blank frame segments are not merged. When the occupied segment range overlaps with an unreadable frame, the occupied segment range is still retained, and an unreadable mark is recorded under the corresponding source material frame number.

[0037] When writing frame-level status records to the media status record library, the task number, media number, source media frame number, and frame-level status record version are used as the write key. If the same write key is submitted repeatedly, the already recorded record will be returned and no duplicate record will be added. Within the same media, submissions are made in ascending order of source media frame number, and different media can be submitted in parallel.

[0038] The old frame-level status record version is read-only and will not be overwritten by the new version. When reorganizing, a new frame-level status record version is generated, and the previous version number, reorganization time, submission account, rule version and field source are retained.

[0039] When a single frame fails to be written, the frame number of the source material is written to be supplemented and the reason for the failure is recorded. Other source material frame numbers in the same batch continue to be submitted. When the number of consecutive writing failures of the same material reaches a preset number, the candidate video material is transferred to be checked and other candidate video materials in the same batch continue to be archived.

[0040] This step only reads the marketing script selling point nodes, target frame constraints, candidate video materials, and historical material usage records within the authorized scope of this batch of tasks. It does not read user profiles, transaction orders, or campaign conversion records. The trace content written to the material status record library includes the fields source, source version, submission time, submission account, and writing result.

[0041] Preferably, in the batch production of vertical screen product short videos, the target final frame size constraint can be set to a width of 1080 pixels, a height of 1920 pixels, and a frame rate of 30 frames per second. The number of candidate video materials in a single batch can be set to 20 to 200 segments, the duration of a single candidate video material can be set to 5 to 180 seconds, the maximum delay for submitting a single batch of files can be set to 30 seconds, the number of retries for write failures can be set to 2, the retry interval can be set to 5 seconds, the number of consecutive write failures can be set to 5, and on-site sampling can be set to sample 30 candidate video materials from each batch, not less than 3% of the total batch size. During the sampling, the frame images of the source material are checked to ensure they match the main subject. The correspondence between regions, subtitle regions, cropping safe zones, and occupied segment intervals; for example, when the frame rate of a candidate video source material is 30 frames per second and the duration is 12 seconds, the source material frame numbers are 0 to 359. If the source material frame numbers 45 to 120 contain the main product region, and the source material frame numbers 80 to 110 contain the existing subtitle region, and the historical material occupancy record shows that the source material frame numbers 60 to 90 have been occupied in the previous effective finished film, then the source material frame numbers 80 to 90 simultaneously record the main product region, subtitle region, and occupied segment interval, and record the corresponding cropping safe zone according to the target finished film aspect ratio constraint of 1080 pixels by 1920 pixels.

[0042] Alternatively, the main body area and the subtitle area can be formed by the manual annotation results during the material entry stage, or by the detection and organization results of the source material frame images during the current batch archiving stage. When both fall into the same frame-level state record according to the source material frame number, pixel coordinates, field source and version caliber, it is an equivalent implementation of this embodiment.

[0043] S2. Input the marketing script selling point nodes into the lightweight large model, output the initial binding relationship between the marketing script selling point nodes and candidate segments, and generate a usable version of the finished product based on the frame-level state record. The specific implementation is as follows: After the marketing script's selling point nodes are version-locked at the material arrangement workstation, they are sent to the lightweight large model. The lightweight large model is deployed in the material arrangement server of the short video material production line, and its runtime is limited to after the frame-level status record is completed and before the editing strategy is generated. The lightweight large model runs with a fixed model version, a fixed prompt rule version, and the task number of this batch. The submitted content includes the node number, node text, node order, suggested exposure time of the marketing script's selling point nodes, as well as the material number, starting source material frame number, ending source material frame number, source material frame image, main area, subtitle area, cropping safety area, and occupied segment range corresponding to the candidate segment. User profiles, transaction orders, and campaign conversion records are not submitted.

[0044] Candidate segments are formed by merging consecutive frame-level status records of source material frame numbers within the same candidate video material. The merging boundary is recorded according to the material number, the starting source material frame number, and the ending source material frame number. When a source material frame number has an unreadable marker, that source material frame number is used as a break point and does not enter any candidate segment. Consecutive source material frame numbers before and after the break point form candidate segments respectively. The conditions for breaking off candidate segments are: main area to be supplemented, subtitle area to be supplemented, cropping safe area to be checked, and occupied segment interval not empty. Instead, these conditions are formed in the subsequent usable state version of the finished video.

[0045] When the lightweight large model reads the selling point nodes of the marketing script, it reads the node text, node order, and suggested exposure time period item by item according to the node number. It also reads the image content, main area position, subtitle area text content, continuous duration of the segment, and occupied segment range of the source material frame image within the candidate segment. The continuous duration of the segment is determined by the number of source material frame numbers and the frame rate of the source material within the candidate segment. The subtitle area text content comes from the sorting result of the text overlay box corresponding to the subtitle area in the frame-level status record. The main area position is read according to the pixel coordinates corresponding to the origin of the upper left corner of the source material frame image.

[0046] The content of the candidate segment is organized according to the main area, surrounding area, and subtitle text content in the readable source material frame image within the candidate segment. When the lightweight large model forms the initial binding relationship, the binding relationship is recorded only when the text content in the node text can correspond to the main area, surrounding area, or subtitle text content. If no correspondence can be established, the initial binding relationship between the candidate segment and the marketing script's selling point node is not generated. The lightweight large model does not change the main area, subtitle area, cropping safe area, or occupied segment interval in the frame-level state record. It only forms the initial binding relationship between the selling point node and the candidate segment based on the correspondence between the node text and the content of the candidate segment's images and subtitle text content.

[0047] The completeness of the main body region is determined by the number of source material frames whose main body region has been recorded within the readable source material frames in the candidate segment and has not exceeded the boundaries of the source material width and height. Source material frames whose main body region is to be supplemented, whose main body region coordinates are out of bounds, or whose main body region cannot be covered by the cropping safe area are not included in the completeness of the main body region. The initial binding relationship is determined by the task number, node number, material number, starting source material frame number, ending source material frame number, binding order, and binding source version record. When the same marketing script selling point node corresponds to multiple candidate segments, all candidate segments are retained first, and then the binding order is determined by the completeness of the main body region within the candidate segment, the continuous duration of the segment, the segment interval occupied, and the starting source material frame number. When the completeness of the main body region is the same, the segment with the longer continuous duration is ranked first. When the continuous duration of the segments is the same, the segment with an empty segment interval is ranked first. If they are still the same, the segment with the smaller starting source material frame number is ranked first.

[0048] When the same candidate segment corresponds to multiple marketing script selling point nodes, the initial binding relationship is recorded according to the node number, and no binding relationship is deleted in this step.

[0049] When generating a usable version of the finished film based on frame-level status records, the candidate segments pointed to by the initial binding relationship are used as objects. The main body area, subtitle area, cropping safe area, and occupied segment range are checked item by item according to the current target finished film aspect ratio constraints. The status value of the usable version of the finished film is limited to one of the following: usable, pending cropping check, pending subtitle recording, pending occupation check, and unusable.

[0050] The status formation order is fixed as follows: first check the main body area, then check the subtitle area, then check the cropping safe area, and finally check the occupied segment range. When each readable source material frame in the candidate segment has a main body area, and the pixel coordinates of the main body area do not exceed the width and height boundaries of the source material, and the main body area is covered by the corresponding cropping safe area, the main body area item passes. When the main body area of ​​any readable source material frame is missing, the coordinates of the main body area are out of bounds, or the main body area cannot be covered by the cropping safe area, the finished product usable status version is recorded as pending cropping verification.

[0051] When there is a "to be added" mark in the subtitle area, the finished product version is marked as "to be added"; when the subtitle area has been completely recorded and there is no pixel coordinate overlap between the subtitle area and the prohibited occlusion area in the target finished product frame constraint, the subtitle area item passes; when there is pixel coordinate overlap, the candidate segment still retains the initial binding relationship, and the finished product version is marked as "to be added".

[0052] If the crop safety zone has been formed in every readable source material frame within the candidate segment, and the version of the crop safety zone is consistent with the version of the target final image frame constraint, the crop safety zone item passes; if there is a pending verification mark in the crop safety zone, or if the version of the crop safety zone is inconsistent with the version of the target final image frame constraint, the final image usable state version is marked as pending crop verification.

[0053] When the occupied segment interval is empty, the occupied segment interval item passes; when the occupied segment interval has overlapping frame numbers at the endpoints of the candidate segment's starting source material frame number to the ending source material frame number, the finished product's usable status version is recorded as pending occupation verification.

[0054] When the main area, subtitle area, cropping safety area, and occupied segment range all pass, the finished product is recorded as usable. When the number of readable source material frames in a candidate segment is 0, or when the material authorization status expires during this step of review, the finished product is recorded as unusable. When the same candidate segment triggers cropping check, subtitle addition check, and occupied check simultaneously, the main status is recorded in the order of cropping check, subtitle addition check, and occupied check, and the reasons for the other statuses are retained.

[0055] The available version of the finished product is written to the material status record library by task number, node number, material number, starting source material frame number, ending source material frame number, initial binding relationship version, frame-level status record version, and target finished product aspect ratio constraint version. When the same write key is submitted repeatedly, the already recorded version is returned and no duplicate version is generated. The old available version of the finished product is read-only and is retained. When it is recreated, only the new version is generated, and the lightweight large model version, prompt rule version, submission account and submission time are retained.

[0056] If the lightweight large model fails to establish an initial binding relationship for a certain marketing script selling point node within the current batch submission window, the marketing script selling point node will be recorded as pending supplementation, which will not affect the formation of initial binding relationships for other marketing script selling point nodes in the same batch. If a candidate segment is missing any of the following: material number, starting source material frame number, ending source material frame number, or frame-level status record version, the candidate segment will be transferred to pending verification and will not participate in the generation of the final usable status version.

[0057] Different marketing script selling point nodes can be sent into the lightweight large model in parallel. Candidate segments under the same marketing script selling point node are submitted in ascending order of material number and starting source material frame number to maintain the stability of the initial binding relationship and the traceability relationship of the same source material frame number in the finished product usable version.

[0058] Preferably, in the batch production of vertical screen product short videos, the lightweight large model can receive 20 to 80 marketing script selling point nodes at a time. The number of candidate segments corresponding to a single marketing script selling point node can be set to 3 to 30 segments. The shortest continuous duration of the candidate segments can be set to 1.2 seconds. The number of frames continuously covered by the main area can be set to no less than 36 frames. The upper limit of the submission delay can be set to 20 seconds. The number of retries for failure can be set to 2. The retry interval can be set to 5 seconds. On-site spot checks can be set to sample no less than 20 initial binding relationships per batch and no less than 5% of the total number of initial binding relationships in the batch. During the spot checks, the node number, material number, starting source material frame number, and ending source material frame number are verified. Consistency of frame-level status records; for example, a marketing script's selling point node has a node number of 3 and a node text of "waterproof test". The lightweight large model initially binds it to the source material frame numbers 45 to 120 of material number A. The frame-level status record shows that each readable source material frame in this range has a main body area, the subtitle area is complete, the cropping safety area is consistent with the target final image aspect ratio constraint version of 1080 pixels by 1920 pixels, and the occupied segment range is an empty range. Then the corresponding final image usable status version record is usable. If there is an occupied segment range of source material frame numbers 60 to 90 in the same range, then the corresponding final image usable status version record is pending occupancy verification.

[0059] Alternatively, the lightweight large model can be deployed on a local material arrangement server or on a controlled computing power node within the same business production line; the source material frame images within the candidate segments can be submitted frame by frame or representative frame images extracted at a fixed frame interval. As long as the marketing script selling point node, candidate segments, initial binding relationship, and finished product usable version all fall into the material status record library according to the task number, node number, material number, source material frame number, and version caliber, it is an equivalent implementation of this embodiment.

[0060] S3. Generate an editing strategy based on the available version of the finished film. The editing strategy includes segment truncation boundaries, cropping windows, subtitle placement, shot order, transition positions, and expected finished film field groups. Specifically, the implementation is as follows: After reading the final, usable version of the completed video, the media arrangement station generates an editing strategy. The execution entity is the editing and arrangement server in the short video media production line, and its effect is limited to the period after the final, usable version of the completed video is generated and before the target short video is generated. The editing and arrangement server reads the task number, node number, media number, starting source media frame number, ending source media frame number, final, usable version of the completed video, initial binding relationship version, frame-level status record version, and target final video aspect ratio constraint version from the media status record library, and uses the task number and the final, usable version of the completed video as the scope of this editing strategy.

[0061] In the finished product's usable version, initial binding relationships with a usable status are included in the editing strategy generation. Initial binding relationships with a status of pending trimming and verification, pending subtitle recording, pending occupation verification, or unusable are only retained for the reason of their status and are not included in this editing strategy. When there are no usable initial binding relationships under the same marketing script's selling point node, that node is recorded as a node to be recorded and is not replaced by a segment to be verified.

[0062] Editing strategies are organized into executable records based on task number, editing strategy version, node number, material number, start frame number, and end frame number. Old editing strategy versions are read-only and are retained. When reorganizing, only new versions are generated, and the version of the finished product in usable state, the version of the initial binding relationship, the version of the frame-level status record, the version of the target finished product aspect ratio constraint, and the submission account are retained.

[0063] The segment truncation boundary is determined by the starting source material frame number, ending source material frame number, and the suggested exposure time of the marketing script's selling point node in the available initial binding relationship. This is recorded as the starting frame number, ending frame number, and the material number to which the segment belongs. The suggested exposure time is recorded in seconds. The editing and arrangement server converts the required number of consecutive source material frames based on the source material frame rate. Within the continuous range of candidate segments, consecutive source material frame segments that meet this number are selected starting from the starting source material frame number. The starting frame number must not be less than the starting source material frame number, and the ending frame number must not be greater than the ending source material frame number. The ending frame number includes the endpoint. If the continuous range of candidate segments is insufficient to cover the suggested exposure time, the complete boundary of the candidate segment is taken, and the reason for the insufficient exposure time is recorded in the editing strategy.

[0064] The cropping window is formed based on the cropping safe zone corresponding to the source material frame number within the segment truncation boundary and the target image frame constraint. The horizontal coordinate of the top left vertex, the vertical coordinate of the top left vertex, the horizontal coordinate of the bottom right vertex, and the vertical coordinate of the bottom right vertex are recorded with the top left corner of the source material frame image as the origin. The cropping window must be located within the cropping safe zone, maintain the same aspect ratio as the target image frame constraint, not cross the width and height boundaries of the source material, and cover the main body area in the corresponding source material frame.

[0065] When multiple cropping windows meet the conditions, first compare the horizontal and vertical pixel distances between the center of the cropping window and the center of the main area, and select the cropping window with the smaller total value of the horizontal and vertical pixel distances; if the total values ​​are the same, select the cropping window with the smaller horizontal coordinate of the top left vertex; if the horizontal coordinates are still the same, select the cropping window with the smaller vertical coordinate of the top left vertex.

[0066] When the position of the main area within the boundary of the same segment changes, the coordinates of the cropping window are recorded separately according to the frame number of the source material, and the cropping windows of multiple source material frames are not merged into a fixed coordinate.

[0067] The subtitle landing point is determined based on the allowed subtitle area, prohibited occlusion area and subtitle area recorded in the frame-level status record of the target final frame. The coordinates of the text box are recorded with the upper left corner of the target final frame as the origin. Before forming the subtitle landing point, the subtitle area within the segment cut boundary is converted to the coordinates of the target final frame according to the corresponding cropping window, so that the subtitle area and the subtitle landing point are in the same coordinate system.

[0068] Subtitles must not fall into the prohibited occlusion area, and must not overlap with the converted subtitle area in pixel coordinates. When multiple placeable areas are allowed within the subtitle area, the first area that can completely accommodate the text box corresponding to the node text is selected from top to bottom and from left to right according to the node order. If a subtitle cannot be formed, the editing strategy corresponding to that node is recorded as pending recording, without changing the editing strategies of other generated nodes.

[0069] The shot order is based on the node order of the marketing script's selling point nodes and the binding order in the initial binding relationship. First, the marketing script's selling point nodes are arranged according to their node order, and then the candidate segments are arranged according to their binding order within the same node. When there are multiple available initial binding relationships for the same node, the initial binding relationship with the earliest binding order that has not been included in this editing strategy by the preceding node is selected, and other available initial binding relationships are reserved as alternative records. If all available initial binding relationships under the same node have been included in this editing strategy by the preceding node, then the node is recorded as a node to be added.

[0070] The transition position is formed based on the boundary of adjacent segments in the shot sequence, and is recorded as the connection position after the end frame number of the previous segment and before the start frame number of the next segment. The transition position only indicates the segment connection position when the target short video is generated, and is not rendered in this step. The number of connecting frames is recorded synchronously in the transition position record. The number of connecting frames is in units of frames. When directly connecting, the number of connecting frames is recorded as 0. The number of connecting frames is only used as the frame number boundary for subsequent transition processing, and is not generated into the final frame in this step.

[0071] The transition position must not fall within the segment corresponding to the suggested exposure time of any marketing script's selling point node; when adjacent segments are directly connected, the transition position is still recorded after the end frame number of the previous segment.

[0072] The expected final cut field group is written into the material status record library along with the editing strategy. The expected final cut field group includes the expected main body visibility ratio, the expected subtitle unobstructed state, the expected highlight exposure duration, the expected material occupancy range, and the expected final cut duration. The expected main body visibility ratio is formed by the pixel area diameter of the main body area covered by the cropping window and recorded as a percentage. The expected subtitle unobstructed state is formed by the overlap of the subtitle landing point with the converted subtitle area and the prohibited obstruction area pixel coordinates and recorded as unobstructed or obstructed. The expected highlight exposure duration is formed by the number of consecutive source material frames from the start frame number to the end frame number of the cropping according to the source material frame rate and is in seconds. The expected material occupancy range is formed by the material number, the start frame number, and the end frame number of the cropping. Both the start frame number and the end frame number of the cropping include the endpoints. The expected final cut duration is formed by merging the corresponding durations of each segment cropping boundary and the transition positions and is in seconds.

[0073] When writing editing strategies, the task number, editing strategy version, node number, material number, start frame number, and end frame number are used as the write keys. If the same write key is submitted repeatedly, the already written record will be returned, and no duplicate record will be added. Within the same task, submissions are made in ascending order of shot sorting, and different tasks can be submitted in parallel.

[0074] For nodes that fail to write, the reason for the failure is recorded and the node is moved to the pending record-keeping stage, without interrupting the editing strategy of other nodes in the same task. When the number of consecutive write failures reaches a preset number, the editing strategy version of this task stops submitting and keeps the already recorded records read-only.

[0075] This step only reads the available version of the finished product, the initial binding relationship, the frame-level status record, the target finished product aspect ratio constraint, and the marketing script selling point node; it does not read user profiles, transaction orders, or campaign conversion records.

[0076] Preferably, in the batch production of vertical screen product short videos, the target final frame size constraint can be set to a width of 1080 pixels and a height of 1920 pixels, the source material frame rate can be set to 30 frames per second, the suggested exposure time for a single marketing script selling point node can be set to 1.2 seconds to 3 seconds, the minimum interval between the subtitle landing point and the converted subtitle area can be set to 24 pixels, the submission delay limit can be set to 15 seconds, the number of retries for failure can be set to 2, the retry interval can be set to 5 seconds, the number of consecutive write failures can be set to 5, and on-site spot checks can be set to sample no less than 20 editing strategies per batch and no less than 5% of the total batch editing strategies. During the spot checks, the segment truncation boundary, the cropping window, and the subtitle landing point are checked. The consistency of the points, shot order, transition positions, expected final cut field group with the available version and frame-level status records of the final cut; for example, if node number 3 corresponds to source material frame numbers 45 to 120 of material number A, the source material frame rate is 30 frames per second, and the recommended exposure time is 2 seconds, the editing strategy can record the clip boundary as source material frame numbers 45 to 104, record the clipping window within the clipping safe zone, record the subtitle landing point in the allowed subtitle area at the bottom of the target final cut screen, place the clip in the 3rd shot position according to the node order, record the transition position after source material frame number 104, and record the expected selling point exposure time of 2 seconds and the expected material occupancy range of 45 to 104 in the expected final cut field group.

[0077] Alternatively, the cropping window can be recorded separately for each source frame, or it can be recorded by sharing the same cropping window for several consecutive source frames; the subtitle landing point can be recorded by a single line of text box, or it can be recorded by multiple lines of text boxes. As long as the cropping window is located within the cropping safe area, the subtitle landing point is located within the allowed subtitle area, and the subtitles are recorded according to the task number, node number, material number, source frame number, and editing strategy version, it is an equivalent implementation of this embodiment.

[0078] S4. According to the editing strategy, the candidate segments are extracted, cropped, overlaid, sorted, and transitioned to generate the target short video. Output frame mapping records are generated simultaneously. The specific implementation is as follows: After receiving the approved editing strategy, the rendering execution station performs cropping, trimming, subtitling, sorting, and transition processing on the candidate segments according to the editing strategy, and generates the target short video and output frame mapping record within the same submission batch. This action is executed by the rendering server in the short video material production line, and its effect is limited to after the editing strategy is version locked and before the target short video enters the output frame retest. The rendering server reads the task number, editing strategy version, node number, material number, start frame number of the cropping, end frame number of the cropping, cropping window, subtitle landing point, shot sorting, transition position, expected final film field group, and target final film aspect ratio constraint version from the material status record library, and reads the source material frame image of the corresponding candidate segment from the material file warehouse.

[0079] During the extraction process, the rendering server locates the media file containing the candidate segment according to the media number in the editing strategy, and reads consecutive source media frames from the start frame number to the end frame number. Both the start frame number and the end frame number include the endpoint. When an unreadable marker is encountered in the source media frame number, the candidate segment stops entering the current target short video generation, and the node number is recorded as pending supplementary recording. Other candidate segments that have been extracted continue to be maintained according to the shot order.

[0080] During cropping, the rendering server reads the corresponding cropping window according to the source material frame number, extracts the image area defined by the cropping window from the source material frame image, and then scales it to the target final image size according to the target final image size constraint. The cropping window must be consistent with the cropping window recorded in the editing strategy version. The cropping window is not reselected in this step. When the coordinates of the cropping window exceed the width and height boundaries of the source material, the candidate segment to which the source material frame belongs stops generating and the reason for the boundary violation is recorded.

[0081] When adding superimposed text, the rendering server writes the node text of the marketing script selling point node into the target final image according to the subtitle landing point in the editing strategy. The subtitle landing point is located at the origin of the coordinates of the upper left corner of the target final image. The superimposed text area must not exceed the constraints of the target final image. If the subtitle landing point cannot fully accommodate the node text, the node number is recorded as pending supplementation and is not replaced by an adjacent subtitle landing point. Superimposed text only uses the subtitle landing point that has been determined in the editing strategy version, and does not avoid the existing subtitle area again in this step, nor does it rewrite the node text of the marketing script selling point node.

[0082] During sorting, the rendering server arranges the candidate segments after trimming, cropping, and text overlay according to the shot order in the editing strategy. Only one candidate segment is allowed to occupy the same shot sorting position. If the candidate segment corresponding to the shot sorting position has not been trimmed, cropped, or text overlaid, an empty position record is formed for that shot sorting position. Empty position records are not generated into finished frames. Subsequent shot sorting positions retain their original numbers, and the target short video is continuously encoded according to the already generated finished frames.

[0083] If the same material number and the same source material frame number range have already been occupied in this target short video, they will not be written again. The duplicate occupied records will be transferred to the pending recording.

[0084] During transition processing, the rendering server creates connecting frames between adjacent candidate segments according to the transition positions in the editing strategy. The transition positions only use the positions recorded in the editing strategy version and do not fall into the suggested exposure time inside the segment. When the editing strategy is recorded as direct connection, the rendering server creates a connection in such a way that the last frame of the previous candidate segment is immediately followed by the first frame of the next candidate segment, without adding new connecting frames.

[0085] The target short video is formed by continuously encoding the finished frames after sorting and transition processing. The first frame of the target short video is numbered 0, and the subsequent output frame numbers are incremented frame by frame according to the order of the finished frames in the target short video. The target short video is stored in the finished video file repository according to the task number, target short video number, editing strategy version, and target finished video aspect ratio constraint version. The target short video number is unique within the same task. Old target short videos are read-only and are retained. When regenerating, only the new target short video number is generated, and the old target short video is not overwritten.

[0086] When synchronously generating output frame mapping records, the rendering server records the target short video number, output frame number, output timestamp, node number, material number, source material frame number, start frame number, end frame number, cropping window, subtitle landing point, shot order, transition position, editing strategy version, and target final frame aspect ratio constraint version for each finished frame; the output timestamp is formed based on the output frame number and the target short video frame rate.

[0087] The finished frame formed directly from candidate segments records the corresponding material number and source material frame number in the output frame mapping record; the connecting frame formed by transition processing records the material number and end source material frame number of the previous candidate segment, and the material number and start source material frame number of the next candidate segment in the output frame mapping record; no additional connecting frame mapping is generated when directly connecting segments.

[0088] The output frame mapping record is stored in the material status record library using the target short video number, output frame number, and editing strategy version as the write key. When the same write key is submitted repeatedly, the already recorded record is returned and no duplicate record is added. Within the same target short video, the output frame number is written in ascending order, and different tasks can generate them in parallel.

[0089] The failure to write the output frame number is recorded as the reason for the failure and the submission of the target short video is suspended. Target short videos that have been generated but whose output frame mapping records have not been fully settled are marked as pending verification and will not enter the output frame retest. Only when the target short video file and all its output frame mapping records have been fully settled will the target short video enter the subsequent output frame retest.

[0090] This step only reads the editing strategy, candidate clips, target final frame constraints, and marketing script selling point nodes; it does not read user profiles, transaction orders, or campaign conversion records.

[0091] Preferably, in the batch production of vertical screen product short videos, the target frame size constraint can be set to a width of 1080 pixels and a height of 1920 pixels, the target short video frame rate can be set to 30 frames per second, the number of shots in a single target short video can be set to 3 to 12, the extraction duration of a single candidate segment can be set to 1.2 seconds to 3 seconds, the font size of overlapping characters can be set to 42 pixels to 72 pixels, the number of connecting frames can be set to 0 to 15 frames, the upper limit of the target short video generation delay can be set to 20 seconds, the number of retries for failed mapping record writing can be set to 2, the retry interval can be set to 5 seconds, and on-site spot checks can be set to randomly select samples from each batch. For fewer than 20 target short videos and no less than 5% of the total batch of target short videos, during random checks, verify the consistency of the output frame number, source material frame number, cropping window, and subtitle landing point with the editing strategy in the target short videos; for example, node number 3 corresponds to the starting frame number 45 and the ending frame number 104 of material number A. The frame rate of the target short video is 30 frames per second. The rendering server captures 60 frames of source material images, generates 60 target finished images according to the corresponding cropping window, writes node text at the subtitle landing point, and associates the output frame numbers 90 to 149 of the target short video with the source material frame numbers 45 to 104 respectively in the output frame mapping record.

[0092] Alternatively, transition processing can employ direct connection or a fixed-frame-count transition; the target short video can be stored in a local finished video file warehouse or in a controlled file warehouse within the same business production line; the frame rate of the target short video can be consistent with the frame rate of the source material, or it can be formed according to the frame order relationship recorded in the editing strategy version. As long as the target short video and output frame mapping records are all recorded according to the target short video number, output frame number, material number, source material frame number, and editing strategy version, it is an equivalent implementation of this embodiment.

[0093] S5. Perform output frame retesting on the target short video to obtain the main visible ratio, subtitle overlap ratio, selling point exposure duration, and material usage range. Compare these with the expected final video field groups to generate feedback difference records. The specific implementation is as follows: After the target short video file and its corresponding output frame mapping record are recorded, the retesting station performs output frame retesting on the target short video. The main body for this execution is the retesting server in the short video material production line, and the time period for this action is limited to after the target short video is generated and before the feedback difference record is written back. The retesting server reads the target short video from the finished product file warehouse, and reads the output frame mapping record, editing strategy version, expected finished product field group, and target finished product aspect ratio constraint version from the material status record library. The target short video number, output frame number, and editing strategy version are used as the traceability index for this retesting.

[0094] The first frame of the target short video is numbered 0. The retest server reads the complete frames in the target short video frame by frame in ascending order of output frame number and checks that each output frame number has a corresponding output frame mapping record. If the target short video file is missing, the target short video number is inconsistent with the output frame mapping record, or any output frame number lacks an output frame mapping record, the target short video is marked as pending verification and will not be included in the feedback difference record generation.

[0095] The subject visibility ratio is obtained frame by frame based on the output frame number. The retest server reads the subject area corresponding to the source material frame number based on the material number, source material frame number, cropping window, and editing strategy version in the output frame mapping record, and converts the subject area to the coordinates of the target final frame according to the cropping window and the target final frame constraints to form the display range. The retest server then checks the actual subject visibility range in the corresponding final frame of the target short video. The actual subject visibility range is based on the identifiable subject boundary within the target final frame. When the subject boundary is cut off by overlapping text, transition frames, or frame boundaries to form multiple separate visible areas, the area of ​​each separate visible area is recorded and summed to the actual subject visibility range area. Areas that are not crossed by obscured areas are merged into continuous areas, and adjacent output frames are not used for padding. The subject visibility ratio is recorded as the percentage of the actual subject visibility range area to the display range area. When the display range is empty, the output frame is recorded as the frame to be checked.

[0096] The subtitle overlap ratio is obtained frame by frame according to the output frame number. The retest server reads the overlapping text area of ​​the marketing script selling point node in the output frame and converts the subtitle area in the source material frame to the target finished screen coordinates according to the output frame mapping record. The subtitle overlap ratio is recorded as the percentage of the overlapping text area and the converted subtitle area where the pixel coordinates overlap. If the overlapping text area exceeds the target finished screen size constraint, the overlapping text area cannot be completely identified, or the source of the subtitle area conversion is missing, the output frame is recorded as a frame to be checked and will not participate in automatic completion.

[0097] The duration of the selling point exposure is obtained from the selling point nodes in the marketing script. The retesting server merges the corresponding output frame numbers in the output frame mapping record according to the node number, and checks whether the node text appears completely in the corresponding output frame in the target short video. The definition of complete appearance of node text is that all characters in the node text are located within the boundary of the target final image, and are not cropped, obscured, or missing words. When the node text appears completely in consecutive output frames and the output frame mapping record points to the same node number, it is included in the selling point exposure frame segment. The duration of the selling point exposure is converted into seconds based on the number of output frames included in the selling point exposure frame segment and the frame rate of the target short video. When the node text is truncated, missing words, or exceeds the boundary of the target final image, the corresponding output frame is not included in the selling point exposure frame segment.

[0098] The material occupancy range is obtained based on the output frame mapping record. The retest server merges the source material frame numbers actually used in the target short video according to the material number to form the material occupancy range. The material occupancy range is recorded according to the material number, the actual starting source material frame number, and the actual ending source material frame number. Both the starting and ending source material frame numbers include the endpoints. When the actual source material frame numbers used under the same material number are not consecutive, multiple material occupancy ranges are formed according to the consecutive source material frame numbers. The occupancy ranges that cross blank frame segments are not merged. The connecting frame formed by the transition processing records the ending source material frame number of the previous candidate segment and the starting source material frame number of the next candidate segment. The connecting frame is not extended into a new consecutive occupancy range.

[0099] After obtaining the main body visibility ratio, subtitle overlap ratio, selling point exposure duration, and material occupation range, the retest server reads the expected final video field group under the same editing strategy version and compares each item with the expected main body visibility ratio, expected subtitle unobstructed state, expected selling point exposure duration, expected material occupation range, and expected final video duration. A main body visibility difference is formed when the main body visibility ratio is lower than the expected main body visibility ratio; a subtitle overlap difference is formed when the expected subtitle unobstructed state is unobstructed and the subtitle overlap ratio is greater than 0; a selling point exposure duration is formed when it is lower than the expected selling point exposure duration; a material occupation difference is formed when the material occupation range is different from the expected material occupation range; and a final video duration difference is formed when the actual duration of the target short video is different from the expected final video duration. The actual duration of the target short video is calculated from the total number of output frames and the frame rate of the target short video.

[0100] Even if no difference is found in the aforementioned comparison, a feedback difference record with the difference type of "no difference" is still generated, so that subsequent write-back can distinguish between the target short video that has been retested and has no difference and the target short video that has not been retested.

[0101] Feedback discrepancy records are written to the material status record library according to the target short video number, output frame number, node number, material number, source material frame number, editing strategy version, and expected final product field group version. The record content includes the main body visibility ratio, subtitle overlap ratio, selling point exposure duration, material occupation range, and corresponding discrepancy type. When the same target short video number, output frame number, and editing strategy version are submitted repeatedly, the already recorded feedback discrepancy record is returned, and no duplicate record is added.

[0102] The feedback difference record is read-only and retained for the old version. When retesting, only the new version is generated, and the retesting time, retesting account, target short video number, output frame mapping record version and editing strategy version are retained. During the retesting process, only the target short video, output frame mapping record and expected final product field group are read, and user profile, transaction order and campaign conversion record are not read.

[0103] The output frame number that failed to write was recorded as the reason for the failure and transferred to pending verification. Target short videos that did not complete the retesting and accounting of all output frame numbers were not included in the subsequent write-back process.

[0104] Preferably, in the mass production of vertical-screen product short videos, the target final frame size constraint can be set to a width of 1080 pixels and a height of 1920 pixels, the target short video frame rate can be set to 30 frames per second, the subject visibility ratio is recorded when it is less than 72%, the subtitle overlap ratio is recorded when it is greater than 0%, the selling point exposure time is recorded when it is more than 0.2 seconds less than the expected value, the maximum delay for retest submission can be set to 15 seconds, the number of failed retries can be set to 2, the retry interval can be set to 5 seconds, and on-site sampling... The check can be set to sample no less than 20 target short videos per batch and no less than 5% of the total target short videos in the batch. During the sampling check, the consistency between the target short video screen, the output frame mapping record, the expected final video field group, and the feedback difference record is verified. For example, in target short video V1, node number 3 corresponds to output frame numbers 90 to 149, and the expected selling point exposure time is 2 seconds. During the retest, only the node text of frame numbers 94 to 145 is fully displayed. Then the selling point exposure time is recorded as 1.73 seconds, and it is compared with the expected final video field group to form the selling point exposure difference.

[0105] Alternatively, the actual visible range of the subject can be formed by manual annotation and retesting, or by controlled image detection and retesting; the subtitle overlap ratio can be formed by checking the text boxes of a single frame, or by merging the text boxes of consecutive output frames. As long as the visible ratio of the subject, the subtitle overlap ratio, the exposure time of the selling points, and the material occupancy range are all recorded according to the target short video number, output frame number, material number, source material frame number, and editing strategy version, it is an equivalent implementation of this embodiment.

[0106] S6. Based on the feedback discrepancy records, write back and correct the croppable boundaries, subtitle avoidance areas, selling point binding status, and material occupancy lock range, and generate the next round of editing permission status. The specific implementation is as follows: After the feedback difference record is completed and recorded, the write-back server reads the target short video ID, output frame ID, node ID, material ID, source material frame ID, editing strategy version, expected final product field group version, main body visibility ratio, subtitle overlap ratio, selling point exposure duration, material occupation range, and difference type. Based on the feedback difference record, it corrects the croppable boundary, subtitle avoidance area, selling point binding status, and material occupation lock range, and generates the next round of editing allowable status. This action occurs at the status write-back station of the short video material production line, and its effective period is limited to after the output frame retest is completed and before the next round of available final product status version is formed. The write-back server reads the feedback difference record, output frame mapping record, frame-level status record, initial binding relationship, and editing strategy version from the material status record library, but does not read user profiles, transaction orders, or campaign conversion records.

[0107] The cropping boundary is written back to the subsequent version of the corresponding frame-level status record according to the material number and the source material frame number. When writing back, the output frame number corresponding to the visible difference of the subject is read first, and then the source material frame number, cropping window and target frame size constraint are traced back according to the output frame mapping record. The actual visible range of the subject under the target frame coordinates is converted back to the source material frame coordinates.

[0108] The cause of the visible difference is determined according to the output frame mapping record, the overlapping text area, and the boundary of the target image. When the actual visible range of the subject touches the boundary of the target image and the display range still extends to the outside of the boundary, it is recorded as image boundary truncation. When the pixel coordinates of the actual visible range of the subject overlap with the overlapping text area, it is recorded as overlapping text occlusion. When the output frame mapping record indicates that the output frame is a transition frame formed by transition processing, it is recorded as a transition frame. When the same output frame satisfies multiple difference causes at the same time, the main difference cause is recorded in the order of image boundary truncation, overlapping text occlusion, and transition frame, and the remaining difference causes are retained. When the visible difference of the subject is formed by image boundary truncation, the edge of the cropping window that caused the truncation is recorded as a prohibited boundary edge, and a new cropping boundary is formed within the original cropping safe area. When the visible difference of the subject is formed by overlapping text occlusion, the cropping boundary is not corrected, and the text avoidance area is entered for rewriting. When the visible difference of the subject is formed by transition frame, the difference cause is retained only at the corresponding transition position, and the cropping boundary is not corrected.

[0109] When multiple feedback discrepancy records correspond to the same material number and the same source material frame number, the multiple new croppable boundaries take the common retention range; when the common retention range is empty, the source material frame number is recorded as pending verification.

[0110] When the feedback difference record shows no difference, the boundary can be trimmed to maintain the original version and the retesting has been passed.

[0111] The subtitle avoidance area is written back according to the target short video number, output frame number, and node number. The write-back server reads the overlapping character area in the subtitle overlap difference and the converted subtitle area, and records the area where the pixel coordinates of the two coincide as the subtitle avoidance area. The subtitle avoidance area records the horizontal coordinate of the top left vertex, the vertical coordinate of the top left vertex, the horizontal coordinate of the bottom right vertex, and the vertical coordinate of the bottom right vertex with the top left corner of the target finished screen as the origin, and associates it with the corresponding material number, source material frame number, node number, and editing strategy version.

[0112] When multiple overlapping subtitles correspond to the same source material frame number, the subtitle avoidance area is retained separately for each overlapping area and is not merged across non-overlapping areas; if the feedback difference record is no difference, no new subtitle avoidance area is added.

[0113] The selling point binding status is written back according to the node number, material number, and source material frame number range. The write-back server reads the selling point exposure duration corresponding to the selling point exposure difference and the expected selling point exposure duration. When the selling point exposure duration is lower than the expected selling point exposure duration, the selling point binding status of the initial binding relationship is recorded as pending extension verification. When the same node number, the same material number, and the same source material frame number range result in selling point exposure differences in two consecutive write-back versions, the selling point binding status is recorded as pending replacement verification. When the feedback difference record is no difference, the selling point binding status is recorded as remaining available.

[0114] The material occupancy lock interval is written back according to the material number, the actual occupancy start source material frame number, and the actual occupancy end source material frame number. Both the start source material frame number and the end source material frame number include the endpoint. As long as there is a material occupancy interval in the feedback difference record, the actual material occupancy interval is written to the material status record database as the material occupancy lock interval, without taking the existence of material occupancy difference as a prerequisite for writing.

[0115] When there are discontinuous source material frame numbers under the same material number, they are written in segments and are not merged across blank frame segments; the transition frame only locks the source material frame number at the end of the previous candidate segment and the source material frame number at the beginning of the next candidate segment, and does not extend to a continuous locking interval.

[0116] When generating the next round of editing permission status, the write-back server uses the task number, node number, material number, source material frame number, and write-back version as the smallest accounting unit to merge and correct the cuttable boundaries, subtitle avoidance areas, selling point binding status, and material occupation lock range; the value of the next round of editing permission status is limited to one of the following: allowed, prohibited from occupation, pending cropping verification, pending subtitle verification, and pending binding verification.

[0117] The status formation order is as follows: first check the material occupancy lock range, then check the croppable boundary, then check the subtitle avoidance area, and finally check the selling point binding status. When the source material frame number falls into the material occupancy lock range, it is recorded as prohibited from occupancy. When the croppable boundary is empty or there is a prohibited boundary edge, it is recorded as pending cropping verification. When the subtitle avoidance area exists, it is recorded as pending subtitle verification. When the selling point binding status is pending extension verification or pending replacement verification, it is recorded as pending binding verification. When none of the above situations exist, it is recorded as allowed. When multiple statuses are triggered at the same time, the main status is recorded in the order of prohibited occupancy, pending cropping verification, pending subtitle verification, and pending binding verification, and the reasons for the remaining statuses are retained.

[0118] The write-back record uses the task number, node number, material number, source material frame number, and feedback difference record version as the write key. When the same write key is submitted repeatedly, the already recorded record is returned and no duplicate record is added. The old write-back version is read-only and retained. When writing back again, only the new version is generated, and the feedback difference record version, output frame mapping record version, submission account, and submission time are retained.

[0119] Frames of source footage that failed to be written are moved to the pending verification stage, and frame numbers of source footage that have not been written back are not allowed to enter the next round of editing.

[0120] Preferably, in the batch production of vertical screen product short videos, when the main subject visibility ratio is less than 72% and the difference is due to screen boundary truncation, a cropping boundary correction is triggered; when the subtitle overlap ratio is greater than 0%, a subtitle avoidance area write-back is triggered; when the selling point exposure time is more than 0.2 seconds less than the expected value, a selling point binding status write-back is triggered; as long as the material occupancy range exists, a material occupancy lock range write-back is triggered; the maximum write-back submission delay can be set to 15 seconds; the number of failed retries can be set to 2; the retry interval can be set to 5 seconds; on-site spot checks can be set to sample no less than 20 write-back records per batch and no less than 5% of all write-back records in the batch; during spot checks, the difference records are verified and reported. The consistency between recording, output frame mapping records, frame-level status records, and the next round of editing permission status is ensured. For example, if the source material frames 45 to 104 of material A correspond to node number 3, the feedback difference record shows that the selling point exposure time is 1.73 seconds and the expected selling point exposure time is 2 seconds. The write-back server records the selling point binding status of the initial binding relationship as pending extension verification, and records the corresponding source material frame number range as pending binding verification in the next round of editing permission status. When the same target short video actually uses source material frames 45 to 104 of material A, this range is synchronously written into the material occupation lock range, and the next round of editing permission status is preferentially recorded as prohibited occupation.

[0121] Alternatively, the cropping boundary can be written back according to a single source material frame number or according to a continuous range of source material frame numbers; the subtitle avoidance area can be written back according to the overlapping area of ​​a single frame or according to the overlapping area of ​​consecutive output frames merged and written back. As long as they are all recorded according to the task number, node number, material number, source material frame number and the written back version, they belong to the equivalent implementation of this embodiment.

[0122] In the operational scenario shown in this embodiment: An e-commerce company creates vertical short video marketing materials for an outdoor waterproof backpack. After receiving task number T20260709001, the short video material production line submits a confirmed planning and arrangement sheet at the script arrangement station. The planning and arrangement sheet includes node number 1 "capacity display", node number 2 "shoulder strap load-bearing capacity", node number 3 "waterproof test" and node number 4 "nighttime reflection". Each marketing script selling point node records the node text, node order and suggested exposure time. The final product specification configuration station synchronously reads the final product template from the vertical screen platform. The target final product aspect ratio constraint is recorded as a width of 1080 pixels, a height of 1920 pixels, and a target short video frame rate of 30 frames per second. It also records the allowed subtitle area and the prohibited occlusion area.

[0123] The media management workstation submitted three candidate video materials, namely material number A, B, and C. Material number A is a 12-second waterproof test material with a source material frame rate of 30 frames per second, resulting in source material frame numbers 0 to 359; material number B is an 8-second capacity display material, resulting in source material frame numbers 0 to 239; and material number C is a 10-second shoulder strap load-bearing material, resulting in source material frame numbers 0 to 299.

[0124] After reading the historical material occupancy records from the material status log library, it is confirmed that the source material frames 60 to 90 of material number A have been occupied in the previous finished film, and the other intervals are empty intervals.

[0125] The archiving server establishes frame-level status records based on task number, material number, and source material frame number. For example, source material frames 45 to 120 for material number A record the main backpack area, source material frames 80 to 110 record the "waterproof test" subtitle area in the original material, and source material frames 60 to 90 record the occupied segment range. A cropping safe area is formed based on the target final image size constraint of 1080 pixels by 1920 pixels.

[0126] Subsequently, the content arrangement server sends the marketing script's selling point nodes to the lightweight large model. The lightweight large model reads the node text, node order, suggested exposure time, and the main area screen, surrounding screen, and subtitle area text content within the candidate clips. For node number 3, "Waterproof Test," the lightweight large model initially binds it to the source material frame numbers 45 to 120 of content number A. However, because this range overlaps with the frame numbers 60 to 90 in the historical content occupancy record, the final usable version records this binding relationship as pending occupancy verification. At the same time, node number 1, "Capacity Display," initially binds it to the source material frame numbers 30 to 110 of content number B, and is recorded as usable because the main area, subtitle area, cropping safety area, and occupied clip range have all been verified. Node number 2, "Shoulder Strap Load Capacity," initially binds it to the source material frame numbers 50 to 140 of content number C and is recorded as usable.

[0127] The editing server reads the initial binding relationships that are available and generates an editing strategy. For node number 1, it selects source material frames 30 to 89 of material number B as the segment trimming boundary. For node number 2, it selects source material frames 50 to 109 of material number C as the segment trimming boundary. It forms a trimming window based on the trimming safety zone, and forms the subtitle landing point based on the allowed subtitle area, the prohibited occlusion area, and the converted subtitle area. It also forms the shot sorting according to the node order. Since the binding relationship of material number A corresponding to node number 3 is pending verification, this segment is not included in the editing strategy in this round, and node number 3 is recorded as a node to be supplemented.

[0128] The rendering server sequentially extracts the corresponding source frames of material number B and material number C according to the editing strategy, generates a target final image of 1080 pixels by 1920 pixels according to the cropping window, writes the corresponding node text "capacity display" and "shoulder strap load" at the subtitle landing point, forms the target short video number V20260709001 according to the shot sorting, and generates output frame mapping records at the same time. For example, the target short video output frame numbers 0 to 59 are respectively associated with the source material frame numbers 30 to 89 of material number B, and the output frame numbers 60 to 119 are respectively associated with the source material frame numbers 50 to 109 of material number C.

[0129] After the target short video file and output frame mapping records are completed, the retest server performs output frame retesting. For each output frame number, the main body visibility ratio, subtitle overlap ratio, selling point exposure duration, and material occupancy range are obtained. The retest results show that the main body visibility ratio of node number 1 is not lower than the expected main body visibility ratio, the subtitle overlap ratio is 0, and the selling point exposure duration is 2 seconds, which is consistent with the expected final video field group, forming a feedback difference record with the difference type of no difference. In output frames 82 to 95, node number 2 has a subtitle overlap ratio greater than 0 because the overlapping text area coincides with the original corner mark text area of ​​the material, forming a subtitle overlap difference.

[0130] After reading the feedback difference record, the write-back server keeps the cuttable boundary, selling point binding status, and subtitle avoidance area unchanged for node number 1, and writes the source material frame numbers 30 to 89 of material number B into the material occupancy lock interval. For node number 2, the write-back server records the area where the pixel coordinates of the overlapping text area in the output frame number 82 to 95 coincides with the converted subtitle area as the subtitle avoidance area. At the same time, it writes the source material frame numbers 50 to 109 of material number C into the material occupancy lock interval and generates the next round of editing permission status. Among them, the source material frame numbers 30 to 89 of material number B and the source material frame numbers 50 to 109 of material number C are preferentially recorded as prohibited in the next round. The source material frame numbers of node number 2 involving the subtitle avoidance area are also retained for the reason of subtitle verification. Node number 3 is still retained as pending supplementary recording because it did not enter the editing strategy in this round.

[0131] When arranging the next batch of materials, the production line can avoid the occupied material area according to the next round of editing permission status, and avoid the subtitle avoidance area that has been written back when processing the "shoulder strap load" node again, so that the same batch or subsequent batches of short video generation process can use the frame-level status records that have been retested and written back.

[0132] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0133] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0134] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0135] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0137] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0139] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0141] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent editing of short video marketing materials based on a lightweight large model, characterized in that, include: S1. Read the marketing script's selling point nodes, target final frame aspect ratio constraints, candidate video materials, and historical material usage records. Establish a frame-level status record based on the source material frame number. The frame-level status record includes the main body area, subtitle area, cropping safety area, and occupied segment interval. Establishing the frame-level status record based on the source material frame number includes: locking the marketing script's selling point node version and target final frame aspect ratio constraint version by task number; reading and merging candidate video materials and historical material usage records by material number; generating source material frame numbers according to the decoding order of candidate video materials; and writing the main body area, subtitle area, cropping safety area, and occupied segment interval into the frame-level status record, using the task number, material number, and source material frame number as the smallest accounting unit. S2. Input the marketing script selling point nodes into the lightweight large model, output the initial binding relationship between the marketing script selling point nodes and the candidate segments, and generate a usable version of the finished product based on the frame-level status records; the candidate segments are formed by merging the frame-level status records of consecutive frame numbers of the source material within the same candidate video material. S3. Generate an editing strategy based on the available state version of the finished film, and read the initial binding relationship with the available state from the available state version of the finished film; the editing strategy includes segment truncation boundaries, cropping windows, subtitle landing points, shot sorting, transition positions, and expected finished film field groups; S4. According to the editing strategy, the candidate segments are extracted, cropped, superimposed, sorted and transitioned to generate the target short video, and the output frame mapping record is generated simultaneously; the output frame number is generated according to the order of the finished frames in the target short video, and the output frame mapping record of the target short video number, output frame number, node number, material number, source material frame number, cropping window, subtitle landing point and editing strategy version is generated simultaneously. S5. Perform output frame retesting on the target short video to obtain the main visible ratio, subtitle overlap ratio, selling point exposure duration, and material occupation range, and compare it with the expected final video field group to generate feedback difference records; the output frame retesting includes: reading the target short video and output frame mapping records according to the target short video number, output frame number, and editing strategy version; tracing the source material frame number and cropping window according to the output frame mapping records, and reading the overlapping text area and the subtitle area converted to the target final video screen coordinates; obtaining the main visible ratio and subtitle overlap ratio according to the output frame number; obtaining the selling point exposure duration according to the node number; and obtaining the material occupation range according to the material number. S6. Based on the feedback difference record, write back and correct the croppable boundary, subtitle avoidance area, selling point binding status, and material occupation lock interval to generate the next round of editing permission status; the generation of the feedback difference record includes: reading the expected final film field group under the same editing strategy version; comparing the main body visibility ratio, subtitle overlap ratio, selling point exposure duration, and material occupation interval with the expected final film field group respectively; generating feedback difference records according to the target short video number, output frame number, node number, material number, source material frame number, and editing strategy version, and writing the difference type in the feedback difference record; the write-back correction includes: reading the feedback difference record and output frame mapping record; truncating the corresponding main body visibility difference according to the screen boundary, converting the actual visible range of the main body from the target final film screen coordinates back to the source material frame coordinates, and then correcting the croppable boundary; writing the subtitle avoidance area according to the subtitle overlap difference; updating the selling point binding status according to the selling point exposure difference; and writing the material occupation lock interval according to the material occupation interval.

2. The intelligent editing method for short video marketing materials based on a lightweight large model according to claim 1, characterized in that, The generation of initial binding relationships and the available state version of the finished product includes: The lightweight large model forms an initial binding relationship between the marketing script's selling point nodes and the candidate segments based on the node text of the selling point nodes, the main area image within the candidate segments, and the text content of the subtitle area. Based on the main area, subtitle area, cropping safe area, and occupied segment range, the candidate segments pointed to by the initial binding relationship are checked to generate a usable version of the finished film.

3. The intelligent editing method for short video marketing materials based on a lightweight large model according to claim 1, characterized in that, The generation of editing strategies includes: Determine the segment truncation boundaries based on the starting source material frame number, the ending source material frame number, and the suggested exposure time period; The cropping window is determined based on the cropping safety area and the target image frame constraints. The subtitle placement point is determined based on the allowed subtitle area, the prohibited occlusion area, and the subtitle area converted to the coordinates of the target final image based on the cropping window in the target final image area constraints; The shot order is determined based on the node order and binding order; Transition positions are determined based on the boundaries between adjacent segments in the shot sequence; The expected final video field group is formed based on the segment extraction boundaries, cropping window, subtitle landing point, shot order, and transition position.

4. The intelligent editing method for short video marketing materials based on a lightweight large model according to claim 1, characterized in that, The generation of the target short video and output frame mapping record includes: Read the source frame of the candidate segment by segment boundary; The target image is formed based on the cropping window and the target image size constraints. Write the node text of the selling point node in the marketing script according to the subtitle's ending point; The target short video is created by sorting shots and determining transition positions.

5. The intelligent editing method for short video marketing materials based on a lightweight large model according to claim 1, characterized in that, The next round of editing allows the generation of states including: The accounting unit is the task number, node number, material number, source material frame number, and write-back version. Check the locked area occupied by the material, the croppable boundary, the subtitle avoidance area, and the binding status of the selling points in sequence; The next round of editing permission status will be determined in the following order: prohibited from use, pending cropping verification, pending subtitle verification, and pending binding verification. If none of the aforementioned states are triggered, it is determined to be allowed.

Images