Method, apparatus and storage medium for scheduling and executing multimedia processing task

Abstract

This application relates to a method, apparatus, device, and storage medium for scheduling and executing multimedia processing tasks. It involves acquiring a multimedia processing instruction template containing fixed instruction fragments and parameter placeholders, along with associated node metadata containing parameter definitions and verification rules corresponding to the parameter placeholders. The method performs validity checks on the input structured parameter set according to the verification rules in the node metadata. Validated parameter values ​​are replaced with the corresponding parameter placeholders in the multimedia processing instruction template and combined with the fixed instruction fragments to generate multimedia processing instructions for scheduling and execution. After scheduling and execution, the generated output file undergoes full-dimensional output verification. If scheduling and execution fail or full-dimensional output verification fails, the captured underlying anomaly information is converted into standardized error messages. Compared with existing technologies, the technical solution of this application can solve the problems of high development costs, significant security risks, and difficulty in troubleshooting associated with hard-coded implementations.

Description

Technical Field

[0001] This application relates to the technical field of electronic digital data processing, and in particular to a method, apparatus, device, and storage medium for scheduling and executing multimedia processing tasks. Background Technology

[0002] Currently, most FFmpeg-based video editing solutions use hard-coded execution instructions, lacking a unified instruction template and standardized parameter parsing mechanism. Adding new features requires rewriting the underlying code, resulting in poor code reusability. Directly concatenating strings to generate commands can easily lead to instruction injection security vulnerabilities and makes it difficult to achieve lightweight and unified access for various editing nodes.

[0003] Existing technologies only set up simple front-end parameter validation, lacking a standardized parameter legality validation process, which easily leads to the issuance of illegal parameters and invalid processing tasks; at the same time, no output file validation process is set up, which cannot identify damaged or non-compliant output files, and can only output low-level raw error information, with obscure prompts, making troubleshooting difficult. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for scheduling and executing multimedia processing tasks, which can solve the problems of high development cost, high security risk, and difficulty in troubleshooting of hard-coded tasks in the prior art.

[0005] In a first aspect, this application provides a method for scheduling and executing a multimedia processing task, comprising: obtaining a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template, wherein the multimedia processing instruction template includes fixed instruction fragments and parameter placeholders, and the node metadata includes parameter definitions and verification rules corresponding to the parameter placeholders; receiving a structured parameter set passed from an upper-layer node, performing a validity check on the structured parameter set according to the verification rules in the node metadata, replacing the parameter values ​​that pass the validity check with the corresponding parameter placeholders in the multimedia processing instruction template, combining the replaced parameter placeholders with the fixed instruction fragments to generate a multimedia processing instruction; scheduling and executing the multimedia processing instruction, and performing a full-dimensional output verification on the generated output file after the scheduling and execution are completed; if the scheduling and execution fail or the full-dimensional output verification fails, capturing underlying exception information and converting the underlying exception information into a standardized error message.

[0006] In one possible implementation, before receiving the structured parameter set from the upper-layer node, the method further includes: performing real-time front-end validation on the user-input parameters according to the validation rules in the node metadata, wherein the real-time front-end validation includes at least format correctness validation, value range compliance validation, and mandatory field completeness validation; if the real-time front-end validation fails, an immediate prompt is made and parameter submission is prohibited; if the real-time front-end validation passes, the structured parameter set is generated based on the user-input parameters and the node metadata, and the step of receiving the structured parameter set from the upper-layer node is executed.

[0007] In one possible implementation, the step of validating the structured parameter set according to the validation rules in the node metadata specifically includes: performing four levels of validity validation on the structured parameter set in a predetermined order according to the validation rules in the node metadata; wherein the validity validation includes a first-level syntax layer validation, a second-level security layer validation, a third-level business layer single-parameter validation, and a fourth-level business layer cross-parameter joint validation. Each level of validation is executed sequentially. If the current level of validation fails, the subsequent validation process is terminated and the next level of validation is not entered. If the current level of validation passes, the next level of validation continues to be executed.

[0008] In one possible implementation, the step of replacing the valid parameter value with the corresponding parameter placeholder in the multimedia processing instruction template, and combining the replaced parameter placeholder with the fixed instruction fragment to generate a multimedia processing instruction, specifically includes: performing regular expression matching on the multimedia processing instruction template to extract all parameter placeholders in the multimedia processing instruction template and generating a placeholder list; matching the parameter values ​​in the structured parameter set with the placeholder list, and assigning parameter values ​​to each parameter placeholder in the placeholder list according to a preset priority; performing type conversion and formatting processing on the matched parameter values ​​according to the parameter type of each parameter placeholder to obtain formatted parameter values; performing security filtering on the formatted parameter values ​​to detect and remove dangerous characters in the formatted parameter values ​​to obtain filtered parameter values; and replacing the filtered parameter values ​​with the corresponding parameter placeholders in the instruction template to generate a complete multimedia processing instruction.

[0009] In one possible implementation, scheduling and executing the multimedia processing instructions specifically includes: encapsulating the multimedia processing instructions into tasks to be executed; constructing multiple priority task queues and allocating the tasks to be executed to the corresponding target priority task queues according to preset priorities; monitoring the resource usage status of the server in real time, the resource usage status including at least CPU utilization, memory utilization, and the number of tasks currently being executed; according to the resource usage status, pulling the tasks to be executed from the target priority task queues for allocation and execution according to preset scheduling rules, and performing state management throughout the entire lifecycle of the tasks to be executed; wherein, the preset scheduling rules include priority-first scheduling rules, resource quota control rules, single task timeout control rules, same-user task rate limiting rules, abnormal task circuit breaker rules, and execution failure retry rules.

[0010] In one possible implementation, the step of performing a full-dimensional output verification on the generated output file after the scheduling execution is completed specifically includes: verifying whether the output file exists in the output path and whether the size of the output file is greater than zero; verifying whether the magic number of the file header of the output file conforms to a preset format standard; reading the metadata and audio / video stream information of the output file through a probe interface to verify the integrity and stream integrity of the output file; verifying whether the difference between the actual duration of the output file and the preset target duration is within a preset fault tolerance threshold; verifying whether the timestamp deviation between the audio stream and the video stream of the output file is within a preset synchronization deviation threshold; if all verifications pass, the output file is determined to be compliant; if any verification fails, the output file is deleted and an exception information capture and conversion process is triggered.

[0011] In one possible implementation, converting the underlying exception information into standardized error messages specifically includes: capturing underlying exception information during the execution of the multimedia processing instructions in real time, wherein the underlying exception information includes at least standard error stream output logs, process exit codes, system signals, and execution timeout information; summarizing the exception information acquired during the real-time capture process into raw exception logs, and performing text cleaning on the raw exception logs to extract exception description statements; constructing a hierarchical exception keyword matching rule base, wherein the hierarchical exception keyword matching rule base is divided into multiple exception levels according to exception type, and each exception level corresponds to a preset error code range, matching keywords, standardized prompts, and troubleshooting suggestions; matching the exception description statements with the matching keywords in the hierarchical exception keyword matching rule base to determine the matching result corresponding to the exception description statements, wherein the matching result includes at least the exception level, error code, standardized prompts, and troubleshooting suggestions; and generating standardized error messages based on the matching results.

[0012] Secondly, this application provides a scheduling and execution device for multimedia processing tasks, comprising: a multimedia processing instruction template acquisition module, a multimedia processing instruction generation module, a scheduling and execution module, and a low-level exception information conversion module; wherein, the multimedia processing instruction template acquisition module is used to acquire a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template, the multimedia processing instruction template includes fixed instruction fragments and parameter placeholders, and the node metadata includes parameter definitions and verification rules corresponding to the parameter placeholders; the multimedia processing instruction generation module is used to receive a structured parameter set passed from an upper-layer node, perform legality verification on the structured parameter set according to the verification rules in the node metadata, replace the parameter values ​​that pass the legality verification with the corresponding parameter placeholders in the multimedia processing instruction template, and combine the replaced parameter placeholders with the fixed instruction fragments to generate multimedia processing instructions; the scheduling and execution module is used to schedule and execute the multimedia processing instructions, and perform full-dimensional output verification on the generated output file after the scheduling and execution are completed; the low-level exception information conversion module is used to capture low-level exception information and convert the low-level exception information into standardized error messages if the scheduling and execution fail or the full-dimensional output verification fails.

[0013] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0015] This application provides a method, apparatus, device, and storage medium for scheduling and executing multimedia processing tasks, which has the following advantages compared with the prior art: By standardizing command templates and replacing parameter placeholders, the generation of underlying commands for video editing functions has been transformed from hard-coded to template-based configuration. Different editing functions only need to be configured with different templates to be integrated, eliminating the need to repeatedly write underlying code and greatly reducing the development cost of adding new functions. Commands are assembled using template placeholders, abandoning the direct string concatenation method and effectively avoiding the security risks of command injection. Furthermore, by validating the legality of structured parameters passed from the upper layer, illegal parameters are intercepted at the source, preventing invalid tasks from being generated. After scheduling and execution, the output files undergo comprehensive checks including existence, format, integrity, duration, and audio / video synchronization. Damaged or non-compliant output files are automatically identified and removed, ensuring that the output results meet business expectations. In addition, when execution fails or output checks fail, the obscure underlying error messages are automatically converted into standardized error messages that users can understand, significantly reducing the difficulty of troubleshooting and improving system maintainability and user experience. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1 This is a flowchart illustrating an embodiment of a multimedia processing task scheduling and execution method provided in this application; Figure 2 This is a schematic diagram of the structure of an embodiment of a multimedia processing task scheduling and execution device provided in this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]." Example 1, see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of a multimedia processing task scheduling and execution method provided in this application, as shown below. Figure 1As shown, the method includes steps 101-104, as detailed below: Step 101: Obtain a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template. The multimedia processing instruction template contains fixed instruction fragments and parameter placeholders. The node metadata contains parameter definitions and verification rules corresponding to the parameter placeholders.

[0026] In one embodiment, the standardized multimedia processing instruction template is a fixed syntax structure predefined and stored in the system for each type of video editing function. Each type of editing function, such as video cutting, image cropping, multi-file splicing, and audio replacement, corresponds to an independent standardized FFmpeg command template. The definition of the multimedia processing instruction template follows a unified general structure specification, consisting of two parts: fixed instruction fragments and parameter placeholders. The parameter placeholders include two types: variable parameter placeholders and optional parameter placeholders. The fixed command fragment is a fixed syntax structure in the native FFmpeg syntax corresponding to the editing function and cannot be modified. The variable parameter placeholders are used to mark the positions of variables that need to be dynamically valued by the upper layer and are required parameters, corresponding to mandatory input items of the corresponding nodes. The optional parameter placeholders are optional parameters and are optional configuration items of the corresponding nodes. When no value is passed, the default value preset by the template is automatically used.

[0027] Specifically, all multimedia processing instruction templates use a unified standardized naming format for placeholders, namely [parameter unique identifier_parameter type]. The parameter type includes five categories: file path, time format, numerical value, text, and string. This provides a structured basis for subsequent parameter parsing, type validation, and security filtering.

[0028] Example Explanation: The multimedia processing command template is for video cutting: ffmpeg -i[input_file_file] -ss[start_time_time] -to[end_time_time] -c:v[video_codec_string] -c:a[audio_codec_string][output_file_file], where -i, -ss, -to, -c:v, and -c:a are fixed command fragments, [input_file_file] is a required variable parameter placeholder, and [video_codec_string] is an optional parameter placeholder. When no value is passed in, the default value of the optional parameter placeholder is copy.

[0029] Specifically, the standardized multimedia processing instruction templates also have template security and compatibility specifications. All multimedia processing instruction templates disable dangerous syntax such as pipes, redirection operators, and system calls, ensuring that only legal syntax related to native FFmpeg editing is included. Simultaneously, the templates are compatible with all versions of FFmpeg 4.0 and above, adapting to syntax differences between different versions to ensure consistent execution across environments. When adding a new type of video editing function, developers only need to predefine the corresponding instruction template according to the above specifications and store it in the multimedia processing instruction template library for subsequent parsing and assembly.

[0030] Preferably, all multimedia processing instruction templates are set with version numbers. When a new template is added or modified, a new version is automatically generated, and the old version template is retained synchronously. This ensures that nodes in the historical workflow can be executed normally and avoids the failure of historical task execution caused by the update of multimedia processing instruction templates.

[0031] In one embodiment, node metadata is standardized descriptive information that accompanies the multimedia processing instruction template, used to support the node's front-end rendering, parameter passing, validation rule definition, and workflow adaptation.

[0032] Specifically, the node metadata is also obtained through predefined storage, with all node metadata centrally stored in a pre-defined node metadata database. The general structure of node metadata is defined as a unified metadata structure for all basic video editing nodes. Node metadata follows a unified general data structure, with core fields including: unique node identifier, node name, canvas display name, function category, bound FFmpeg command template version, input parameter list, output parameter definition, parameter validation rules, function description, online status, and permission identifier.

[0033] Specifically, in the node metadata, each input parameter in its input parameter list has a one-to-one correspondence with a parameter placeholder in the multimedia processing instruction template. The core definition fields of each input parameter include: parameter unique identifier, display name, parameter type, required field flag, default value, value range, validation rules, bound placeholder, front-end interactive control type, and whether it supports input from upstream nodes.

[0034] Specifically, when the upper layer needs to obtain the metadata of a certain type of edit node, the system queries and loads from the metadata database based on the node's unique identifier, and returns the complete parameter definition and validation rules together.

[0035] Specifically, all video basic editing nodes support three types of input files: local video files, platform media asset library files, and video files output from upstream nodes. All files use a unique file identifier for parameter passing. All video basic editing nodes output files in MP4 format (H.264 video encoding + AAC audio encoding), and output metadata such as file duration, file size, frame rate, and bit rate. This metadata can be directly used as input for other nodes, enabling standardized connection between nodes.

[0036] Specifically, through the aforementioned mechanism of separating and unifying the storage of multimedia processing instruction templates and node metadata, standardized acquisition of multimedia processing instruction templates and node metadata is achieved, providing a complete and reliable data foundation for subsequent parameter parsing, verification, replacement, and instruction generation.

[0037] In one embodiment, the multimedia processing instruction template and node metadata are associated and bound through two dimensions of information: first, the template version number, which explicitly records the template version of the multimedia processing instruction template bound to the node in the node metadata, ensuring that even if the multimedia processing instruction template is iteratively updated, nodes in the historical workflow can still use the old version of the multimedia processing instruction template they were originally bound to to continue execution; second, the parameter unique identifier, which is completely consistent with the parameter unique identifier of the input parameter list in the node metadata and the parameter unique identifier of the placeholder in the template. This design enables the subsequent parsing and assembly to accurately match the parameter values ​​passed by the user with the placeholders in the template, completing precise replacement. Through the dual mechanism of template version number association and parameter unique identifier matching, a traceable and verifiable close association is formed between the multimedia processing instruction template and the node metadata.

[0038] In one embodiment, in the actual execution process, the acquisition action occurs at the following times: after the engine receives the standardized parameters passed from the upper layer, it first extracts the node unique identifier and template version number from the request; then, using the node unique identifier as an index, it queries and loads the metadata definition of the node corresponding to the node unique identifier from the node metadata database; at the same time, using the template version number as an index, it pulls the corresponding version of the instruction template from the multimedia processing instruction template library.

[0039] Specifically, both are managed through structured storage, supporting online visual configuration, version updates, and online / offline operations. When adding video editing functionality, integration is complete simply by defining instruction templates in the multimedia processing instruction template library and configuring corresponding metadata records in the node metadata database, without requiring modification to the core engine or front-end code. After configuration, the system automatically synchronizes the newly added node information to the front-end canvas, enabling dynamic rendering and lightweight integration of nodes.

[0040] Step 102: Receive the structured parameter set passed from the upper-layer node, perform legality verification on the structured parameter set according to the verification rules in the node metadata, replace the parameter values ​​that pass the legality verification with the corresponding parameter placeholders in the multimedia processing instruction template, and combine the replaced parameter placeholders with the fixed instruction fragment to generate a multimedia processing instruction.

[0041] In one embodiment, before receiving the structured parameter set from the upper-layer node, the method further includes: performing front-end real-time validation on the user-input parameters according to the validation rules in the node metadata, wherein the front-end real-time validation includes at least format correctness validation, value range compliance validation, and mandatory field completeness validation; if the front-end real-time validation fails, an immediate prompt is made and parameter submission is prohibited; if the front-end real-time validation passes, the structured parameter set is generated based on the user-input parameters and the node metadata, and the step of receiving the structured parameter set from the upper-layer node is executed.

[0042] Specifically, during the process of configuring node parameters on the front-end canvas, the system performs real-time validation of the user-input parameters according to the predefined validation rules in the node metadata. This validation occurs during the user operation phase and includes three dimensions: First, format correctness validation, which determines whether the user input conforms to the format requirements based on the parameter type, including whether numeric parameters are valid numbers, whether time parameters are in a valid time format, whether text parameters exceed the length limit, and whether a valid file is selected for file types. Regular expressions are used to match the format of the parameter content, and parameters that do not meet the format requirements are immediately highlighted in red. Second, value range compliance validation, which performs real-time validation of numeric and time parameters according to the preset value ranges in the node metadata. This includes ensuring that the video cutting deadline value is greater than the start time value, the video frame rate value is within the preset frame rate range, and the font size is within the preset font size range. Values ​​exceeding the range are immediately indicated. Third, mandatory field completeness validation, which checks in real-time whether all mandatory parameters have been filled with valid content. If any mandatory field is missing, the submit button is grayed out, and the missing parameter is highlighted.

[0043] Preferably, the preset frame rate range is 1-60fps, and the preset font size range is 12-120.

[0044] Specifically, if any item fails the real-time front-end validation, the system immediately provides a prompt and prohibits parameter submission. The user can only continue after correcting the error. Only if all front-end validations pass will the user be allowed to submit the task. At this point, the system encapsulates the user-configured parameter values, unique node identifiers, bound template versions, output path definitions, and other information into the structured parameter set based on the user-input parameters and the node metadata. The system then receives the structured parameter set and proceeds to the back-end processing flow. Real-time front-end validation can intercept obviously non-compliant parameter inputs during the user configuration phase, reducing invalid task submissions at the source.

[0045] In one embodiment, the structured parameter set passed in by the upper-layer node is used to describe all the input parameters required for the multimedia processing task to be executed by the upper-layer node.

[0046] In one embodiment, the step of validating the structured parameter set according to the verification rules in the node metadata specifically includes: performing four levels of validity verification on the structured parameter set in a predetermined order according to the verification rules in the node metadata; wherein, the validity verification includes a first-level syntax layer verification, a second-level security layer verification, a third-level business layer single-parameter verification, and a fourth-level business layer cross-parameter joint verification. Each level of verification is executed sequentially. If the current level of verification fails, the subsequent verification process is terminated and the next level of verification is not entered. If the current level of verification passes, the next level of verification continues to be executed.

[0047] Specifically, after the user completes the parameter configuration and submits the task, the engine performs four levels of validity checks on the received structured parameter set according to the verification rules in the node metadata, in a predetermined order throughout the entire process of parameter parsing and placeholder replacement. Each level of verification is executed sequentially, and the next level of verification only proceeds after all previous level checks have passed; if none of the previous level checks have passed, the subsequent verification process is immediately terminated, and the next level of verification is not initiated.

[0048] Specifically, the first level is syntax layer validation: the core validation checks whether each parameter in the parsed structured parameter set conforms to the syntax requirements of the underlying multimedia processing execution instructions. This includes: verifying whether the parameter type of each parameter in the structured parameter set is consistent with the type defined by the placeholders, whether the parameter values ​​are properly standardized and formatted, whether special characters are properly escaped, and whether required parameters have valid values. If the validation fails, the process is terminated directly, and a standardized message indicating a parameter format error is returned, clearly marking the erroneous parameter item and the reason for the error.

[0049] Specifically, the second level is the security layer verification: the core verification checks whether each parameter in the structured parameter set has security risks, including: verifying whether each parameter in the structured parameter set contains dangerous characters and illegal statements, whether the file path is within the preset legal directory range and whether there is a risk of path crossing, and whether the parameter length exceeds the system's preset maximum threshold; if the verification fails, the process is terminated directly, a security alarm is triggered, and a detailed security log is recorded.

[0050] Specifically, the third level is single-parameter validation at the business layer: the core validation is the business rationality of a single parameter, including performing input file integrity validation and time range rationality validation on the structured parameter set; wherein, the input file integrity validation includes: detecting whether the structured parameter set exists on the server and whether the structured parameter set can be read normally based on the file path; extracting the magic number of the file header of the structured parameter set, validating whether the structured parameter set is a valid audio and video file, filtering non-video files and corrupted files; reading node metadata in the structured parameter set through the probe interface, including duration, frame rate, bitrate, audio and video stream information, to verify whether the structured parameter set is complete and whether there is any audio and video stream corruption; verifying whether the size of the structured parameter set exceeds the preset maximum processing threshold to avoid server resource exhaustion caused by excessively large files; if the validation fails, the process is terminated, and a standardized prompt of structured parameter set not existing / corrupted / format not supported is returned; The time range validity check includes: verifying whether the start time parameter value and the end time parameter value are valid values ​​greater than or equal to zero, and whether the end time parameter value is greater than the start time parameter value; obtaining the total duration of the structured parameter set through the probe interface, and verifying whether the end time parameter value does not exceed the total duration of the structured parameter set and whether the start time parameter value is less than the total duration; verifying whether the time interval duration is greater than the preset minimum processing duration to avoid invalid empty content processing; if the check fails, the process is terminated, a standardized prompt indicating that the time range does not meet the requirements is returned, and the total duration of the source file and the valid time range are clearly marked.

[0051] Specifically, the fourth level is cross-parameter joint verification at the business layer: the core verification checks the logical rationality between multiple parameters, including: performing video resolution aspect ratio and encoder upper limit verification on the structured parameter set, and video bitrate / frame rate / resolution matching verification, respectively determining whether the width and height of the video resolution parameter meet the preset aspect ratio requirements and whether they exceed the maximum resolution supported by the encoder; whether the video bitrate, frame rate, and resolution match to avoid invalid configurations of ultra-high-definition resolution with extremely low bitrate; in multi-file splicing scenarios, verifying whether the format and encoding of the structured parameter set are consistent and whether direct stream copying and splicing is supported; in audio replacement scenarios, verifying whether the duration of the audio file in the structured parameter set matches the video file and whether automatic adaptation is supported; if the verification fails, the process is terminated and the corresponding standardized prompt is returned, while providing compliant parameter configuration suggestions.

[0052] Specifically, through the above four-level progressive backend verification, from syntax security and single-parameter business rationality to cross-parameter joint logic, the full-dimensional legality verification of the structured parameter set is completed layer by layer to ensure that all parameters entering the instruction generation and scheduling execution stage meet the compliance requirements.

[0053] In one embodiment, the step of replacing the parameter values ​​that have passed the legality check with the corresponding parameter placeholders in the multimedia processing instruction template, and combining the replaced parameter placeholders with the fixed instruction fragments to generate a multimedia processing instruction, specifically includes: performing regular expression matching on the multimedia processing instruction template to extract all parameter placeholders in the multimedia processing instruction template and generating a placeholder list; matching the parameter values ​​in the structured parameter set with the placeholder list, and assigning parameter values ​​to each parameter placeholder in the placeholder list according to a preset priority; performing type conversion and formatting processing on the matched parameter values ​​according to the parameter type of each parameter placeholder to obtain formatted parameter values; performing security filtering on the formatted parameter values ​​to detect and remove dangerous characters in the formatted parameter values ​​to obtain filtered parameter values; and replacing the filtered parameter values ​​with the corresponding parameter placeholders in the instruction template to generate a complete multimedia processing instruction.

[0054] Specifically, the engine performs regular expression matching on the multimedia processing instruction template to identify and extract all parameter placeholders, including variable parameter placeholders and optional parameter placeholders. Simultaneously, it parses the unique parameter identifier and parameter type of each parameter placeholder, classifying them as required or optional, and generating a placeholder list. After extraction, the engine also performs a matching verification between the placeholder list and the input parameter list in the node metadata: if a parameter placeholder has no corresponding parameter definition in the node metadata, a template exception is triggered and the process terminates; if the matching verification passes, the placeholder list is used as the basis for subsequent parameter value matching. Through the above extraction and verification, the standardization and completeness of the multimedia processing instruction template itself are ensured.

[0055] Specifically, after generating the placeholder list, the engine needs to match the corresponding parameter value for each parameter placeholder in the placeholder list. During this process, the engine iterates through the placeholder list, matching the parameter values ​​in the structured parameter set with the unique parameter identifier of each parameter placeholder, and assigns parameter values ​​to each parameter placeholder according to a preset priority. The preset priority is as follows: parameter values ​​passed from the upper-level node have the highest priority and are used first; if the upper-level node does not pass a corresponding parameter value, the parameter value manually configured by the user is used; if the parameter value manually configured by the user also does not exist, the default value preset in the node metadata is used, i.e., parameter value passed from the upstream node > parameter value manually configured by the user > default value. For required variable parameter placeholders, if no valid parameter value is found after matching according to the above preset priority, a parameter missing exception is directly triggered, the process is terminated, and a standardized prompt is returned; for optional parameter placeholders, if no valid parameter value is found, the default value preset in the node metadata is automatically filled.

[0056] Specifically, after matching and assigning parameter values ​​to each parameter placeholder, the engine performs type conversion and formatting on the matched parameter values ​​according to the parameter type of each placeholder, obtaining formatted parameter values ​​to ensure that each parameter value conforms to the syntax requirements of the multimedia processing instructions. During this process, for file type parameters: they are automatically converted to the server's local absolute path, and special characters such as spaces and Chinese characters in the server's local absolute path are escaped to avoid command execution errors caused by spaces, Chinese characters, etc.; for time type parameters: they are automatically and uniformly formatted into the standard time format of hours:minutes:seconds.milliseconds, and support automatic conversion of user-inputted second, minute:second, and hour:minute:second formats; for numeric type parameters: they are automatically converted to numeric types, removing redundant non-numeric characters and retaining the specified number of decimal places as required; for text or string type parameters: special characters such as single quotes, double quotes, and backslashes are automatically escaped to avoid disrupting the syntax structure of the multimedia processing instructions.

[0057] Specifically, after the formatting process is completed, the engine performs security filtering on all formatting parameter values, detecting and removing dangerous characters from the formatting parameter values ​​to obtain filtered parameter values. During this process, the engine uses regular expression matching to detect whether each formatting parameter value contains dangerous characters, including pipe symbols, redirection symbols, system command calls, and SQL injection statements. If a dangerous character is detected, a security exception is triggered directly, the process is terminated, and a security log is recorded. Simultaneously, the engine also performs secondary validation on the length and range of the formatting parameter values ​​to avoid command execution anomalies caused by excessively long or out-of-range parameters. Through security filtering, the risk of command injection security is eliminated at the source.

[0058] Specifically, after the security filter passes, the engine traverses the placeholder list and replaces the filter parameter values ​​one by one with the corresponding parameter placeholder positions in the multimedia processing instruction template to generate the complete multimedia processing instruction. That is, the replaced parameter values ​​and the fixed instruction fragments are combined according to the original arrangement order in the multimedia processing instruction template, and all the original parameter placeholders are replaced by the corresponding filter parameter values.

[0059] Specifically, after the multimedia processing instructions are assembled, the engine performs a syntax pre-validation on the assembled multimedia processing instructions through the -h syntax checking interface provided by FFmpeg. If the validation passes, the multimedia processing instructions are stored in the task execution queue for subsequent scheduling and execution; if the validation fails, an instruction syntax exception is triggered and the process is terminated. Through the sequential execution of the above five steps, the automatic conversion from valid parameters to complete executable instructions is achieved.

[0060] Step 103: Schedule and execute the multimedia processing instructions, and perform full-dimensional output verification on the generated output file after the scheduling and execution are completed.

[0061] In one embodiment, scheduling and executing the multimedia processing instructions specifically includes: encapsulating the multimedia processing instructions into tasks to be executed; constructing multiple priority task queues and allocating the tasks to be executed to the corresponding target priority task queues according to preset priorities; monitoring the resource usage status of the server in real time, the resource usage status including at least CPU utilization, memory utilization, and the number of tasks currently being executed; according to the resource usage status, pulling the tasks to be executed from the target priority task queues for allocation and execution according to preset scheduling rules, and performing state management throughout the entire lifecycle of the tasks to be executed; wherein, the preset scheduling rules include priority-first scheduling rules, resource quota control rules, single task timeout control rules, same-user task rate limiting rules, abnormal task circuit breaker rules, and execution failure retry rules.

[0062] Specifically, once the multimedia processing instruction is generated, the engine first encapsulates it into an executable task. The executable task includes the multimedia processing instruction itself, as well as task metadata information such as task identifier, preset priority, user identifier, and submission time, giving it complete attributes that allow it to be scheduled, trackable, and managed.

[0063] Specifically, multiple priority task queues are constructed, namely, high-priority, medium-priority, and low-priority task queues. The high-priority queue is used to store real-time preview tasks, the medium-priority queue is used to store workflow automation execution tasks, and the low-priority queue is used to store batch background processing tasks. The tasks to be executed are placed into the corresponding queues according to their preset priorities and await scheduling. The construction of multi-level queues and the precise queuing of tasks lay the foundation for subsequent differentiated scheduling and execution.

[0064] Specifically, after the tasks to be executed are enqueued, the engine starts the resource monitoring module to monitor the server's resource usage status in real time. The resource usage status includes at least CPU utilization, memory utilization, and the number of tasks currently being executed. In addition, the resource monitoring module also collects GPU utilization, disk I / O utilization, and the specific resource usage of each task in real time, providing comprehensive data support for scheduling decisions.

[0065] Specifically, the resource monitoring module employs a periodic data collection mechanism, refreshing the server's resource usage data at a fixed frequency and reporting the collected real-time data to the scheduler. Based on the resource usage status and pre-defined scheduling rules, the scheduler dynamically decides when to retrieve the tasks to be executed from the target priority task queue for allocation and execution.

[0066] Specifically, the priority scheduling rule divides task priorities into three levels: high, medium, and low. Real-time preview tasks are high priority, workflow automation tasks are medium priority, and batch background processing tasks are low priority. The scheduler always prioritizes tasks from the high-priority queue for execution. Only when the high-priority queue is empty will it pull tasks from the medium-priority queue, and only when the medium-priority queue is empty will it pull tasks from the low-priority queue. At the same time, high-priority tasks can preempt the resource quotas of low-priority tasks to ensure the response speed of real-time user operations.

[0067] Specifically, the resource quota control rules are based on three core quotas: the maximum number of parallel tasks, the maximum resource consumption per task, and the global resource utilization threshold. The scheduler monitors the server resource status in real time. Only when the current CPU utilization is below 80%, the memory utilization is below 85%, and the number of tasks being executed is less than the maximum number of parallel tasks will a new task be pulled from the queue for execution. When the global resource utilization exceeds the threshold, the scheduling of tasks in the low-priority queue is suspended, and only high-priority tasks are processed. When the resource utilization exceeds 90%, the scheduling of all new tasks is suspended until the resource utilization falls back to a safe range.

[0068] Specifically, the single-task timeout control rules are as follows: preset maximum execution timeout for different types and priorities of tasks; the timeout for high-priority real-time preview tasks is 30 seconds, the timeout for medium-priority workflow tasks is 10 minutes, and the timeout for low-priority batch tasks is 30 minutes. After a task starts execution, the scheduler starts a timeout timer. If the task execution time exceeds the timeout period, the FFmpeg process is forcibly terminated to release the occupied system resources. At the same time, a standardized prompt indicating that the task execution has timed out is returned to avoid zombie processes occupying server resources.

[0069] Specifically, the user task rate limiting rule is to preset the maximum number of tasks that a single user can execute at the same time. The maximum number of parallel tasks for ordinary users is 3, and the maximum number of parallel tasks for VIP users is 10. When the scheduler pulls tasks, if the number of tasks that the user is currently executing has reached the limit, the new tasks of the user will not be scheduled until the user has a task to complete, so as to avoid a single user occupying too many server resources and ensure service fairness in multi-user scenarios.

[0070] Specifically, the abnormal task circuit breaker rule is to implement a circuit breaker mechanism for consecutive failed tasks on the same node and for the same user: if a single user's task on the same type of node fails to execute 3 times in a row, the task scheduling of that user's task on that type of node will be suspended for 1 minute, and an abnormal prompt will be pushed to guide the user to check the parameter configuration; if the same type of node fails to execute 10 times in a row across the entire platform, the node will be automatically taken offline temporarily, and the administrator will be notified to investigate template and engine issues to avoid large-scale task execution failures.

[0071] Specifically, the retry rule for execution failures is to automatically implement a retry mechanism for occasional execution failures caused by non-parameter errors or security issues: After a task fails, the failure type is first determined. If it is an occasional issue such as server resource fluctuations or temporary disk I / O anomalies, the task is automatically put back into the corresponding priority queue and retried a maximum of 2 times. If the task still fails after retrying, the task is terminated and a failure message is returned to avoid resource waste caused by infinite retries.

[0072] Specifically, the scheduler manages the state of each task throughout its entire lifecycle, dividing tasks into seven states: pending scheduling, pending execution, in execution, successful execution, failed execution, terminated, and timed out. Each state transition has a clear triggering condition. For example, a task enters the pending scheduling state when it is enqueued, enters the pending execution state when it is pulled and assigned by the scheduler, enters the in execution state when execution begins, and enters the successful execution state after execution is completed and the output verification passes. At the same time, each state change is recorded in detail, ensuring that the entire task execution process is traceable and troubleshootable.

[0073] Specifically, after a task is completed, regardless of whether it succeeds or fails, the scheduler will immediately release the system resources occupied by the task, close the underlying multimedia processing process, and update the resource monitoring data.

[0074] In one embodiment, the scheduling and execution of the multimedia processing instructions further includes: dividing the entire execution process of the multimedia processing instructions into multiple execution stages, assigning a corresponding progress weight to each execution stage, with the sum of the progress weights of each execution stage being 100%; for the core execution stage of the multimedia processing instructions, before execution begins, obtaining the processing baseline value of the input source file through a probe interface; during execution, capturing the currently processed progress data in real time through the progress output mechanism of the underlying multimedia processing tool; calculating the real-time progress of the core execution stage based on the processing baseline value and the currently processed progress data; and calculating the global progress value in conjunction with the progress weight of the core execution stage; pushing the task execution status and progress information to the front end in real time through a full-duplex long-distance connection channel; wherein, when the task status changes, immediately pushing the latest task status and the corresponding global progress value; during the execution of the core execution stage, continuously pushing real-time progress data according to a preset push cycle; and pushing corresponding result information or exception information when the task is completed, encounters an error, or is terminated.

[0075] Specifically, the multiple execution stages include: parameter verification and command assembly stage (weight 5%), input file probe and preprocessing stage (weight 10%), multimedia processing core execution stage (weight 75%), output file verification stage (weight 8%), and result storage and return stage (weight 2%). After each stage is completed, the engine automatically accumulates the progress value corresponding to the weight and pushes it to the front end through a full-duplex long-distance connection channel, allowing users to clearly perceive the overall progress of the task execution. Through the above-mentioned execution stage division and progress weight allocation mechanism, the originally coarse-grained task status is refined into quantifiable stage progress nodes, providing a structured data foundation for subsequent accurate progress calculation and real-time push.

[0076] Specifically, for the core execution phase of the multimedia processing instructions, which accounts for the largest proportion, the engine adopts a real-time progress calculation model based on processing duration. The specific execution logic is as follows: First, before the multimedia processing instructions begin execution, the engine obtains the processing baseline value of the input source file through a probe interface. The processing baseline value includes the total duration or total number of frames of the input source file, which serves as the baseline denominator for subsequent progress calculations. Second, during the execution of the multimedia processing instructions, the engine captures the currently processed progress data in real time through the progress output mechanism of the underlying multimedia processing tool. For example, a progress output parameter is added when the underlying multimedia processing process is started, enabling the underlying multimedia processing tool to output information such as processing progress, processed duration, frame rate, and bit rate to the engine through a pipeline in real time during execution. The engine captures and parses the progress output stream in real time according to a preset parsing period (e.g., every 200ms) to extract the currently processed video duration or processed frame count. Then, based on the processing baseline value and the currently processed progress data, the engine calculates the real-time progress of the core execution stage according to the formula: Core Execution Stage Real-time Progress = (Processed Duration / Total Duration of Source Files) × 100% or Core Execution Stage Real-time Progress = (Processed Frames / Total Frames) × 100%. Combining this with the progress weight (75%) of the core execution stage, the engine calculates the corresponding global progress value for that stage, i.e., Core Execution Stage Global Progress = Core Execution Stage Real-time Progress × 75%. For scenarios with multiple input source files, such as multi-file concatenation and multi-track mixing, the engine uses the total duration of all input source files as the processing baseline value and calculates the progress based on the total processed duration. For scenarios without input source file duration, such as image-to-video generation and audio visualization, the engine uses a preset total number of frames as the processing baseline value and calculates the progress based on the number of processed frames. Through the above progress calculation model, accurate quantification of the progress of the core execution stage of the multimedia processing instructions is achieved, providing reliable data support for the smooth updating of the progress bar.

[0077] Specifically, the engine pushes task execution status and progress information to the front end in real time through a full-duplex connection channel; when a user enters the canvas or task list page, the front end and back end automatically establish a full-duplex connection, binding the user's unique identifier and the task's unique identifier, ensuring that the front end can receive various status and progress data during the execution of the multimedia processing instructions in real time.

[0078] Specifically, when a task status changes, such as from pending scheduling to executing, from executing to executing completed, or executing failed, the engine immediately pushes the latest task status and corresponding global progress value to the front end through the full-duplex long-term connection channel. Simultaneously, it pushes stage description information, enabling the front end interface to respond instantly to changes in user operation permissions. During the core execution stage of the multimedia processing instructions, the engine continuously pushes real-time progress data to the front end according to a preset push cycle, such as every 200ms. This data includes real-time progress value, processed time, and estimated remaining time, ensuring smooth updates of the front end progress bar without any stuttering or jumps. When the task is completed, the engine pushes the preview address, download address, and file metadata of the output file to the front end, while simultaneously closing the progress push channel for that task, retaining the full-duplex long-term connection for synchronization of subsequent tasks. When an exception, timeout, or termination occurs during task execution, the engine immediately pushes exception information, error codes, and standardized prompts to the front end, along with corresponding solutions, helping users quickly understand the cause of the failure and take appropriate measures.

[0079] Specifically, if a full-duplex connection is lost due to a front-end page refresh or network interruption, the front-end can retrieve the latest status and progress of the multimedia processing instructions from the back-end through the task's unique identifier after reconnection. This completes the progress information during the interruption, ensuring that progress synchronization is not lost and enhancing system reliability and user experience.

[0080] Specifically, after receiving progress and status pushes, the front end updates the node's status indicator, progress bar, and operation button status in real time: in the execution state, the node's parameter modification and resubmit buttons are disabled, while the pause and stop buttons are displayed; in the execution success state, the output file preview and download buttons are displayed; in the execution failure state, the error message is highlighted and the resubmit button is displayed, realizing full-process visual interaction.

[0081] In one embodiment, after the scheduling execution is completed, the generated output file is subjected to full-dimensional output verification, specifically including: verifying whether the output file exists under the output path and whether the size of the output file is greater than zero; verifying whether the file header magic number of the output file conforms to a preset format standard; reading the metadata and audio / video stream information of the output file through a probe interface to verify the integrity and stream integrity of the output file; verifying whether the difference between the actual duration of the output file and the preset target duration is within a preset fault tolerance threshold; verifying whether the timestamp deviation between the audio stream and the video stream of the output file is within a preset synchronization deviation threshold; if all verifications pass, the output file is determined to be compliant; if any verification fails, the output file is deleted and an abnormal information capture and conversion process is triggered.

[0082] Specifically, after the multimedia processing instruction scheduling and execution is completed, the engine first checks whether the output file has been generated under the output path, and simultaneously verifies whether the size of the output file is greater than zero. If the output file does not exist or its size is not greater than zero, it is directly determined that the verification fails, triggering an exception that the output file has not been generated, and terminating the subsequent verification process.

[0083] Specifically, the engine reads the file header magic number of the output file and verifies whether the file header magic number conforms to a preset format standard, such as MP4 format, to filter out files with incorrect format or forged file extensions. At the same time, the engine reads the encoding information of the output file through a probe interface to verify whether the video encoding and audio encoding conform to preset encoding standards, such as H.264 video encoding and AAC audio encoding, to ensure that the output file conforms to the preset standardized format and is compatible with the standardized processing of players on all platforms and subsequent nodes.

[0084] Specifically, the engine reads the full metadata of the output file through the probe interface to check whether the file has problems such as index corruption, missing audio and video streams, and corrupted frame data; at the same time, it performs a tail frame reading verification on the output file to verify whether the file has been written completely without truncation or damage, ensuring that there is no truncation or damage; if file corruption or missing audio and video streams are detected, it is determined that the verification fails, triggering an exception that the output file is corrupted, and terminating the subsequent verification process.

[0085] Specifically, the engine reads the actual total duration of the output file and compares it with the preset target duration in the task parameters to verify whether the difference between the actual duration and the preset target duration is within a preset fault tolerance threshold. If the difference between the actual duration and the target duration exceeds the preset fault tolerance threshold, for example, ±0.5 seconds, the verification is deemed unsuccessful, triggering an anomaly that the output file duration does not meet expectations, and terminating the subsequent verification process. For scenarios with multiple input source files, such as multi-file concatenation and audio replacement, the engine also simultaneously verifies whether the output duration matches the total duration of all input files to ensure that the output file meets business expectations in terms of duration.

[0086] Specifically, the engine parses the audio and video timestamps of the output file and verifies whether the timestamp deviation between the audio stream and the video stream is within a preset synchronization deviation threshold, such as 100ms. If the synchronization deviation between the audio stream and the video stream exceeds the preset synchronization deviation threshold, the verification fails, triggering an audio-visual desynchronization anomaly. Simultaneously, it also verifies whether the video frame rate and bitrate are consistent with the parameter configuration to avoid issues such as stuttering or blurring.

[0087] Specifically, if all verification items pass, the output file is deemed compliant, its information is stored in the platform's media asset library, the task status is updated to successful execution, and the execution result, including the preview and download addresses of the output file, is pushed to the front end. If any verification item fails, the task is deemed to have failed, the engine automatically deletes the output file to free up disk space, and simultaneously triggers an exception information capture and conversion process to convert the underlying exception information into standardized error messages and push them to the front end.

[0088] Step 104: If the scheduling execution fails or the full-dimensional output verification fails, capture the underlying exception information and convert the underlying exception information into a standardized error message.

[0089] In one embodiment, converting the underlying anomaly information into standardized error messages specifically includes: capturing underlying anomaly information during the execution of the multimedia processing instructions in real time, wherein the underlying anomaly information includes at least standard error stream output logs, process exit codes, system signals, and execution timeout information; summarizing the anomaly information acquired during the real-time capture process into a raw anomaly log, and performing text cleaning on the raw anomaly log to extract anomaly description statements; constructing a hierarchical anomaly keyword matching rule base, wherein the hierarchical anomaly keyword matching rule base is divided into multiple anomaly levels according to anomaly type, and each anomaly level corresponds to a preset error code range, matching keywords, standardized prompts, and troubleshooting suggestions; matching the anomaly description statements with the matching keywords in the hierarchical anomaly keyword matching rule base to determine the matching result corresponding to the anomaly description statements, wherein the matching result includes at least an anomaly level, error code, standardized prompts, and troubleshooting suggestions; and generating standardized error messages based on the matching results.

[0090] Specifically, during the execution of the multimedia processing instructions, the engine captures exception information in real time through the standard error stream. This exception information includes at least the standard error stream output log, process exit codes, system signals, and execution timeout information. Simultaneously, the engine also captures operating system-level exceptions, including system-level exceptions such as insufficient resources.

[0091] Specifically, after a task fails, the engine aggregates all exception information acquired during the real-time capture process into a raw exception log and stores it in the task execution record for subsequent troubleshooting and exception matching. This full real-time capture mechanism ensures that all exception information generated during the underlying execution process is completely recorded, providing a comprehensive data foundation for subsequent standardized transformations.

[0092] Specifically, the engine pre-builds a hierarchical error keyword matching rule base. This rule base is divided into multiple error levels based on error type, with each level corresponding to a pre-defined unique error code, matching keyword, standardized prompt, and troubleshooting suggestions.

[0093] Specifically, the multiple exception levels include: parameter configuration exceptions, input file exceptions, system resource exceptions, and system internal exceptions.

[0094] Specifically, for parameter configuration errors: error codes range from 1001 to 1999, corresponding matching keywords include invalid parameters, unsupported format, timestamp out of range, no encoder, and resolution not supported. Standardized prompts clearly inform the user of the incorrect parameter and the correct configuration method.

[0095] Specifically, for input file-related errors: error codes range from 2001 to 2999, corresponding matching keywords include file not found, unable to open input file, corrupted file, no audio or video stream, and format not recognized. Standardized prompts clearly inform users of the input file problem and solutions.

[0096] Specifically, for system resource exceptions: error codes in the range of 3001-3999, corresponding matching keywords include insufficient memory, insufficient disk space, permission denied, process terminated, and execution timeout. Standardized prompts inform users of the current service status and guide users to try again later or contact the administrator.

[0097] Specifically, for internal system exceptions: error codes in the range of 4001-4999, corresponding matching keywords include command syntax error, template exception, multimedia processing tool not found, and pipe error. The standardized prompt message informs the user of the internal service exception, that the administrator has been notified to investigate, and a detailed exception log is recorded.

[0098] Specifically, by constructing the hierarchical abnormal keyword matching rule base, the obscure abnormal information at the bottom layer is classified according to type, and a standardized conversion template is preset for each type of abnormality, providing a structured basis for subsequent abnormal matching and conversion.

[0099] Specifically, after a task fails, the engine performs a standardized transformation process on the original exception log. First, the engine cleanses the original exception log, removing irrelevant debugging information, timestamps, and redundant characters, and extracts the exception description statement. Second, the engine performs regular expression matching between the exception description statement and the matching keywords in the hierarchical exception keyword matching rule base, according to the priority order of exception categories; it prioritizes matching higher-priority exception categories, such as parameter configuration categories over input file categories, and system resource categories over system internal categories. Once a corresponding exception rule is matched, the matching stops, and the matching result corresponding to the exception description statement is determined. The matching result includes at least the anomaly level, error code, standardized prompt, and troubleshooting suggestions. If the match is successful, the engine extracts the error code, standardized prompt, and troubleshooting suggestions from the matched rules, and combines them with the task parameter information to generate the final standardized error message. For example, the original anomaly message "Invalid time duration 00:05:00 exceeds input file total duration 00:03:20" is converted to "[Error Code 1002] Time range configuration error. The total duration of the input video is 3 minutes and 20 seconds. The end time cannot exceed the total duration of the source file. Please adjust the time parameters and resubmit." If no preset rule is matched, the engine classifies it as an unknown anomaly, assigns error code 4001, generates a standardized prompt, and if the task fails, the anomaly information is recorded and the administrator is notified to investigate. Please try again later. At the same time, the original anomaly log is pushed to the administrator's alarm channel.

[0100] Specifically, all exception information is bound to a unique task identifier and stored in the execution log database, allowing administrators to query and statistically analyze by error code, task type, and time range. At the same time, for frequently occurring exceptions, statistical reports are automatically generated and pushed to R&D and operations personnel to optimize node parameter verification rules, command templates, and user operation guidance.

[0101] Specifically, after generating a standardized error message based on the matching results, the engine returns the standardized error message to the front end, so that the user can clearly understand the reason for the failure and take corresponding measures.

[0102] In this embodiment, all video editing nodes follow the platform's unified workflow node specifications. Specifically, the input and output parameters of each video editing node are defined according to a unified metadata structure, ensuring full compatibility between the video editing nodes and the platform's AI generation nodes, logic control nodes, and material input nodes. The output of a node can be directly used as input for other nodes without manual format conversion or adaptation development; through this standardized design, seamless plug-and-play integration is achieved between nodes of different functional types.

[0103] In one embodiment, users can drag and drop multiple video editing nodes in a visual canvas and freely connect them with AI digital human nodes, text-based video nodes, intelligent subtitle nodes, and audio generation nodes to build a custom automated video production workflow.

[0104] Specifically, users can define upstream and downstream dependencies between nodes by dragging and dropping on the canvas, forming a directed acyclic execution flow chart. This allows users to complete the arrangement of complex video production processes without writing code, lowering the barrier to entry for non-technical users.

[0105] In one embodiment, during workflow execution, the system automatically generates an execution topology based on the node connection relationships. During execution, after the current node completes its execution, the system automatically passes the output of the current node to the next node in the topology, triggering parameter parsing, verification, and execution in the next node; that is, the next node automatically receives the output and incorporates it into its own parameter parsing process, without requiring manual uploading or configuration by the user. Through automated node-by-node concatenation, fully unattended automated video production is achieved.

[0106] In one embodiment, workflow template saving is supported, allowing users to save the completed video production workflow as a template. The next time the workflow is used, users only need to replace the input materials, such as changing the video source file or replacing files in specific material nodes, to execute the entire video production process with a single click. This template-based mechanism adapts to batch video production scenarios, significantly improving batch production efficiency.

[0107] Example 2, see Figure 2 , Figure 2 This is a schematic diagram of an embodiment of a multimedia processing task scheduling and execution device provided in this application. Corresponding to the above-described multimedia processing task scheduling and execution method, this application also provides a multimedia processing task scheduling and execution device. This multimedia processing task scheduling and execution device includes modules for executing the above-described multimedia processing task scheduling and execution method, and can be configured in a desktop computer, tablet computer, laptop computer, or other terminal. Specifically, the multimedia processing task scheduling and execution device includes a multimedia processing instruction template acquisition module 201, a multimedia processing instruction generation module 202, a scheduling and execution module 203, and a low-level exception information conversion module 204.

[0108] The multimedia processing instruction template acquisition module 201 is used to acquire a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template. The multimedia processing instruction template includes fixed instruction fragments and parameter placeholders, and the node metadata includes parameter definitions and verification rules corresponding to the parameter placeholders.

[0109] The multimedia processing instruction generation module 202 is used to receive a set of structured parameters passed in by the upper-layer node, perform a validity check on the set of structured parameters according to the verification rules in the node metadata, replace the parameter values ​​that pass the validity check with the corresponding parameter placeholders in the multimedia processing instruction template, and combine the replaced parameter placeholders with the fixed instruction fragment to generate a multimedia processing instruction.

[0110] The scheduling and execution module 203 is used to schedule and execute the multimedia processing instructions, and to perform full-dimensional output verification on the generated output file after the scheduling and execution is completed.

[0111] The underlying exception information conversion module 204 is used to capture underlying exception information and convert it into a standardized error message if the scheduling execution fails or the full-dimensional output verification fails.

[0112] In one embodiment, before receiving the structured parameter set from the upper-layer node, the multimedia processing instruction generation module 202 further includes: performing front-end real-time verification of the user-input parameters according to the verification rules in the node metadata, wherein the front-end real-time verification includes at least format correctness verification, value range compliance verification, and required field completeness verification; if the front-end real-time verification fails, an immediate prompt is made and parameter submission is prohibited; if the front-end real-time verification passes, the structured parameter set is encapsulated and generated according to the user-input parameters and the node metadata, and the step of receiving the structured parameter set from the upper-layer node is executed.

[0113] In one embodiment, the multimedia processing instruction generation module 202 is used to perform legality verification on the structured parameter set according to the verification rules in the node metadata. Specifically, it includes: performing four levels of legality verification on the structured parameter set in a predetermined order according to the verification rules in the node metadata; wherein, the legality verification includes a first-level syntax layer verification, a second-level security layer verification, a third-level business layer single-parameter verification, and a fourth-level business layer cross-parameter joint verification. Each level of verification is executed sequentially. If the current level of verification fails, the subsequent verification process is terminated and the next level of verification is not entered. If the current level of verification passes, the next level of verification continues to be executed.

[0114] In one embodiment, the multimedia processing instruction generation module 202 is used to replace the parameter values ​​that have passed the legality check with the corresponding parameter placeholders in the multimedia processing instruction template, and combine the replaced parameter placeholders with the fixed instruction fragment to generate a multimedia processing instruction. Specifically, this includes: performing regular expression matching on the multimedia processing instruction template to extract all parameter placeholders in the multimedia processing instruction template and generating a placeholder list; matching the parameter values ​​in the structured parameter set with the placeholder list, and assigning parameter values ​​to each parameter placeholder in the placeholder list according to a preset priority; performing type conversion and formatting processing on the matched parameter values ​​according to the parameter type of each parameter placeholder to obtain formatted parameter values; performing security filtering on the formatted parameter values ​​to detect and remove dangerous characters in the formatted parameter values ​​to obtain filtered parameter values; and replacing the filtered parameter values ​​with the corresponding parameter placeholders in the instruction template to generate a complete multimedia processing instruction.

[0115] In one embodiment, the scheduling and execution module 203 is used to schedule and execute the multimedia processing instructions, specifically including: encapsulating the multimedia processing instructions into tasks to be executed; constructing multiple priority task queues and allocating the tasks to be executed to the corresponding target priority task queues according to preset priorities; monitoring the resource usage status of the server in real time, the resource usage status including at least CPU utilization, memory utilization, and the number of tasks currently being executed; according to the resource usage status, pulling the tasks to be executed from the target priority task queues for allocation and execution according to preset scheduling rules, and performing state management for the entire lifecycle of the tasks to be executed; wherein, the preset scheduling rules include priority-first scheduling rules, resource quota control rules, single task timeout control rules, same-user task rate limiting rules, abnormal task circuit breaker rules, and execution failure retry rules.

[0116] In one embodiment, the scheduling execution module 203 is used to perform full-dimensional output verification on the generated output file after the scheduling execution is completed. Specifically, this includes: verifying whether the output file exists under the output path and whether the size of the output file is greater than zero; verifying whether the magic number of the file header of the output file conforms to a preset format standard; reading the metadata and audio / video stream information of the output file through a probe interface to verify the integrity and stream integrity of the output file; verifying whether the difference between the actual duration of the output file and the preset target duration is within a preset fault tolerance threshold; verifying whether the timestamp deviation between the audio stream and the video stream of the output file is within a preset synchronization deviation threshold; if all verifications pass, the output file is determined to be compliant; if any verification fails, the output file is deleted and an abnormal information capture and conversion process is triggered.

[0117] In one embodiment, the underlying exception information conversion module 204 is used to convert the underlying exception information into standardized error messages, specifically including: capturing underlying exception information during the execution of the multimedia processing instructions in real time, wherein the underlying exception information includes at least standard error stream output logs, process exit codes, system signals, and execution timeout information; summarizing the exception information acquired during the real-time capture process into raw exception logs, and performing text cleaning on the raw exception logs to extract exception description statements; constructing a hierarchical exception keyword matching rule library, wherein the hierarchical exception keyword matching rule library is divided into multiple exception levels according to exception type, and each exception level corresponds to a preset error code range, matching keywords, standardized prompts, and troubleshooting suggestions; matching the exception description statements with the matching keywords in the hierarchical exception keyword matching rule library to determine the matching result corresponding to the exception description statements, wherein the matching result includes at least the exception level, error code, standardized prompts, and troubleshooting suggestions; and generating standardized error messages based on the matching results.

[0118] The multimedia processing task scheduling and execution device described above can implement the multimedia processing task scheduling and execution method of the above method embodiments. The options in the above method embodiments are also applicable to this embodiment, and will not be detailed here.

[0119] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a computer device provided in this application; it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 communicate with each other through the communication bus 114, and the memory 113 is used to store computer programs.

[0120] In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the multimedia processing task scheduling and execution method provided in any of the foregoing method embodiments.

[0121] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0122] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the multimedia processing task scheduling and execution method provided in any of the foregoing method embodiments.

[0123] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0126] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0127] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0128] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0129] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for scheduling and executing multimedia processing tasks, characterized in that, include: Obtain a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template. The multimedia processing instruction template includes fixed instruction fragments and parameter placeholders. The node metadata includes parameter definitions and verification rules corresponding to the parameter placeholders. The system receives a set of structured parameters from an upper-layer node, performs a validity check on the set of structured parameters according to the validation rules in the node metadata, replaces the parameter values ​​that pass the validity check with the corresponding parameter placeholders in the multimedia processing instruction template, and combines the replaced parameter placeholders with the fixed instruction fragment to generate a multimedia processing instruction. The multimedia processing instructions are scheduled and executed, and the generated output files are fully verified after the scheduling and execution are completed. If the scheduling execution fails or the full-dimensional output verification fails, the underlying exception information is captured and converted into a standardized error message.

2. The method according to claim 1, characterized in that, Before receiving the structured parameter set from the upper-layer node, the method further includes: According to the verification rules in the node metadata, the parameters input by the user are subject to real-time front-end verification during the user configuration phase. The real-time front-end verification includes at least format correctness verification, value range compliance verification, and mandatory field completeness verification. If the front-end real-time validation fails, an immediate message will be displayed and parameter submission will be prohibited. If the front-end real-time verification passes, the structured parameter set is encapsulated and generated based on the user-input parameters and the node metadata, and the step of receiving the structured parameter set passed from the upper-layer node is executed.

3. The method according to claim 1, characterized in that, The step of validating the structured parameter set according to the validation rules in the node metadata specifically includes: Based on the verification rules in the node metadata, the structured parameter set is subjected to four levels of validity verification in a predetermined order; The validity verification includes a first-level syntax layer verification, a second-level security layer verification, a third-level business layer single-parameter verification, and a fourth-level business layer cross-parameter joint verification. Each level of verification is executed sequentially. If the current level of verification fails, the subsequent verification process is terminated and the next level of verification is not entered. If the current level of verification passes, the next level of verification continues.

4. The method according to claim 1, characterized in that, The step of replacing the valid parameter values ​​with the corresponding parameter placeholders in the multimedia processing instruction template, and combining the replaced parameter placeholders with the fixed instruction fragments to generate multimedia processing instructions, specifically includes: Perform regular expression matching on the multimedia processing instruction template to extract all parameter placeholders in the multimedia processing instruction template and generate a placeholder list; The parameter values ​​in the structured parameter set are matched with the placeholder list, and parameter values ​​are assigned to each parameter placeholder in the placeholder list according to a preset priority. Based on the parameter type of each parameter placeholder, perform type conversion and formatting on the matched parameter values ​​to obtain formatted parameter values; Perform security filtering on the formatting parameter value, detect and remove dangerous characters in the formatting parameter value, and obtain the filtered parameter value; Replace the filter parameter values ​​with the corresponding parameter placeholders in the instruction template to generate a complete multimedia processing instruction.

5. The method according to claim 1, characterized in that, The scheduling and execution of the multimedia processing instructions specifically includes: The multimedia processing instructions are encapsulated into a task to be executed; Construct multiple priority task queues and assign tasks to be executed to the corresponding target priority task queues according to preset priorities; Real-time monitoring of server resource usage status, including at least CPU utilization, memory utilization, and the number of tasks currently being executed; Based on the resource usage status, the tasks to be executed are pulled from the target priority task queue according to the preset scheduling rules and allocated for execution, and the status management of the entire life cycle of the tasks to be executed is performed. The preset scheduling rules include priority-first scheduling rules, resource quota control rules, single task timeout control rules, same-user task rate limiting rules, abnormal task circuit breaker rules, and execution failure retry rules.

6. The method according to claim 1, characterized in that, The step of performing full-dimensional output verification on the generated output file after the scheduling execution is completed specifically includes: Verify that the output file exists in the output path and that the size of the output file is greater than zero. Verify whether the magic number of the output file header conforms to the preset format standard; The metadata and audio / video stream information of the output file are read through the probe interface to verify the integrity of the output file and the stream integrity. Verify whether the difference between the actual duration of the output file and the preset target duration is within the preset fault tolerance threshold range; Verify whether the timestamp deviation between the audio stream and video stream of the output file is within a preset synchronization deviation threshold range; If all checks pass, the output file is deemed compliant. If any verification fails, the output file is deleted and the exception information capture and conversion process is triggered.

7. The method according to claim 1, characterized in that, The process of converting the underlying anomaly information into a standardized error message specifically includes: Real-time capture of low-level exception information during the execution of the multimedia processing instructions, including at least standard error stream output logs, process exit codes, system signals, and execution timeout information; The abnormal information acquired during the real-time capture process is summarized into a raw abnormal log, and the raw abnormal log is cleaned to extract abnormal description statements. A hierarchical abnormal keyword matching rule base is constructed. The hierarchical abnormal keyword matching rule base is divided into multiple abnormal levels according to the abnormal type. Each abnormal level corresponds to a preset error code range, matching keywords, standardized prompts and investigation suggestions. The abnormal description statement is matched with the matching keywords in the hierarchical abnormal keyword matching rule base to determine the matching result corresponding to the abnormal description statement. The matching result includes at least the abnormal level, error code, standardized prompt and investigation suggestions. A standardized error message is generated based on the matching results.

8. A scheduling and execution device for multimedia processing tasks, characterized in that, include: The module includes a multimedia processing instruction template acquisition module, a multimedia processing instruction generation module, a scheduling and execution module, and a low-level exception information conversion module. The multimedia processing instruction template acquisition module is used to acquire a standardized multimedia processing instruction template and node metadata associated with the multimedia processing instruction template. The multimedia processing instruction template includes fixed instruction fragments and parameter placeholders, and the node metadata includes parameter definitions and verification rules corresponding to the parameter placeholders. The multimedia processing instruction generation module is used to receive a set of structured parameters passed in by the upper-layer node, perform a validity check on the set of structured parameters according to the verification rules in the node metadata, replace the parameter values ​​that pass the validity check with the corresponding parameter placeholders in the multimedia processing instruction template, and combine the replaced parameter placeholders with the fixed instruction fragment to generate a multimedia processing instruction. The scheduling and execution module is used to schedule and execute the multimedia processing instructions, and to perform full-dimensional output verification on the generated output file after the scheduling and execution is completed. The underlying exception information conversion module is used to capture underlying exception information and convert it into standardized error messages if the scheduling execution fails or the full-dimensional output verification fails.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images