H5 animation generation and quality evaluation method, related device and program product

Abstract

The application discloses an H5 animation generation and quality evaluation method, related equipment and program products. The application is aimed at H5 animation code generated by a large model, renders the animation and captures image frame sequences during the animation playing process, and obtains DOM element information corresponding to each image. For each piece of explanation text in the code, text features are extracted, visual features of each image, layout features of DOM element information are extracted, and the three are mapped to the same alignment space. Based on the multi-modal features, the animation quality score is evaluated from at least one of the three dimensions of time sequence consistency, spatial consistency and layout rationality, ensuring the accuracy of the code quality evaluation. In the animation generation process, the large model generates multiple candidate H5 animation codes, selects the H5 animation code with the highest quality score and not lower than the set quality score threshold, and uses the H5 animation code to generate the H5 animation. The quality of the finally selected H5 animation code can be significantly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method for generating and evaluating the quality of H5 animations, as well as related equipment and program products. Background Technology

[0002] H5 animations are widely used in educational presentations, visualizations, and interactive web pages. In recent years, Large Models (LLMs) can automatically generate H5 animation code from text, but the generated results often suffer from problems such as animation timing mismatch with the rhythm of the text explanation, element positions, sizes, or animation transitions not conforming to visual specifications, lack of spatial consistency between different animations, and poor user experience.

[0003] Therefore, there is an urgent need for a quality assessment method for H5 animation generation that can accurately evaluate the quality of H5 animation code generated from large models, in order to support decisions on the use strategy of H5 animation code in subsequent steps, such as generating multiple times and selecting the best one, optimizing the H5 animation code, or discarding it. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide an H5 animation generation and quality assessment method, related equipment, and program products, so as to achieve accurate quality assessment of H5 animation code generated from large models. The specific solution is as follows:

[0005] Firstly, a method for evaluating the quality of H5 animation generation is provided, including:

[0006] Obtain the H5 animation code generated by the large model based on the animation generation instructions. The H5 animation code includes one or more explanatory texts.

[0007] The H5 animation code is executed in the rendering environment to render and generate the animation. During the animation playback process, multiple frames of images are captured and the document object model (DOM) element information corresponding to each frame of image is obtained.

[0008] Extract the text features of each explanatory text, extract the visual features of each frame image, and extract the layout features of the DOM element information corresponding to each frame image, and map the text features, the visual features, and the layout features to the same alignment space.

[0009] Based on the mapped text features, visual features, and layout features, an animation quality score is calculated from at least one of the three dimensions: temporal consistency, spatial consistency, and layout rationality. Temporal consistency measures the time synchronization between the animation content and the explanatory text. Spatial consistency measures the consistency between the spatial position of elements within the animation and the semantics of the explanatory text. Layout rationality measures the rationality of the animation screen layout.

[0010] Secondly, an H5 animation generation method is provided, including:

[0011] Obtain the animation generation instructions input by the user, and call the large model to generate multiple candidate H5 animation codes based on the animation generation instructions;

[0012] For each candidate H5 animation code, the H5 animation generation quality evaluation method described in the first aspect is used to calculate the quality score of each candidate H5 animation code;

[0013] If the highest quality score among all candidate H5 animation codes is not lower than the set quality score threshold, then the candidate H5 animation code with the highest quality score will be used as the final H5 animation code to generate the H5 animation.

[0014] Thirdly, an electronic device is provided, comprising: a memory and a processor;

[0015] The memory is used to store programs;

[0016] The processor is configured to execute the program to implement the steps of the H5 animation generation quality evaluation method as described in the first aspect, or to implement the steps of the H5 animation generation method as described in the second aspect.

[0017] Fourthly, a readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the H5 animation generation quality evaluation method as described in the first aspect, or implements the steps of the H5 animation generation method as described in the second aspect.

[0018] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the H5 animation generation quality evaluation method as described in the first aspect, or implements the steps of the H5 animation generation method as described in the second aspect.

[0019] Using the above technical solution, this application executes the obtained H5 animation code generated from a large model in a rendering environment to generate animation. During animation playback, it captures image frame sequences and obtains the DOM element information corresponding to each frame. For each explanatory text segment in the H5 animation code, textual features are extracted, as well as visual features and layout features of the DOM element information for each frame. These three are mapped to the same alignment space. Based on this, the animation quality score can be evaluated from at least one of three dimensions: temporal consistency, spatial consistency, and layout rationality. Temporal consistency measures the time synchronization between the animation content and the explanatory text; spatial consistency measures the consistency between the spatial position of elements within the animation and the semantics of the explanatory text; and layout rationality measures the rationality of the animation's layout. Clearly, this application's solution can evaluate animation quality from at least one of these dimensions, thus assessing the quality of the H5 animation code generated from the large model and ensuring the accuracy of the H5 animation code quality evaluation.

[0020] Building upon this foundation, during the H5 animation generation process, diversity sampling can be used to allow the large model to generate multiple candidate H5 animation codes based on animation generation instructions. For each candidate H5 animation code, a quality score is obtained using the aforementioned method. The candidate H5 animation code with the highest quality score that is not lower than a set quality score threshold is selected as the final H5 animation code used to generate the H5 animation. This significantly improves the quality of the final selected H5 animation code. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0022] Figure 1 A schematic diagram of an implementation system architecture for the H5 animation generation and quality assessment method provided in this application embodiment;

[0023] Figure 2 This is a schematic diagram of a method for evaluating the quality of H5 animation generation, provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram of an H5 animation generation method provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0028] The current approach of generating H5 animation code using large models cannot determine the quality of the generated code through quantitative indicators, and therefore cannot guide the application of subsequent H5 animation code.

[0029] To this end, this application provides an H5 animation generation quality assessment scheme, and further provides an H5 animation generation method based on the quality assessment scheme, which can improve the quality of the generated H5 animation code and improve the quality of H5 animation.

[0030] This application provides a method for generating and evaluating the quality of H5 animations, which can be applied to, for example... Figure 1 The system architecture shown includes a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0031] Either terminal 100 or server 200 can be used independently to execute the H5 animation generation and quality evaluation method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the H5 animation generation and quality evaluation method provided in the embodiments of this application.

[0032] The following description Figure 1 The product form of the mid-terminal 100;

[0033] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, teaching large screen, wearable device, learning machine, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0034] First, the H5 animation generation quality assessment method provided in the embodiments of this application will be introduced, taking the application of this method to a computer device as an example. Specifically, the computer device can be... Figure 1 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 2 The H5 animation generation quality assessment method specifically includes the following steps:

[0035] Step S100: Obtain the H5 animation code generated by the large model based on the animation generation instructions. The H5 animation code includes one or more explanatory texts.

[0036] The H5 animation code to be evaluated in this step is generated by the large model. Specifically, the animation generation command can be sent to the large model, and the large model will generate the H5 animation code in response to the animation generation command. The H5 animation code contains more than one explanatory text.

[0037] The H5 animation code to be evaluated in this step can be code from various scenarios, such as educational explanations and visual report displays.

[0038] Step S110: Execute H5 animation code in the rendering environment to render and generate animation. During the animation playback process, capture multiple frames of images and obtain the Document Object Model (DOM) element information corresponding to each frame of image.

[0039] Specifically, the H5 animation code obtained in the previous step can be executed in the rendering environment to generate an animation. For example, the H5 animation code can be rendered into an animation via a browser (such as the Chromium Embedded Framework). During animation playback, a sequence of image frames F={f1,f2,…, f…} can be captured. T For example, during the animation playback, a frame is captured at set intervals, resulting in a sequence of captured image frames F.

[0040] The DOM (Document Object Model) is a tree-structured representation of HTML page content by the browser. All elements within an H5 animation page (such as text, images, buttons, graphics, SVG, canvas, etc.) correspond to a single DOM tree within the browser.

[0041] During animation playback, the DOM element information corresponding to each frame can be obtained by calling code in the browser to retrieve DOM element information for each DOM element, such as getBoundingClientRect(), to obtain the DOM element information E corresponding to each frame. t :

[0042]

[0043] in, This represents the spatial position and size of the i-th DOM element.

[0044] This indicates the type information of the i-th DOM element, that is, the tag type of the DOM element in HTML. For example, the type of the "div" DOM ​​element is: container element; the type of the "img" DOM ​​element is: image element.

[0045] This represents the set of style parameters for the i-th DOM element, used to describe the element's visual appearance, spatial layout, and animation properties.

[0046] This represents the text semantic content carried in the i-th DOM element.

[0047] Step S120: Extract the text features of each explanatory text, extract the visual features of each frame image, and extract the layout features of the DOM element information corresponding to each frame image, and map the text features, the visual features, and the layout features to the same alignment space.

[0048] This step involves the extraction of multimodal features, specifically:

[0049] For each explanatory text T in the H5 animation code k (T) k (representing the k-th explanatory text), text features s can be extracted. k .

[0050] When extracting text features, pre-trained large language models can be used, such as Mistral-7B-instruct and Qwen2-7B-Chat, to extract features from each explanatory text T. k Extracting text features:

[0051]

[0052] For each frame image f t Visual Transformers (such as CLIP-ViT-L / 14, SigLIPencoder, etc.) can be used to extract global and local visual features of each frame of the image. t :

[0053]

[0054] For the DOM element information E corresponding to each frame image tThe layout features d can be obtained by encoding through the LayoutFormer network. t :

[0055]

[0056] For the multimodal features extracted above: text features s k Visual features v t and layout features d t It can be projected to the same alignment space, for example, text features s k Visual features v t Projected onto layout feature d t The feature space in which it resides.

[0057] By using text features k Visual features v t and layout features d t Projecting them onto the same alignment space facilitates the processing of multimodal features in subsequent steps, and the animation quality score is calculated based on one or more of these modal features.

[0058] Step S130: Based on the mapped text features, visual features, and layout features, calculate the animation quality score from at least one of the three dimensions: temporal consistency, spatial consistency, and layout rationality.

[0059] Among them, temporal consistency is used to measure the time synchronization between animation content and explanatory text, spatial consistency is used to measure the consistency between the spatial position of elements in the animation and the semantics of the explanatory text, and layout rationality is used to measure the rationality of the animation screen layout.

[0060] In this step, based on the mapped text features, visual features, and layout features, the animation quality is quantified from at least one of the three dimensions: time alignment (temporal consistency), spatial alignment (spatial consistency), and layout rationality. This allows the animation quality score to reflect the performance of the animation and the explanatory content in terms of logic, coherence, and / or visual balance, ensuring the accuracy of the H5 animation code quality assessment.

[0061] In some possible implementations, the animation quality score can be calculated simultaneously from three dimensions: temporal consistency, spatial consistency, and layout rationality. That is, the animation quality score R can be expressed as:

[0062] R=α·S temporal +β·S spatial +γ·S layout

[0063] Among them, S temporal S spatial S layoutThese represent the temporal consistency score, spatial consistency score, and layout rationality score, respectively, with α, β, and γ being the set weight parameters.

[0064] By using the above three dimensions of scoring, the quality of H5 animation code can be evaluated from the perspectives of the consistency between the explanation content and the animation in terms of time and space, as well as the rationality of the layout of elements within the animation, thus improving the accuracy of quality assessment.

[0065] Next, the specific calculation process of the temporal consistency score, spatial consistency score, and layout rationality score will be explained.

[0066] 1. Timing consistency score

[0067] Based on the textual features s of each explanatory text k and the visual features v of each frame of the image t Determine the first alignment distribution of each explanatory text segment and each frame image. :

[0068]

[0069] in, For temperature parameters. The text feature s in the formula. k and visual features v t All of these are features projected onto the same alignment space.

[0070] Based on the first alignment distribution And the ideal time distribution of the explanatory text and each frame of the image. Calculate the temporal consistency score S between the explanatory text and the image. temporal .

[0071] Among them, the ideal time distribution of the explanatory text and each frame image. It can be calculated using the DOM animation timeline during the execution of H5 animation code. Details are as follows:

[0072] The process of executing H5 animation code in the rendering environment to render and generate animation, locating the k-th explanatory text T. kThe corresponding DOM element set (e.g., div / p / span containing the text) is then used to extract the time intervals (visible intervals) for the appearance and disappearance of each DOM element from the animation control information of each DOM element in the DOM element set. This includes, but is not limited to, CSS animation / transition parameters (animation-delay, animation-duration, transition-duration, etc.), startTime and duration returned by the WebAnimations API, and the control of visibility by the JS timeline (setTimeout, requestAnimationFrame, or animation library timeline).

[0073] If the k-th explanatory text T k If the corresponding DOM element set contains multiple DOM elements, then the visible regions of each element in the DOM element set are merged to obtain the k-th explanatory text T. k Overall ideal time window .

[0074] Mapping the aforementioned overall ideal time window onto the frame sequence index, for each frame t, if the frame time falls within... If a frame is selected within a given frame, it is assigned a positive weight; otherwise, it is assigned a weight of 0. Finally, the weights of all frames are normalized to form a probability distribution over time, thus obtaining the ideal temporal distribution of the k-th explanatory text and each frame's image. :

[0075]

[0076]

[0077] In one alternative example, KL divergence can be used to explain the temporal consistency score S between text and images. temporal :

[0078] .

[0079] This embodiment calculates the first alignment distribution between each segment of explanatory text and each frame of image, and calculates the temporal consistency score between the explanatory text and the animated image based on the first alignment distribution and the ideal time distribution, thereby measuring the temporal alignment between the explanatory content and the animated image.

[0080] 2. Spatial Consistency Score

[0081] At the spatial level, this embodiment calculates the overlap consistency between the text-related elements in the image frame and the DOM element regions of the image frame.

[0082] Specifically, the visual regions in the current frame image that best represent the semantics of the explanatory text segment (such as text, formulas, graphics, highlighted areas, etc. corresponding to the explanatory content) are identified, thus obtaining the "semantically corresponding visual salient regions". These visual salient regions are then aligned to a collection of DOM elements, thereby completing the cross-modal association of "semantics-visual-structure".

[0083] A visual saliency map is a spatial heat map of the regions in a given frame of an image that are most relevant to a specific semantic meaning (text). It is used to describe "which regions in the image this text mainly corresponds to".

[0084] In this embodiment, the visual saliency map of each explanatory text segment on each frame image is calculated.

[0085] Based on the visual saliency map and the DOM element mask information of each frame image, the spatial alignment score between the visual saliency map of the explanatory text and the DOM element region is calculated.

[0086] A DOM element mask is a representation of a DOM element's spatial area on the screen as a binary image (or set of pixels), used to measure the spatial overlap between the element and visually salient areas.

[0087] Define M as the visual saliency map of the k-th explanatory text in the animation sequence composed of frames. k The mask B for the i-th DOM element i It can be obtained by aggregating the masks of the i-th DOM element in each frame of the image according to the time sequence.

[0088] The spatial alignment score is then expressed as:

[0089]

[0090] Where N is the number of DOM elements.

[0091] In some possible implementations, the visual saliency map M corresponding to the k-th explanatory text in each frame image. k It can be obtained in the following ways:

[0092] S1. Using the text features of the k-th explanatory text as the query, perform cross-attention processing on the visual features of each frame image to obtain the attention weights that represent the degree of attention the k-th explanatory text pays to different regions in each frame image.

[0093] Specifically, for the rendered first Frame Image Perform visual encoding and classify into By analyzing local regions (patches), a set of visual features for that frame can be obtained. At the same time, for the first Explanation of the text Semantic encoding is performed and mapped to a uniform alignment space to obtain text features. .Will As a query, visual features Using this as the key, we calculate the attention weights of the explanatory text for each local region of the frame, i.e., the frame-level saliency distribution:

[0094]

[0095] S2. The attention weights are converted into visual saliency maps of the k-th explanatory text in each frame image to obtain frame-level visual saliency maps.

[0096] Specifically, the attention weights (i.e., the frame-level saliency distribution) The spatial grid structure of the local area is reconstructed into a two-dimensional heatmap to obtain a frame-level visual saliency map. Furthermore, upsampling can be used to obtain saliency maps with the same resolution as the frame image. .

[0097] S3. Using the first alignment distribution of the k-th explanatory text and each frame image as weights, perform cross-time weighted aggregation on the frame-level visual saliency map to obtain the visual saliency map of the k-th explanatory text on each frame image.

[0098] Specifically, to obtain a cross-temporal saliency map for the entire explanatory text, the first alignment distribution obtained from the aforementioned temporal alignment analysis is used. The frame-level visual saliency maps are then aggregated across time using a weighted aggregation method to obtain the final visual saliency map.

[0099]

[0100] income Used to characterize the main visual corresponding area of ​​the k-th explanatory text in the animation sequence, and compared with the DOM element mask. Perform spatial overlap consistency measurement to complete spatial alignment analysis.

[0101] 3. Layout rationality score

[0102] In this embodiment, a layout rationality evaluation model can be pre-trained, which can employ the LayoutEvalNet network. During the training phase, training samples labeled with layout rationality scores are used. These training samples include frame-level visual feature samples and layout feature samples of DOM element information.

[0103] Therefore, when calculating the layout rationality score, frame-level visual features v can be used. t Layout features of DOM element information t The data is fed into the layout rationality assessment model to obtain the layout rationality score output by the model.

[0104] The layout rationality score can include scores for at least one of the following four dimensions:

[0105] q1: Alignment score;

[0106] q2: Overlap penalty score;

[0107] Q3: Score based on the amount of white space used;

[0108] q4: Rhythm balance score.

[0109] Among them, the alignment score is used to indicate the degree of spatiotemporal alignment between the explanatory text segment and the image frame; the overlap penalty score is used to indicate the degree of penalty for the overlap of DOM elements within the image frame; the white space ratio score is used to indicate the degree to which the white space ratio within the image frame approaches a reasonable range; and the rhythm balance score is used to indicate the reasonableness of the distribution of element playback events on the timeline in the animation.

[0110] The white space ratio is not necessarily better the higher it is, nor is it better the lower it is; rather, there is a reasonable range, and the goal of system evaluation is to bring it close to the target range.

[0111] Insufficient white space: Pages become crowded, information density is too high, and the probability of element overlap increases;

[0112] Excessive white space results in sparse information, low expressive efficiency, and a hollow visual rhythm.

[0113] Therefore, the scoring strategy for the white space ratio in this application is usually as follows:

[0114] A higher score is achieved when the white space ratio falls within the empirical range (e.g., 20%–50%), and a lower score is achieved when it deviates from this range.

[0115] Rhythm balance refers to whether the distribution of events such as the appearance, movement, fade-in and fade-out of elements in the animation is even, coherent, and in line with the rhythm that users understand.

[0116] Specifically, this means that animated events should not be excessively concentrated in a short period of time (causing an "explosive appearance"), nor should there be a long period without key events (causing a "stagnation"). The pace of textual explanation should match the pace of visual change.

[0117] In one alternative implementation, the layout rationality score can be expressed as:

[0118]

[0119] in, This indicates that weight parameters are set.

[0120] By simultaneously considering alignment score, overlap penalty score, white space ratio score, and rhythm balance score, a more accurate layout rationality score can be obtained.

[0121] In some embodiments of this application, based on the H5 animation generation quality evaluation method described in the foregoing embodiments, an H5 animation generation method is further provided.

[0122] Reference Figure 3 As shown, the H5 animation generation method may include the following steps:

[0123] Step S200: Obtain the animation generation instructions input by the user, and call the large model to generate multiple candidate H5 animation codes based on the animation generation instructions.

[0124] In this step, multiple candidate H5 animation codes can be generated based on animation generation instructions using a large model and diverse sampling.

[0125] Step S210: Calculate the quality score for each candidate H5 animation code.

[0126] Specifically, for each candidate H5 animation code obtained in the previous step, the H5 animation generation quality evaluation method described in the aforementioned embodiments can be used to calculate the quality score R of the candidate H5 animation code.

[0127] Step S220: If the highest quality score among all candidate H5 animation codes is not lower than the set quality score threshold, then the candidate H5 animation code with the highest quality score is used as the final H5 animation code to generate the H5 animation.

[0128] Select the H5 animation code with the highest quality score from all candidate H5 animation codes. If the highest quality score is not lower than the set quality score threshold R... min Then, the candidate H5 animation code with the highest quality score can be used as the final H5 animation code to generate the H5 animation.

[0129] In the H5 animation generation method provided in this embodiment, multiple candidate H5 animation codes can be obtained through diversity sampling, generated from a large model. Based on this, the quality of each candidate H5 animation code is calculated using the aforementioned quality assessment method. The one with the highest score can be selected. If the highest score is not lower than a set quality score threshold, the candidate H5 animation code with the highest score can be used as the final result, thus improving the quality of the finally selected H5 animation code.

[0130] Further, optionally, if the highest quality score among all candidate H5 animation codes is lower than a set quality score threshold R...min This indicates that the quality of all candidate H5 animation codes generated by the large model is too low to be directly used. This embodiment further provides a code optimization scheme, specifically:

[0131] The candidate H5 animation code with the highest quality score is used as the baseline code to be optimized, and the parameter x to be optimized is determined from the baseline code.

[0132] In some possible implementations, the scope of optimization can be constrained, for example, by limiting the type of the parameter to be optimized to element position, animation delay, font size and style, etc.

[0133] Based on this, the quality score R of the optimized H5 animation code is used as the optimization objective. The parameters to be optimized are solved to obtain the optimized H5 animation code after one round of optimization.

[0134] Specifically: The optimization goal is to improve the quality score (R) of H5 animation code.

[0135]

[0136] To ensure that the optimization process is interpretable, controllable, and does not compromise the semantics and usability of the animation, this application introduces several constraints while maximizing the quality score R. These constraints are used to limit the optimization range and avoid side effects such as "occlusion, disorder, or unpredictable style changes in pursuit of a higher score." This ensures that the output meets the requirements for engineering implementation in terms of visual readability, rhythm consistency, and code security.

[0137] The first constraint in the above optimization objective This means limiting the overlap between any two DOM elements in screen space to no more than a threshold. .in, These represent the spatial regions of the i-th and j-th DOM elements in the rendering frame (which can be represented by the element's bounding rectangle or pixel mask). The intersection-union ratio (IUGR) measures the overlap strength between two regions. This constraint is used to prevent issues such as text and graphics occlusion and element stacking caused by moving or scaling elements during the repair process, ensuring the readability of the animation and the stability of the layout.

[0138] The second constraint is: This means that the timing parameters of elements or animation events are limited to fine-tuning within a certain tolerance range to avoid disrupting the original explanation order and rhythmic logic. This represents the time parameters (such as occurrence time, delay, keyframe trigger time, etc.) of the i-th element (or key event) after repair. This indicates the reference time parameter (which can be the original time configuration before the repair, or the ideal time arrangement derived from the DOM animation timeline). This is the maximum allowed time offset threshold.

[0139] The third constraint is: This means that optimization is restricted to modifying only the parameter types or value ranges within a predefined "safety style set," where... This represents the set of style parameters for the i-th element (such as position, size, font size, opacity, layer, animation duration / easing, etc.). This indicates the adjustable style categories and scope, used to prevent the system from modifying high-risk attributes that may cause semantic drift, structural damage, or script anomalies, thereby ensuring that the repair process is safe, controllable, and reversible.

[0140] When solving the problem, you can use an optimizer based on gradient approximation (such as L-BFGS or AdamW) or a feasible solution search based on constraint-guided sampling.

[0141] If, after one or more rounds of optimization, the quality score of the final optimized H5 animation code is not lower than the set quality score threshold, then the final optimized H5 animation code will be used as the final H5 animation code.

[0142] The method in this embodiment aims to improve the quality score of H5 animation code. It optimizes the parameters of H5 animation code and can automatically correct problems such as element overlap, timing delay, layout misalignment and visual drift without manual intervention, thereby improving the quality of the final H5 animation code.

[0143] In some possible implementations, after each round of optimization, a quality score R' can be calculated for the optimized H5 animation code. If R' is greater than the quality score of the previous round of H5 animation code, the current round of optimized H5 animation code is updated to the new version; otherwise, it can be rolled back. Through one or more rounds of iterative optimization, the quality of H5 animation code can be continuously improved.

[0144] In some embodiments of this application, the aforementioned step S200 is a process of calling a large model to generate multiple candidate H5 animation codes based on animation generation instructions. The large model called can be a large model that has been trained in advance on the H5 animation code generation task.

[0145] During training, the loss function can include a first loss, which is the negative of the quality score of the H5 animation code generated by the large model, i.e., -R, where R = α·S. temporal +β·S spatial +γ·S layout .

[0146] By using the inverse of the quality score as the loss function, frame-level temporal synchronization, element-level spatial consistency, and semantic-level visual association can be optimized simultaneously during the training process.

[0147] Furthermore, to mitigate the local offset alignment error introduced during the time-series consistency score calculation process, a second loss L can be added to the loss function. time That is, the first alignment distribution Compared with ideal time distribution Soft Dynamic Time Warping (SoftDTW):

[0148] L time =SoftDTW( , )

[0149] Local offset alignment error refers to a situation where the generated animation and the explanatory text are logically consistent, but there is a phenomenon of "overall translation" or "local misalignment" in certain segments. That is, the order is consistent, but the overall timing is delayed / advanced. For example, the explanatory text appears at 10-15 seconds, but the corresponding content in the animation appears at 12-17 seconds.

[0150] In this case, since KL divergence is more sensitive to "strict frame-by-frame correspondence" and not robust enough to overall translation, only KL divergence is used to calculate S. temporal This may result in a significant penalty. To mitigate this local offset alignment error, the aforementioned second loss L can be introduced into the loss function. time .

[0151] Furthermore, a third loss L can be added to the loss function. overlap The third loss is used to penalize overlapping DOM elements in the rendered H5 animation code generated from a large model. The third loss L overlap It can be represented as:

[0152]

[0153] Among them, B i B j This refers to two DOM elements in the animation. This represents the Intersection over Union (IoU) threshold for DOM elements.

[0154] The overall loss function can be expressed as:

[0155] L total =-R+λ1·L time +λ2·L overlap

[0156] Where λ1 and λ2 are weight parameters.

[0157] By using the loss function described above to train the large model, the quality of the H5 animation code generated by the large model can be improved.

[0158] In some embodiments of this application, an optimization process for large models may also be added.

[0159] Specifically, when the set optimization conditions for the large model are met, the quality score R of the large model for the generated H5 animation code is used as a reinforcement signal to optimize and train the large model, thereby adjusting the strategy preferences and improving the capabilities of the large model.

[0160] By using the quality score R as a reinforcement signal to continuously optimize the large model, the large model can gradually converge towards high-quality H5 animation generation in a closed loop of continuous generation-quality evaluation-optimization.

[0161] This embodiment can employ various optimization strategies:

[0162] The first type is Reinforcement Fine-Tuning (RFT).

[0163] In RFT mode, the large model is treated as a policy network, and its output H5 animation code serves as the action. This application utilizes a quality score R to construct a reward function, and then uses reinforcement learning to fine-tune the large model, increasing the probability that the large model generates high-quality rated H5 animation code.

[0164] The second method is Direct Preference Optimization (DPO), which is based on preference modeling.

[0165] During the training phase, multiple sets of candidate H5 animation codes are generated for the same input animation generation instruction, and a quality score is calculated for each candidate H5 animation code. A preference pair is automatically constructed based on the score. The high-scoring H5 animation code was used as a positive sample. Low-scoring H5 animation code as negative samples This results in a preference dataset. Based on this preference data, a large model is continuously optimized and trained, with the following preference loss:

[0166]

[0167] This method can continuously improve the intrinsic preferences of large models under unsupervised conditions. Without explicit policy gradients, it directly learns the preference relationship that "high-quality code should be better than low-quality code," making the structure of H5 animation code generated by large models more stable and conforming to quality constraints.

[0168] In some embodiments of this application, a self-learning mechanism is further provided.

[0169] The self-learning module aims to build a generation system capable of continuous iterative optimization during long-term operation, enabling the large model to possess "dynamic evolution" capabilities in real-world deployment environments. By collecting multi-source feedback data during system operation, it automatically trains the reward model and the large model, allowing the large model to gradually improve the generation quality of H5 animation code without human intervention.

[0170] Specifically, the self-learning module is mainly responsible for the following tasks:

[0171] 1. The self-learning module quantifies user behavior feedback into training signals to support the optimized training of large models.

[0172] The self-learning module can convert user behavior during use (e.g., satisfaction level, version selection, export, continued modification, dwell time, exit) into "quality preference signals." For example, the same input animation generation command typically generates multiple candidate H5 animation codes. If the user ultimately selects one version, the system considers that version a "high-quality sample," and the unselected versions a "low-quality sample." This allows the system to generate a large amount of comparative data on "high-quality and low-quality samples," which can then guide subsequent training.

[0173] Even if the user does not make an explicit choice, the system can still use some implicit behaviors (such as exporting, saving, playback completion rate, fewer edits, etc.) to infer the user's preferred version, thereby obtaining a training signal.

[0174] Ultimately, these training signals will be used to optimize the large model, making it more likely to generate H5 animation code that satisfies users in the future. The large model optimization training process can use reinforcement learning training algorithms such as RFT or GRPO (Generalized Reward Policy Optimization).

[0175] 2. The self-learning module can continuously update the quality scoring model. The quality scoring model is the formula R mentioned earlier used to calculate the quality of H5 animation code.

[0176] The self-learning module continuously collects paired samples of "high-quality H5 animation code" and "low-quality H5 animation code" and periodically fine-tunes the scoring model.

[0177] The process of fine-tuning and updating the scoring model R based on sample pairs can employ a contrastive learning method. The loss function for contrastive learning is as follows:

[0178]

[0179] Based on the contrastive learning loss function described above, the parameters of the scoring model R are periodically adjusted. Update.

[0180] The collected paired samples of "high-quality H5 animation code" and "low-quality H5 animation code" mainly came from three types of channels:

[0181] (1) Based on the user's selection results among multiple candidate H5 animation code versions generated from the large model;

[0182] (2) Comparison of H5 animation code before and after parameter optimization;

[0183] (3) After the system internally reorders multiple candidate H5 animation codes according to quality scores, the versions with the highest scores and the versions with the lowest scores are compared.

[0184] By continuously training the rating model using these paired samples, the model's ability to judge "which animation is better" can be gradually improved, making the rating more consistent with real quality and closer to user preferences.

[0185] 3. The self-learning module establishes a stable and reliable online update mechanism to ensure that the system's performance continues to improve during iteration and that it has the ability to roll back.

[0186] Through the aforementioned self-learning mechanism, the system can ultimately achieve a self-improvement process similar to that of a reinforcement learning agent, so that the generation effect of large models naturally increases with the number of uses.

[0187] After training, the self-learning module deploys the new scoring model and the main model version to the production environment. To ensure system stability, the self-learning module monitors the average animation score, code optimization success rate, user satisfaction, model prediction stability, and animation layout error rate in real time. If system performance degradation exceeds a preset threshold (e.g., 5%–10%), a rollback mechanism is triggered to restore the scoring model and the main model to the previous version to prevent the degradation from spreading.

[0188] Through a continuous cycle of "data collection, training, deployment, and monitoring," the system achieves long-term adaptive evolution. This mechanism enables the model to continuously learn new preference patterns and layout rules in real user environments, significantly improving the animation generation effect and stability.

[0189] This application also provides an electronic device in its embodiments. (See reference...) Figure 4 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, tablets, large-screen teaching displays, wearable devices, etc. Figure 4The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0190] like Figure 4 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the H5 animation generation and quality evaluation method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0191] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0192] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the H5 animation generation and quality assessment methods provided in this application.

[0193] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the H5 animation generation and quality assessment methods provided in this application.

[0194] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0195] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0196] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0197] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0198] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A method for evaluating the quality of H5 animation generation, characterized in that, include: Obtain the H5 animation code generated by the large model based on the animation generation instructions. The H5 animation code includes one or more explanatory texts. The H5 animation code is executed in the rendering environment to render and generate the animation. During the animation playback process, multiple frames of images are captured and the document object model (DOM) element information corresponding to each frame of image is obtained. Extract the text features of each explanatory text, extract the visual features of each frame image, and extract the layout features of the DOM element information corresponding to each frame image, and map the text features, the visual features, and the layout features to the same alignment space. Based on the mapped text features, visual features, and layout features, an animation quality score is calculated from at least one of the three dimensions: temporal consistency, spatial consistency, and layout rationality. Temporal consistency measures the time synchronization between the animation content and the explanatory text. Spatial consistency measures the consistency between the spatial position of elements within the animation and the semantics of the explanatory text. Layout rationality measures the rationality of the animation screen layout.

2. The method according to claim 1, characterized in that, The process of calculating animation quality scores based on temporal consistency includes: Based on the textual features of each explanatory text segment and the visual features of each image frame, the first alignment distribution of each explanatory text segment and each image frame is determined. Based on the first alignment distribution and the ideal time distribution of the explanatory text and each frame image, the temporal consistency score of the explanatory text and the image is calculated. The ideal time distribution is calculated based on the DOM animation timeline when the H5 animation code is executed.

3. The method according to claim 1, characterized in that, The process of calculating animation quality scores from the spatial consistency dimension includes: Calculate the visual saliency map of each explanatory text segment on each frame of the image; Based on the visual saliency map and the DOM element mask information of each frame image, the spatial alignment score between the visual saliency map of the explanatory text and the DOM element region is calculated.

4. The method according to claim 3, characterized in that, The process of calculating the visual saliency map of the k-th explanatory text in each frame image includes: Using the text features of the k-th explanatory text as the query, cross-attention processing is performed on the visual features of each frame image to obtain attention weights that represent the degree of attention the k-th explanatory text pays to different regions in each frame image. The attention weights are converted into visual saliency maps of the k-th explanatory text in each frame of the image to obtain frame-level visual saliency maps. Using the first alignment distribution of the k-th explanatory text and each frame image as weights, the frame-level visual saliency map is weighted and aggregated across time to obtain the visual saliency map of the k-th explanatory text on each frame image.

5. The method according to claim 1, characterized in that, The process of calculating animation quality scores based on layout rationality includes: The layout features of frame-level visual features and DOM element information are fed into the configured layout rationality evaluation model to obtain the layout rationality score output by the model. The layout rationality evaluation model is trained using training samples labeled with layout rationality score tags. The training samples include frame-level visual feature samples and layout feature samples of DOM element information.

6. The method according to claim 5, characterized in that, The layout rationality score includes scores for at least one of the following four dimensions: Alignment score, overlap penalty score, white space ratio score, rhythm balance score; The alignment score is used to represent the spatiotemporal alignment between the explanatory text segment and the image frame. The overlap penalty score is used to represent the degree of penalty for the overlap of DOM elements within the image frame. The white space ratio score is used to represent the degree to which the white space ratio within the image frame approaches a reasonable range. The rhythm balance score is used to represent the reasonableness of the distribution of element playback events on the timeline in the animation.

7. A method for generating H5 animations, characterized in that, include: Obtain the animation generation instructions input by the user, and call the large model to generate multiple candidate H5 animation codes based on the animation generation instructions; For each candidate H5 animation code, the quality score of each candidate H5 animation code is calculated by using the H5 animation generation quality evaluation method described in any one of claims 1-6. If the highest quality score among all candidate H5 animation codes is not lower than the set quality score threshold, then the candidate H5 animation code with the highest quality score will be used as the final H5 animation code to generate the H5 animation.

8. The method according to claim 7, characterized in that, Also includes: If the highest quality score among all candidate H5 animation codes is lower than the set quality score threshold, then the candidate H5 animation code with the highest quality score is used as the benchmark code, and the parameters to be optimized are determined from the benchmark code. With the goal of improving the quality score of the H5 animation code after parameter optimization, the parameters to be optimized are solved to obtain the H5 animation code after one round of optimization. If, after one or more rounds of optimization, the quality score of the final optimized H5 animation code is not lower than the set quality score threshold, then the final optimized H5 animation code will be used as the final H5 animation code.

9. The method according to claim 7, characterized in that, The large model is obtained through pre-training, and the loss function during the training process includes a first loss, a second loss, and a third loss. The first loss is the inverse of the quality score of the H5 animation code generated by the large model; The second loss is the soft dynamic time warping loss (SoftDTW) between the first alignment distribution and the ideal time distribution. The first alignment distribution is the time alignment distribution between each segment of explanatory text and each frame of the rendered animation in the H5 animation code generated by the large model. The ideal time distribution is calculated based on the DOM animation timeline when the H5 animation code generated by the large model is executed. The third loss is used to penalize the overlap of DOM elements in the animation after the H5 animation code generated by the large model is rendered.

10. The method according to any one of claims 7-9, characterized in that, Also includes: Under the condition of setting large model optimization, the quality score of the large model for the generated H5 animation code is used as a reinforcement signal to optimize and train the large model.

11. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the H5 animation generation quality evaluation method as described in any one of claims 1 to 6, or to implement the steps of the H5 animation generation method as described in any one of claims 7 to 10.

Images