Mapping operator-based automatic generation method of sub-mirror rough sketch

Abstract

The invention relates to the technical field of machine learning, and discloses a mapping operator-based automatic generation method for a sub-mirror rough sketch, which comprises the following steps of: carrying out hierarchical analysis on a sub-mirror script text by adopting a power attention-Transform model; generating a visual composition scheme through a semantic visual mapping operator; wherein an improved Bezier curve method of a four-parameter Bernstein substrate is introduced into an action track, flexible reconstruction and key frame extraction of complex actions are realized, and picture colors, illumination and line styles can be adaptively adjusted by emotional style parameters to ensure that a narrative atmosphere is consistent with a director intention; and finally, rendering into a complete sub-mirror sketch sequence. According to the method, efficient, accurate and uniform-style sub-mirror sketch generation can be realized in film and television, animation and advertisement production, and creation efficiency and picture expressive force are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and particularly relates to a shot script automatic generation method based on a mapping operator. BACKGROUND

[0002] With the increasing requirements of the film and television, animation and digital media industries on production efficiency and quality, more and more intelligent shot script generation systems have been proposed and tried to be applied to actual projects. Such systems can generally automatically generate basic pictures or shot drafts according to input script texts, and to some extent, improve the automation degree of shot creation. However, the existing intelligent systems still have obvious deficiencies in complex scene environments. On the one hand, for scripts containing multi-role interaction, large-scale scene switching or high dynamic action, the generated shot pictures often lack fine portrayal of environmental elements, role relationships and action rhythm, resulting in single picture expression and lack of narrative coherence. On the other hand, in terms of emotional atmosphere or style expression, the existing systems mostly rely on general templates or fixed parameters, and it is difficult to adapt to the lighting, color and rendering requirements under different script contexts, and the generated effect deviates from the creative intention of the director or designer. Therefore, how to realize accurate mapping and controllable expression of complex semantic structures and multi-dimensional environmental factors while maintaining the efficiency of automatic generation has become a key problem to be broken through in the current intelligent shot script generation technology. SUMMARY

[0003] The application provides a shot script automatic generation method based on a mapping operator, aiming at solving the problems of insufficient narrative coherence, rough action depiction and lack of emotional atmosphere expression of existing shot generation systems in complex scenes; the method can accurately capture environmental elements, character relationships and emotional context in multi-character interaction, large scene switching and complex action sequences through hierarchical analysis of the script text by a power attention-Transformer model, and convert the analysis results into executable visual composition parameters such as shot language, character layout, action trajectory and emotional style by using a semantic visual mapping operator, so that the system can automatically generate shot scripts with coherent narrative, realistic action and unified style when facing group action in war films, multi-scene switching in urban dramas or high dynamic action scenes in animation works; wherein the action trajectory reconstruction method based on the four-parameter Bernstein base can realize flexible control of action amplitude and rhythm in action fighting scenes or dance performance scripts, and the generation of emotional style parameters can adaptively adjust color and lighting effects in suspense, romantic or epic themes, ensuring that the picture atmosphere is consistent with the script text; through the above design, the application not only improves the automation level and generation quality of shot production, but also significantly shortens the production cycle, reduces labor input and enhances the consistency of the finished script with the director's intention in application scenarios such as pre-production of film and television projects, animation shot drawing and rapid output of advertising shorts.

[0004] The application provides a shot script automatic generation method based on a mapping operator, characterized by comprising the following steps:

[0005] Step S1: constructing a semantic structure: obtaining a shot script text sequence, constructing a power attention-Transformer model, performing hierarchical analysis on the shot script text sequence by the power attention-Transformer model, and establishing a scene semantic structure; the power attention-Transformer model is constructed in the following way: constructing a Transformer model, introducing power attention to replace the Softmax attention of the Transformer model, and constructing a power attention-Transformer model;

[0006] Step S2: semantic visual mapping: mapping the scene semantic structure into a visual composition scheme by using a semantic visual mapping conversion operator; the semantic visual mapping conversion operator is composed of a shot language parameter generation unit, a character layout parameter generation unit, an action visualization parameter generation unit, an emotional style parameter generation unit and a combined output unit;

[0007] Step S3: visual rendering generation: converting the visual composition scheme into specific shot script frames, and generating a complete shot script sequence.

[0008] Further, step S1 specifically comprises the following steps:

[0009] Step S11: Tokenize and encode the split shot script text sequence, map each token to a word vector representation, and form an initial word vector matrix;

[0010] Step S12: Input the initial word vector matrix into the Transformer model to generate a context feature representation matrix of the split shot script text sequence, the Transformer model including a self-attention sublayer and a feedforward neural network sublayer;

[0011] Step S13: Optimize the attention weight calculation mechanism of the self-attention sublayer and the context dependency modeling capability by power attention, improve the generation effect of the context feature representation matrix, and obtain an enhanced context feature representation matrix;

[0012] Step S14: Perform hierarchical semantic modeling on the split shot script text sequence using the enhanced context feature representation matrix to obtain scene-level representation layer, character-level representation layer, action-level representation layer, and dialogue / emotion-level representation layer, and obtain scene environment, character information, action sequence, and emotion label;

[0013] Step S15: Integrate the scene environment, character information, action sequence, and emotion label to construct a scene semantic structure.

[0014] Further, step S13 specifically comprises the following steps:

[0015] Step S131: Linearly transform the context feature representation matrix to generate Query matrix, Key matrix, and Value matrix; and map Query matrix and Key matrix using power attention mapping function to generate power mapping result; the power attention mapping function is any one of TPOW mapping, SPOW mapping, or TSPOW mapping;

[0016] Step S132: Based on the power mapping result, calculate the power attention weight of the self-attention sublayer and weight the Value matrix to obtain a weighted context representation matrix;

[0017] Step S133: Combine the weighted context representation matrix and the power mapping result to fuse the context information, and output the enhanced context feature representation matrix.

[0018] Further, step S2 specifically comprises the following steps:

[0019] Step S21: The shot language parameter generation unit calculates the shot type and camera angle according to the scene environment in the scene semantic structure, and outputs the shot language parameter;

[0020] Step S22: The role layout parameter generation unit allocates the picture area according to the golden section principle according to the role information in the scene semantic structure, calculates the role position and orientation by using the polar coordinate layout formula, and outputs the role layout parameter;

[0021] Step S23: The action visualization parameter generation unit converts the action in the action sequence into a motion trajectory by using an action trajectory reconstruction method based on a four-parameter Bernstein base, extracts a key action frame, and outputs the action visualization parameter;

[0022] Step S24: The emotion style parameter generation unit outputs the emotion style parameter according to the emotion label in the scene semantic structure.

[0023] Step S25: The combination output unit stores the shot language parameter, the role layout parameter, the action visualization parameter, and the emotion style parameter as a visual composition scheme.

[0024] Further, step S23 specifically includes the following steps:

[0025] Step S231: The action sequence is parsed to generate continuous action instructions, and the continuous action instructions are discretized into action feature points.

[0026] Step S232: According to the action feature points, an action control point set is constructed, a four-parameter Bernstein base function is introduced, a smooth control parameter, an offset control parameter, a scale control parameter, and a curvature control parameter are added in the Bernstein base, the multidimensional adjustment of the curve shape of the Bezier curve is performed, the flexibility and controllability of the curve in the complex action sequence modeling are improved while the geometric characteristics such as the convex hull property and the endpoint interpolation property of the Bezier curve are maintained, the Bezier curve is optimized, an improved Bezier curve is constructed, the action control point set is fitted and interpolated according to the improved Bezier curve, and the continuous trajectory of the action sequence is calculated.

[0027] Step S233: According to the continuous trajectory of the action sequence, a spatial path in the form of an action curve is obtained.

[0028] Step S234: According to the action feature points, the action amplitude is calculated, according to the spatial path in the form of the action curve, the trajectory curvature change rate is calculated, based on the joint discrimination of the action amplitude and the trajectory curvature change rate, the continuous trajectory of the action sequence is differentially sampled to generate a differential motion trajectory, and a key action frame is extracted.

[0029] Step S235: The differential motion trajectory and the key action frame are fused and converted to generate the action visualization parameter.

[0030] By using the above scheme, the application has the following beneficial effects:

[0031] The present application realizes deep analysis of script text and capture of long-distance dependence relationship by introducing a power attention-Transformer model, improves semantic understanding ability in multi-role interaction, large scene switching and complex narrative context, and solves the problem of insufficient scene environment, role relationship and emotional atmosphere description of the existing shot generation method; with the help of the model, the present application can generate a shot script with more coherent narrative and more complete logic, thereby enhancing the accuracy and integrity of the picture in the mapping of script intention.

[0032] The present application realizes automatic conversion from semantic structure to visual composition through a semantic visual mapping operator, effectively improves the parameterization expression ability of shot language, role layout, action trajectory and emotional style, and solves the limitation that the picture generated by the previous system is single and difficult to reflect the director's creative intention; in specific application, the system can automatically generate a shot scheme in accordance with the visual aesthetics and narrative logic of different scene requirements such as war films, urban dramas or animation films, and significantly enhance the practical value of the shot script in scene adaptability and expressiveness.

[0033] The present application realizes further breakthrough in action and style expression: through the action trajectory reconstruction method based on the four-parameter Bernstein base, flexible control and accurate key frame extraction of complex actions are realized, and the trajectory restoration degree and rhythm in high dynamic action scenes such as fighting and dancing are improved; through adaptive generation of emotional style parameters, the lack of flexibility in color, lighting and line expression of existing systems is solved, and the atmosphere building ability of the shot script in suspense, romance or epic scripts is enhanced; the above design not only improves the efficiency and quality of shot generation, but also shortens the preparation period of film, television, animation and advertising production, reduces the labor input cost, and provides a more efficient, reliable and expressive creation tool for creators. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A module schematic diagram of the semantic visual mapping conversion operator proposed in step S2 of embodiment one;

[0035] Figure 2 A dimension change trend chart of the power mapping method proposed in embodiments three, four and five.

[0036] Figure 2 The figure shows the mapping dimension change trend of the three power mapping methods of TPOW, SPOW and TSPOW when the input dimension d=64, tile=8 increases with the power order p, in which the blue dotted line represents the TPOW mapping dimension, the orange square line represents the SPOW mapping dimension, and the green triangular line represents the TSPOW mapping dimension; horizontal axis (X axis): power order p; vertical axis (Y axis): mapping dimension (logarithmic scale). DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0038] In an embodiment, the present application provides a script generation method based on a mapping operator, and the method comprises the following steps: Figure 1 In an embodiment, the present application provides a script generation method based on a mapping operator, and the method comprises the following steps:

[0039] Step S1: constructing a semantic structure, obtaining a sequence of screenplay texts, constructing a power attention-Transformer model, performing hierarchical analysis on the sequence of screenplay texts by using the power attention-Transformer model, extracting a scene environment, a role list, an action sequence and an emotion label, and establishing a scene semantic structure; the power attention-Transformer model is constructed in the following manner: constructing a Transformer model, introducing power attention to replace the Softmax attention of the Transformer model, and constructing a power attention-Transformer model;

[0040] Step S2: semantic visual mapping, mapping the scene semantic structure into a visual composition scheme by using a semantic visual mapping conversion operator, wherein the visual composition scheme comprises shot language parameters, role layout parameters, action visualization parameters and emotion style parameters; the semantic visual mapping conversion operator comprises a shot language parameter generation unit, a role layout parameter generation unit, an action visualization parameter generation unit, an emotion style parameter generation unit and a combined output unit;

[0041] Step S3: visual rendering generation, converting the visual composition scheme into specific screenplay frames, and generating a complete sequence of screenplay frames.

[0042] In an embodiment, the present application provides a script generation method based on a mapping operator, and the method comprises the following steps:

[0043] Step S11: performing word segmentation and coding on the sequence of screenplay texts, mapping each word unit into a word vector representation, and forming an initial word vector matrix;

[0044] Step S12: input the initial word vector matrix into the Transformer model to generate a context feature representation matrix of the shot script text sequence, the Transformer model including a self-attention sublayer and a feedforward neural network sublayer, and realizing layer-by-layer extraction and transmission of multi-layer semantic features under the action of a residual connection and a normalization mechanism;

[0045] Step S13: optimizing the attention weight calculation mechanism and context dependency modeling capability of the self-attention sublayer by power attention to replace the traditional Softmax attention, improving the generation effect of the context feature representation matrix, and obtaining an enhanced context feature representation matrix; the power attention performs polynomial expansion on the similarity of Query and Key through a power mapping operator, avoids the normalization operation of the exponential function, can expand the state representation scale under the condition of a long sequence, improve the capture capability of long-distance dependency, and maintain a high weight-state balance in the weight calculation and state updating process, thereby realizing efficient modeling of long context semantics;

[0046] Step S14: using the enhanced context feature representation matrix to perform hierarchical semantic modeling on the shot script text sequence to obtain a scene-level representation layer, a role-level representation layer, an action-level representation layer, and a dialogue / emotion-level representation layer, and obtain scene environment, role information, action sequence, and emotion label;

[0047] At the scene-level representation layer, the time, place, light, and background elements in the text are analyzed to form the scene environment;

[0048] At the role-level representation layer, the role list and attribute information appearing in the text are identified to obtain the role information;

[0049] At the action-level representation layer, the action sequence of the role is extracted;

[0050] At the dialogue / emotion-level representation layer, the dialogue content and corresponding emotion label of the role are captured in combination with the context;

[0051] Step S15: integrating the scene environment, role information, action sequence, and emotion label to construct a scene semantic structure.

[0052] Embodiment Three, according to Figure 2 , this embodiment is based on Embodiment Two, in this embodiment, step S13 specifically includes the following steps:

[0053] Step S131: linearly transforming the context feature representation matrix to generate a Query matrix, a Key matrix, and a Value matrix; and mapping the Query matrix and the Key matrix by using a power attention mapping function to generate a power mapping result; the power attention mapping function selects a TPOW mapping;

[0054] The power mapping result is a new matrix obtained by processing the Query- Key similarity matrix through the power mapping function, which can change the attention distribution and make the model more biased to highlight strong correlations or promote weakly correlated features.

[0055] TPOW mapping: the mapping is realized by using the Tensor Power of Word (TPOW) method, which is defined as:

[0056] , ;

[0057] wherein, represents the power order, represents the input vector, and the dimension is ; , Query and Key vectors respectively; represents the power mapping function, represents the mapping result obtained by performing the tensor power expansion of the input vector to the order, and the mapping dimension is , which contains all product combination items of the components of the vector in the order space, so as to explicitly model the interaction relationship between high-order features; represents the dimension of the vector space after TPOW mapping; represents the transpose of the Query vector, represents the inner product of Query and Key in the power mapping space; shows that the inner product is calculated in the original space and raised to the power;

[0058] The advantage is that the form is simple and easy to calculate, but there is a problem of high dimension redundancy;

[0059] Step S132: based on the power mapping result, calculate the power attention weight of the self-attention sublayer and weight the Value matrix to obtain the weighted context representation matrix, and the formula is as follows:

[0060] ;

[0061] wherein, represents the weighted context representation matrix obtained by calculating the power similarity of Query and all historical Key and weighting and combining the corresponding Value vector for the th position of the input sequence under the power attention mechanism; represents the A key vector at each position, No. Value vectors at each position; This represents the inner product similarity between the Query and the Key, measuring the similarity of the first and second parts. The position and the first The strength of the correlation between the positions; This represents the weights after applying a power-law mapping to the inner product similarity;

[0062] Step S133: Combine the weighted context representation matrix and the power mapping result to perform context information fusion and output the enhanced context feature representation matrix.

[0063] Step S2 specifically includes the following steps:

[0064] Step S21: The lens language parameter generation unit calculates the lens type and camera angle based on the scene environment in the scene semantic structure, and outputs the lens language parameters. The lens language parameters include lens category (long shot, medium shot, close shot), lens angle (overhead shot, low shot, eye-level shot) and movement mode (push, pull, pan, tilt).

[0065] Step S22: The character layout parameter generation unit allocates the screen area according to the golden ratio principle based on the character information in the scene semantic structure, calculates the character position and orientation using the polar coordinate layout formula, and outputs the character layout parameters, including the character's coordinate position, orientation, and foreground and background layers.

[0066] Step S23: The motion visualization parameter generation unit uses a motion trajectory reconstruction method based on a four-parameter Bernstein basis to convert the motion in the motion sequence into a motion trajectory, then extracts key motion frames, and outputs motion visualization parameters. The motion visualization parameters include the character's motion path, key frame timing, and motion amplitude, which are used to drive the dynamic performance of subsequent storyboard drawings. The construction method of the motion trajectory reconstruction method based on a four-parameter Bernstein basis is as follows: by introducing four types of control parameters—smoothing, offset, scale, and curvature—into the traditional Bernstein basis function, the multi-dimensional controllability model of the motion control point set is performed. While maintaining the convex hull property and endpoint interpolation property of the Bézier curve, the fine reconstruction and flexible adjustment of the motion sequence trajectory are achieved.

[0067] Step S24: The emotional style parameter generation unit outputs emotional style parameters based on the emotional tags in the scene semantic structure. The emotional style parameters include the color tone of the picture, the distribution of light intensity, and the line rendering style.

[0068] Step S25: The combined output unit stores the camera language parameters, character layout parameters, motion visualization parameters, and emotional style parameters as a visual composition scheme.

[0069] Step S23 specifically comprises the following steps:

[0070] Step S231: parsing the action sequence to generate continuous action instructions, discretizing the continuous action instructions into action feature points, the action feature points containing timestamp, spatial position and action amplitude information;

[0071] Step S232: constructing an action control point set according to the action feature points, introducing a four-parameter Bernstein base function, adding a smooth control parameter, an offset control parameter, a scale control parameter and a curvature control parameter in the Bernstein base, adjusting the multidimensionality of the curve shape of the Bézier curve, maintaining the convex hull property and endpoint interpolation property of the Bézier curve, improving the flexibility and controllability of the curve in complex action sequence modeling, optimizing the Bézier curve, constructing an improved Bézier curve, fitting and interpolating the action control point set according to the improved Bézier curve, calculating the continuous trajectory of the action sequence, and the formula is as follows:

[0072] Bézier curve formula of four-parameter Bernstein base:

[0073] , z∈[0,1] ;

[0074] wherein, represents a smooth control parameter (affecting the degree of curve flatness), represents an offset control parameter (controlling the offset of the relative convex hull of the curve), represents a scale control parameter (controlling local stretching or compression), represents a curvature control parameter, adjusting the bending degree of the curve; represents a curve parameter, controlling the progress of the curve, represents the order of the curve; represents an improved Bézier curve based on four parameters; represents the index of the control point, represents the control point, used to determine the geometric position of the curve; represents a four-parameter Bernstein base function, represents a curvature correction function, used to describe the additional adjustment ability of the base function in the direction of curvature change;

[0075] Step S233: obtaining a spatial path in the form of an action curve according to the continuous trajectory of the action sequence, the spatial path in the form of the action curve having stronger shape controllability and robustness compared to the traditional Bézier trajectory;

[0076] Step S234: calculating a motion amplitude according to the action feature points, calculating a trajectory curvature change rate according to a spatial path of a motion curve form, performing differential sampling on a continuous trajectory of the motion sequence based on a joint determination of the motion amplitude and the trajectory curvature change rate, generating a differential motion trajectory, and extracting a key action frame; the key action frame is used to represent main turning points, amplitude peaks, and rhythm nodes of the motion;

[0077] Step S235: fusing and converting the differential motion trajectory and the key action frame to generate a motion visual parameter, which is used for subsequent composition and rendering of a shot script.

[0078] Embodiment Four, according to Figure 2 , this embodiment is based on Embodiment Two, and the difference between this embodiment and Embodiment Three is that the power attention mapping function is selected as SPOW mapping.

[0079] SPOW mapping: the mapping is realized in a manner of symmetric tensor power expansion (Symmetric Power of Word, SPOW), which is defined as:

[0080] , ;

[0081] wherein, represents a mapping result obtained by performing a symmetric tensor power expansion of order n on an input vector x, represents a binomial coefficient, and represents a number of independent monomials when a symmetric polynomial with n variables has a degree of k. represents a dimension of a vector space after SPOW mapping, and the dimension is equal to the binomial coefficient; the mapping greatly reduces the expansion dimension compared with TPOW by eliminating redundant terms caused by different arrangements, and ensures that an inner product calculation in the mapping space is strictly equivalent to a power of n of a dot product in an original space. This manner significantly reduces the dimension of the mapping result by constraining multiple indexes to be in non-decreasing order and eliminating redundant combination terms, thereby avoiding the high redundancy problem of TPOW; the advantage is higher storage and calculation efficiency, and the disadvantage is a more complex structure and slightly less hardware friendliness. Embodiment Five, according to , this embodiment is based on Embodiment Two, and the difference between this embodiment and Embodiment Three is that the power attention mapping function is selected as TSPOW mapping.

[0082]

[0083] Figure 2

[0084] ​​​​​TSPOW mapping: the mapping is implemented in a tiled symmetric power of word (TSPOW) manner, the input vector is divided into a plurality of tiles; the SPOW is used in the tile to maintain the non-decreasing index constraint, and a GPU-friendly parallel structure is used between the tiles to implement:

[0085] , ;

[0086] wherein, represents the mapping result obtained by performing the order n block symmetric tensor power expansion on the input vector , that is, the input vector is divided into a plurality of subblocks, the symmetric tensor power expansion is used in the subblock to remove the permutation redundancy, and a parallel-friendly combination mode is used between the subblocks to expand; represents the dimension size of a single tile (subblock), represents the dimension of the high-dimensional feature vector obtained by the TSPOW mapping;

[0087] This mode takes into account the dimension compression advantage of SPOW and the hardware friendliness of TPOW, while ensuring the calculation equivalence, the execution efficiency on the GPU is improved;

[0088] Embodiments three, four and five all satisfy the power kernel equivalence relationship, but there are differences in the mapping dimension, storage overhead and hardware adaptability: the TPOW form is simple but has high dimension redundancy, the SPOW has strong dimension compression capability but relatively high calculation complexity, and the TSPOW takes into account the dimension compression and GPU parallel execution efficiency while ensuring the mapping equivalence.

[0089] The above describes the present application and its embodiments, which are not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto; in summary, if a person skilled in the art is inspired thereby, without departing from the purpose of the present application, without creative design, similar structure and embodiments of the technical solution are obtained, which should belong to the protection scope of the present application.​

Claims

1. A method for automatically generating a shot script based on a mapping operator, characterized in that: The method comprises the following steps: Step S1: obtaining a sequence of shot script texts, constructing a power attention-Transformer model, and performing hierarchical analysis on the sequence of shot script texts through the power attention-Transformer model to establish a scene semantic structure; Step S2: mapping the scene semantic structure into a visual composition scheme by using a semantic visual mapping conversion operator; The semantic visual mapping conversion operator comprises a shot language parameter generation unit, a role layout parameter generation unit, an action visualization parameter generation unit, an emotional style parameter generation unit and a combination output unit; Step S3: converting the visual composition scheme into a sequence of shot storyboard frames.

2. The method according to claim 1, wherein the method is characterized by: The power attention-Transformer model is constructed in the following manner: a Transformer model is constructed, a power attention is introduced to replace the Softmax attention of the Transformer model, and a power attention-Transformer model is constructed.

3. The method according to claim 2, wherein: Step S1, specifically comprising the following steps: Step S11: performing word segmentation and coding on the sequence of shot script texts to form an initial word vector matrix; Step S12: inputting the initial word vector matrix into the Transformer model to generate a context feature representation matrix of the sequence of shot script texts, wherein the Transformer model comprises a self-attention sublayer; Step S13: optimizing the attention weight calculation mechanism of the self-attention sublayer through power attention to improve the generation effect of the context feature representation matrix, and obtaining an enhanced context feature representation matrix; Step S14: performing hierarchical semantic modeling on the sequence of shot script texts by using the enhanced context feature representation matrix to generate a scene environment, role information, action sequence and emotional label; Step S15: uniformly integrating the scene environment, role information, action sequence and emotional label to construct a scene semantic structure.

4. The method according to claim 3, wherein: Step S13, specifically comprising the following steps: Step S131: performing linear transformation on the context feature representation matrix to generate a Query matrix, a Key matrix and a Value matrix respectively; and mapping the Query matrix and the Key matrix by using a power attention mapping function to generate a power mapping result; Step S132: calculating the power attention weight of the self-attention sublayer based on the power mapping result and weighting the Value matrix to obtain a weighted context representation matrix; Step S133: combining the weighted context representation matrix and the power mapping result to fuse the context information, and outputting an enhanced context feature representation matrix.

5. The method according to claim 4, wherein the mapping operator is defined as: ###0002### where: ###0003### and ###0004### are the first and second derivatives of the mapping operator, respectively, and ###0005### is the second derivative of the mapping operator. The power attention mapping function is any one of a TPOW mapping, a SPOW mapping or a TSPOW mapping.

6. The method according to claim 3, wherein the method is characterized by: Step S2, specifically comprising the following steps: Step S21: the shot language parameter generation unit calculates a shot type and a camera angle according to the scene environment, and outputs a shot language parameter; Step S22: the role layout parameter generation unit outputs a role layout parameter according to the role information; Step S23: The action visual parameter generation unit converts the action in the action sequence into a motion trajectory by using a four-parameter Bernstein base-based action trajectory reconstruction method, extracts key action frames, and outputs action visual parameters; Step S24: The emotion style parameter generation unit outputs emotion style parameters according to emotion labels; Step S25: The combination output unit stores the shot language parameters, the role layout parameters, the action visual parameters, and the emotion style parameters as a visual composition scheme.

7. The method according to claim 6, wherein the method is characterized by: Step S23 specifically includes the following steps: Step S231: The action sequence is parsed to generate continuous action instructions, and the continuous action instructions are discretized into action feature points; Step S232: According to the action feature points, an action control point set is constructed, a four-parameter Bernstein base function is introduced, a Bezier curve is optimized, an improved Bezier curve is constructed, the action control point set is fitted and interpolated according to the improved Bezier curve, and the continuous trajectory of the action sequence is calculated; Step S233: According to the continuous trajectory of the action sequence, a spatial path in the form of an action curve is obtained; Step S234: The action amplitude is calculated according to the action feature points, the trajectory curvature change rate is calculated according to the spatial path in the form of the action curve, the continuous trajectory of the action sequence is differentially sampled based on the joint discrimination of the action amplitude and the trajectory curvature change rate, a differential motion trajectory is generated, and key action frames are extracted; Step S235: The differential motion trajectory and the key action frames are fused and converted to generate action visual parameters.

Images