Intelligent shooting method for scene understanding and script analysis driven by large science and technology movie and television model

Abstract

The invention relates to an intelligent shooting method for scene understanding and script analysis driven by a large science and technology movie and television model, and belongs to the technical field of machine vision. The method comprises the steps that script text features are extracted and decomposed to obtain plot development, emotional fluctuation and artistic style information, a reference film and television work with the maximum overall matching score is selected based on a decomposition result, and an optimal shooting strategy vector is extracted; analyzing the shot scene to generate visual features; fusing the text features and the visual features to obtain a multi-modal semantic representation, and generating a dynamic scene-task knowledge graph according to the optimized multi-modal semantic representation so as to generate a shot scheduling strategy; the shooting process is tracked, the shooting sequence and the lens application mode are monitored in real time, and when it is detected that the shooting sequence deviates, lens connection deviates or visual expression does not conform to expectation, an intelligent optimization mechanism is triggered; and when the deviation exceeds a set threshold value, a manual intervention prompt is given out. The shooting cost can be reduced, and the manufacturing efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology and relates to an intelligent shooting method for scene understanding and script analysis driven by a large model of science and technology films and television. Background Art

[0002] In the field of film and television production, script analysis, scene understanding, and shot scheduling are the core links that determine the artistic quality and production efficiency of a work. The traditional shooting process relies heavily on the experience of creative personnel such as directors, photographers, and editors for scene planning and shot design. This involves manually analyzing the emotional context of the script, designing the shot language, and planning the shooting sequence. This approach has significant limitations: First, human decision-making is easily influenced by subjective factors. In complex narrative structures or multi-line parallel scenes, it is difficult to maintain consistency between shot scheduling and narrative logic, resulting in an imbalance in plot rhythm and a break in visual style. Second, the inheritance of experience is individual and inconsistent in execution standards during team collaboration. Third, when faced with large-scale scene changes or high-density shooting tasks, manual planning is inefficient and has weak fault tolerance, and the confusion in shooting sequence often leads to a surge in rework costs.

[0003] In recent years, artificial intelligence (AI) technologies have been gradually applied to film and television production processes. For example, natural language processing can extract plot keywords and sentiment tags from scripts, while computer vision can identify the attributes and spatial relationships of objects in scenes. However, existing solutions suffer from systemic flaws. For example, text parsing and visual analysis typically operate as independent modules, lacking a deep correlation between script semantics and scene elements. This makes it impossible to map the emotional dynamics and artistic style of a script to the visual elements of a scene, which can easily lead to shooting decisions that deviate from narrative intent. Existing technologies also suffer from insufficient integration of empirical knowledge. The experience gained from shot scheduling in classic film and television works has not been effectively transformed into inferable semantic knowledge, making it difficult to dynamically adapt to the artistic requirements of new scripts. Furthermore, existing intelligent solutions primarily focus on camera mechanical control or post-production editing optimization, lacking a systematic planning and optimization mechanism for aligning shot usage, narrative logic, and the director's intent. Furthermore, they lack a comprehensive monitoring mechanism throughout the entire filming process, making it impossible to monitor the rationality of shot transitions, deviations from the shooting sequence, and consistency of visual style in real time. Furthermore, they lack dynamic correction capabilities.

[0004] Therefore, how to parse the script, deeply understand the scene and combine the script semantics and scene elements, how to draw on the shot scheduling experience of classic film and television works to form a systematic planning scheme, and at the same time be able to optimize the shooting strategy in real time to dynamically correct errors, has become an important challenge for the current intelligent development of film and television production. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an intelligent shooting method for scene understanding and script analysis driven by a large model of science and technology films and television. Through script analysis, scene understanding, knowledge graph modeling and intelligent shooting strategy optimization, and through multimodal perception and adaptive control technology, full-process intelligent shooting planning and execution supervision are realized, effectively improving the intelligence level, execution efficiency and artistic coherence of film and television shooting.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An intelligent filming method for scene understanding and script analysis driven by a large model of science and technology films and television, the method comprising:

[0008] Obtain the script text, extract text features, and decompose them to obtain plot development, emotional fluctuations, and artistic style information. This information is then matched with reference film and television works. The reference film and television work with the highest overall matching score is selected and the optimal shooting strategy vector is extracted to provide a reference basis for intelligent shooting planning. The shooting scene is analyzed to identify key elements, object features, and spatial layout in the shooting scene to generate visual features.

[0009] Cross-modally fuse textual and visual features to compute a multimodal semantic representation of text and scene, and further optimize the multimodal semantic representation based on adaptive feature transformation and semantic alignment networks. Generate a dynamic scene-task knowledge graph based on the optimized multimodal semantic representation, and generate a shot scheduling strategy based on this knowledge graph.

[0010] Track the shooting progress, monitor the shooting order and lens usage in real time, analyze the degree of match between the current shooting scene and the script setting, and trigger the intelligent optimization mechanism when it detects deviations in the shooting order, deviations in the lens connection, or discrepancies between the visual expression and expectations; and issue a manual intervention prompt when the deviation exceeds the set threshold.

[0011] Furthermore, the global text features of the script text are extracted through a pre-trained text encoding model, and the global text features are decomposed through a decoupling function to obtain plot features that characterize plot development, emotional features that characterize emotional fluctuations, and style features that characterize artistic style; based on the plot features, emotional features, and style features, reference film and television work samples are selected from a film and television database for matching, and the reference film and television work with the largest overall matching score is selected by calculating the similarity, and the optimal shooting strategy vector is extracted based on the reference film and television work with the largest similarity.

[0012] Among them, the reference film and television work with the largest overall matching score is selected by calculating the similarity, as shown in the following formula:

[0013]

[0014] Where V C For plot features, V E is the emotional feature, V F is the style feature; R i is the reference film and television work sample in the film and television database, n is the total number of samples; α and β are parameters for adjusting the influence weights of emotional features and style features respectively; is the similarity calculation function.

[0015] Furthermore, a captured scene image I is obtained, and a visual feature extraction model calculates the global visual features of image I. A multi-granular visual entity perception model is used to identify key elements in image I, extract the features and spatial layout of each object in image I, and fuse the global visual features, object features, and spatial layout to generate scene visual features:

[0016]

[0017] Where G is the global visual feature; h u 、h j is the feature of objects u and j in the scene; d uj Represents the spatial relationship value between object u and object j, where U and J are the total number of objects; is the spatial weight function, α is the parameter that controls the steepness of the function, and τ is the threshold value of the relationship significance.

[0018] Furthermore, the text features are cross-modally fused with the visual features to calculate the multimodal semantic representation of the text and the scene:

[0019] L=η·Γ1(V A ,K)+(1-η)·Γ2(V A ,K)

[0020] Where V A is the global text feature extracted from the script text, Γ1(V A ,K) represents the difference dimension between text features and visual features; Γ2(V A ,K) represents the enhancement dimension between text features and visual features; η represents the modality adaptation weight;

[0021] The multimodal semantic representation is further optimized based on adaptive feature conversion and semantic alignment network. The similarity score N between text and scene semantics is first calculated to measure the matching degree between the current scene and the script:

[0022]

[0023] Where, L i is the i-th semantic dimension, μ i is the semantic center of the modal prior; γi is the nonlinear sensitivity control parameter; α i is the semantic attention weight; is the normalization coefficient; when N is lower than the threshold The semantic completion mechanism is triggered when the element is not matched, and the unmatched elements are enhanced to obtain the optimized multimodal semantic representation: Represents the semantic completion mechanism.

[0024] Furthermore, a dynamic scene-task knowledge graph is constructed based on the optimized multimodal semantic representation L′, and the node set of the knowledge graph is represented as X={x1,x2,...,x h}, for each node x i Assign weights:

[0025]

[0026] Where, ρ(x i ,L′) is the node x i and the semantic attraction function between L′; ξ(x i ) is the node x i The narrative tension weight of h is the number of nodes. In the above formula, the numerator represents a node x i The original weight score of ; the denominator represents the sum of the total scores of all nodes, that is, the weighted sum of the comprehensive attraction and tension of all nodes, which is for normalization.

[0027] The node association modeling function is used to calculate the semantic association matrix Z between graph nodes = {z ij}:

[0028]

[0029] Where, ΔS s (x i →x j ) means that under the sth semantic channel, node x i To x j The narrative energy transfer value; Δt ij is the optimal time advancement step between the two; represents the lens coordination entropy under the sth channel; β k 、μ k is the control factor of semantic flow and lens entropy; m is the number of channels;

[0030] Generate a shot scheduling strategy S based on the dynamic scene-task knowledge graph:

[0031]

[0032] Where, is the ideal lens tensor template; T ij Represents the slave node x i to x j The scheduling feature vector of z ij is the semantic association vector between nodes; Represents the overall difference between the current shot scheduling and the ideal strategy; represents a subset of potential shot paths, Represents the set of all candidate shot nodes, represents the shot planning loss function.

[0033] Furthermore, the shooting process is tracked, the shooting sequence and the use of lenses are monitored in real time, and the matching degree between the current shooting scene and the script setting is analyzed. This includes: during the shooting process, the current lens is compared with the script requirements. Suppose the shooting lens sequence is A = {a1, ..., a i ,...,a q}, a q Denote the visual features of the qth shot, and let the expected shot sequence in the script be expressed as E = {e1,...,e i ,...,e q}, e q For the visual features of the qth shot set in the script, calculate the similarity score between the current shot and the script preset shot:

[0034]

[0035] Where, ψ(a i ) is the semantic feature vector of the current shot; The feature representation of the script preset shooting shot; α is the parameter that controls the degree of smoothness; when W is lower than the set threshold It is determined that there is a deviation in the shooting content;

[0036] Assume that the timestamp sequence of the current shot is: t={t1,...,t q}, t q Indicates the timestamp of the qth shot; the theoretically correct time sequence is: t * ={t1 * ,...,t q *}, t q * Represents the theoretical timestamp of the qth shot; calculate the time series deviation:

[0037]

[0038] When Δ t Above the preset threshold δ t, then it is determined that the shooting order is wrong;

[0039] Assume the lens switching matrix is c i,i+1 Represents the switching characteristics from the i-th shot to the i+1-th shot, and the expected shot connection matrix is Calculate the lens switching error E:

[0040]

[0041] Where, || || 2 Indicates calculating the difference between two vectors; when E is higher than the preset threshold θ E , it is determined that the shooting shot switching is not coherent.

[0042] Furthermore, when the similarity score W and the time series deviation Δ t When any of the lens switching errors E exceeds the corresponding threshold, the lens scheduling strategy is readjusted, and the adjustment optimization matrix is ​​set to R = [r i,i+1 ],r i,i+1 Indicates the weight relationship of adjusting the i-th lens to the i+1-th lens:

[0043] R=λ1W+λ2Δ t +λ3E

[0044] Where λ1, λ2, and λ3 are the weighted coefficients of the deviation term;

[0045] The re-adjusted lens scheduling strategy is: S*=R·S, S={s1,s2,...,s q} is the initial shot scheduling strategy, s q is the scheduling strategy for the qth shot, S * ={s1 * ,s2 * ,…,s q *} is the optimized lens scheduling strategy, s q * The qth shot scheduling strategy after optimization adjustment;

[0046] Evaluate the matching degree between the optimized shot scheduling strategy and the script requirements and calculate the multi-dimensional deviation:

[0047] Δ=α1·Δ sem +α2·Δ tim +α3·Δ vis

[0048] Where α1, α2, and α3 are the weighting coefficients of each deviation; Represents the average semantic difference from the script shot sequence, Sim sem represents the semantic similarity measurement function; Indicates the sequential offset of the lens position, Pos indicates the position number of the lens; Represents the coherence error of the visual language between shots, Sim vis represents the visual similarity measurement function; s i ′ represents the i-th shooting shot preset in the script; when the multidimensional deviation Δ exceeds the severe deviation threshold Δ s Prompts manual intervention.

[0049] The beneficial effects of the present invention are:

[0050] 1) Multimodal Intelligent Fusion: This invention combines script analysis, scene understanding, and knowledge graph modeling to achieve deep fusion of multimodal information such as text and vision, so that shooting planning can conform to narrative logic and the director's intention.

[0051] 2) Knowledge graph-driven shooting optimization: This invention constructs a dynamic scene-task knowledge graph to analyze the matching degree between the scene and the script, thereby optimizing the shot scheduling and shooting sequence, and improving the intelligence and adaptability of shooting decisions.

[0052] 3) Intelligent shooting supervision and real-time adjustment: This invention is based on a large-scale model-driven intelligent shooting strategy supervision network, which can monitor the shooting progress in real time, automatically detect problems such as shooting sequence deviation and improper lens connection, and dynamically optimize the shooting strategy to ensure the rationality and consistency of the shooting process.

[0053] 4) Wide Applicability and Efficient Execution: The present invention can be widely applied to various scenarios such as movies, TV series, advertisements, and virtual shooting. It can reduce shooting costs, improve production efficiency, and enhance the artistic consistency and visual expression of film and television works.

[0054] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0056] Figure 1 This is a flowchart of the overall architecture of the intelligent shooting method for scene understanding and script analysis driven by a large model of science and technology films and television provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0058] The purpose of this invention is to propose an intelligent shooting method for science and technology film and television driven by large-scale model scene understanding and script analysis. The method mainly consists of: first, using the script parsing network to extract plot, emotion and style features, and combining classic film and television works to analyze and optimize shooting strategies. At the same time, based on the large-scale model-driven scene understanding network, key elements, object relationships and spatial layout are extracted to generate visual feature vectors; then, a dynamic scene-task knowledge graph is constructed to analyze the deep semantic relationship between the scene and the script, and optimize shot scheduling and task planning; finally, based on the intelligent shooting strategy supervision network, the shooting sequence, shot connection and visual style are monitored and dynamically adjusted in real time to ensure that the shooting process conforms to the narrative logic of the script and the director's intention. Through script parsing, scene understanding, knowledge graph modeling and intelligent shooting strategy optimization, this method realizes intelligent shooting planning and execution supervision throughout the entire process, which can improve the intelligence level, execution efficiency and artistic coherence of film and television shooting, and is widely applicable to the fields of film, TV series, advertising, virtual shooting and so on.

[0059] like Figure 1 As shown, an intelligent filming method for scene understanding and script analysis driven by a large model of science and technology films provided by an embodiment of the present invention is as follows:

[0060] S1: First, a large-scale model-based plot-emotion-style decoupling analysis network is used to extract key information from the script, including plot development, emotional fluctuations, and artistic style. Subsequently, a multi-knowledge semantic mining and reasoning network is used to compare this key information with classic film and television works, analyzing similar situations and corresponding shooting strategies in historical works. This approach draws on the creative experience of classic works and integrates the script's own artistic style to optimize shooting decisions. Based on this extracted information, a text feature vector is generated to capture the script's structure and emotional changes, ultimately deriving a preliminary shooting plan.

[0061] Secondly, through the scene understanding and visual analysis network guided by the large model, the shooting scene is deeply analyzed to identify the key elements, object relationships and spatial layout, and generate the corresponding visual feature vectors.

[0062] 1) For the extraction of text modal features, based on the plot-emotion-style decoupling analysis network driven by a large model, the script text is analyzed for multi-dimensional features to obtain key information such as plot development, emotional fluctuations and artistic style. Let the input script text be T, first use the pre-trained text encoding model Extract global text features V A : In order to analyze the different dimensions of the text, we further adopt the plot-emotion-style decoupling function For the global text feature V A Decompose and get the plot feature V C , emotional characteristics V E and style features V F :

[0063]

[0064] Among them, the plot feature V C Characterizes the plot structure of the script and reflects the narrative logic of the story; emotional characteristics V E Describe the trend of emotional fluctuations and analyze the emotional change pattern of the plot; style characteristics V F Describe the script's language style, rhythm, and visual tendencies.

[0065] 2) Comparison between the script and classic film and television works: In order to optimize the shooting strategy, the multi-knowledge semantic mining reasoning network is used to extract the feature V C 、V E 、V F With the film and television database R={R1,R2,...,R n Reference film and television sample R in} i Match and calculate the similarity score. Suppose the similarity calculation function is Define the overall matching score E i as follows:

[0066]

[0067] Here, θ represents the angle between X and Y in vector space. The closer cos(θ) is to 1, the closer the two vectors are in their directions, and the more similar they are. α and β are parameters that adjust the weights of the influence of sentiment and style features.

[0068] According to the overall matching score E i Select the film and television works with the highest similarity: R F =argmaxE i , and extract the optimal shooting strategy vector from it: in, represents the shooting strategy extraction function, This function extracts shooting strategy knowledge elements from reference film and television works and fuses them to generate vector representations, ω1 c ω2 E ω3 F This is the weight matrix or vector of strategy fusion, representing the importance of each strategy dimension to the final shooting vector. F contains information including, but not limited to, shot usage (e.g., long takes, quick cuts), shooting sequence (scene arrangement), and narrative rhythm (how the plot rhythm is visually presented). This information provides a reference for subsequent intelligent shooting planning.

[0069] 3) Visual modality scene understanding and feature extraction: Film and television shooting relies on the visual features of the scene, such as key elements, object relationships, and spatial layout. To ensure that the shooting plan meets the requirements of the script, the scene understanding and visual analysis network guided by the large model analyzes the scene image, extracts visual information, and generates a visual feature vector for optimizing shooting decisions. Suppose the scene input image I is used, and the visual feature extraction model is used to extract the scene image I. Compute global visual features of an image: Then, a multi-granularity visual entity perception model is used to identify key elements in the scene and extract the feature set and spatial relationship matrix of each object in the scene. Finally, the global visual features, object features and spatial layout are integrated to generate the scene visual features:

[0070]

[0071] Among them, h u is the feature of object u in the scene; d uj Represents the spatial relationship value between object u and object j (common types include but are not limited to: characters, main props, spatial structural elements, environmental background, light source and shadow areas, interactive objects); is a spatial weight function, α (α > 0) controls the steepness of the function, and τ is the threshold for relationship significance. U and J are the total number of objects. K serves as the final visual feature representation, ensuring that the shooting plan conforms to the visual style and scene layout of the script.

[0072] S2. Utilize a large model of cross-modal scene-text understanding and knowledge mapping to extract the semantic associations between script content and shooting scenes, and parse their corresponding relationships. Based on adaptive feature conversion and semantic alignment networks, a unified semantic representation model is further constructed. On this basis, a dynamic scene-task knowledge graph is generated, and through the dynamic knowledge graph temporal reasoning network, corresponding semantic weights are assigned to each knowledge node, mapping it to a specific scene or task. At the same time, the causal relationships, sequential dependencies, and shot scheduling strategies are represented through the associated edges between nodes. This knowledge graph can dynamically adapt to changes in script content and continuously optimize scene scheduling based on historical shooting experience, thereby improving execution efficiency and shooting quality.

[0073] 1) Scene-text semantic association extraction: First, using the cross-modal understanding network, from the global text feature V A The deep semantic association between the scene and the script is extracted from the visual feature K. Specifically, a unified representation of the text and the scene can be calculated through a cross-modal feature fusion model:

[0074] L=η·Γ1(V A ,K)+(1-η)·Γ2(V A ,K) (4)

[0075] Among them, Γ1(V A ,K) represents the difference dimension between text features and visual features; Γ2(V A ,K) represents the enhancement dimension between text features and visual features; η∈(0,1) represents the modality adaptation weight, which is controlled and adjusted by the large model. L represents the fused multimodal semantic representation, which includes key elements such as scene description, character action, and emotional information, and is used for subsequent knowledge graph construction.

[0076] 2) Semantic Representation Model Construction: To describe the scene-text relationship, the fused multimodal semantic representation is further optimized based on adaptive feature conversion and semantic alignment networks. The similarity score N between the text and scene semantics is calculated to measure the degree of match between the current scene and the script:

[0077]

[0078] Where L=[L1,L2,...,L k ] is the fused multimodal semantic vector, k represents the vector dimension, μ i The semantic center of the modal prior (obtained through training statistics and model dynamic estimation, which guides the system to preserve the stability and continuity of modal features when constructing multimodal semantic representation); γ i (γ i >1) is a nonlinear sensitivity control parameter used to emphasize semantic dimension differences (such as amplifying key semantic dimensions); α i is the semantic attention weight, reflecting the importance of the i-th dimension in matching; is the normalization coefficient used to keep the output range in [0,1].

[0079] When N is lower than the threshold The semantic completion mechanism is triggered when Unmatched elements are enhanced to improve semantic consistency and generate optimized multimodal semantic representations:

[0080] 3) Dynamic scene-task knowledge graph generation: Based on the optimized fusion multimodal semantic representation L′, a dynamic scene-task knowledge graph is constructed. The nodes in the knowledge graph represent tasks or scenes, and the edges represent causal relationships, sequential dependencies, and shot scheduling strategies. Let the knowledge graph node set be X = {x1, x2, ..., x h}, weights are assigned to each node through a node weight assignment function based on semantic core appeal and narrative tension:

[0081]

[0082] Among them, ρ(x i ,L′) is the node x i and the semantic attraction function between the semantic representation L′; ξx i ) is the node x i The narrative tension weight, which is modeled based on factors such as plot twists and character conflicts; Y i is the node weight, which represents the importance of different task nodes, such as key plot points, character interaction events, etc., to ensure that the shot scheduling conforms to the narrative logic; h is the number of nodes.

[0083] At the same time, the system uses the node association modeling function to calculate the semantic association matrix Z between the graph nodes = {z ij}, where nodes include tasks and scenes. The correlation matrix reflects the temporal dependencies, causal chains, and shot scheduling strategies between tasks, between scenes, and between tasks and scenes:

[0084]

[0085] Where, ΔS s (x i →x j ) means that under the sth semantic channel, node x i To x j The narrative energy transfer value; Δt ij is the optimal time advancement step between two nodes (which can be obtained by script temporal reasoning); represents the lens coordination entropy under the s-th channel (i.e., the uncertainty of lens conversion); β s 、μ s is the control factor of semantic flow and lens entropy. ij It reflects the temporal dependency, causal chain and shot scheduling rules between nodes to ensure that the shooting order conforms to the narrative rhythm.

[0086] 4) Temporal reasoning and shot scheduling optimization: Based on the constructed dynamic scene-task knowledge graph, the dynamic knowledge graph temporal reasoning network is used to calculate the shot scheduling strategy S:

[0087]

[0088] in, is the ideal lens tensor template, generated by the director's intention, lens style template or user preset; T ij Represents the slave node x i to x j During the process, the comprehensive scheduling feature vector in terms of time rhythm, camera angle and emotional changes is used to describe the narrative coherence and scheduling logic of the transition between shots; i is the node weight; z ij is the semantic association vector between nodes; Represents the overall difference between the current shot scheduling and the ideal strategy, used for global optimization. represents a subset of potential shot paths, Represents the set of all candidate shot nodes, Denotes the shot planning loss function. The calculated shot scheduling strategy S reflects the final shooting plan, including shot switching methods, shooting order, and shot angles, ensuring that the shooting strategy meets the director's intentions and is used for intelligent shooting planning and execution.

[0089] S3: Based on a multimodal scene-task knowledge graph and script parsing results, a large-scale model-driven intelligent shooting strategy supervision network provides real-time supervision of the shooting sequence and lens usage, ensuring that the shooting process adheres to the script's narrative logic, visual style, and directorial intent. Combining dynamic scene understanding and temporal consistency analysis, it continuously tracks shooting progress, determines whether the current shots are being executed according to the predetermined sequence, and evaluates whether the shot transitions meet the script's plot rhythm and visual language requirements. During the shooting process, multimodal semantic reasoning and adaptive adjustment mechanisms dynamically analyze the degree of match between the current scene and the script settings, ensuring the expression of key scenes, the smoothness of shot transitions, and the consistency of the visual style. Intelligent optimization mechanisms are triggered when deviations in the shooting sequence, unreasonable shot arrangements, or discrepancies between visual expressions and expectations are detected. Manual intervention prompts are issued when deviations exceed set thresholds to ensure the rationality, consistency, and artistic consistency of the shooting process.

[0090] 1) During the shooting process, compare the current shot with the script requirements to ensure that the sequence and connection are in line with the established plan. Let the shooting shot sequence be A = {a1,...,a i ,...,a q}, where a q Denote the visual features of the qth shot, and let the expected shot sequence in the script be expressed as E = {e1,...,e i ,...,e q}, where e qThe visual features of the qth shot set in the script. Calculate the similarity score between the current shot and the script preset shot:

[0091]

[0092] Among them, there are q shooting shots and q preset shots, and there are q matching pairs, such as the total number of shots to shots and scenes to tasks; ψ(a i ) is the semantic feature vector of the current shot (such as subject action, composition, emotion); The feature representation of the preset shooting shots in the script; α(α∈(0,2)) is used to control the degree of smoothness. When W is lower than the set threshold This indicates that there is a deviation in the shooting content, triggering the optimization mechanism.

[0093] 2) During the shooting process, it is necessary not only to match the script settings, but also to ensure the rationality of the time sequence and the continuity of the shot switching. Suppose the timestamp sequence of the current shooting is: t={t1,...,t q}, where t i Indicates the timestamp of the i-th shot; the theoretically correct time sequence t * Given by the script plan: t * ={t1 * ,...,t q *}. Calculate the time series deviation:

[0094]

[0095] Among them, Δ t It is the time sequence deviation, which is used to evaluate whether the actual shooting time sequence is consistent with the time sequence set in the script. It means averaging the deviations of all adjacent shots, making the indicator globally representative. t Above the preset threshold δ t , it means that the shooting order is wrong and needs to be adjusted.

[0096] 3) Shot transitions not only involve time sequence, but also visual language and artistic style. Let the shot switching matrix be c i,i+1 Represents the switching characteristics from the i-th shot to the i+1-th shot, and the expected shot connection matrix is Determined by the director's intention and the script style. Calculate the shot switching error E:

[0097]

[0098] Among them, ∑ represents the traversal of all switching pairs between different shots; || || 2Used to measure the difference between two vectors. When E is higher than the preset threshold θ E , it means that the switching is not coherent and the shot connection needs to be readjusted.

[0099] 4) When problems are detected in steps 1) to 3) (shooting sequence or lens connection), the intelligent optimization mechanism needs to be triggered to calculate a new lens scheduling strategy. Let the adjustment optimization matrix be R = [r i,i+1 ], indicating the lens adjustment operation, where r i,i+1 It represents the weight or rearrangement relationship of adjusting the i-th lens to the i+1-th lens. The optimization matrix is ​​calculated by combining multiple deviation functions:

[0100] R=λ1·W+λ2·Δ t +λ3·E (12)

[0101] Among them, λ1, λ2, and λ3 are weighted coefficients of the deviation term, satisfying λ1+λ2+λ3=1.

[0102] Finally, the updated shot scheduling strategy is:

[0103] S * =R·S (13)

[0104] Where S={s1,s2,...,s q} is the initial shot scheduling strategy, s q is the scheduling strategy for the qth shot, S * ={s1 * ,s2 * ,…,s q *} is the optimized lens scheduling strategy, s q * The optimized adjustment is for the qth shot scheduling strategy. After completing the shot adjustment, evaluate the matching degree between the optimized shot scheduling strategy and the script requirements and calculate the multi-dimensional deviation:

[0105] Δ=α1·Δ sem +α2·Δ tim +α3·Δ vis (14)

[0106] Among them, α1+α2+α3=1, which is the weighting coefficient of each deviation; Represents the average semantic difference from the script shot sequence, Sim sem represents the semantic similarity measurement function; Indicates the sequential offset of the lens position, Pos indicates the position number of the lens; Represents the coherence error of the visual language between shots, Sim visrepresents the visual similarity measurement function, s i ′ represents the i-th shot preset in the script. When the multidimensional deviation Δ exceeds the severe deviation threshold Δ s Prompts for manual intervention in real time to ensure that the final shooting process conforms to the narrative logic and artistic style of the script.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. An intelligent filming method for scene understanding and script analysis driven by a large model of science and technology films, characterized by: The method includes: Obtain the script text, extract text features, and decompose them to obtain plot development, emotional fluctuations, and artistic style information. This information is then matched with reference film and television works. The reference film and television work with the highest overall matching score is selected and the optimal shooting strategy vector is extracted to provide a reference basis for intelligent shooting planning. The shooting scene is analyzed to identify key elements, object features, and spatial layout in the shooting scene to generate visual features. Cross-modally fuse textual and visual features to compute a multimodal semantic representation of text and scene, and further optimize the multimodal semantic representation based on adaptive feature transformation and semantic alignment networks. Generate a dynamic scene-task knowledge graph based on the optimized multimodal semantic representation, and generate a shot scheduling strategy based on this knowledge graph. Track the shooting progress, monitor the shooting order and lens usage in real time, analyze the degree of match between the current shooting scene and the script setting, and trigger the intelligent optimization mechanism when it detects deviations in the shooting order, deviations in the lens connection, or discrepancies between the visual expression and expectations; and issue a manual intervention prompt when the deviation exceeds the set threshold.

2. The method according to claim 1, characterized in that The global text features of the script text are extracted through a pre-trained text encoding model, and the global text features are decomposed through a decoupling function to obtain plot features that characterize plot development, emotional features that characterize emotional fluctuations, and style features that characterize artistic style; based on the plot features, emotional features, and style features, reference film and television work samples are selected from a film and television database for matching, and the reference film and television work with the largest overall matching score is selected by calculating the similarity, and the optimal shooting strategy vector is extracted based on the reference film and television work with the largest similarity.

3. The method according to claim 2, characterized in that The reference film and television work with the largest overall matching score is selected by calculating the similarity, which is calculated using the following formula: Where V C For plot features, V E is the emotional feature, V F is the style feature; R i is the reference film and television work sample in the film and television database, n is the total number of samples; α and β are parameters for adjusting the influence weights of emotional features and style features respectively; is the similarity calculation function.

4. The method according to claim 2, characterized in that Obtain a captured scene image I, and use a visual feature extraction model to calculate the global visual features of the image I; A multi-granularity visual entity perception model is used to identify key elements in image I and extract the features and spatial layout of each object in image I; Fusion of global visual features, object features, and spatial layout to generate scene visual features: Where G is the global visual feature; h u 、h j is the feature of objects u and j in the scene; d uj Represents the spatial relationship value between object u and object j, where U and J are the total number of objects; is the spatial weight function, α is the parameter that controls the steepness of the function, and τ is the threshold value of the relationship significance.

5. The method according to claim 4, characterized in that Cross-modal fusion of textual features and visual features to calculate multimodal semantic representations of text and scenes: L=η·Γ1(V A ,K)+(1-η)·Γ2(V A ,K) Where V A is the global text feature extracted from the script text, Γ1(V A ,K) represents the difference dimension between text features and visual features; Γ2(V A ,K) represents the enhancement dimension between text features and visual features; η represents the modality adaptation weight; The multimodal semantic representation is further optimized based on adaptive feature conversion and semantic alignment network. The similarity score N between text and scene semantics is first calculated to measure the matching degree between the current scene and the script: Where, L i is the i-th semantic dimension, μ i is the semantic center of the modal prior; γ i is the nonlinear sensitivity control parameter; α i is the semantic attention weight; is the normalization coefficient; when N is lower than the threshold The semantic completion mechanism is triggered when the element is not matched, and the unmatched elements are enhanced to obtain the optimized multimodal semantic representation: Represents the semantic completion mechanism.

6. The method according to claim 5, characterized in that Based on the optimized multimodal semantic representation L′, a dynamic scene-task knowledge graph is constructed. The node set of the knowledge graph is represented as X = {x1, x2, ..., x h }, for each node x i Assign weights: Where, ρ(x i ,L′) is the node x i and the semantic attraction function between L′; ξ(x i ) is the node x i The weight of narrative tension; h is the number of nodes; The node association modeling function is used to calculate the semantic association matrix Z between graph nodes = {z ij }: Where, ΔS s (x i →x j ) means that under the sth semantic channel, node x i To x j The narrative energy transfer value; Δt ij is the optimal time advancement step between two nodes; represents the lens coordination entropy under the s-th channel; β k 、μ k is the control factor of semantic flow and lens entropy; m is the number of channels; Generate a shot scheduling strategy S based on the dynamic scene-task knowledge graph: Where, is the ideal lens tensor template; T ij Represents the slave node x i to x j The scheduling feature vector of z ij is the semantic association vector between nodes; Represents the overall difference between the current shot scheduling and the ideal strategy; represents a subset of potential shot paths, Represents the set of all candidate shot nodes, represents the shot planning loss function.

7. The method according to claim 6, characterized in that Track the shooting process, monitor the shooting sequence and lens usage in real time, and analyze the matching degree between the current shooting scene and the script setting, including: During the shooting process, compare the current lens with the script requirements. Suppose the shooting lens sequence is A={a1,...,a i ,...,a q }, a q Denote the visual features of the qth shot, and let the expected shot sequence in the script be expressed as E = {e1,...,e i ,...,e q }, e q For the visual features of the qth shot set in the script, calculate the similarity score between the current shot and the script preset shot: Where, ψ(a i ) is the semantic feature vector of the current shot; The feature representation of the script preset shooting shot; α is the parameter that controls the degree of smoothness; when W is lower than the set threshold It is determined that there is a deviation in the shooting content; Assume that the timestamp sequence of the current shot is: t={t1,...,t q }, t q Indicates the timestamp of the qth shot; the theoretically correct time sequence is: t * ={t1 * ,...,t q * }, t q * Represents the theoretical timestamp of the qth shot; calculate the time series deviation: When Δ t Above the preset threshold δ t , then it is determined that the shooting order is wrong; Assume the lens switching matrix is c i,i+1 Represents the switching characteristics from the i-th shot to the i+1-th shot, and the expected shot connection matrix is Calculate the lens switching error E: Where, || || 2 Indicates calculating the difference between two vectors; when E is higher than the preset threshold θ E , it is determined that the shooting shot switching is not coherent.

8. The method according to claim 7, characterized in that When the similarity score W and time series deviation Δ t When any of the lens switching errors E exceeds the corresponding threshold, the lens scheduling strategy is readjusted, and the adjustment optimization matrix is ​​set to R = [r i,i+1 ],r i,i+1 Indicates the weight relationship of adjusting the i-th lens to the i+1-th lens: R=λ1·W+λ2·Δ t +λ3·E Where λ1, λ2, and λ3 are the weighted coefficients of the deviation term; The re-adjusted lens scheduling strategy is: S * =R·S, S={s1,s2,...,s q } is the initial shot scheduling strategy, s q is the scheduling strategy for the qth shot, S * ={s1 * ,s2 * ,...,s q * } is the lens scheduling strategy after optimization adjustment, s q * The qth shot scheduling strategy after optimization adjustment; Evaluate the matching degree between the optimized shot scheduling strategy and the script requirements and calculate the multi-dimensional deviation: Δ=α1·Δ sem +α2·D tim +α3·D vis Where α1, α2, and α3 are the weighting coefficients of each deviation; Represents the average semantic difference from the script shot sequence, Sim sem represents the semantic similarity measurement function; Indicates the sequential offset of the lens position, Pos indicates the position number of the lens; Represents the coherence error of the visual language between shots, Sim vis represents the visual similarity measurement function; s i ′ represents the i-th shooting shot preset in the script; When the multidimensional deviation Δ exceeds the severe deviation threshold Δ s Prompts manual intervention.

Images