Multi-modal interactive creation system based on film and television knowledge graph

Abstract

The invention discloses a multi-mode interactive creation system based on a movie and television knowledge graph, and particularly relates to the technical field of multi-mode interactive creation systems, which comprises an AI analysis module, a LoRA fine tuning module, a sub-mirror generation module, a multi-mode conversion module, a user interaction module, a data storage module and a security authentication module. According to the multi-mode interactive creation system based on the movie and television knowledge graph, an AI model is finely tuned through an over-low rank adaptation LoRA, the GPU video memory occupation is reduced from 24 GB to 8 GB, the amplitude is reduced by 66.7%, meanwhile, a single machine is supported to process 100 thousand concurrent WebSocket connection epoll I / O multiplexing, the resource utilization rate is remarkably increased, the real-time processing capacity can be achieved, 100 thousand files are processed every day, the average 10 MB / file is processed, the transmission speed reaches 1 MB / s, and the real-time processing capacity is achieved. The split mirror generation response time is less than or equal to 2 seconds, and the efficient creation requirement of the movie and television industry is met.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal interactive creation systems, and in particular to a multimodal interactive creation system based on a film and television knowledge graph. Background Art

[0002] The film and television industry has long faced challenges with low storyboard production efficiency and insufficient multimodal data integration. Traditional storyboard production relies on manual drawing and scene design, which is time-consuming and labor-intensive, and hinders rapid iteration. Existing AI-assisted tools are often limited to single-modality processing, such as text or image generation, and lack the multimodal association capabilities built on knowledge graphs. This results in a disconnect between the generated content and the emotional logic of the script. Furthermore, existing systems commonly suffer from the following issues: large-scale AI model fine-tuning consumes significant GPU memory, typically exceeding 24GB, limiting lightweight deployment; data synchronization latency during cross-platform collaboration is high, often exceeding 500 milliseconds, and version management relies on manual record-keeping; sensitive script data lacks dynamic encryption and sharded storage mechanisms, posing a risk of leakage; and the generated content suffers from rigid style, preventing flexible adjustment of shot switching frequency and composition ratios based on the director's preferences. While some research has attempted to incorporate knowledge graphs to enhance semantic understanding, the combined drive of emotional heatmaps and character relationship graphs has not been achieved. For example, the underapplication of BERT self-attention weight visualization and PageRank weighted directed graphs results in storyboard generation lacking dynamic emotional expression and character interaction logic. Summary of the Invention

[0003] The main purpose of the present invention is to provide a multimodal interactive creation system based on film and television knowledge graphs, which can effectively solve the problems of high data synchronization delay during cross-platform collaboration and insufficient application of BERT self-attention weight visualization and PageRank weighted directed graphs, resulting in the lack of dynamic emotional expression and character interaction logic in storyboard generation.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] A multimodal interactive creation system based on film and television knowledge graph, including:

[0006] AI parsing module, LoRA fine-tuning module, storyboard generation module, multimodal conversion module, user interaction module, data storage module and security authentication module, including:

[0007] The AI ​​parsing module is used to analyze the emotional characteristics, character relationships and scene logic in the text script through a deep learning model;

[0008] The LoRA fine-tuning module uses low-rank adaptation technology to perform lightweight parameter adjustments on the AI ​​model, supporting multimodal applications across text and image generation.

[0009] The storyboard generation module automatically generates a storyboard script table and visualization content including shot switching, composition and timeline based on the analysis results;

[0010] The multimodal conversion module realizes style transfer and theme adaptation from text to image;

[0011] The user interaction module supports manual intervention and adjustment throughout the entire process, and provides a cross-platform operation interface for mobile phones and computers;

[0012] The data storage module uses a MongoDB database to store user scripts, storyboards, and historical modification records;

[0013] The security authentication module uses a JWT token based on the HMAC-SHA256 algorithm to implement user identity authentication and API access control, where the token signing key is 256 bits long and valid for 1 hour.

[0014] Preferably, the AI ​​parsing module includes:

[0015] Sentiment analysis submodule: This module uses the BERT model to extract the emotional intensity and changing trends of the script text, calculates the emotional weight of each paragraph through the self-attention mechanism, and generates an emotional heat map;

[0016] Scene segmentation submodule: This module divides shot boundaries based on a semantic relevance threshold, where the semantic relevance threshold is cosine similarity ≥ 0.85, and combines a time window sliding algorithm to identify scene transition nodes.

[0017] Role relationship graph submodule: After constructing the role co-occurrence matrix, the PageRank algorithm is used to calculate the role interaction weights and generate a weighted directed relationship graph.

[0018] Preferably, the LoRA fine-tuning module includes:

[0019] Parameter rank optimization submodule: Decompose the fully connected layer fine-tuning parameter matrix into two low-rank matrices, where the matrix rank is set to 8, satisfying that the parameter matrix W is equal to the initial matrix W0 plus the low-rank matrix product BA, where the dimension of matrix B is the input dimension d multiplied by the rank r, and the dimension of matrix A is the rank r multiplied by the output dimension k;

[0020] Style transfer submodule: By adjusting the LoRA weight matrices A and B, the director's style feature vector is injected to control the shot switching frequency and composition preferences of the generated storyboards;

[0021] Cross-modal adaptation submodule: maps the text feature vector to the latent space U-Net layer of the StableDiffusion model via LoRA to ensure the consistency of image generation and text theme.

[0022] Preferably, the storyboard generation module includes:

[0023] Table Generator submodule: Outputs storyboards using commercial specifications, including incremental shot IDs, durations accurate to 0.1 second, medium and full shots for close-ups, and push-pull and pan / tilt camera movements.

[0024] Visual rendering submodule: uses Matplotlib to generate dynamic previews of storyboards and supports interactive timeline dragging, and integrates the Plotly3D engine to render spatial motion trajectories;

[0025] Double review sub-module: Compares the AI-generated version with the manually revised version through the Diff algorithm, highlights the differences and generates a modification suggestion report.

[0026] Preferably, the multimodal conversion module includes:

[0027] Text-to-image generation submodule: Based on the StableDiffusion 2.1 model, it combines LoRA weight fine-tuning to generate scene sketches. Input prompts include shot type and scene, atmosphere and emotional labels, and director style dynamic parameters.

[0028] Style consistency controller submodule: uses the CLIP model to calculate the similarity score between the generated image and the script text. If the score is lower than the threshold of 0.7, the LoRA weight adjustment is triggered;

[0029] Real-time feedback submodule: allows users to adjust the hue, saturation, and brightness parameters of the HSV color space and the nine-square grid composition ratio through sliders.

[0030] Preferably, the user interaction module includes:

[0031] Full-process intervention sub-module: supports inserting manual modification instructions in the five stages of storyboard generation, including emotion analysis, scene segmentation, storyboard generation, visual rendering, and output review;

[0032] Multi-terminal synchronization submodule: synchronizes mobile and computer operations through WebSocket persistent connections, with a delay of ≤ 200 milliseconds, and data packets are encapsulated using the MsgPack binary protocol;

[0033] Version management submodule: records each modification based on the Git mechanism and generates a difference report, supporting rollback to any historical version.

[0034] Preferably, the security authentication module includes:

[0035] Token refresh submodule: automatically refreshes the token through the OAuth2.0 protocol 30 minutes before the JWT expires, and the refresh token validity period is set to 7 days;

[0036] Permission classification submodule: defines user roles as guest editor and administrator. The editor role can only modify the storyboard content, while the administrator role can export and delete projects.

[0037] Transmission encryption submodule: AES-256-GCM is used to encrypt sensitive data such as undisclosed scripts. The initialization vector length is 12 bytes and the authentication tag length is 16 bytes.

[0038] Preferably, the performance optimization module includes:

[0039] Real-time communication submodule: maintains 100,000 concurrent connections through WebSocket and uses the epoll I / O multiplexing mechanism to achieve a single-machine throughput of no less than 10,000 connections per second;

[0040] Resource scheduling submodule: Dynamically allocates GPU computing power based on Docker Swarm, prioritizing storyboard generation tasks over visualization rendering and data backup.

[0041] Disaster recovery submodule: 100,000 files are stored in a remote active-active dual-location system every day. The CRUSH algorithm is used to shard and store the data in at least three availability zones, with three copies.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention uses a low-rank adaptive LoRA fine-tuning AI model to reduce GPU memory usage from 24GB to 8GB, a decrease of 66.7%. At the same time, it supports single-machine processing of 100,000 concurrent WebSocket connections and epoll I / O multiplexing, significantly improving resource utilization. It also has real-time processing capabilities: 100,000 files are processed daily at an average of 10MB / file, with a transmission speed of 1MB / s and a storyboard generation response time of ≤2 seconds, meeting the efficient creation needs of the film and television industry.

[0044] 2. This invention integrates BERT sentiment analysis, PageRank role relationship graph and StableDiffusion multimodal conversion to realize the automatic generation from text to storyboard scripts (including professional tables + visual content), avoiding templated output. At the same time, it adopts flexible style adaptation and injects director style parameters (such as shot switching frequency and composition ratio) through LoRA. The similarity score between the generated content and the director's style is ≥0.7 (CLIP model verification), which enhances the uniqueness of the creation.

[0045] 3. This invention adopts JWT token (HMAC-SHA256 encryption) and AES-256-GCM transmission encryption, supports 7-day automatic refresh token, sharded storage of sensitive data (CRUSH algorithm, 3 availability zones + 3 replicas), RTO ≤ 5 minutes, RPO = 0, and supports real-time synchronization of mobile phones / computers (WebSocket delay ≤ 200ms) through cross-platform collaboration. Git-based version management enables multi-person collaborative editing (conflict intelligent merge threshold ≤ 5%), thereby improving team collaboration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0047] Figure 2 A schematic diagram of the three-layer architecture and data flow of the system of the present invention; DETAILED DESCRIPTION

[0048] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0049] like Figure 1 As shown in the figure, a multimodal interactive creation system based on film and television knowledge graph includes:

[0050] AI parsing module, LoRA fine-tuning module, storyboard generation module, multimodal conversion module, user interaction module, data storage module and security authentication module, including:

[0051] The AI ​​parsing module is used to analyze the emotional characteristics, character relationships and scene logic in the text script through a deep learning model;

[0052] The LoRA fine-tuning module uses low-rank adaptation technology to perform lightweight parameter adjustments on the AI ​​model, supporting multimodal applications across text and image generation.

[0053] The storyboard generation module automatically generates a storyboard script table and visualization content including shot switching, composition and timeline based on the analysis results;

[0054] The multimodal conversion module realizes style transfer and theme adaptation from text to image;

[0055] The user interaction module supports manual intervention and adjustment throughout the entire process, and provides a cross-platform operation interface for mobile phones and computers;

[0056] The data storage module uses a MongoDB database to store user scripts, storyboards, and historical modification records;

[0057] The security authentication module uses a JWT token based on the HMAC-SHA256 algorithm to implement user identity authentication and API access control, where the token signing key is 256 bits long and valid for 1 hour.

[0058] Furthermore, the AI ​​parsing module includes:

[0059] Sentiment analysis submodule: This module uses the BERT model to extract the emotional intensity and changing trends of the script text, calculates the emotional weight of each paragraph through the self-attention mechanism, and generates an emotional heat map;

[0060] Scene segmentation submodule: This module divides shot boundaries based on a semantic relevance threshold, where the semantic relevance threshold is cosine similarity ≥ 0.85, and combines a time window sliding algorithm to identify scene transition nodes.

[0061] Role relationship graph submodule: After constructing the role co-occurrence matrix, the PageRank algorithm is used to calculate the role interaction weights and generate a weighted directed relationship graph.

[0062] Furthermore, the LoRA fine-tuning module includes:

[0063] Parameter rank optimization submodule: Decompose the fully connected layer fine-tuning parameter matrix into two low-rank matrices, where the matrix rank is set to 8, satisfying that the parameter matrix W is equal to the initial matrix W0 plus the low-rank matrix product BA, where the dimension of matrix B is the input dimension d multiplied by the rank r, and the dimension of matrix A is the rank r multiplied by the output dimension k;

[0064] Style transfer submodule: By adjusting the LoRA weight matrices A and B, the director's style feature vector is injected to control the shot switching frequency and composition preferences of the generated storyboards;

[0065] Cross-modal adaptation submodule: maps the text feature vector to the latent space U-Net layer of the StableDiffusion model via LoRA to ensure the consistency of image generation and text theme.

[0066] Furthermore, the storyboard generation module includes:

[0067] Table Generator submodule: Outputs storyboards using commercial specifications, including incremental shot IDs, durations accurate to 0.1 second, medium and full shots for close-ups, and push-pull and pan / tilt camera movements.

[0068] Visual rendering submodule: uses Matplotlib to generate dynamic previews of storyboards and supports interactive timeline dragging, and integrates the Plotly3D engine to render spatial motion trajectories;

[0069] Double review sub-module: Compares the AI-generated version with the manually revised version through the Diff algorithm, highlights the differences and generates a modification suggestion report.

[0070] Furthermore, the multimodal conversion module includes:

[0071] Text-to-image generation submodule: Based on the StableDiffusion 2.1 model, it combines LoRA weight fine-tuning to generate scene sketches. Input prompts include shot type and scene, atmosphere and emotional labels, and director style dynamic parameters.

[0072] Style consistency controller submodule: uses the CLIP model to calculate the similarity score between the generated image and the script text. If the score is lower than the threshold of 0.7, the LoRA weight adjustment is triggered;

[0073] Real-time feedback submodule: allows users to adjust the hue, saturation, and brightness parameters of the HSV color space and the nine-square grid composition ratio through sliders.

[0074] Furthermore, the user interaction module includes:

[0075] Full-process intervention sub-module: supports inserting manual modification instructions in the five stages of storyboard generation, including emotion analysis, scene segmentation, storyboard generation, visual rendering, and output review;

[0076] Multi-terminal synchronization submodule: synchronizes mobile and computer operations through WebSocket persistent connections, with a delay of ≤ 200 milliseconds, and data packets are encapsulated using the MsgPack binary protocol;

[0077] Version management submodule: records each modification based on the Git mechanism and generates a difference report, supporting rollback to any historical version.

[0078] Furthermore, the security authentication module includes:

[0079] Token refresh submodule: automatically refreshes the token through the OAuth2.0 protocol 30 minutes before the JWT expires, and the refresh token validity period is set to 7 days;

[0080] Permission classification submodule: defines user roles as guest editor and administrator. The editor role can only modify the storyboard content, while the administrator role can export and delete projects.

[0081] Transmission encryption submodule: AES-256-GCM is used to encrypt sensitive data such as undisclosed scripts. The initialization vector length is 12 bytes and the authentication tag length is 16 bytes.

[0082] Furthermore, the performance optimization module includes:

[0083] Real-time communication submodule: maintains 100,000 concurrent connections through WebSocket and uses the epoll I / O multiplexing mechanism to achieve a single-machine throughput of no less than 10,000 connections per second;

[0084] Resource scheduling submodule: Dynamically allocates GPU computing power based on Docker Swarm, prioritizing storyboard generation tasks over visualization rendering and data backup.

[0085] Disaster recovery submodule: 100,000 files are stored in a remote active-active dual-location system every day. The CRUSH algorithm is used to shard and store the data in at least three availability zones, with three copies.

[0086] Example 1: System Architecture and Operation Process

[0087] This embodiment relates to the field of artificial intelligence and film and television creation technology, specifically a multimodal interactive creation system based on a film and television knowledge graph, which integrates an AI large language model, LoRA fine-tuning technology and multimodal generation capabilities to achieve automatic generation and interactive creation from text scripts to storyboards and video sketches.

[0088] Example 2: Core module technology implementation

[0089] (1) Overall system architecture

[0090] Based on the cloud computing platform (Alibaba Cloud ECS instance, GPU model V100) and MongoDB database (sharding cluster mode), the following three-tier architecture is constructed, such as Figure 2 As shown:

[0091] 1. Platform data layer:

[0092] Cloud computing platform: elastically allocates GPU computing power, dynamically schedules based on Docker Swarm, and supports 100,000 concurrent WebSocket connections per machine;

[0093] Data storage: MongoDB stores user scripts (JSON format), storyboards (BSON format), and historical versions;

[0094] (100,000 files are processed daily, and the shard key is user_id+timestamp);

[0095] 2. Logical model layer:

[0096] AI parsing module: BERT model (HuggingFaceTransformers library) sentiment analysis, PageRank role relationship graph construction;

[0097] LoRA fine-tuning module: low-rank matrix factorization (rank r=8) based on PyTorch;

[0098] Storyboard generation module: Matplotlib visualization rendering (resolution 1920×1080), Plotly3D trajectory generation (60 frames per second).

[0099] 3. Application layer:

[0100] Cross-platform interactive interface: supports simultaneous operation on mobile phones (ReactNative framework) and computers (Electron framework);

[0101] 4. Security authentication: JWT token (HMAC-SHA256 algorithm, key length 256 bits), API access frequency is limited to 1000 times / minute.

[0102] (2) Technical details of the AI ​​analysis module

[0103] 1. Sentiment Analysis Submodule

[0104] Model formula:

[0105]

[0106] Where Q, K, V: query matrix, key matrix, value matrix (dimension = 768);

[0107] d k : Scaling factor (default = 64) to prevent gradient explosion.

[0108] 2. Scene segmentation submodule

[0109] Semantic relevance calculation:

[0110]

[0111] Where A, B: BERT embedding vectors of adjacent paragraph text (dimension = 768);

[0112] Threshold setting: Cosine similarity ≥ 0.85 triggers scene switching.

[0113] 3. Role Relationship Graph Submodule

[0114] PageRank algorithm formula:

[0115]

[0116] Where d: damping coefficient (default value = 0.85);

[0117] L(p j ): Role p j The number of outgoing links;

[0118] Output: A weighted directed graph, where the size of the node indicates the importance of the role (PR value ≥ 0.6 is a key role).

[0119] (3) Technical details of LoRA fine-tuning module

[0120] 1. Parameter rank optimization submodule

[0121] Matrix decomposition formula:

[0122] W=W0+B·A

[0123] Where W0: the initial weight matrix of the pre-trained model (dimension = input dimension d × output dimension k);

[0124] B: low-rank matrix (dimension = d × r, r = 8);

[0125] A: low-rank matrix (dimension = r × k);

[0126] Typical values: When d = 1024 and k = 512, the number of parameters is reduced to 0.78% of the original model (original number of parameters = 1024 × 512 = 524,288; LoRA number of parameters = 1024 × 8 + 8 × 512 = 12,288).

[0127] 2. Style Transfer Submodule

[0128] Director Style Infusion Formula:

[0129] W new =W0+α·B·A

[0130] Where α: style intensity coefficient (default value = 0.5, range 0 to 1);

[0131] Style vector: Extract the characteristics of the director's works through PCA dimensionality reduction (e.g., "Nolan style" corresponds to fast editing and high contrast).

[0132] 3. Cross-modal adapter module

[0133] Text to image mapping formula:

[0134] z=LoRA(E text (x))

[0135] Among them E text :Text encoder (CLIPViT-B / 32);

[0136] z: U-Net latent space vector of StableDiffusion (dimension = 768).

[0137] Technical details of the storyboard generation module

[0138] 4. Table Generator Submodule

[0139] Storyboard table fields:

[0140] Field Name Data Type Example Value Lens ID Incrementing an integer SC001 Duration floating point numbers 3.2 seconds (precision 0.1) Scenery Enumeration value Medium shot / Close-up / Panoramic shot Camera movement Multiple-select tags Push-pull + pan

[0141] 5. Visual rendering submodule

[0142] Dynamic preview generation:

[0143] Matplotlib generates storyboards, renders 24 frames per second, and supports interactive timeline dragging (response time ≤ 100ms).

[0144] 6. Double Audit Submodule

[0145] Difference comparison algorithm:

[0146]

[0147] Where δ: difference function (Levenshtein distance is used for text comparison, and SSIM structural similarity is used for image comparison);

[0148] Output report: highlight the modified points (e.g., the shot duration is adjusted from 3.2 seconds to 4.5 seconds).

[0149] Example 3: System performance verification and comparison

[0150] (1) Comparison of generated content quality is shown in the following table:

[0151] Function items This system Competitor (Flexclip) Storyboard Including shot number, duration, scene, and camera movement Script text only Multi-terminal compatibility Mobile phone + computer (WebSocket synchronization) PC only Style Transfer Support director style parameter injection Preset templates only

[0152] (2) Performance indicators

[0153] File processing: 100,000 files (average file size 10MB) processed daily, with a transfer speed of 1MB / s (accelerated by Alibaba Cloud OSS);

[0154] Resource consumption: LoRA fine-tuning reduces GPU memory usage from 24GB to 8GB (NVIDIA V100);

[0155] Real-time communication: WebSocket supports 100,000 concurrent connections on a single machine (epoll I / O multiplexing, throughput 10,000 connections / second).

[0156] Example 4: Security and Disaster Recovery Design

[0157] 1. Security authentication module

[0158] JWT token refresh logic:

[0159] The token is valid for 1 hour and automatically refreshed 30 minutes before expiration (OAuth2.0 protocol);

[0160] The refresh token is valid for 7 days and is stored in the Redis cluster (TTL = 604800 seconds).

[0161] 2. Disaster recovery and backup submodule

[0162] CRUSH algorithm storage sharding:

[0163] Placement = CRUSH (file_id, storage pool topology)

[0164] The data shard size is 64MB and is stored in three availability zones (number of replicas = 3).

[0165] Recovery time objective (RTO) ≤ 5 minutes, recovery point objective (RPO) = 0 (real-time synchronization).

[0166] Example 5: Commercial application scenario

[0167] 1. Collaboration with film and television studios:

[0168] Supports multiple people editing the same storyboard project at the same time (based on Git version management), automatically merging conflicts (intelligent fusion when the difference rate is ≤5%).

[0169] 2. Short video creators:

[0170] The mobile phone generates storyboard scripts in real time (response time ≤ 2 seconds) and supports exporting to Premiere Pro project files (XML format).

[0171] It should be noted that the specific installation methods of,, and the circuit connection methods and control methods used in the present invention are all conventional designs and will not be elaborated in detail in the present invention.

[0172] The working principle of the present invention is:

[0173] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal interactive creation system based on film and television knowledge graph, characterized by: include: AI parsing module, LoRA fine-tuning module, storyboard generation module, multimodal conversion module, user interaction module, data storage module and security authentication module, including: The AI ​​parsing module is used to analyze the emotional characteristics, character relationships and scene logic in the text script through a deep learning model; The LoRA fine-tuning module uses low-rank adaptation technology to perform lightweight parameter adjustments on the AI ​​model, supporting multimodal applications across text and image generation. The storyboard generation module automatically generates a storyboard script table and visualization content including shot switching, composition and timeline based on the analysis results; The multimodal conversion module realizes style transfer and theme adaptation from text to image; The user interaction module supports manual intervention and adjustment throughout the entire process, and provides a cross-platform operation interface for mobile phones and computers; The data storage module uses a MongoDB database to store user scripts, storyboards, and historical modification records; The security authentication module uses a JWT token based on the HMAC-SHA256 algorithm to implement user identity authentication and API access control, where the token signing key is 256 bits long and valid for 1 hour.

2. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The AI ​​parsing module includes: Sentiment analysis submodule: This module uses the BERT model to extract the emotional intensity and changing trends of the script text, calculates the emotional weight of each paragraph through the self-attention mechanism, and generates an emotional heat map; Scene segmentation submodule: This module divides shot boundaries based on a semantic relevance threshold, where the semantic relevance threshold is cosine similarity ≥ 0.85, and combines a time window sliding algorithm to identify scene transition nodes. Role relationship graph submodule: After constructing the role co-occurrence matrix, the PageRank algorithm is used to calculate the role interaction weights and generate a weighted directed relationship graph.

3. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The LoRA fine-tuning module includes: Parameter rank optimization submodule: Decompose the fully connected layer fine-tuning parameter matrix into two low-rank matrices, where the matrix rank is set to 8, satisfying that the parameter matrix W is equal to the initial matrix W0 plus the low-rank matrix product BA, where the dimension of matrix B is the input dimension d multiplied by the rank r, and the dimension of matrix A is the rank r multiplied by the output dimension k; Style transfer submodule: By adjusting the LoRA weight matrices A and B, the director's style feature vector is injected to control the shot switching frequency and composition preferences of the generated storyboards; Cross-modal adaptation submodule: maps the text feature vector to the latent space U-Net layer of the StableDiffusion model via LoRA to ensure the consistency of image generation and text theme.

4. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The storyboard generation module includes: Table Generator submodule: Outputs storyboards using commercial specifications, including incremental shot IDs, durations accurate to 0.1 second, medium and full shots for close-ups, and push-pull and pan / tilt camera movements. Visual rendering submodule: uses Matplotlib to generate dynamic previews of storyboards and supports interactive timeline dragging, and integrates the Plotly3D engine to render spatial motion trajectories; Double review sub-module: Compares the AI-generated version with the manually revised version through the Diff algorithm, highlights the differences and generates a modification suggestion report.

5. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The multimodal conversion module includes: Text-to-image generation submodule: Based on the StableDiffusion 2.1 model, it combines LoRA weight fine-tuning to generate scene sketches. Input prompts include shot type and scene, atmosphere and emotional labels, and director style dynamic parameters. Style consistency controller submodule: uses the CLIP model to calculate the similarity score between the generated image and the script text. If the score is lower than the threshold of 0.7, the LoRA weight adjustment is triggered; Real-time feedback submodule: allows users to adjust the hue, saturation, and brightness parameters of the HSV color space and the nine-square grid composition ratio through sliders.

6. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The user interaction module includes: Full-process intervention sub-module: supports inserting manual modification instructions in the five stages of storyboard generation, including emotion analysis, scene segmentation, storyboard generation, visual rendering, and output review; Multi-terminal synchronization submodule: synchronizes mobile and computer operations through WebSocket persistent connections, with a delay of ≤ 200 milliseconds, and data packets are encapsulated using the MsgPack binary protocol; Version management submodule: records each modification based on the Git mechanism and generates a difference report, supporting rollback to any historical version.

7. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The security authentication module includes: Token refresh submodule: automatically refreshes the token through the OAuth2.0 protocol 30 minutes before the JWT expires, and the refresh token validity period is set to 7 days; Permission classification submodule: defines user roles as guest editor and administrator. The editor role can only modify the storyboard content, while the administrator role can export and delete projects. Transmission encryption submodule: AES-256-GCM is used to encrypt sensitive data such as undisclosed scripts. The initialization vector length is 12 bytes and the authentication tag length is 16 bytes.

8. The multimodal interactive creation system based on film and television knowledge graph according to claim 1 is characterized in that: The performance optimization module includes: Real-time communication submodule: maintains 100,000 concurrent connections through WebSocket and uses the epol l I / O multiplexing mechanism to achieve a single-machine throughput of no less than 10,000 connections per second; Resource scheduling submodule: Dynamically allocates GPU computing power based on Docker Swarm, prioritizing storyboard generation tasks over visualization rendering and data backup. Disaster recovery submodule: 100,000 files are stored in a remote active-active dual-location system every day. The CRUSH algorithm is used to shard and store the data in at least three availability zones, with three copies.

Images