Media generation method and device and storage medium
By employing an end-to-end media generation method based on multimodal intelligent agent collaboration, the problems of low image-text matching and poor logical consistency in traditional news systems are solved, achieving efficient integration and generation of image-text content and improving the quality and efficiency of news generation.
Patent Information
- Application Number
- CN202511334301.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-18
- Publication Date
- 2026-02-10
AI Technical Summary
Traditional news generation systems lack in-depth modeling of the semantic relationship between images and text, resulting in low image-text matching and poor logical consistency in the generated news content. Furthermore, they suffer from low efficiency in task decomposition and resource allocation, as well as insufficient model adaptability and scalability.
An end-to-end media generation method based on multimodal agent collaboration is adopted. The planning agent retrieves text and image draft sets, the multimodal typesetting agent performs deep semantic analysis and image-text layout optimization, and combined with large language models and cross-modal retrieval technology, the deep integration and efficient generation of image and text content are achieved.
It significantly improves the matching degree of text and images in media documents, enhances the quality and efficiency of news generation, and ensures the logical consistency of content and the efficient use of computing resources.
Smart Images
Figure CN121502004A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a media generation method, apparatus and storage medium. Background Technology
[0002] With the rapid development of information technology and the diversified evolution of media forms, the dissemination of sports news has gradually shifted from traditional single-text reporting to rich media formats that integrate text, images, videos, and other modalities. In sports event reporting, multimodal content can significantly improve the readability and information delivery efficiency of news, meeting users' needs for an immersive reading experience. Meanwhile, advancements in artificial intelligence technology, particularly the development of multimodal large language models, have provided new technological pathways for automated news generation, enabling computers to simulate the creative logic of human editors and achieve end-to-end generation from multimodal data input to structured news output.
[0003] However, traditional news generation systems typically process text and image content independently, lacking in-depth modeling of the semantic relationship between text and images. This results in problems such as low matching degree between text and images and poor logical consistency in the generated news content. Summary of the Invention
[0004] To address the aforementioned technical problems, this application proposes a media generation method, apparatus, and storage medium.
[0005] To address the aforementioned technical problems, this application proposes a media generation method, comprising: a planning agent retrieving a set of text drafts and a set of image drafts based on a media query; a multimodal typesetting agent analyzing the semantic information of each text draft in the text draft set; the multimodal typesetting agent sorting and merging the text draft set into a long media text based on the semantic information of all text drafts; the multimodal typesetting agent marking image insertion points in the long media text based on the semantic information of adjacent text drafts after sorting, and generating title text for the image insertion points; the multimodal typesetting agent matching target images in the image draft set with the title text of each image insertion point, and generating a target image set; the multimodal typesetting agent generating a target media graphic layout based on the target image set and the long media text; and the multimodal typesetting agent embedding the target image set and the long media text into the target media graphic layout to generate a media document.
[0006] The multimodal typesetting agent generates a target media graphic layout based on the target image set and the long media text, including: the multimodal typesetting agent generates a text interface layout of the long media text according to a preset interface size; the multimodal typesetting agent generates the insertion method of each target image in the target image set at the corresponding image insertion point in the text interface layout, generating candidate media graphic layouts; the multimodal typesetting agent performs visual analysis on the candidate media graphic layouts to obtain layout defect identification results; the multimodal typesetting agent iteratively optimizes the candidate media graphic layouts based on the layout defect identification results until there are no layout defects or a preset number of iterations is reached, and determines the target media graphic layout.
[0007] The multimodal typesetting agent matches a target image in the image draft set based on the title text of each image insertion point, including: the multimodal typesetting agent matching several candidate images in the image draft set based on the title text of each image insertion point; the multimodal typesetting agent analyzing the theme information and sentiment information of the long media text; the multimodal typesetting agent obtaining the context information of the matched images of the long media text; and the multimodal typesetting agent determining at least one image as the target image from the several candidate images based on the theme information, the sentiment information, and the context information.
[0008] The planning agent retrieves a set of text drafts and a set of image drafts based on a media query, including: the planning agent retrieves a set of multimodal information units based on the media query; wherein the set of multimodal information units consists of a set of related texts and a set of image drafts; the planning agent calls a large language model to generate independent text drafts for the multimodal information units, forming the set of text drafts.
[0009] The media generation method further includes the following steps: after the planning agent retrieves a set of text drafts and a set of image drafts based on media queries, the planning agent performs dynamic quality assessment on each text draft in the set of text drafts; in response to any text draft having a quality score lower than a preset threshold, the planning agent calls an external media model to generate supplementary content for the multimodal information units corresponding to the text draft; and the planning agent regenerates an independent text draft based on the multimodal information units after incorporating the supplementary content.
[0010] The large language model is determined by the media complexity level of the media query or the media subqueries obtained by decomposing the media query.
[0011] The media generation method further includes: an intent decomposition agent decomposing the media query into several media subqueries, wherein the media subqueries include key information and modal features of the media query.
[0012] The media query can be an image media query, a text media query, or a graphic-text media query. The planning agent retrieves a set of text drafts and a set of image drafts based on the media query, including: the planning agent uses cross-modal retrieval technology to retrieve the set of text drafts and the set of image drafts related to the media query; the cross-modal retrieval technology includes text-to-text retrieval, text-to-image retrieval, graphic-text retrieval, and / or graphic-to-image retrieval.
[0013] To address the aforementioned technical problems, this application proposes a media generation apparatus, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the aforementioned media generation method.
[0014] To address the aforementioned technical problems, this application proposes a computer storage medium for storing program data, which, when executed by a processor, is used to implement the aforementioned media generation method.
[0015] Compared with existing technologies, the beneficial effects of this application are as follows: The planning agent retrieves a set of text drafts and a set of image drafts based on media queries; the multimodal typesetting agent analyzes the semantic information of each text draft in the text draft set; the multimodal typesetting agent sorts and merges the text draft set into a long media text based on the semantic information of all text drafts; the multimodal typesetting agent marks image insertion points in the long media text based on the semantic information of adjacent text drafts after sorting, generating title text for each image insertion point; the multimodal typesetting agent matches target images in the image draft set with the title text of each image insertion point, generating a target image set; the multimodal typesetting agent generates a target media graphic layout based on the target image set and the long media text; the multimodal typesetting agent embeds the target image set and the long media text into the target media graphic layout, generating a media document. Through the above methods, multimodal information integration is achieved, improving the graphic matching degree of the media document. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] in:
[0018] Figure 1 This is a flowchart illustrating the first embodiment of the media generation method provided in this application;
[0019] Figure 2 This is a schematic diagram of an embodiment of the overall process of the media generation method provided in this application;
[0020] Figure 3 This is a flowchart illustrating a second embodiment of the media generation method provided in this application;
[0021] Figure 4 This is a flowchart illustrating the third embodiment of the media generation method provided in this application;
[0022] Figure 5 This is a schematic diagram of another embodiment of the overall process of the media generation method provided in this application;
[0023] Figure 6 This is a schematic diagram of the structure of an embodiment of the media generation apparatus provided in this application;
[0024] Figure 7 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation
[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0026] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0027] The video data processing method of this application is applied to a video data processing apparatus, wherein the video data processing apparatus can be a server, a terminal device, or a system in which the server and the terminal device cooperate with each other. Accordingly, the various parts of the video data processing apparatus, such as various units, sub-units, modules, and sub-modules, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.
[0028] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed servers, or as a single software program or software module; no specific limitations are made here.
[0029] Currently, automated media generation, especially sports news generation systems, still faces many technical bottlenecks:
[0030] Insufficient integration of multimodal information: Traditional media generation systems typically process text and image content independently, lacking in-depth modeling of the semantic relationship between text and images. This results in problems such as low image-text matching and poor logical consistency in the generated content. For example, key action images and text descriptions in sports event reports may deviate due to modal fragmentation.
[0031] Inefficient task decomposition and resource allocation: Existing methods often employ fixed processes to handle user queries, making it difficult to dynamically adjust task decomposition strategies and computational resource allocation based on query complexity. For example, simple and complex queries require differentiated processing, but existing systems lack adaptive task planning capabilities, easily leading to wasted computational resources or unstable generation quality.
[0032] Insufficient model adaptability and scalability: Most generative systems are based on fine-tuning a single pre-trained model, and cannot flexibly call models of different scales according to the difficulty of the task. For example, simple description generation and deep analysis require models with different computing power, but existing solutions often suffer from performance bottlenecks or resource overload due to model rigidity.
[0033] This application addresses the problems of insufficient image-text fusion, poor content logic consistency, and low efficiency of manual typesetting in existing multimodal sports news generation processes. It proposes an end-to-end media generation method based on multimodal intelligent agent collaboration. By constructing a multi-agent task orchestration system, the entire process from multimodal query input to structured news document output is automated, significantly improving the quality and efficiency of sports news generation.
[0034] Please see details. Figures 1-2 , Figure 1This is a flowchart illustrating the first embodiment of the media generation method provided in this application; Figure 2 This is a schematic diagram of an embodiment of the overall process of the media generation method provided in this application.
[0035] like Figure 1 As shown, the specific steps are as follows:
[0036] Step S11: The planning agent retrieves the text draft set and the image draft set based on media queries.
[0037] Among them, the planning agent refers to an intelligent system with a large language model as its core reasoning engine. It utilizes the powerful world knowledge, contextual understanding, logical reasoning, and sequence generation capabilities of the large model to understand complex goals, formulate, execute, and dynamically adjust step-by-step plans.
[0038] The generation of planning agents is an iterative, context-based cycle of understanding, decomposition, reflection, and action guided by cueing engineering.
[0039] In this embodiment, a planner agent is constructed with the Qwen2.5-VL-72B large language model as the core, serving as the central scheduling module of the system to coordinate the collaborative execution of intent decomposition, information retrieval, content generation, and layout optimization.
[0040] In other implementations of this application, other large language models may also be used, and this application does not specifically limit the type of large model.
[0041] Specifically, in this embodiment of the application, the planning agent receives a media query input by the user and retrieves a set of text drafts and a set of image drafts from the database.
[0042] Media queries include, but are not limited to, image media queries, text media queries, or graphic media queries.
[0043] The planning agent retrieves a set of multimodal information units based on the media query.
[0044] The multimodal information unit set consists of a set of relevant texts and a set of image drafts.
[0045] The planning agent calls upon the multimodal information units of the large language model to generate independent text drafts, forming a text draft set.
[0046] In one embodiment of this application, the planning agent uses cross-modal retrieval technology to retrieve a set of text drafts and a set of image drafts related to the media query.
[0047] Cross-modal retrieval technologies include text-to-text retrieval, text-to-image retrieval, image-to-text retrieval, and / or image-to-image retrieval.
[0048] Cross-modal retrieval is a core technology designed to bridge the semantic gap between heterogeneous modal data. Its core task is to enable mutual retrieval between different modalities, such as text, images, audio, and video content, by mapping data from each modality to a shared semantic embedding space.
[0049] In spatial representation, semantically related samples, regardless of their original modalities, will have high-dimensional vector representations that are close to each other; while unrelated samples will be far apart. Achieving this mapping typically relies on deep representation learning architectures, such as dual-tower neural networks, which extract high-level semantic features through modality-specific encoders and jointly optimize them using metric learning strategies to ensure that the distance between positive sample pairs is smaller than the distance between negative sample pairs.
[0050] Ultimately, the retrieval process is transformed into a nearest neighbor search problem in this common vector space. By calculating the cosine or Euclidean similarity between the query vector and the candidate vector set, the most relevant result is returned. Current research challenges focus on fine-grained alignment, robust learning under modality loss, and the transparency and interpretability of the model.
[0051] In this embodiment, the planning agent retrieves a set of text drafts and a set of image drafts based on media queries. Then, the planning agent evaluates the quality to ensure that the final output content meets the expected standards. Please refer to [link to details]. Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of the media generation method provided in this application.
[0052] like Figure 3 As shown, the specific steps are as follows:
[0053] Step S21: The planning agent performs a dynamic quality assessment on each text draft in the text draft set.
[0054] The planner agent generates a set of text drafts S. D Perform dynamic quality assessment and calculate D for each draft. i Quality ratings i , where s i ∈[0,1], and perform content filtering and model switching according to the preset threshold τ.
[0055] Step S22: In response to any text draft having a quality score lower than a preset threshold, the planning agent calls an external media model to generate supplementary content for the multimodal information unit corresponding to the text draft.
[0056] Specifically, if the score satisfies s i If the score is ≥τ, then the draft is selected as the output. i<τ, then call the external model for the multimodal information unit U i Generate supplementary content to ensure that the quality of the final output meets the expected standards.
[0057] Step S23: The planning agent regenerates an independent text draft based on the multimodal information units after incorporating the supplementary content.
[0058] The media generation device integrates and optimizes a collection of text drafts, ultimately outputting high-quality, structured multimodal text drafts.
[0059] By using the methods described above, we can ensure that the quality of the final output meets the expected standards.
[0060] Step S12: The multimodal typesetting agent analyzes the semantic information of each text draft in the text draft set.
[0061] In this embodiment, the multimodal typesetting agent relies on a pre-trained language model to perform deep semantic encoding on the input text draft, transforming it into a high-dimensional vector representation, thereby capturing the deep semantic features of the text.
[0062] Based on this, a parallel computing architecture is used to simultaneously execute multi-dimensional natural language processing tasks such as named entity recognition, keyword extraction, sentiment polarity analysis, syntactic parsing, and functional role classification. Ultimately, a structured semantic object is generated for each draft, which integrates semantic vectors, entity labels, topic labels, sentiment weights, and functional identifiers. This provides a precise semantic understanding foundation for subsequent cross-modal sorting, content fusion, and intelligent typesetting.
[0063] Step S13: The multimodal typesetting agent sorts and merges the set of text drafts into a long media text based on the semantic information of all text drafts.
[0064] Specifically, the multimodal typesetting agent processes the set of text drafts S D All text drafts D i Semantic analysis and merging were performed to extract the chapter themes and key information points of each draft.
[0065] In one embodiment of this application, the media generation device eliminates repetitive and redundant sentences and reorders paragraph logic to form a unified and coherent long media text T. final .
[0066] By using a natural language processing model to deeply understand the semantic content of all text drafts, and conducting analysis and calculation based on three dimensions—topic relevance, logical coherence, and information importance—drafts with similar semantics are first clustered together. Then, a neural network sorting algorithm is used to determine the optimal narrative order. Finally, a text generation model is used to add transitional content, forming a long media text with a complete structure and semantic coherence.
[0067] Step S14: The multimodal typesetting agent marks the image insertion point in the long media text according to the semantic information of the sorted adjacent text drafts, and generates the title text of the image insertion point.
[0068] Since media texts do not include images after every sentence, this embodiment of the application requires finding the most suitable anchor point to insert an image.
[0069] Multimodal typesetting agents in T final The system marks potential image insertion points and generates concise title sentences for each insertion point for subsequent image and text matching and optimization.
[0070] The multimodal typesetting agent uses a visual language model to perform cross-modal semantic alignment based on the semantic coherence of adjacent text drafts after sorting.
[0071] First, the fused semantic features of adjacent text segments are extracted and projected into a shared semantic embedding space. Then, similarity calculations are performed between this space and image features in the media library. When the text semantics are detected to constitute a complete and visually identifiable conceptual unit, it is automatically labeled as an image insertion point.
[0072] Meanwhile, using abstract generation technology, the core content of the semantic unit is condensed to generate a precise title text rich in keywords. This title not only summarizes the content of the insertion point, but also provides a query basis for subsequent image retrieval and matching.
[0073] Step S15: The multimodal typesetting agent matches the target image in the image draft set with the title text of each image insertion point to generate a target image set.
[0074] A multimodal typesetting agent performs initial screening of candidate images for potential titles generated at each insertion point. Using the potential title at each insertion point as the query, a visual encoder based on jina-clip-v2 is used to compute the complete set of images I retrieved from the knowledge base. all The similarity between each image and the title phrase is used to select a Top-m candidate set I for each insertion point. top .
[0075] In a specific embodiment of this application, a method for determining a target image set is provided, as follows:
[0076] The multimodal typesetting agent matches several candidate images from the image draft set based on the title text of each image insertion point; the multimodal typesetting agent analyzes the theme information and sentiment information of the long media text; the multimodal typesetting agent obtains the context information of the matched images of the long media text; the multimodal typesetting agent determines at least one image as the target image from the several candidate images based on the theme information, the sentiment information, and the context information.
[0077] The modal typesetting agent further evaluates the candidate image set I for each insertion point. top Combined with the full text T final Taking into account the overall theme, emotional tone, and other selected image contexts, a unique optimal image is chosen from the candidates for each insertion point, ultimately forming a preferred subset of images.
[0078] Step S16: The multimodal typesetting agent generates a target media graphic layout based on the target image set and the media long text.
[0079] Specifically, this application provides an embodiment for determining the layout of graphics and text. Please refer to the following for details. Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the media generation method provided in this application.
[0080] like Figure 4 As shown, the specific steps are as follows:
[0081] Step S31: The multimodal typesetting agent generates the text interface layout of the long media text according to the preset interface size.
[0082] like Figure 5 As shown, Figure 5 This is a schematic diagram of another embodiment of the overall process of the media generation method provided in this application.
[0083] The multimodal typesetting agent, based on the physical constraints of preset interface dimensions, such as width, height, and resolution, first performs semantic hierarchical parsing on the generated long media text, identifying the logical relationships between elements such as titles, body text, and chart annotations. Then, it dynamically calculates the optimal layout scheme by combining visual design rules, including the number of columns, font hierarchy, spacing ratio, and text-image mixing strategies. A responsive algorithm ensures cross-device adaptation, ultimately generating a precise text interface layout that conforms to aesthetic principles and guarantees a superior reading experience.
[0084] Step S32: The multimodal typesetting agent generates the insertion method of each target image in the target image set at the corresponding image insertion point in the text interface layout, and generates a candidate media graphic layout.
[0085] The multimodal typesetting intelligent agent dynamically generates multiple candidate insertion methods for each image insertion point based on the visual flow and semantic nodes of the text interface layout. It then performs weighted evaluation by calculating multiple indicators such as visual salience, image-text relevance, color harmony, and layout stability, ultimately outputting a set of candidate media image-text layouts that both conform to aesthetic principles and enhance information expression.
[0086] The insertion methods include, but are not limited to, embedded, floating, full-screen, and background insertion.
[0087] Step S33: The multimodal layout agent performs visual analysis on the candidate media graphic layout to obtain layout defect identification results.
[0088] The multimodal typesetting agent performs visual analysis on the draft to identify potential layout defects in the news typesetting draft, including text overflow, element overlap, too much or too little blank space, and poor alignment of text and images.
[0089] Step S34: The multimodal typesetting agent iteratively optimizes the candidate media graphic layout based on the layout defect identification result until there are no layout defects or the preset number of iterations is reached, and determines the target media graphic layout.
[0090] The news draft is iteratively optimized based on the analysis results. If a defect is identified, the agent generates specific modification instructions. Upon receiving this feedback, the layout parameters are adjusted, and the process of rendering and defect identification is restarted. This continues until the agent determines that the layout has no obvious defects or the preset maximum number of iterations has been reached.
[0091] After determining that the layout has no obvious defects, the multimodal typesetting agent outputs a high-quality structured multimodal media document M and related multimodal resource metadata.
[0092] Step S17: The multimodal typesetting agent embeds the target image set and the media long text into the target media graphic layout to generate a media document.
[0093] The large language model is determined by the media complexity level of the media query or the media subqueries obtained by decomposing the media query.
[0094] In this embodiment of the application, the intent decomposes the media query into several media subqueries, the media subqueries including key information and modal features of the media query.
[0095] For different text complexity levels h i From the pre-set large language model set LLM set Select the corresponding level of model LLM. teUsed for subsequent total event generation, where LLM set ={Qwen3-30B-A3B,Qwen3-32B,Qwen2.5-VL-32B-Instruct},LLM te The format is as follows:
[0096] LLM te =LLM set [h i #(1)
[0097] The selected LLM in use te Combined with the text generation instruction p t For each multimodal information unit U i Generate independent text drafts D i ,satisfy:
[0098] D i =LLM te (U i ,p t )#(2)
[0099] This yields the text draft set S. D ={D1,D2,…,D k};
[0100] To implement the above media generation method, this application also proposes a media generation apparatus, for details please refer to [link / reference needed]. Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the media generation apparatus provided in this application.
[0101] The media generation apparatus 400 in this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.
[0102] The processor 41, memory 42, and input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the media generation method described in the above embodiments.
[0103] In this embodiment, processor 41 can also be referred to as a CPU (Central Processing Unit). Processor 41 may be an integrated circuit chip with signal processing capabilities. Processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 41 can be any conventional processor.
[0104] This application also provides a computer storage medium; please refer to the following: Figure 7 , Figure 7 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the media generation method of the above embodiment.
[0105] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0106] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A media generation method, characterized in that, The media generation method includes: The planning agent retrieves text draft sets and image draft sets based on media queries; The multimodal typesetting agent analyzes the semantic information of each text draft in the text draft set; The multimodal typesetting agent sorts and merges the set of text drafts into a long media text based on the semantic information of all text drafts. The multimodal typesetting agent marks the image insertion point in the long media text based on the semantic information of the sorted adjacent text drafts, and generates the title text of the image insertion point; The multimodal typesetting agent matches the target image in the image draft set with the title text of each image insertion point to generate a target image set; The multimodal typesetting agent generates a target media graphic layout based on the target image set and the media long text; The multimodal typesetting agent embeds the target image set and the media long text into the target media graphic layout to generate a media document.
2. The media generation method according to claim 1, characterized in that, The multimodal typesetting agent generates a target media graphic layout based on the target image set and the media long text, including: The multimodal typesetting agent generates the text interface layout of the long media text according to the preset interface size; The multimodal typesetting agent generates the insertion method of each target image in the target image set at the corresponding image insertion point in the text interface layout, and generates candidate media graphic layout; The multimodal typesetting agent performs visual analysis on the candidate media graphic layout to obtain layout defect identification results; The multimodal typesetting agent iteratively optimizes the candidate media graphic layout based on the layout defect identification results until there are no layout defects or the preset number of iterations is reached, and then determines the target media graphic layout.
3. The media generation method according to claim 1 or 2, characterized in that, The multimodal typesetting agent matches the target image in the image draft set based on the title text of each image insertion point, including: The multimodal typesetting agent matches several candidate images from the image draft set based on the title text of each image insertion point; The multimodal typesetting agent analyzes the thematic and sentiment information of the long media text; The multimodal typesetting agent obtains the context information of the matched images of the long media text; The multimodal typesetting agent determines at least one image as the target image from the candidate images based on the topic information, the emotion information, and the context information.
4. The media generation method according to claim 1, characterized in that, The planning agent retrieves text draft sets and image draft sets based on media queries, including: The planning agent retrieves a set of multimodal information units based on the media query; wherein the set of multimodal information units consists of a set of relevant texts and a set of image drafts; The planning agent invokes a large language model to generate independent text drafts for the multimodal information units, forming the text draft set.
5. The media generation method according to claim 4, characterized in that, The planning agent, based on media queries, retrieves a set of text drafts and a set of image drafts. The media generation method further includes: The planning agent performs dynamic quality assessment on each text draft in the text draft set. In response to any text draft having a quality score lower than a preset threshold, the planning agent calls an external media model to generate supplementary content for the multimodal information units corresponding to the text draft; The planning agent regenerates an independent text draft based on the multimodal information units after incorporating the supplementary content.
6. The media generation method according to claim 4, characterized in that, The large language model is determined by the media complexity level of the media query or the media subqueries obtained by decomposing the media query.
7. The media generation method according to claim 1, characterized in that, The media generation method further includes: The intent decomposes the media query into several media subqueries, each including key information and modal features of the media query.
8. The media generation method according to claim 7, characterized in that, The media query can be an image media query, a text media query, or a graphic and text media query. The planning agent retrieves text draft sets and image draft sets based on media queries, including: The planning agent uses cross-modal retrieval technology to retrieve the text draft set and image draft set related to the media query; The cross-modal retrieval technology includes text-to-text retrieval, text-to-image retrieval, image-to-text retrieval, and / or image-to-image retrieval.
9. A media generation device, characterized in that, The media generation apparatus includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the media generation method as described in any one of claims 1 to 8.
10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by a processor, is used to implement the media generation method as described in any one of claims 1 to 8.