Content generation method and system based on multi-modal large model

Abstract

The invention provides a multi-modal large model-based content generation method and system, and belongs to the technical field of digital content creation and dissemination, and the method comprises the steps: firstly building a demand association network comprising a user demand node, a content element node and a scene form node, and marking the association strength; constructing a multi-modal element coupling pool comprising a text description unit, a visual display unit and a semantic association unit on the basis, enabling the multi-modal element coupling pool to form mapping with demand association network nodes, calling a multi-modal large model to carry out situational mapping processing on the multi-modal element coupling pool, and generating an initial content framework set; and performing element collaborative optimization on the initial content framework set to obtain an optimized content framework conforming to content element node constraints, generating a target content set according to the optimized content framework, and pushing the target content set to a content distribution node to complete propagation, thereby efficiently generating and accurately propagating the multi-modal content.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital content creation and dissemination, and in particular to a content generation method and system based on a multi-modal large model. BACKGROUND

[0002] In the field of digital content creation and dissemination, with the rapid development of information technology, users' demands for content are increasingly diversified and personalized. Traditional content generation methods are often limited to a single modality, such as focusing only on text creation or image design, making it difficult to meet users' needs for the integration of multiple modalities in different scenarios.

[0003] Existing multi-modal content generation techniques have achieved the combination of text, image, audio and other modalities to some extent, but most lack a deep understanding and accurate grasp of user needs. In the content generation process, different modal elements are simply stacked without considering the internal relationship and logical consistency between elements, resulting in inconsistent content quality and failing to truly meet users' actual needs. In addition, traditional methods lack effective demand correlation analysis and element coordination optimization mechanisms when dealing with complex content generation tasks, making it difficult to generate high-quality, personalized target content according to different scenario forms and content element constraints, greatly reducing the effectiveness of content dissemination. SUMMARY

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present application, a content generation method based on a multi-modal large model is provided, the method comprising: establishing a demand correlation network for content generation, the demand correlation network comprising a plurality of nodes, the plurality of nodes including user demand nodes, content element nodes and scenario form nodes, each node being connected by an association edge and labeled with an association strength; constructing a multi-modal element coupling pool based on the demand correlation network, the multi-modal element coupling pool comprising a plurality of element units, the plurality of element units including text description units, visual display units and semantic association units, each element unit forming a mapping relationship with the nodes of the demand correlation network; calling a multi-modal large model to perform situational mapping processing on the multi-modal element coupling pool to generate an initial content framework set matching the demand correlation network; performing element coordination optimization processing on the initial content framework set to obtain an optimized content framework that meets the content element node constraints; generating a target content set according to the optimized content framework and pushing the target content set to the corresponding content distribution node to complete the content dissemination operation.

[0005] In still another aspect, the present application also provides a content generation system based on a multi-modal large model, comprising a processor, a machine-readable storage medium, the machine-readable storage medium being connected with the processor, the machine-readable storage medium being used for storing programs, instructions or codes, and the processor being used for executing the programs, instructions or codes in the machine-readable storage medium to realize the above-mentioned method.

[0006] Based on the above aspects, by establishing a demand association network for content generation, the user demand nodes, content element nodes and scene form nodes are connected by association edges and the association strength is marked, a multi-modal element coupling pool is constructed based on the demand association network, the element units such as the text description unit, the visual display unit and the semantic association unit are mapped with the nodes of the demand association network, the organic integration and ordered organization of the multi-modal elements are realized, the multi-modal large model is called to perform situational mapping processing on the multi-modal element coupling pool, an initial content framework set matched with the demand association network is generated, the powerful capability of the multi-modal large model can be fully utilized, the multi-modal elements are deeply fused with the user demand and the scene form, the initial content framework conforming to the situation is generated, the element collaborative optimization processing is performed on the initial content framework set, the optimized content framework conforming to the content element node constraint is obtained, the quality and consistency of the content are further improved, it is ensured that the generated content can accurately meet the requirements of the user demand and the content element, the target content set is generated according to the optimized content framework, and the target content set is pushed to the corresponding content distribution node to complete the content propagation operation, the efficiency and quality of the content generation are effectively improved, the pertinence and effectiveness of the content propagation are enhanced, and a more high-quality and personalized content experience is provided for the user. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is the execution flow diagram of the content generation method based on a multi-modal large model provided by an embodiment of the present application.

[0008] Figure 2 is the schematic diagram of exemplary hardware and software components of the content generation system based on a multi-modal large model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0009] The present application will be described in detail below with reference to the accompanying drawings, Figure 1 is the flow diagram of the content generation method based on a multi-modal large model provided by an embodiment of the present application, and the content generation method based on a multi-modal large model will be described in detail below.

[0010] In step S110, a demand association network for content generation is established, the demand association network contains a plurality of nodes, the plurality of nodes include user demand nodes, content element nodes and scene form nodes, and each node is connected by an association edge and the association strength is marked.

[0011] In this embodiment, the generation of promotion content for clothing products in the e-commerce field is taken as a unified application scenario. In this scenario, promotion content that meets the purchase needs of different consumers, covers specific attribute elements of clothing, and is suitable for various display scenarios needs to be generated, such as generating multi-modal promotion content suitable for different scenarios such as product detail pages, short video promotion, and live display for a summer dress.

[0012] In step S111, user historical interaction records and content detail documents are collected, user behavior keywords and feedback phrases are extracted from the user historical interaction records, and content core attribute words and feature descriptions are extracted from the content detail documents.

[0013] When collecting user historical interaction records, consumer behavior data related to summer dresses in the e-commerce platform is collected. The user historical interaction records include the page duration of the consumer browsing the summer dress on the platform, the number of times of clicking different styles of dresses, the comment content published in the product evaluation area, and the description of the purchased dress review, etc. During the collection of these data, the personal purchase records and other privacy sensitive data of consumers are involved. Data desensitization technology is used to anonymize the personal identification information of consumers such as name and mobile phone number, and through encryption algorithm, the personal identification is converted into meaningless character sequence. At the same time, data access control technology is used to limit that only authorized personnel can access sensitive data, so as to ensure that the privacy information of consumers is not leaked.

[0014] User behavior keywords and feedback phrases are extracted from user historical interaction records. For example, the consumer inputs “slimming summer dress” in the search bar, and extracts “slimming” and “summer dress” as user behavior keywords; the consumer comments “I hope the dress material is breathable, suitable for summer”, and extracts “breathable fabric” and “suitable for summer” as feedback phrases.

[0015] The content detail documents include product design instructions, fabric composition reports, size specification tables, and washing instructions of summer dresses. From these documents, content core attribute words and feature descriptions are extracted, such as “fabric material”, “pattern design”, “color and style”, and “size” which belong to content core attribute words; “made of light and breathable fabric” and “A-line design with waistline” which belong to feature descriptions.

[0016] In step S112, the user behavior keywords and feedback phrases are clustered to generate a plurality of user demand nodes, each user demand node including a demand type label and a demand description.

[0017] The extracted user behavior keywords and feedback phrases are clustered. First, the keywords and phrases are preprocessed to remove meaningless virtual words (such as "ah" and "oh") and to merge synonyms (such as "slim" and "slim and trim" into "slim and trim"). Then, a clustering algorithm based on semantic similarity is used to cluster the keywords and phrases according to their meaning associations.

[0018] For example, keywords and phrases including "slim", "slim and trim", and "tailored" are clustered into one category to generate a user demand node with a demand type label of "version slim demand" and a demand description of "consumers hope that summer dresses have a slim and tailored version effect to modify their body shape". Keywords and phrases including "breathable fabric", "comfortable fabric", and "skin-friendly fabric" are clustered into another category to generate a user demand node with a demand type label of "fabric comfort demand" and a demand description of "consumers require that the fabric of summer dresses be breathable, comfortable, and skin-friendly, suitable for summer wear". In addition, "fashionable style demand" and "reasonable price demand" user demand nodes may also be generated to correspond to the consumers' demand for fashionable and stylish dresses and within an acceptable price range, respectively.

[0019] In step S113, the content core attribute keywords and feature descriptions are hierarchically divided to construct a content element node system, which includes a main content category node, a sub-content attribute node, and a content parameter node.

[0020] The content core attribute keywords and feature descriptions of summer dresses are hierarchically divided. The main content category node is set to "summer dresses".

[0021] Under the main content category node, multiple sub-content attribute nodes are divided. For example, the "fabric attribute" sub-content attribute node covers content related to the fabric of the dress; the "version attribute" sub-content attribute node relates to the version design of the dress; the "style attribute" sub-content attribute node contains the style details of the dress; the "size attribute" sub-content attribute node relates to various size parameters of the dress, etc.

[0022] Under each sub-content attribute node, content parameter nodes are set. The content parameter nodes under the "fabric attribute" sub-content attribute node include "fabric composition", "fabric thickness", "fabric breathability", etc.; the content parameter nodes under the "version attribute" sub-content attribute node include "overall version", "waist design", "skirt design", etc.; the content parameter nodes under the "style attribute" sub-content attribute node include "collar design", "sleeve design", "pattern decoration", etc.; the content parameter nodes under the "size attribute" sub-content attribute node include "dress length", "bust circumference", "waist circumference", etc.

[0023] Step S114, analyze the content samples in the content library that match the user demand node and the content element node, and extract the presentation mode, structure type, and modal combination mode of the content samples as the constituent elements of the scene form node.

[0024] The content samples related to summer dresses in the content library include existing product detail pages, promotional short videos, live broadcast playback segments, etc. Analyze the matching of these content samples with the above user demand node and content element node.

[0025] For content samples matching the "version type slimming demand" and "version type attribute" content element nodes, such as product detail pages and short videos showing dress version types, the presentation mode is mostly multiple-angle real photo graphs combined with detailed descriptions, showing the version effect of the dress through pictures from different angles; the structure type is usually "overall version display-local detail design-wearing effect demonstration"; the modal combination mode is the combination of text and static images.

[0026] For content samples matching the "fabric comfort demand" and "fabric attribute" content element nodes, such as live broadcast segments and detail page content introducing dress fabrics, the presentation mode is mostly close-up display of fabric texture with oral explanation; the structure type is "fabric appearance display-fabric texture description-fabric property explanation"; the modal combination mode is the combination of video, audio, and text, with video showing fabric details, audio explaining, and text presenting fabric composition information.

[0027] Based on these analyses, the presentation mode, structure type, and modal combination mode are extracted as the constituent elements of the scene form node, generating scene form nodes such as "product detail page scene", "short video promotion scene", and "live broadcast display scene". The constituent elements of the "product detail page scene" are: the presentation mode is a combination of multi-picture display and detailed text description, the structure type is "overall product display-detail feature introduction-specification parameter explanation-user evaluation display", and the modal combination mode is the combination of text, static images, and dynamic images; the constituent elements of the "short video promotion scene" are: the presentation mode is a combination of dynamic video display and background music, the structure type is "eye-catching beginning-product highlights display-purchase guidance", and the modal combination mode is the combination of video, audio, and text subtitles; the constituent elements of the "live broadcast display scene" are: the presentation mode is a combination of real-time demonstration and interactive Q&A, the structure type is "product on-body effect display-detail explanation-promotion activity introduction-interactive Q&A", and the modal combination mode is the combination of video, audio, and real-time interactive text.

[0028] Step S115, calculate the co-occurrence frequency between the user demand node and the content element node through an association rule mining algorithm, and set the first association strength parameter according to the co-occurrence frequency as the labeling information of the association edge between them.

[0029] The FP-Growth algorithm in the association rule mining algorithm is used to calculate the co-occurrence frequency between the user demand node and the content element node. First, the behavior data related to the user demand node and the content element node in the user historical interaction record is converted into a transaction set, and each transaction represents a combination of the user demand node and the content element node involved by a consumer in one shopping process.

[0030] For example, a consumer embodies both the "version thin demand" and the "overall version", "waist design" and other content element nodes in one shopping, which constitutes a transaction. By counting the number of times the "version thin demand" user demand node and the "waist design" content parameter node appear together in multiple transactions, the co-occurrence frequency is obtained.

[0031] The first association strength parameter is set according to the co-occurrence frequency. The higher the co-occurrence frequency, the greater the first association strength parameter, which ranges from 0 to 1. For example, the co-occurrence frequency of the "version thin demand" user demand node and the "waist design" content parameter node is high, and the first association strength parameter is set to a high value; the co-occurrence frequency of the "version thin demand" and the "loose sleeve" content parameter node is low, and the first association strength parameter is set to a low value.

[0032] In step S116, the adaptation degree between the content element node and the scene form node is calculated, and the second association strength parameter is set according to the adaptation degree as the labeling information of the association edge between them.

[0033] When calculating the adaptation degree between the content element node and the scene form node, the clarity of content display, the effectiveness of information transmission and other aspects are considered. For the "fabric composition" content parameter node and the "product detail page scene" scene form node, "fabric composition" needs detailed textual description and close-up picture display, and the product detail page can better meet this demand, so the adaptation degree is high; while the adaptation degree of "fabric composition" and "short video promotion scene" is relatively low, because the short video is short in length and it is difficult to show the fabric composition in detail.

[0034] In the specific calculation, an adaptation degree evaluation index system is constructed, including the matching degree of content information density and scene capacity, the fit degree of content display demand and scene presentation ability, etc. Each index is scored, and then the scores of each index are weighted according to the set weight to obtain the overall adaptation degree. The second association strength parameter is set according to the adaptation degree. The higher the adaptation degree, the greater the second association strength parameter, which also ranges from 0 to 1. For example, the adaptation degree calculation result of "fabric composition" and "product detail page scene" is a high value, and the corresponding second association strength parameter is a high value; the adaptation degree calculation result of "overall version display" and "short video promotion scene" is a high value, and the corresponding second association strength parameter is a high value.

[0035] In step S117, the satisfaction degree between the user demand node and the scene form node is calculated, and a third correlation strength parameter is set according to the satisfaction degree as the labeling information of the correlation edge between them.

[0036] The satisfaction degree between the user demand node and the scene form node is calculated mainly considering whether the scene form node can effectively meet the demand described by the user demand node. For example, the "fashion style demand" user demand node and the "short video promotion scene" scene form node, the short video can highlight the fashion style of the dress through dynamic display and fashionable shooting method, and the satisfaction degree is relatively high; while the "fashion style demand" and the "product detail page scene" have a relatively low satisfaction degree, because the detail page is mainly static display.

[0037] When calculating the satisfaction degree, the conversion rate and user satisfaction of the user demand node under the corresponding scene form node in the historical data are analyzed. If the proportion of consumers who purchase the dress due to the fashion style in the "short video promotion scene" is high, and the satisfaction degree of the fashion style is high, the satisfaction degree of the "fashion style demand" and the "short video promotion scene" is high. According to the satisfaction degree, the third correlation strength parameter is set, the higher the satisfaction degree, the greater the third correlation strength parameter, which is in the range of 0 to 1. For example, the third correlation strength parameter corresponding to the satisfaction degree of the "fashion style demand" and the "short video promotion scene" is set to a high value; the satisfaction degree of the "material comfort demand" and the "product detail page scene" is relatively high, because the detail page can display the texture and composition of the material in detail, and the corresponding third correlation strength parameter is also set to a high value, while the satisfaction degree of the "material comfort demand" and the "short video promotion scene" is relatively low, and the third correlation strength parameter is set to a low value.

[0038] In step S118, the user demand node, the content element node, the scene form node and the correlation edge labeled with the first correlation strength parameter, the second correlation strength parameter and the third correlation strength parameter are networked and integrated to generate a demand correlation network with a three-layer node structure.

[0039] The user demand node, the content element node and the scene form node generated above are connected according to the correlation relationship among them to form a mutually correlated network structure. In the network, each node in the user demand node layer is connected to the corresponding node in the content element node layer through a correlation edge, and the first correlation strength parameter is labeled on the correlation edge; the node in the content element node layer is connected to the node in the scene form node layer through a correlation edge, and the second correlation strength parameter is labeled on the correlation edge; the node in the user demand node layer is also directly connected to the node in the scene form node layer through a correlation edge, and the third correlation strength parameter is labeled on the correlation edge.

[0040] For example, the "fashion style demand" node is connected with the "dress style design" and "pattern element" content parameter nodes through an association edge marked with a first association strength parameter; the "dress fabric material" content parameter node is connected with the "product detail page scene" node through an association edge marked with a second association strength parameter; and the "cost-effective demand" node is connected with the "promotion activity page scene" node through an association edge marked with a third association strength parameter. Through the above network integration, a complete demand association network is formed, which can accurately reflect the association relationship and strength between user demand, content elements and scene forms.

[0041] In step S120, a multi-modal element coupling pool is constructed based on the demand association network, the multi-modal element coupling pool including a plurality of element units, the plurality of element units including a text description unit, a visual display unit and a semantic association unit, and each element unit forming a mapping relationship with a node of the demand association network.

[0042] In step S121, based on the user demand nodes in the demand association network, description words, sentence templates and chapter structure fragments in a text corpus that match the demand type label are retrieved, and then classified according to the demand description to generate a text description unit, each text description unit including a classification identifier and corresponding text content.

[0043] The text corpus stores a large amount of text information related to dresses, including product titles, detail descriptions, user reviews, fashion information, etc. According to the demand type label of the user demand node in the demand association network, matching description words, sentence templates and chapter structure fragments are retrieved from the text corpus.

[0044] For the "fashion style demand" user demand node, the retrieved description words are "trend design", "fashion cut", "personal pattern", "unique collar", etc.; the sentence templates include "This dress adopts the current popular [pattern] design, combined with [cutting] technology, showing a fashionable atmosphere", "The unique [collar] design makes this dress stand out among many styles", etc.; and the chapter structure fragments include "First introduce the overall style of the dress, then describe its design highlights, and finally explain the suitable dressing scene".

[0045] According to the level and depth of the demand description, it is classified into basic level, advanced level and professional level. The basic level text description unit mainly introduces the simple style, and the classification mark is T1, such as T1-“This dress is fashionable and suitable for daily wear”; the advanced level text description unit will involve more design details, and the classification mark is T2, such as T2-“This dress adopts fashionable A-line style, and is matched with polka dot pattern, and the neckline is round collar design, which shows elegant and fashionable”; the professional level text description unit will analyze the fashion trend, and the classification mark is T3, such as T3-“This dress follows the current fashion trend, and its irregular skirt design is derived from the latest fashion week elements, and is matched with color blocking, which shows unique fashion attitude”.

[0046] In step S122, according to the content element node in the demand associated network, the image segments, video clips and display models corresponding to the content core attribute vocabulary and feature description in the visual resource library are screened, and the visual display unit containing the content feature label is generated, and each visual display unit is associated with the corresponding content element node.

[0047] The visual resource library contains a large amount of visual materials related to dresses, such as product pictures from different angles, wearing effect videos, fabric detail close-ups, 3D display models, etc. According to the content core attribute vocabulary and feature description of the content element node, the corresponding visual resources are screened out.

[0048] For the “dress fabric material” content parameter node, the close-up image segments of the fabric (such as the texture of cotton, the light quality of chiffon), the stretch video clips of the fabric (showing the elasticity of the fabric), and the 3D fabric display model (can view the fabric details in 360 degrees) are screened out. The features of the image segments are extracted, such as color (color of the fabric), texture (texture of the fabric surface), gloss (degree of light reflection of the fabric), etc.; the dynamic features of the video clips are extracted, such as the state of the fabric flowing, key frame image features, etc.; the model structure features of the display model are extracted, such as whether the details can be interactively viewed, material simulation features (such as simulating the tactile feedback visual effect of the fabric), etc.

[0049] The visual display unit containing the content feature label is generated, and the corresponding content element node is associated. For example, the visual display unit V1-image segment, the content feature label is “cotton, light color system, clear texture”, and it is associated with the “dress fabric material” content parameter node; the visual display unit V2-video clip, the content feature label is “chiffon fabric, light and flowing, good air permeability”, and it is associated with the “dress fabric material” content parameter node; the visual display unit V3-display model, the content feature label is “knitted fabric, good elasticity, 360 degree view”, and it is associated with the “dress fabric material” content parameter node.

[0050] Step S123, analyze the association edges and association strength parameters between nodes in the requirement association network, extract the semantic correspondence rules between the user requirement nodes and the content element nodes, the presentation logic rules between the content element nodes and the scene form nodes, and the matching rules between the user requirement nodes and the scene form nodes, and then convert the semantic correspondence rules, the presentation logic rules and the matching rules into semantic association units respectively, the semantic association units including rule type description and strength threshold range.

[0051] The association edges and association strength parameters between nodes in the requirement association network are analyzed, and various rules are extracted. The semantic correspondence rules between the user requirement nodes and the content element nodes, for example, the association strength between the "material comfort requirement" and the "dress fabric material" content parameter node is high, and the semantic correspondence rule is extracted as "when the user has a material comfort requirement, preferentially associate the content elements related to the dress fabric material", and the strength threshold range is determined according to the first association strength parameter, that is, when the first association strength parameter is within the strength threshold range, the rule is applicable.

[0052] The presentation logic rules between the content element nodes and the scene form nodes, for example, the association strength between the "dress size parameter" content parameter node and the "product detail page scene" node is high, and the presentation logic rule is extracted as "in the product detail page scene, the dress size parameter content element is preferentially presented in the form of a table combined with a schematic diagram", and the strength threshold range is determined according to the second association strength parameter.

[0053] The matching rules between the user requirement nodes and the scene form nodes, for example, the association strength between the "cost performance requirement" and the "promotion activity page scene" node is high, and the matching rule is extracted as "the cost performance requirement has a high matching degree with the promotion activity page scene, and is suitable for highlighting the price advantage and preferential information in the scene", and the strength threshold range is determined according to the third association strength parameter.

[0054] The rules are converted into semantic association units respectively, such as semantic association unit R1, the rule type is described as "user demand-content element semantic corresponding rule: when the user has a material comfort demand, the content element related to the dress fabric material is preferentially associated", the intensity threshold range is the applicable range corresponding to the first association strength parameter; semantic association unit R2, the rule type is described as "content element-scene form presentation logic rule: in the commodity detail page scene, the dress size parameter content element is preferentially presented in the form of a table combined with a schematic diagram", the intensity threshold range is the applicable range corresponding to the second association strength parameter; semantic association unit R3, the rule type is described as "user demand-scene form matching rule: the matching degree of cost performance demand and promotion activity page scene is high, and it is suitable to highlight the display of price advantage and preferential information in the scene", the intensity threshold range is the applicable range corresponding to the third association strength parameter.

[0055] In step S124, a mapping relationship table between the text description unit and the user demand node, the content element node and the scene form node is established, and the corresponding node and the mapping weight of each text description unit are determined.

[0056] The mapping relationship between each text description unit and the user demand node, the content element node and the scene form node is established. By analyzing the content of the text description unit, the association degree with each node is judged, and then the mapping weight is determined. The mapping weight reflects the close degree of association between the text description unit and the corresponding node, and ranges from 0 to 1.

[0057] For example, the text description unit T2-"This dress adopts a fashionable A-shaped version, is matched with a wave pattern, and has a round collar design, showing elegant fashion", has a close association with the "fashionable style demand" user demand node, and the mapping weight is set to a high value; it also has a relatively close association with the "dress style design" and "pattern element" content parameter nodes, and the mapping weight is set to a high value; it has a certain association with the "short video promotion scene" and "commodity detail page scene" scene form nodes, and the mapping weight is set to a corresponding value according to the fitting degree. The mapping relationship is arranged into a mapping relationship table, and the corresponding node and the mapping weight of each text description unit are clearly recorded.

[0058] In step S125, a mapping relationship table between the visual display unit and the user demand node, the content element node and the scene form node is established, and the corresponding node and the mapping weight of each visual display unit are determined.

[0059] Similarly, the mapping relationship between each visual display unit and the user demand node, the content element node and the scene form node is established. According to the content feature label of the visual display unit, the association degree with each node is analyzed, and the mapping weight is determined.

[0060] For example, the visual display unit V1-image segment (the content feature label is "cotton, light color, clear texture"), closely associated with the "material comfort demand" user demand node, with a high mapping weight setting; closely associated with the "dress fabric material" content parameter node, with a high mapping weight setting; relatively closely associated with the "product detail page scene" node, with a high mapping weight setting, and relatively weakly associated with the "short video promotion scene" node, with a low mapping weight setting. These mapping relationships are sorted into a mapping relationship table for subsequent use.

[0061] Step S126, a mapping relationship table between the semantic association unit and the association edge in the demand association network is established, and the corresponding association edge and the association strength parameter range of each semantic association unit are determined.

[0062] The rules represented by each semantic association unit are analyzed to determine the association edge in the demand association network corresponding to the semantic association unit, and the association strength parameter range of the association edge is recorded.

[0063] For example, the semantic association unit R1 corresponds to the association edge between the "material comfort demand" user demand node and the "dress fabric material" content parameter node, and the association strength parameter range thereof is the first association strength parameter applicable range of the association edge; the semantic association unit R2 corresponds to the association edge between the "dress size parameter" content parameter node and the "product detail page scene" node, and the association strength parameter range thereof is the second association strength parameter applicable range of the association edge; the semantic association unit R3 corresponds to the association edge between the "cost performance demand" user demand node and the "promotion activity page scene" node, and the association strength parameter range thereof is the third association strength parameter applicable range of the association edge. These mapping relationships are sorted into a mapping relationship table to clearly indicate the association edge and the association strength parameter range corresponding to each semantic association unit.

[0064] Step S127, the text description unit, the visual display unit, the semantic association unit and their mapping relationship table are integrated to construct a multi-modal element coupling pool.

[0065] The generated text description unit, visual display unit, semantic association unit and their respective mapping relationship tables are integrated to form a unified multi-modal element coupling pool. In the coupling pool, various element units are associated with each other, and the corresponding element units can be quickly retrieved and called according to the nodes and association relationships in the demand association network.

[0066] Step S130, calling the multi-modal large model to perform situational mapping processing on the multi-modal element coupling pool to generate an initial content framework set matched with the demand association network.

[0067] Step S131, the context analysis layer of the multi-modal large model inputs the demand association network, identifies core demand keywords of the user demand node, key attribute features of the content element node, and preference types of the scene form node through a node feature extraction algorithm, and generates a context feature vector.

[0068] In this embodiment, the context analysis layer of the multi-modal large model is used to deeply analyze the input demand association network to extract key features. First, the user demand node is subjected to feature extraction, and core demand keywords are identified from the demand type label and demand description of each user demand node, such as “fashion style demand” node extracting “fashion style” and “material comfort demand” node extracting “material” and “comfort”. Then, the content element node is processed, and key attribute features are extracted from the main content category, sub-content attribute, and content parameter node, including core attributes such as “fashion style”, “fabric”, and “size”. The scene form node is analyzed to determine the preference for dynamic display in the “short video promotion scene” and the preference for combination of text and images in the “product detail page scene”. After standardizing these features into calculable feature values, a target feature vector is generated by setting weights according to importance, and finally mapped to a high-dimensional semantic space to obtain a context feature vector.

[0069] Step S1311, the user demand node in the demand association network is subjected to feature extraction, and the demand type label, demand description, and associated content element node information of each user demand node are extracted. Core demand keywords are mined from the information through a topic model algorithm, and the core demand keywords are used to represent the demand points that the user pays most attention to.

[0070] The user demand node in the demand association network is scanned comprehensively, and the basic information of each node is extracted. For example, the “fashion style demand” node extracts its demand type label “fashion style”, demand description “consumers hope that the dress design is novel and conforms to the trend”, and associated content element node information such as “dress design” and “pattern elements”. The LDA topic model algorithm is used to mine the topics from the information, and through calculating the co-occurrence probability and topic distribution of words in the text, core demand keywords such as “fashion style”, “fashion”, “trend”, and “design” are identified, and the core demands of the user in terms of fashion style are accurately positioned. For the “material comfort demand” node, the core demand keywords such as “material”, “comfort”, “breathability”, and “skin-friendly” are extracted after topic model analysis.

[0071] Step S1312, the content element node in the demand association network is subjected to feature extraction, and the category label of the main content category node, the attribute description of the sub-content attribute node, and the specific parameter value of the content parameter node are extracted. Key attribute features that affect content generation are selected through a feature selection algorithm, and the key attribute features include the function, material, and specification of the content.

[0072] Feature extraction is performed on the content element node system. The category label "dress" is extracted from the main content category node "summer dress". The attribute descriptions "fabric material description" and "design features" are extracted from the sub-content attribute nodes "fabric attribute" and "style attribute". The specific parameter values "80% cotton and 20% polyester", "A-line style", and "85 cm long" are extracted from the content parameter nodes. The weight values of each feature are calculated using the ReliefF feature selection algorithm, and the top 30% of the features are selected as key attribute features. For example, in the fabric attribute, "fabric composition", "breathability level", and "skin-friendliness" are selected as key attribute features because they significantly affect content generation. In the style attribute, "overall style type", "waist design method", and "skirt shape" are selected as key attribute features, which directly determine the focus direction of content generation.

[0073] In step S1313, feature extraction is performed on the scene form nodes in the demand association network. The presentation mode, structure type, and modal combination mode of the scene form nodes are extracted as feature parameters. Cluster analysis is performed to determine the scene form preference types, which include detail page form, short video form, and segment form.

[0074] The core feature parameters of each scene form node are extracted. The presentation mode parameter of the "product detail page scene" is "multi-image display + text description", the structure type parameter is "hierarchical progressive", and the modal combination mode parameter is "text + static image + dynamic image". The presentation mode parameter of the "short video promotion scene" is "dynamic video + background music", the structure type parameter is "highlight preposition", and the modal combination mode parameter is "video + audio + text subtitle". K-means clustering algorithm is used to perform cluster analysis on these feature parameters, and the Euclidean distance between different scene node feature parameters is calculated. Nodes with close distances are classified into the same category. Finally, three preference types are aggregated: detail page form (emphasis on information completeness), short video form (emphasis on visual impact), and live segment form (emphasis on real-time interactivity), each corresponding to a group of typical feature parameter combinations.

[0075] In step S1314, the core demand keywords, key attribute features, and scene form preference types are standardized and represented to generate computable feature values.

[0076] The unstructured text type features are standardized, the core demand keyword "style" is mapped to the value 1, and the "fashion" is mapped to the value 2; the key attribute feature "waist A type" is coded as the value 101, and the "cotton and hemp blended fabric" is coded as the value 203. The scene form preference type is encoded by using a one-hot encoding method, the detail page form is represented as [1, 0, 0], the short video form is represented as [0, 1, 0], and the live segment form is represented as [0, 0, 1]. For the numerical parameters such as "air permeability grade", it is converted to the [0, 1] interval by Min-Max standardization, so that different types of features have a unified computable form, laying a foundation for subsequent weighted processing.

[0077] In step S1315, the weight parameters are set according to the importance of the core demand keywords, the key attribute features and the scene form preference types in content generation, and the feature values are weighted to generate a target feature vector.

[0078] The weight parameters of each feature are determined by the analytic hierarchy process. The core demand keywords are directly reflected by the user demand weight and set to 0.4; the key attribute features are the basic elements of content generation and set to 0.35; and the scene form preference types affect the content presentation effect and set to 0.25. The weighted calculation is performed on the standardized feature values. For example, in a certain demand association network, the core demand keyword feature value vector is [0.8, 0.6, 0.9], the key attribute feature value vector is [0.7, 0.85, 0.75], and the scene form preference type feature value is [1, 0, 0]. After weighted calculation, the target feature vector is obtained: [0.8x0.4+0.7x0.35+1x0.25, 0.6x0.4+0.85x0.35+0x0.25, 0.9x0.4+0.75x0.35+0x0.25], forming a target vector containing weighted features of each dimension.

[0079] In step S1316, the target feature vector is mapped to a high-dimensional semantic space to generate a context feature vector.

[0080] The pre-trained BERT model is used as a mapping tool, the target feature vector is input into the semantic encoding layer of the model, and the feature conversion and semantic enhancement are performed by using the multi-layer Transformer structure. The model maps the original feature vector from the low-dimensional space to the 768-dimensional high-dimensional semantic space, and the semantic association relationship between the features is preserved in the mapping process, such as the internal relationship between "breathable fabric" and "comfort demand". The context feature vector finally generated not only contains the numerical information of the original features, but also contains rich semantic association information, which can represent the overall context of the demand association network.

[0081] Step S132, input the text description unit, visual display unit and semantic association unit in the multi-modal element coupling pool into the element encoding layer of the multi-modal large model, respectively perform text encoding, visual encoding and semantic rule encoding processing, and generate an element feature vector set composed of a text feature vector, a visual feature vector and a semantic association feature vector.

[0082] The element encoding layer of the multi-modal large model encodes each type of element unit in the multi-modal element coupling pool. For the text description unit, a pre-trained language model is used for text encoding to convert the text content into a fixed-dimensional text feature vector, which can capture the semantic information of the text. For example, after encoding the text description unit T2, the generated text feature vector can reflect the semantic information of "A-shaped pattern", "dot pattern" and "round collar design".

[0083] For the visual display unit, a convolutional neural network is used for visual encoding to extract deep features of the visual resource and generate a visual feature vector. For example, the image segment of the visual display unit V1 is encoded to generate a visual feature vector that can reflect the texture and color of the cotton material.

[0084] For the semantic association unit, a rule encoding method is used to convert the rule type description and intensity threshold range into a semantic association feature vector, which can represent the core meaning and applicable range of the rule. For example, the semantic association unit R1 is encoded to generate a semantic association feature vector that can reflect the semantic correspondence and intensity threshold range between "material comfort demand" and "dress fabric material".

[0085] The generated text feature vector, visual feature vector and semantic association feature vector are integrated together to form an element feature vector set.

[0086] Step S133, using the mapping matching layer of the multi-modal large model, calculate the matching degree of each element feature vector and the context feature vector based on the mapping relationship table of the multi-modal element coupling pool, and select the candidate element feature vector that meets the preset condition.

[0087] The main function of the mapping matching layer is to match the element feature vector with the context feature vector. According to the mapping relationship table in the multi-modal element coupling pool, the association relationship between each element feature vector and the node in the demand association network is determined. Then, the similarity between the element feature vector and the context feature vector is calculated to determine the matching degree. The higher the similarity, the higher the matching degree.

[0088] For example, the cosine similarity of the text feature vector and the context feature vector is calculated. The greater the value of the cosine similarity, the higher the matching degree. For the visual feature vector and the context feature vector, a suitable similarity calculation method is used to obtain the matching degree.

[0089] In this embodiment, a preset condition is set, such as a matching degree greater than or equal to a certain set threshold, and the element feature vectors meeting the preset condition are selected as candidate element feature vectors. The element units corresponding to these candidate element feature vectors have a high matching degree with the context of the demand association network, and can be used as the basis for subsequent content framework construction.

[0090] In step S134, the candidate element feature vectors are weighted and sorted according to the association strength parameters of the association edges in the demand association network, and the priority of the text description unit, the visual display unit and the semantic association unit in content generation is determined, and a priority sorting result is obtained.

[0091] The candidate element feature vectors are weighted according to the association strength parameters of the association edges in the demand association network. The higher the association strength parameter, the greater the weight of the corresponding element feature vector in the sorting. After weighting calculation, the candidate element feature vectors are sorted to determine the priority of the corresponding text description unit, visual display unit and semantic association unit in content generation.

[0092] For example, the candidate element feature vector corresponding to the association edge with a higher association strength parameter has a higher priority; otherwise, the priority is lower. Through the above weighting and sorting, a priority sorting result is obtained, which clearly defines the order and importance of each type of element unit in generating the content framework.

[0093] In step S135, the framework construction layer of the multi-modal large model combines the text feature vector, the visual feature vector and the semantic association feature vector according to the semantic logic of content presentation based on the priority sorting result, and generates a plurality of initial content frameworks, each of which contains an element combination sequence and a framework structure description.

[0094] The framework construction layer combines different types of feature vectors according to the semantic logic of content presentation based on the priority sorting result. First, the main line logic of content presentation is determined, such as the logical order of "attracting attention-introducing core selling points-detailed explanation-promoting purchase". Then, the text feature vector, the visual feature vector and the semantic association feature vector are selected according to the priority from high to low, and they are combined into a content framework with a certain structure.

[0095] Each initial content framework contains a sequence of element combinations and a framework structure description. The sequence of element combinations records the order of selected text description units, visual presentation units, and semantic association units; the framework structure description describes the overall structure of the content framework, such as being divided into several parts, the main content of each part, and the presentation mode, etc. For example, the sequence of element combinations of an initial content framework can be "visual presentation unit V2 - text description unit T2 - semantic association unit R2 - visual presentation unit V1 - text description unit T1", and the framework structure description is "first show the wearing effect of the dress through a video, then introduce the fabric material and comfort, then explain the matching relationship between the wearing effect and the fabric material through a semantic association unit, then show the detail design pictures of the dress, and finally supplement the unique features of the detail design".

[0096] For another initial content framework, the sequence of element combinations can be "text description unit T3 - visual presentation unit V4 - semantic association unit R1 - text description unit T5 - visual presentation unit V3", and the framework structure description is "first highlight the fashion design style of the dress in words, then show the overall appearance pictures from different angles, then associate the design style with the appearance features through a semantic association unit, then introduce the matching suggestions in words, and finally show the overall effect pictures after matching".

[0097] During the generation of these initial content frameworks, the framework construction layer ensures the semantic coherence between different element units and avoids logical discontinuity. For example, when combining a visual presentation unit with a text description unit, it can be ensured that the information presented by the visual content and the content described by the text can echo each other, and there will be no situation where the visual presentation is of a dress of style A while the text introduction is of a dress of style B. At the same time, semantic association units are interspersed at appropriate positions to play a bridging and explaining role, making the logic of the entire content framework more clear.

[0098] In step S136, the integrity of each initial content framework is checked to check whether it covers the core user demand nodes, key content element nodes, and important scene form nodes in the demand association network. If there is a missing, the corresponding element feature vector is selected from the multi-modal element coupling pool for supplementation.

[0099] For each generated initial content framework, integrity verification is required. First, the core user demand nodes in the demand association network are determined, such as "fashion demand for style", "comfort demand for fabric", "reasonable demand for price", etc.; the key content element nodes include "dress style design", "fabric material", "price range", etc.; and the important scene form nodes are, for example, "short video promotion scene" and "product detail page scene".

[0100] During the checking, whether the initial content framework covers these core nodes is checked one by one. For example, for a certain initial content framework, whether it contains the element unit related to "fashion demand" is checked. If it is found that only fabric material is introduced in the framework, without involving the fashion design related content, that is, the core user demand node of "fashion demand" and the key content element node of "dress fashion design" are missing, the feature vectors of the text description unit, the visual display unit and the semantic association unit related to the fashion design are selected from the multi-modal element coupling pool to supplement.

[0101] During the supplement, it is ensured that the newly added element feature vectors are consistent in semantics with the element feature vectors in the original framework, without destroying the original logical structure. For example, the original framework is mainly based on the introduction of fabric comfort, and when the fashion design related content is supplemented, the two are connected through a suitable semantic association unit to explain the combination advantages of comfortable fabric and fashionable style.

[0102] In step S137, the initial content frameworks that pass the integrity check are integrated into an initial content framework set, and each initial content framework corresponds to a matching degree score of the demand association network in the initial content framework set.

[0103] All initial content frameworks that pass the integrity check are integrated together to form an initial content framework set. In order to facilitate subsequent element collaborative optimization processing, each initial content framework needs to be assigned a matching degree score of the demand association network.

[0104] The calculation of the matching degree score mainly depends on the association degree of the element units contained in the initial content framework and the nodes in the demand association network, and the association strength parameters of the associated edges. For example, if a certain initial content framework covers more core user demand nodes, key content element nodes and important scene form nodes, and the association strength parameters between these nodes are high, the matching degree score of the framework is relatively high; on the contrary, if the covered nodes are less or the association strength parameters are low, the matching degree score is relatively low.

[0105] The matching degree score of each initial content framework is recorded in the initial content framework set. For example, the matching degree score of the initial content framework F1 is 0.85, and the matching degree score of the initial content framework F2 is 0.72. These matching degree scores will intuitively reflect the degree of fit of each framework with the demand association network.

[0106] In step S140, the initial content framework set is subjected to element collaborative optimization processing to obtain an optimized content framework that meets the content element node constraint condition.

[0107] Step S141, analyze the content element node constraint conditions in the requirement association network, determine the presentation requirements of the content core attributes, the matching standards of the content specification parameters, and the creation rules of the content style features.

[0108] Analyzing the content element node constraint conditions in the requirement association network is the premise of the element collaborative optimization processing. The content element node constraint conditions are formulated based on the actual requirements of the clothing product promotion, and are used to ensure that the generated content framework meets the core attributes and promotion requirements of the product.

[0109] The presentation requirements of the content core attributes mainly involve the display methods of the core attributes such as the style, fabric, and color of the dress. For example, for the style attribute, it is required to clearly show the details of the dress such as the collar type, sleeve type, and skirt design; for the fabric attribute, it is required to explain the composition, texture, and care method of the fabric.

[0110] The matching standards of the content specification parameters mainly refer to the need for the parameters such as the size and pattern of the dress to match the characteristics of the target consumer group. For example, for the dress targeting young women, the size should cover common small, medium, and large sizes, and the pattern design should meet the body characteristics and aesthetic preferences of this group.

[0111] The creation rules of the content style features are to ensure that the overall style of the content framework is coordinated with the product positioning and promotion scenario. For example, if the product positioning of the dress is high-end fashion, then the style of the content framework should be elegant and delicate, the language expression should be concise and atmospheric, and the visual presentation should focus on texture and details.

[0112] Step S142, extract the text description unit, visual display unit, and semantic association unit contained in each initial content framework from the initial content framework set, analyze the matching degree of each element unit with the content element node constraint conditions, and the matching degree is calculated by the fit degree of the content of the element unit and the constraint conditions.

[0113] Each initial content framework is extracted from the initial content framework set one by one to extract the text description unit, visual display unit, and semantic association unit contained in each initial content framework. Then, the content of the above element units is compared with the content element node constraint conditions to analyze their matching degree.

[0114] The calculation of the matching degree is determined by the fit degree of the content of the element unit and the constraint conditions. For example, for the text description unit, if its content details the fabric composition and care method of the dress, which is highly consistent with the presentation requirements of the fabric attribute in the content core attributes, then the matching degree of the text description unit is high; if its content only mentions the fabric without involving specific composition and care method, then the matching degree is low.

[0115] For the visual display unit, if it clearly displays the collar type, sleeve type and other style details of the dress, and meets the presentation requirements of the style attributes, the matching degree is higher; if the visual content is blurred and the style details cannot be clearly distinguished, the matching degree is lower.

[0116] For the semantic association unit, if it can accurately associate different attributes of the dress and meet the creation rules of the content style characteristics, the matching degree is higher; if the association logic is chaotic or does not conform to the overall style, the matching degree is lower.

[0117] Step S143, replace the element unit with insufficient matching degree, select the element unit that meets the content element node constraint condition from the multi-modal element coupling pool to replace, and keep the semantic association between the text description unit and the visual display unit during the replacement process.

[0118] If the matching degree of a certain element unit is lower than the preset threshold, it needs to be replaced. For example, a certain text description unit does not describe the fabric composition of the dress in detail, only mentions that the fabric is comfortable, and does not specify the specific composition, which does not meet the matching degree of the presentation requirements of the content core attributes. At this time, a text description unit that can describe the fabric composition (such as the proportion of cotton and polyester fiber) in detail is selected from the multi-modal element coupling pool to replace.

[0119] When replacing the visual display unit, if the color of a certain visual display unit displayed on the dress deviates from the actual product and does not meet the matching standard of the content specification parameters, a visual display unit that can accurately present the color of the product is selected to replace it.

[0120] During the replacement process, the semantic association between the text description unit and the visual display unit must be maintained. For example, if the replaced text description unit emphasizes the "chiffon fabric, light and breathable" of the dress, the corresponding visual display unit should display pictures or videos that can reflect the light and breathable feeling of the chiffon fabric, such as the dynamic effect of the dress floating in the wind, to ensure that the information conveyed by the text description and the visual display is consistent.

[0121] Step S144, adjust the combination order and association strength of the text description unit, visual display unit and semantic association unit, so that the content logic of the text description unit and the presentation logic of the visual display unit are consistent, and the semantic association unit can accurately reflect the logical relationship between the text and the visual.

[0122] After the replacement of the element units is completed, the combination order and the correlation strength of each element unit need to be adjusted. When adjusting the combination order, it is necessary to ensure that the content logic of the text description unit is consistent with the presentation logic of the visual display unit. For example, if the text description unit describes the order of "fabric composition - style design - wearing effect", the visual display unit should also present the order of showing fabric details - showing overall style - showing wearing effect.

[0123] When adjusting the correlation strength, for those element units that can highlight the core selling points of the dress, the correlation strength between them should be enhanced. For example, if "silk fabric" is one of the core selling points of the dress, the correlation strength between the text description unit describing the silk fabric and the visual display unit showing the texture of the silk fabric should be appropriately increased, and the corresponding semantic association unit should also emphasize the relationship between the two.

[0124] The adjustment of the semantic association unit should accurately reflect the logical relationship between the text and the visual. For example, when the text description unit mentions "the waist-hugging design of the dress can highlight the figure", and the visual display unit shows the details of the waist-hugging design, the semantic association unit should clearly state that "the waist-hugging design shown in the picture corresponds to the highlighting effect of the figure described in the text, and the design achieves the modification of the figure through XX method".

[0125] In step S145, the element coordination of the adjusted content framework is evaluated, and the coherence score between the text description units, the style consistency score between the visual display units, and the semantic matching degree score between the text description units and the visual display units are calculated.

[0126] In step S1451, the sequence of text description units in the adjusted content framework is extracted, and the lexical cohesion information, grammatical association information, and semantic coherence information between adjacent text description units are analyzed. The coherence score is calculated through a pre-trained language model, and the coherence score is taken as the coherence score between the text description units.

[0127] The sequence of text description units in the adjusted content framework is extracted, such as "T1-T2-T3-T4". The lexical cohesion between T1 and T2 is analyzed. If "silk fabric" appears in T1 and "the fabric" is used for reference in T2, it indicates that the lexical cohesion is smooth. If T1 discusses the fabric and T2 suddenly shifts to the size without reasonable transition, there is a problem with lexical cohesion.

[0128] The grammatical association information is analyzed to see if the sentence patterns of adjacent text description units are coordinated and if there are any grammatical conflicts. For example, T1 uses a declarative sentence to introduce the fabric, and T2 uses a question sentence to introduce the style. If the transition is natural, the grammatical association is good. If the sentence pattern is chaotic and lacks logical connection, the grammatical association is poor.

[0129] Analyze semantic coherence to determine whether the semantics expressed by adjacent text description units are related and whether they can form a coherent semantic flow. For example, if T1 describes the fabric as lightweight and T2 then describes it as suitable for summer wear, the semantics are coherent. However, if T1 describes the fabric as heavy and T2 then describes it as suitable for summer wear, the semantics are incoherent.

[0130] This information is input into the pre-trained language model, and the model will calculate a coherence score based on the language rules and semantic association knowledge learned during its training process. The coherence score is the coherence score between text description units.

[0131] Step S1452: extract the sequence of visual display units in the adjusted content frame, analyze the consistency of the color tone, composition style and element layout of the visual display units, calculate the style similarity through the visual feature comparison algorithm, and use the style similarity as the style consistency score between the visual display units.

[0132] Extract the sequence of visual display units in the adjusted content framework, such as "V1-V2-V3-V4." Analyze the color tone. If all visual display units are primarily soft, light colors, the color tone is consistent. However, if V1 is cool and V2 suddenly changes to warm tones without a reasonable transition, the color tone is inconsistent.

[0133] Analyze the composition style. If all visual display units use a centered composition to highlight the main body of the dress, the composition style is consistent; if V1 uses a panoramic composition and V2 suddenly uses a local close-up and the composition methods are too different, the composition style is inconsistent.

[0134] Analyze the element layout. If the placement of the dresses and the matching of background elements in all visual display units are consistent, such as a simple solid-color background, the element layout is consistent. If the background of V1 is complex and the background of V2 is simple and unrelated, the element layout is inconsistent.

[0135] These visual features are compared through a visual feature comparison algorithm to calculate the style similarity, which is the style consistency score between the visual display units.

[0136] Step S1453: establishing a correspondence between the text description unit and the visual presentation unit, wherein each text description unit corresponds to one or more visual presentation units, and extracting the core semantics of the text description unit and the visual semantics of the visual presentation unit.

[0137] To establish a corresponding relationship between the text description unit and the visual display unit, for example, the text description unit T1 (introducing the light texture of chiffon fabric) corresponds to the visual display unit V1 (close-up picture of chiffon fabric) and V2 (video clip of the flowing dress); the text description unit T2 (introducing the waistline design) corresponds to the visual display unit V3 (detail picture of the waistline).

[0138] Extract the core semantics of the text description unit, the core semantics of T1 is "light chiffon fabric"; the core semantics of T2 is "waistline design to modify the body shape".

[0139] Extract the visual semantics of the visual display unit, the visual semantics of V1 is "delicate texture and light transmission of chiffon fabric"; the visual semantics of V2 is "lightness of the dress in motion"; the visual semantics of V3 is "line design of the waistline and optimization of body proportion".

[0140] Step S1454, calculate the semantic distance between the core semantics and the visual semantics, which is calculated by the Euclidean distance in the semantic vector space.

[0141] Convert the core semantics of the text description unit and the visual semantics of the visual display unit into semantic vectors and map them into the same semantic vector space. For example, convert the core semantics of "light chiffon fabric" into vector A, convert the visual semantics of "delicate texture and light transmission of chiffon fabric" into vector B, and convert the visual semantics of "lightness of the dress in motion" into vector C.

[0142] Calculate the Euclidean distance between vector A and vector B, and the Euclidean distance between vector A and vector C, which are the semantic distances between the core semantics and the visual semantics.

[0143] Step S1455, convert the semantic distance into a semantic matching degree score according to the semantic distance, and the semantic matching degree score is negatively related to the semantic distance.

[0144] The smaller the semantic distance, the higher the matching degree between the core semantics and the visual semantics, and the higher the corresponding semantic matching degree score; on the contrary, the larger the semantic distance, the lower the semantic matching degree score. For example, if the semantic distance between vector A and vector B is small, the corresponding semantic matching degree score is high; if the distance between vector A and another visual semantic vector is large, the semantic matching degree score is low.

[0145] Step S1456, weight the coherence score between the text description units, the style consistency score between the visual display units, and the semantic matching degree score between the text description units and the visual display units according to the preset weight parameters to obtain the comprehensive score of the element cooperativity evaluation.

[0146] The preset weight parameters of the coherence score, the style consistency score and the semantic matching degree score are, for example, the coherence score weight is 0.3, the style consistency score weight is 0.3, and the semantic matching degree score weight is 0.4.

[0147] Suppose that the coherence score of a certain adjusted content framework is 0.8, the style consistency score is 0.7, and the semantic matching degree score is 0.9, then the comprehensive score = 0.8*0.3 + 0.7*0.3 + 0.9*0.4, and the comprehensive score is obtained by calculation, which is the comprehensive score of the element synergy evaluation.

[0148] In step S146, if the element synergy evaluation result meets the preset threshold, the adjusted content framework is taken as a candidate optimization framework; if not, the replacement and adjustment operations are repeatedly performed until the element synergy evaluation result meets the standard.

[0149] The preset threshold of the comprehensive score of the element synergy evaluation is set, if the comprehensive score of the adjusted content framework reaches or exceeds the threshold, it means that the element synergy of the framework is good, and it is taken as a candidate optimization framework; if the comprehensive score is lower than the threshold, the replacement and adjustment of the element units need to be performed again, and the element synergy evaluation is performed again until the comprehensive score meets the standard.

[0150] For example, if the preset threshold is 0.75, the comprehensive score of a certain adjusted content framework is 0.82, which meets the threshold requirement and becomes a candidate optimization framework; the comprehensive score of another framework is 0.7, so the replacement and adjustment need to be performed again, such as replacing part of the visual display unit to improve the style consistency, or adjusting the order of the text description unit to enhance the coherence, and then the evaluation is performed again until the score reaches 0.75 or above.

[0151] In step S147, the framework with the highest matching degree with the content element node constraint condition is selected from the candidate optimization frameworks as an optimized content framework, and the optimized content framework includes the text description sequence, the visual display sequence and the semantic association rule set which are synergistically optimized.

[0152] The matching degrees of all candidate optimization frameworks with the content element node constraint condition are evaluated again, and the evaluation dimensions include the completeness of the core attribute presentation, the matching accuracy of the specification parameter, the consistency of the style feature, etc.

[0153] The candidate optimization framework with the best performance in these dimensions, i.e., the highest matching degree, is selected as the final optimized content framework. The text description sequence, the visual display sequence and the semantic association rule set in the optimized content framework are synergistically optimized and can well meet the constraint conditions of the content element nodes.

[0154] Step S150, generating a target content set according to the optimized content framework, and pushing the target content set to the corresponding content distribution node to complete the content propagation operation.

[0155] Step S151, parsing the text description sequence in the optimized content framework, and combining each text description unit into complete text content according to semantic logic, the language style of the text content matching the user demand node in the demand association network.

[0156] Step S1511, extracting each text description unit of the text description sequence in the optimized content framework, and parsing the hierarchical identifier, text content and corresponding user demand node information of each text description unit.

[0157] Each text description unit is extracted from the text description sequence of the optimized content framework, such as "T1-T2-T3-T4". The hierarchical identifier of each unit is parsed, the hierarchical identifier of T1 is "basic level", T2 is "advanced level", T3 is "basic level", and T4 is "advanced level".

[0158] The text content is parsed, T1: "This dress uses high-quality chiffon fabric"; T2: "Chiffon fabric has the characteristics of light and breathable, suitable for summer wear"; T3: "The dress uses a waistline design"; T4: "The waistline design can effectively modify the body proportion and highlight the elegant temperament".

[0159] The corresponding user demand node information is parsed, T1 and T2 correspond to "fabric comfort demand"; T3 and T4 correspond to "style fashion demand".

[0160] Step S1512, determining the combination order of the text description units according to the semantic association relationship between the text description units.

[0161] The semantic association between the text description units is analyzed, T1 introduces the fabric material, T2 further explains the advantages of the fabric in the dress design, such as "breathable cotton and linen fabric in the waistline A-shaped dress design can better highlight the lightness of the dress", so T1 and T2 have progressive association, T1 should be placed before T2. T3 describes the dressing scene of the dress, its content needs to be based on the basic information of fabric and pattern, for example, "breathable cotton and linen waistline A-shaped dress is suitable for matching with a suit jacket for commuting and with a canvas shoe for leisure", so T3 needs to be after T1 and T2. T4 involves washing and maintenance method, which is directly related to the fabric material, such as "cotton and linen fabric needs to be washed gently with cold water and avoid sun exposure", so T4 should be followed by T1 which introduces the fabric, forming a logical chain of "fabric characteristics-washing method-pattern advantages-dressing scene".

[0162] Through the above semantic association analysis, the combination order of the text description units is determined as T1-T4-T2-T3. The above order not only conforms to the cognitive logic of consumers from basic attributes to use details, but also enables the text content to remain coherent when delivering information, avoiding information jumps or logical breaks.

[0163] In step S1513, the text description units in the combination process are connected, and preset conjunctions, transition sentences or paragraphs are added.

[0164] Between T1 and T4, the transition sentence "After understanding the fabric properties of this dress, the correct washing method can better maintain its texture" is added to naturally transition from fabric introduction to washing and maintenance content. Between T4 and T2, the conjunction "in addition" is used, and the transition sentence "In addition to the comfortable experience brought by the fabric, its design is also worth paying attention to" is used to achieve a smooth transition from washing information to design advantages. Between T2 and T3, the conjunction "based on the above design and fabric" is used to introduce the transition sentence "This dress can show unique charm in different scenes", which closely links the design information and the scene description.

[0165] These connection processes not only avoid the harshness of text unit combination, but also strengthen the logical relationship between the contents of each part, making the entire text content read smoothly and naturally, and facilitating consumer understanding and information reception.

[0166] In step S1514, the demand type label and demand description of the user demand node in the demand association network are analyzed to determine the corresponding language style features, including formality, emotional tendency and conciseness.

[0167] In the demand association network, the demand description of the "fabric comfort demand" node is "consumers hope that the dress fabric is skin-friendly, breathable, and suitable for long-term wear", and the demand type label is "practical demand". This type of demand focuses more on the accuracy and practicality of information, so the language style should be formal and objective, avoiding excessive modification. The demand description of the "design slimming demand" node is "consumers pursue dresses that can modify body shape and highlight body advantages", and the demand type label is "aesthetically pleasing demand". The language style can appropriately add positive emotional tendencies, such as using "elegant outline" and "clever modification", to enhance appeal. The demand description of the "scene adaptation demand" node is "consumers need dresses that can meet the needs of various occasions and improve the convenience of dressing", and the demand type label is "multi-functional demand". The language style needs to be concise and clear, using concise sentences to explain dressing suggestions for different scenes, making it easy for consumers to quickly obtain key information.

[0168] According to the characteristics of the demand nodes, the overall language style is determined to be formal with moderate positive emotion, while ensuring that the information is concise and easy to understand, which meets the requirements of accuracy for practical needs, the pursuit of attractiveness for aesthetic needs, and the expectation of information efficiency for multi-functional needs.

[0169] In step S1515, the vocabulary selection, sentence structure, and expression tone of the text content are adjusted according to the language style characteristics, and the complete text content is output. The text content includes multiple paragraphs, each corresponding to a core semantic theme.

[0170] Regarding the formal degree requirement, avoid colloquial expressions in vocabulary selection, such as not using "wear very comfortable" but using "wear experience comfortable"; instead of "slim effect good", use "body modification effect significant". The sentence structure is mainly in the form of declarative sentences, reducing the use of exclamatory and interrogative sentences, such as changing "this dress is very versatile!" to "this dress has strong versatility".

[0171] Considering the emotional tendency, when describing the advantages of the version, use expressions with positive emotions such as "elegant waist design cleverly outlines the waist line, A-shaped skirt naturally falls, showing dynamic temperament"; when introducing the dressing scene, use sentences that can evoke consumers' good associations, such as "whether it is a professional commuting style or a weekend casual style, it can be easily mastered".

[0172] To meet the requirement of simplicity, each paragraph is controlled to three to five sentences, with the core semantics highlighted. For example, the paragraph about the fabric: "This dress uses breathable cotton and linen fabric, lightweight, skin-friendly, suitable for long-term wear. The fiber structure of the fabric is loose, with good moisture-wicking performance, even in hot weather, it can keep the wearer dry and comfortable." The paragraph about the version: "The dress uses a waist A-shaped version design, the waist tightening process can modify the waistline, the A-shaped skirt falls naturally from the waist, effectively hiding the hip and thigh lines, and has good adaptability to various body shapes." After adjustment, the complete text content is divided into four paragraphs, corresponding to four core semantic themes of fabric characteristics, washing and maintenance, version advantages, and dressing scene. The language style of each paragraph meets the determined characteristics, and the overall content accurately conveys product information and meets the expectations of consumers.

[0173] In step S152, the visual display sequence in the optimized content framework is parsed, and each visual display unit is integrated according to the content composition rules to generate visual content that corresponds to the text content. The presentation style of the visual content meets the requirements of the content element nodes in the demand association network.

[0174] The visual presentation sequence in the optimized content framework is analyzed to clarify the content and role of each visual presentation unit. For example, visual presentation unit V1 is a close-up of the dress fabric details, V2 is an overall dress effect drawing (front), V3 is an overall dress effect drawing (side), and V4 is a dress demonstration drawing in different scenes (office, leisure).

[0175] The content composition rules include subject prominence, complementary perspective, scene matching, etc. When integrating, V1 is placed at the beginning of the visual content, highlighting the texture and quality of the fabric through close-up shots, echoing the description of fabric characteristics in the text; V2 and V3 are arranged in an up-down manner, with the front view showing the overall style of the dress, and the side view highlighting the effect of the waistline version, complementing the content of the version advantage in the text; V4 is arranged in the order of office scene first and leisure scene last, corresponding to the description order of the dress scene in the text.

[0176] The "fabric attribute" content element node in the demand association network requires visual content to clearly show fabric details, so the composition of V1 should focus on fabric texture and avoid interference from irrelevant elements; the "version design" content element node requires highlighting version lines, so the composition of V2 and V3 should use a simple background to make the outline and lines of the dress the visual focus; the "scene adaptation" content element node requires visual content to reflect the atmosphere of different scenes, so the office scene in V4 is set in an office environment, and the leisure scene is set in a coffee shop or park, making the visual presentation style consistent with the requirements of each content element node.

[0177] The integrated visual content has a natural transition between units, maintaining overall coordination through unified color tones (such as soft pastel colors) and consistent shooting styles (such as natural light shooting), while each unit can effectively echo the corresponding part of the text content, enhancing the intuitiveness of information transmission.

[0178] In step S153, according to the semantic association rule set in the optimized content framework, the text content and visual content are associated and combined to form multi-modal content units, each containing corresponding user demand node labels, content element node labels, and scene form node labels.

[0179] The semantic association rule set in the optimized content framework includes rules such as "fabric characteristic text corresponds to fabric detail visual", "version advantage text corresponds to version display visual", and "dress scene text corresponds to scene dress visual". According to these rules, the fabric paragraph of the text content is combined with V1 to form the first multi-modal content unit, the version paragraph is combined with V2 and V3 to form the second multi-modal content unit, and the dress scene paragraph is combined with V4 to form the third multi-modal content unit.

[0180] Each multimodal content unit is labeled with corresponding labels. The first unit is labeled with the "fabric comfort demand" user demand node label, the "fabric attribute" content element node label, and the "product detail page scene" scene form node label; the second unit is labeled with the "slim-fit demand" user demand node label, the "pattern design" content element node label, and the "product detail page scene" scene form node label; and the third unit is labeled with the "scene adaptation demand" user demand node label, the "scene adaptation" content element node label, and the "short video promotion scene" scene form node label.

[0181] The addition of these labels enables the multimodal content units to accurately correspond to the nodes in the demand association network, ensuring that different content units can be pushed to appropriate scenes and channels.

[0182] Step S154, arrange and combine the plurality of multimodal content units according to the structure type of the scene form node to generate a target content set containing different modal combinations.

[0183] The structure type of the scene form node includes "product detail page structure" and "short video promotion structure". The "product detail page structure" requires content units to be arranged in the order of "basic attribute-core advantage-use scene", so the first multimodal content unit (fabric), the second multimodal content unit (pattern), and the third multimodal content unit (outfit scene) are arranged in turn to form a content combination suitable for a product detail page.

[0184] The "short video promotion structure" requires attention-grabbing visual content at the beginning, key information in the middle, and action-guiding content at the end. Accordingly, V2 (overall outfit front view) is used as the beginning, followed by a simplified version of the second multimodal content unit (text highlighting pattern advantages and V3), then the third multimodal content unit (outfit scene), and finally the guiding text "click to view details" to form a content combination suitable for short video promotion.

[0185] The target content set contains two different modal combinations, each of which maintains logical coherence between content units and meets the structural requirements of the corresponding scene form node, and can meet the needs of different promotion scenarios.

[0186] Step S155, retrieve content distribution node information matching the labels of the multimodal content units in the target content set, which includes node identification, distribution range, and receiving preferences.

[0187] The content distribution node information is retrieved through tag matching. The multi-modal content unit labeled with the "product detail page scene" tag is matched to the distribution nodes with node identifiers "PC detail page node" and "mobile detail page node", whose distribution ranges are website PC end and mobile APP end respectively, and the receiving preference is detailed content with combination of text and images.

[0188] The multi-modal content unit labeled with the "short video promotion scene" tag is matched to the distribution nodes with node identifiers "short video platform node A" and "short video platform node B", whose distribution ranges are corresponding short video platform user groups, and the receiving preference is content with short duration and strong visual impact.

[0189] Meanwhile, the distribution node information also includes user portrait data of each node, such as that the users of the "mobile detail page node" are mostly young women, and the users of the "short video platform node A" are mainly young groups in second- and third-tier cities.

[0190] In step S156, the multi-modal content units in the target content set are screened according to the receiving preference of the content distribution node, so that the multi-modal content units pushed to the content distribution node meet the receiving requirements of the node.

[0191] The "PC detail page node" and "mobile detail page node" prefer detailed text and image content, so the content combination suitable for product detail pages is selected from the target content set, and the complete text and visual units are retained to ensure the comprehensiveness of the information. For the "mobile detail page node", considering the small size of the mobile phone screen, the size of the visual content is adjusted to be suitable for the mobile phone screen, and the text font is set to a readable size.

[0192] The "short video platform node A" and "short video platform node B" prefer short duration and strong visual content, so the content combination suitable for short video promotion is screened, the text content is simplified, the core information is retained, the video duration is controlled within the range required by the platform, and the visual content is highlighted. For example, some detailed description of the fabric is deleted, and only the key sentence "breathable cotton and linen, comfortable and versatile" is retained.

[0193] The screened content units all meet the receiving preference of each distribution node and can be better presented in different distribution channels.

[0194] In step S157, the screened target content set is pushed to the corresponding content distribution node, and the content distribution node performs content display and propagation operations after receiving the target content set.

[0195] The content combination suitable for the product detail page is pushed to the "PC detail page node" and the "mobile terminal detail page node" respectively, and after the two nodes receive the content, the dress product page on the website PC terminal and the mobile phone APP terminal displays the content, including complete text description and ordered visual content, for consumers to browse and understand product details.

[0196] The content combination suitable for short video promotion is pushed to the "short video platform node A" and the "short video platform node B", and after the nodes receive the content, the content is processed into a short video in the format of the platform, and is recommended and spread on the platform, attracts users' attention through dynamic vision and concise information, and guides users to click the link to enter the product detail page.

[0197] When each content distribution node performs the display and spread operation, it also feeds back the exposure and click volume of the content in real time, and these data can be used for subsequent optimization and adjustment of the content generation method, and improve the effect of content promotion.

[0198] Figure 2 A schematic diagram of exemplary hardware and software components of a content generation system 100 based on a multi-modal large model that can implement the idea of the present application is shown. For example, the processor 120 can be used in the content generation system 100 based on a multi-modal large model, and is used to execute the functions in the present application.

[0199] For example, the content generation system 100 based on a multi-modal large model can include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as a disk, a ROM, or a RAM, or any combination thereof. Exemplarily, the content generation system 100 based on a multi-modal large model can also include program instructions stored in a ROM, a RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The content generation system 100 based on a multi-modal large model also includes an I / O interface 150 between the computer and other input / output devices.

[0200] In addition, the present application also provides a readable storage medium, wherein computer executable instructions are pre-set in the readable storage medium, and when a processor executes the computer executable instructions, the content generation method based on a multi-modal large model is realized.

[0201] It should be noted that, in order to simplify the description of the present application and help understand one or more embodiments of the present application, in the foregoing description of embodiments of the present application, various features are sometimes combined into one embodiment, drawing or description thereof.

Claims

1. A content generation method based on a multi-modal large model, characterized in that, The method comprises: establishing a demand correlation network of content generation, the demand correlation network comprising a plurality of nodes, the plurality of nodes including user demand nodes, content element nodes and scene form nodes, each node being connected by an association edge and being marked with an association strength; constructing a multi-modal element coupling pool based on the demand correlation network, the multi-modal element coupling pool comprising a plurality of element units, the plurality of element units including text description units, visual display units and semantic association units, each element unit forming a mapping relationship with the nodes of the demand correlation network; calling a multi-modal large model to perform situational mapping processing on the multi-modal element coupling pool to generate an initial content framework set matched with the demand correlation network; performing element collaborative optimization processing on the initial content framework set to obtain an optimized content framework conforming to the content element node constraints; generating a target content set according to the optimized content framework, and pushing the target content set to a corresponding content distribution node to complete a content propagation operation.

2. The content generation method based on a multi-modal large model according to claim 1, characterized in that, The method comprises: collecting user historical interaction records and content detail documents, extracting user behavior keywords and feedback phrases from the user historical interaction records, and extracting content core attribute words and feature descriptions from the content detail documents; performing clustering processing on the user behavior keywords and feedback phrases to generate a plurality of user demand nodes, each user demand node including a demand type label and a demand description; performing hierarchical division on the content core attribute words and feature descriptions to construct a content element node system, the content element node system including main content category nodes, sub-content attribute nodes and content parameter nodes; analyzing content samples in a content library that match the user demand nodes and the content element nodes, and extracting presentation modes, structure types and modality combination modes of the content samples as constituent elements of scene form nodes; calculating the co-occurrence frequency between the user demand nodes and the content element nodes by an association rule mining algorithm, setting a first association strength parameter according to the co-occurrence frequency as the marking information of the association edge between them; calculating the adaptation degree between the content element nodes and the scene form nodes, setting a second association strength parameter according to the adaptation degree as the marking information of the association edge between them; calculating the satisfaction degree between the user demand nodes and the scene form nodes, setting a third association strength parameter according to the satisfaction degree as the marking information of the association edge between them; networking the user demand nodes, content element nodes, scene form nodes and association edges marked with the first association strength parameter, the second association strength parameter and the third association strength parameter to generate a demand correlation network with a three-layer node structure. 3.The content generation method based on a multi-modal large model according to claim 2, wherein, The method comprises: Retrieving description vocabularies, sentence templates and chapter structure fragments in a text corpus that match the demand type label according to the user demand node in the demand association network, generating text description units according to the demand description, each text description unit containing a hierarchical identifier and corresponding text content; According to the content element node in the demand association network, filtering image fragments, video clips and display models in a visual resource library that correspond to the content core attribute vocabulary and feature description, and then performing feature extraction, generating a visual display unit containing a content feature label, each visual display unit being associated with a corresponding content element node; Analyzing the association edges and association strength parameters between the nodes in the demand association network, extracting the semantic correspondence rules between the user demand node and the content element node, the presentation logic rules between the content element node and the scene form node, and the matching rules between the user demand node and the scene form node, and then converting the semantic correspondence rules, the presentation logic rules and the matching rules into semantic association units respectively, the semantic association units containing rule type descriptions and strength threshold ranges; Establishing a mapping relationship table between the text description units and the user demand nodes, content element nodes and scene form nodes, and determining the corresponding nodes and mapping weights of each text description unit; Establishing a mapping relationship table between the visual display units and the user demand nodes, content element nodes and scene form nodes, and determining the corresponding nodes and mapping weights of each visual display unit; Establishing a mapping relationship table between the semantic association units and the association edges in the demand association network, and determining the corresponding association edges and association strength parameter ranges of each semantic association unit; Integrating the text description units, visual display units, semantic association units and their mapping relationship tables to construct a multi-modal element coupling pool. 4.The content generation method based on a multi-modal large model according to claim 1, wherein, The multi-modal large model is called to perform situational mapping processing on the multi-modal element coupling pool to generate an initial content framework set matching the demand association network, including: Inputting the demand association network into the situational analysis layer of the multi-modal large model, identifying the core demand keywords of the user demand node, the key attribute features of the content element node and the preference type of the scene form node through a node feature extraction algorithm, and generating a situational feature vector; Inputting the text description units, visual display units and semantic association units in the multi-modal element coupling pool into the element encoding layer of the multi-modal large model to perform text encoding, visual encoding and semantic rule encoding processing respectively, generating an element feature vector set composed of a text feature vector, a visual feature vector and a semantic association feature vector; Using the mapping matching layer of the multi-modal large model to calculate the matching degree of each element feature vector and the situational feature vector based on the mapping relationship table of the multi-modal element coupling pool, and filtering out candidate element feature vectors that meet the preset conditions. According to the association strength parameters of the association edges in the requirement association network, the candidate element feature vectors are weighted and sorted to determine the priority of the text description unit, the visual display unit and the semantic association unit in content generation, and a priority sorting result is obtained; Based on the priority sorting result, the text feature vectors, visual feature vectors and semantic association feature vectors are combined according to the semantic logic of content presentation by using the framework construction layer of the multi-modal large model, and a plurality of initial content frameworks are generated, each initial content framework including an element combination sequence and a framework structure description; Each initial content framework is subjected to integrity checking to check whether it covers the core user demand nodes, key content element nodes and important scene form nodes in the requirement association network, and if there is a missing, corresponding element feature vectors are selected from the multi-modal element coupling pool for supplementation; The initial content frameworks that pass the integrity checking are integrated into an initial content framework set, and each initial content framework corresponds to a matching degree score of the requirement association network in the initial content framework set.

5. The content generation method based on a multi-modal large model according to claim 4, characterized in that, The context analysis layer of inputting the requirement association network into the multi-modal large model identifies the core demand keywords of the user demand nodes, the key attribute features of the content element nodes and the preference types of the scene form nodes by a node feature extraction algorithm, and generates a context feature vector, including: Feature extraction is performed on the user demand nodes in the requirement association network, and the demand type label, demand description and associated content element node information of each user demand node are extracted, and the core demand keywords are mined by a topic model algorithm, the core demand keywords are used to represent the demand points that the user pays most attention to; Feature extraction is performed on the content element nodes in the requirement association network, and the class label of the main content category node, the attribute description of the sub-content attribute node and the specific parameter value of the content parameter node are extracted, and the key attribute features that affect content generation are selected by a feature selection algorithm, the key attribute features include the function, material and specification of the content; Feature extraction is performed on the scene form nodes in the requirement association network, and the feature parameters of the presentation mode, structure type and modality combination mode of the scene form nodes are extracted, and the scene form preference type is determined by cluster analysis, the preference type includes the detail page form, the short video form and the segment form; The core demand keywords, key attribute features and scene form preference types are standardized to generate computable feature values; According to the importance of the core demand keywords, key attribute features and scene form preference types in content generation, weight parameters are set to weight the feature values, and a target feature vector is generated; The target feature vector is mapped to a high-dimensional semantic space to generate a context feature vector. 6.The content generation method based on a multi-modal large model according to claim 1, wherein, The element collaborative optimization processing is performed on the initial content framework set to obtain an optimized content framework that meets the constraints of the content element nodes, including: Analyzing the content element node constraint condition in the requirement association network, determining the presentation requirement of the content core attribute, the matching standard of the content specification parameter, and the creation rule of the content style feature; Extracting the text description unit, visual display unit, and semantic association unit contained in each initial content framework from the initial content framework set, analyzing the matching degree of each element unit and the content element node constraint condition, and calculating the matching degree by the content of the element unit and the fitting degree of the constraint condition; Replacing the element unit that does not meet the matching degree, selecting the element unit that meets the content element node constraint condition from the multi-modal element coupling pool for replacement, and maintaining the semantic association between the text description unit and the visual display unit during the replacement process; Adjusting the combination order and association strength of the text description unit, visual display unit, and semantic association unit, so that the content logic of the text description unit and the presentation logic of the visual display unit remain consistent, and the semantic association unit can accurately reflect the logical relationship between text and vision; Evaluating the element cooperativity of the adjusted content framework, calculating the coherence score between the text description units, the style consistency score between the visual display units, and the semantic matching degree score between the text description units and the visual display units; If the element cooperativity evaluation result meets the preset threshold, the adjusted content framework is selected as the candidate optimization framework; if not, the replacement and adjustment operations are repeated until the element cooperativity evaluation result meets the standard; Selecting the framework with the highest matching degree with the content element node constraint condition from the candidate optimization framework as the optimized content framework, which contains the cooperatively optimized text description sequence, visual display sequence, and semantic association rule set.

7. The content generation method based on a multi-modal large model according to claim 6, characterized in that, The element cooperativity evaluation of the adjusted content framework, the coherence score between the text description units, the style consistency score between the visual display units, and the semantic matching degree score between the text description units and the visual display units, include: Extracting the text description unit sequence in the adjusted content framework, analyzing the lexical cohesion information, grammatical association information, and semantic coherence information between adjacent text description units, calculating the coherence score through a pre-trained language model, and taking the coherence score as the coherence score between the text description units; Extracting the visual display unit sequence in the adjusted content framework, analyzing the consistency of the color keynote, composition style, and element layout of the visual display unit, calculating the style similarity through a visual feature comparison algorithm, and taking the style similarity as the style consistency score between the visual display units; Establishing the correspondence between the text description unit and the visual display unit, each text description unit corresponding to one or more visual display units, extracting the core semantics of the text description unit and the visual semantics of the visual display unit; Calculating the semantic distance between the core semantics and the visual semantics, the semantic distance being calculated by the Euclidean distance in the semantic vector space; According to the semantic distance is converted into semantic matching degree score, the semantic matching degree score and semantic distance is negatively correlated; The coherence score between the text description units, the style consistency score between the visual display units, and the semantic matching degree score between the text description units and the visual display units are weighted and calculated according to the preset weight parameters to obtain a comprehensive score of the element collaboration evaluation. 8.The content generation method based on a multi-modal large model according to claim 1, wherein, The target content set is generated according to the optimized content framework, and the target content set is pushed to the corresponding content distribution node to complete the content propagation operation, including: Parsing the text description sequence in the optimized content framework, combining each text description unit into complete text content according to semantic logic, the language style of the text content matches the user demand node in the demand association network; Parsing the visual display sequence in the optimized content framework, integrating each visual display unit according to the content composition rule to generate visual content that echoes the text content, and the presentation style of the visual content meets the requirements of the content element node in the demand association network; According to the semantic association rule set in the optimized content framework, the text content and the visual content are associated and combined to form a multi-modal content unit, each multi-modal content unit contains corresponding user demand node label, content element node label and scene form node label; Arranging and combining a plurality of multi-modal content units according to the structure type of the scene form node to generate a target content set containing different modal combinations; Retrieving content distribution node information matching the multi-modal content unit label in the target content set, the content distribution node information including node identification, distribution range and receiving preference; According to the receiving preference of the content distribution node, the multi-modal content units in the target content set are screened so that the multi-modal content units pushed to the content distribution node meet the receiving requirements of the node; The filtered target content set is pushed to the corresponding content distribution node, and the content distribution node receives the target content set and performs content display and propagation operations.

9. The content generation method based on a multi-modal large model according to claim 8, characterized in that, The parsing of the text description sequence in the optimized content framework, combining each text description unit into complete text content according to semantic logic, includes: Extracting each text description unit of the text description sequence in the optimized content framework, parsing the hierarchical identifier, text content and corresponding user demand node information of each text description unit; According to the semantic association relationship between the text description units, determine the combination order of the text description units; The text description units in the combination process are connected and handled, and preset connecting words, transition sentences or paragraphs are added; Analyze the demand type label and demand description of the user demand node in the demand association network, determine the corresponding language style features, the language style features include formal degree, emotional tendency and brevity; According to the language style features, adjust the vocabulary selection, sentence structure and expression tone of the text content, output complete text content, the text content contains multiple paragraphs, each paragraph corresponds to a core semantic theme.

10. A multi-modal large model based content generation system, characterized by, The application relates to a device for implementing the content generation method based on a multimodal large model, comprising a processor and a memory, wherein the memory is connected with the processor, the memory is used for storing programs, instructions or codes, and the processor is used for executing the programs, instructions or codes in the memory to realize the content generation method based on a multimodal large model according to any one of claims 1-9.

Images