Intelligent social media post generation method and system based on multi-modal architecture

Abstract

The invention relates to an intelligent social media post generation system and method based on a multi-modal architecture, and the method comprises the steps: firstly carrying out the subdivision of a complex demand inputted by a user through a user demand disassembly module, dividing a task into a plurality of subtasks, and constructing a preliminary thinking chain; on the basis, a multi-angle demand analysis module is used for carrying out emotion and style preference analysis on the demand, and a sub-task list conforming to the intention of the user is generated. Next, the system enters a multi-modal content interpretation module, comprehensively analyzes the uploaded data through a multi-modal large language model, and extracts a feature vector of each modal; and the content vectorization and RAG matching module performs retrieval enhancement generation matching on vectorization features, and extracts content similar to requirements from social media as a reference. And the thinking chain construction module performs logic organization and multi-angle optimization analysis on the reference content. And the social media post generation module performs style and emotion adaptive adjustment on the logically structured content to generate high-quality social media posts meeting user requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent social media post generation, and in particular to an intelligent social media post generation method and system based on a multimodal architecture. Background Art

[0002] The traditional way of generating social media posts is for the author to conceive, write, edit and proofread the content. This method can ensure the originality and personalization of the content and better express personal opinions and style. Editing and proofreading can ensure the accuracy and quality of the content, but it requires high creativity and time investment, and is inefficient. Some social media platforms provide template functions, where users can select a suitable template, fill in the corresponding content, and generate posts. This method can save time, but may lack personalization.

[0003] Therefore, the present invention hopes to improve the above-mentioned traditional way of generating social media posts, and construct more attractive and influential posts by analyzing and imitating high-quality content in social media. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent social media post generation method and system based on a multimodal architecture, so as to improve the above-mentioned traditional social media post generation method, and construct more attractive and disseminating posts by analyzing and imitating high-quality content in social media.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] The intelligent social media post generation method based on multimodal architecture includes the following steps:

[0007] S1: The user inputs the demand through the demand input module, and the data identification module identifies the demand data input by the user and determines whether the user input is text input. If so, execute step S2, if not, execute step S4;

[0008] S2: Based on the user's input requirements, the input requirements are decomposed into complex tasks, all the complex tasks of the user's requirements are decomposed into multiple subtasks, and a subtask thinking chain is constructed, and it is determined whether the subtask thinking chain captures all the user's requirements. If so, step S3 is executed, if not, the subtask thinking chain is reconstructed until the subtask thinking chain captures all the user's requirements;

[0009] S3: Decomposing all complex tasks required by users into multiple subtasks to create a subtask list, constructing and training a multimodal analysis model, and inputting the subtasks into the multimodal analysis model in sequence based on the subtask list;

[0010] S4: the multimodal analysis model performs multimodal content understanding analysis on the user input requirements, extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode;

[0011] S5: creating a retrieval enhancement generation matching model, and performing retrieval enhancement generation matching on the feature vector of each modality through the retrieval enhancement generation matching model, and extracting content matching the user input requirements from social media as reference content;

[0012] S6: The content generation large language model semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles;

[0013] S7: The social media post generation module adapts the style and emotion of the logically structured content optimized from multiple angles, generates and outputs social media posts that meet user needs.

[0014] Preferably, the specific process in step S2 is as follows:

[0015] S21: Develop task decomposition standards based on user input requirements;

[0016] S22: Decomposing the user's needs based on the formulated task decomposition standard to obtain multiple subtasks, wherein the subtasks are tasks of different dimensions, thereby achieving dimensionality reduction processing of the initial complex task;

[0017] S23: Construct a subtask thinking chain from all subtasks based on the specified order, where there are logical connections and dependencies between adjacent subtasks in the subtask thinking chain, and the response of the previous subtask in the subtask thinking chain serves as the input of the next subtask.

[0018] Preferably, there are specified logical relationships between subtasks of different dimensions, and all requirements of the initial complex task are met after all subtasks are logically organized by induction or deduction.

[0019] Preferably, the following specific process is also included before step S4:

[0020] S411: Initializing parameters of the multimodal analysis model;

[0021] S412: inputting known different types of data into the initialized multimodal analysis model to train it, and optimizing the parameters of the initialized multimodal analysis model;

[0022] Among them, step S421 optimizes the parameters of the initialized multimodal analysis model by freezing the specified basic parameters of the multimodal analysis model, and after introducing the specified specific parameters, inputting known different types of data to perform multimodal analysis model training, thereby achieving parameter optimization of the multimodal analysis model.

[0023] Preferably, step S4 includes the following specific process:

[0024] S421: performing multimodal content understanding analysis on the user input demand based on the multimodal analysis model after parameter optimization, obtaining specific modal information of the user input demand, and dividing it into different modal data sets including a text modal data set, a video modal data set, or an audio modal data set;

[0025] S422: performing text feature extraction on the text modality dataset, performing video feature extraction on the video modality dataset, or performing audio feature extraction on the audio modality dataset;

[0026] S423: Perform feature conversion on the extracted text features, video features, or audio features to obtain a text modality feature vector, a video modality feature vector, or an audio modality feature vector.

[0027] Preferably, the process of extracting text features is:

[0028] Count the keywords in the text and the frequencies corresponding to the keywords, and calculate the information entropy of each keyword. The information entropy calculation formula is: H(X)=E[I(X)]=E[-ln(P(X))];

[0029] Where I(X) is the information content of keyword X, E[x] is the expected function of variable x, and P(x) is the probability mass function of variable x;

[0030] The information gain of each keyword is calculated based on the information entropy of each keyword, and the calculation formula of the information gain is: G=H(X)-H(X|a);

[0031] Among them, H(X|a) represents the conditional entropy of keyword X under given condition a;

[0032] Sort by the information gain of each keyword and extract the first n keywords as the features of the text.

[0033] Preferably, the process of extracting audio features is:

[0034] Cut the audio content into several segments of specified length, perform Fourier transform on each audio segment, and obtain the spectrum information of each audio segment. The specific formula for Fourier transform on each audio segment is:

[0035]

[0036] Among them, s(τ) is the audio signal, w(t-τ) is the window function, and f is the audio frequency;

[0037] The process of video feature extraction is as follows:

[0038] The video is decomposed into single-frame images, and features are extracted for the specified frame images based on a convolutional neural network model, wherein the convolutional neural network model is a 3D convolutional neural network model, which includes 3 convolutional layers, 1 fully connected layer and 2 pooling layers, at least two convolution kernels of the three convolutional layers are 3D convolutional layers, and the sizes of the convolution kernels of the two 3D convolutional layers are 7×6×3 and 7×7×3, each node of the fully connected layer is connected to all nodes of its previous layer, and the pooling layer calculates the average value of all values ​​in the window as the representative value of the specified image area.

[0039] Preferably, the specific process of the retrieval enhancement generation matching model in step S5 performing retrieval enhancement generation matching on the feature vector of each modality is as follows:

[0040] S51: converting the video modality feature vector or the audio modality feature vector into a text feature vector, and inputting the text feature vector into the retrieval enhancement generation matching model;

[0041] S52: obtaining a specified text segment from the text feature vector, and performing block processing of a specified size on the specified text segment;

[0042] S53: Representing the text segments after block processing as a number of coordinate points in a continuous feature vector space, and matching similar objects by calculating the distances between the coordinate points corresponding to the specified feature vectors.

[0043] In a second aspect, a smart social media post generation system based on a multimodal architecture is provided, which is used to implement any one of the smart social media post generation methods based on a multimodal architecture, including a data input module, a data identification module, a task decomposition module, a content generation large language model, a feature vector conversion module, a retrieval enhancement generation matching model, and an adaptation adjustment module, wherein the data input module is connected to the data identification module, the data identification module is connected to the task decomposition module, the task decomposition module is connected to the content generation large language model, the content generation large language model is connected to the feature vector conversion module, the feature vector conversion module is connected to the retrieval enhancement generation matching model, and the retrieval enhancement generation matching model is connected to the adaptation adjustment module;

[0044] The data input module is used to input user demand data;

[0045] The data identification module is used to identify the required data input by the user and determine whether the user input is text input;

[0046] The task decomposition module is used to decompose the input requirements into complex tasks based on the user's input requirements, obtain multiple subtasks, create a subtask list, construct a subtask thinking chain, and determine whether the subtask thinking chain captures all the user's requirements;

[0047] The content generates a large language model, performs multimodal content understanding and analysis on the user input requirements, and extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode;

[0048] The retrieval enhancement generation matching model performs retrieval enhancement generation matching on the feature vector of each modality, extracts content matching the user input requirements from social media as reference content, and semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles;

[0049] The adaptation and adjustment module is used to adapt the style and emotion of the logically structured content after multi-angle optimization, generate and output social media posts that meet user needs.

[0050] The beneficial effects of the present invention include:

[0051] The intelligent social media post generation method and system based on a multimodal architecture provided by the present invention first subdivides the complex needs input by the user through the user demand decomposition module, divides the task into multiple subtasks and constructs a preliminary thinking chain. On this basis, the multi-angle demand analysis module is used to analyze the emotions and style preferences of the needs, and a subtask list that meets the user's intentions is generated. Next, the system enters the multimodal content interpretation module, and through the multimodal large language model, the uploaded data is comprehensively analyzed to extract the feature vectors of each modality. The content vectorization and RAG matching module retrieves, enhances and generates matches for the vectorized features, and extracts content similar to the needs from social media as a reference. The thinking chain construction module logically organizes and optimizes the reference content from multiple angles. The social media post generation module adapts the style and emotion of the logically structured content to generate high-quality social media posts that meet user needs.

[0052] First, based on the user's input requirements, a task decomposition standard is formulated; based on the formulated task decomposition standard, the user's requirements are decomposed to obtain multiple subtasks, which are tasks of different dimensions, realizing the dimensionality reduction of the initial complex tasks and effectively reducing the difficulty of demand processing. Based on the specified order, all subtasks are constructed into a subtask thinking chain. There are logical connections and dependencies between adjacent subtasks in the subtask thinking chain. The response of the previous subtask in the subtask thinking chain is used as the input of the next subtask, avoiding errors in the final matching of similar objects due to task decomposition.

[0053] Secondly, the feature extraction module is set up with text feature extraction sub-module, audio feature extraction sub-module, video feature extraction sub-module and interactive feature extraction sub-module to perform targeted feature extraction on text features, audio features, video features and interactive features respectively, effectively improving the accuracy of feature extraction, and combined with the subsequent similarity calculation module to perform similarity calculation on different features respectively, to further ensure the accuracy of the similarity calculation results, and further ensure the accuracy of the recommendation results of the recommendation module.

[0054] Thirdly, the parameters of the multimodal analysis model are initialized; known different types of data are input into the initialized multimodal analysis model to train it, and the parameters of the initialized multimodal analysis model are optimized to promote the improvement of the accuracy of the subsequent object matching process.

[0055] Finally, the video modality feature vector or the audio modality feature vector is converted into a text feature vector and input into the retrieval enhancement generation matching model; the specified text segment is obtained from the text feature vector, and the specified text segment is divided into blocks of specified size; the text segment after block processing is represented as a number of coordinate points in a continuous feature vector space, and similar objects are matched by calculating the distance between the coordinate points corresponding to the specified feature vector, which effectively improves the accuracy of matching similar objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the process of the intelligent social media post generation method based on a multimodal architecture of the present invention.

[0057] Figure 2 A schematic diagram of the process of searching, enhancing and generating matches for feature vectors of each modality according to the present invention. DETAILED DESCRIPTION

[0058] The following is combined with Figure 1-Figure 2 The present invention is further described in detail:

[0059] Example 1

[0060] See attached Figure 1As shown, the intelligent social media post generation method based on the multimodal architecture includes the following steps:

[0061] S1: The user inputs the demand through the demand input module, and the data identification module identifies the demand data input by the user and determines whether the user input is text input. If so, execute step S2, if not, execute step S4;

[0062] S2: Based on the user's input requirements, the input requirements are decomposed into complex tasks, all the complex tasks of the user's requirements are decomposed into multiple subtasks, and a subtask thinking chain is constructed, and it is determined whether the subtask thinking chain captures all the user's requirements. If so, step S3 is executed, if not, the subtask thinking chain is reconstructed until the subtask thinking chain captures all the user's requirements;

[0063] S3: Decomposing all complex tasks required by users into multiple subtasks to create a subtask list, constructing and training a multimodal analysis model, and inputting the subtasks into the multimodal analysis model in sequence based on the subtask list;

[0064] S4: the multimodal analysis model performs multimodal content understanding analysis on the user input requirements, extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode;

[0065] S5: creating a retrieval enhancement generation matching model, and performing retrieval enhancement generation matching on the feature vector of each modality through the retrieval enhancement generation matching model, and extracting content matching the user input requirements from social media as reference content;

[0066] S6: The content generation large language model semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles;

[0067] S7: The social media post generation module adapts the style and emotion of the logically structured content optimized from multiple angles, generates and outputs social media posts that meet user needs.

[0068] The intelligent social media post generation method based on multimodal architecture of the present invention first subdivides the complex needs input by the user through the user demand disassembly module, divides the task into multiple subtasks and constructs a preliminary thinking chain. On this basis, the multi-angle demand analysis module is used to analyze the emotions and style preferences of the needs, and a subtask list that meets the user's intention is generated. Next, the system enters the multimodal content interpretation module, and the uploaded text, pictures, videos and audio content are comprehensively analyzed through the multimodal large language model to extract the feature vectors of each modality. Subsequently, the content vectorization and RAG matching module performs retrieval enhancement generation (RAG) matching on these vectorized features, and extracts content similar to the needs from social media as a reference. The thinking chain construction module logically organizes the reference content, semantically expands the content through the chain reasoning model, and performs multi-angle optimization analysis from aspects such as attractiveness, readability and emotional resonance. Finally, the social media post generation module adapts the style and emotion of the logically structured content according to the characteristics of different platforms, generates high-quality social media posts that meet user needs, enhances user interaction, and effectively reduces the time and energy required for content creation.

[0069] In this embodiment, the specific process in step S2 is as follows:

[0070] S21: Develop task decomposition standards based on user input requirements;

[0071] S22: Decomposing the user's needs based on the formulated task decomposition standard to obtain multiple subtasks, wherein the subtasks are tasks of different dimensions, thereby achieving dimensionality reduction processing of the initial complex task;

[0072] S23: Construct a subtask thinking chain from all subtasks based on the specified order. There are logical connections and dependencies between adjacent subtasks on the subtask thinking chain. The response of the previous subtask in the subtask thinking chain is used as the input of the next subtask. There are specified logical relationships between subtasks of different dimensions. After all subtasks are logically organized by induction or deduction, all requirements of the initial complex task are met. Based on the user's input requirements, a task decomposition standard is formulated; based on the formulated task decomposition standard, the user's requirements are decomposed to obtain multiple subtasks, and the subtasks are tasks of different dimensions, which realizes the dimensionality reduction of the initial complex task and effectively reduces the difficulty of demand processing. Construct a subtask thinking chain from all subtasks based on the specified order. There are logical connections and dependencies between adjacent subtasks on the subtask thinking chain. The response of the previous subtask in the subtask thinking chain is used as the input of the next subtask, avoiding errors in the final matching of similar objects due to task decomposition.

[0073] Example 2

[0074] On the basis of Example 1, the following specific process is also included before step S4:

[0075] S411: Initializing parameters of the multimodal analysis model;

[0076] S412: inputting known different types of data into the initialized multimodal analysis model to train it, and optimizing the parameters of the initialized multimodal analysis model;

[0077] Among them, step S421 optimizes the parameters of the initialized multimodal analysis model by freezing the specified basic parameters of the multimodal analysis model, and after introducing the specified specific parameters, inputting known different types of data to perform multimodal analysis model training, thereby achieving parameter optimization of the multimodal analysis model.

[0078] Initialize the parameters of the multimodal analysis model; input known different types of data into the initialized multimodal analysis model to train it, optimize the parameters of the initialized multimodal analysis model, and promote the improvement of the accuracy of the subsequent object matching process.

[0079] In this embodiment, step S4 includes the following specific processes:

[0080] S421: performing multimodal content understanding analysis on the user input demand based on the multimodal analysis model after parameter optimization, obtaining specific modal information of the user input demand, and dividing it into different modal data sets including a text modal data set, a video modal data set, or an audio modal data set;

[0081] S422: performing text feature extraction on the text modality dataset, performing video feature extraction on the video modality dataset, or performing audio feature extraction on the audio modality dataset;

[0082] S423: Perform feature conversion on the extracted text features, video features, or audio features to obtain a text modality feature vector, a video modality feature vector, or an audio modality feature vector.

[0083] Example 3

[0084] On the basis of Example 1 or Example 2, the process of extracting text features is as follows:

[0085] Count the keywords in the text and the frequencies corresponding to the keywords, and calculate the information entropy of each keyword. The information entropy calculation formula is: H(X)=E[I(X)]=E[-ln(P(X))];

[0086] Where I(X) is the information content of keyword X, E[x] is the expected function of variable x, and P(x) is the probability mass function of variable x;

[0087] The information gain of each keyword is calculated based on the information entropy of each keyword, and the calculation formula of the information gain is: G=H(X)-H(X|a);

[0088] Among them, H(X|a) represents the conditional entropy of keyword X under given condition a;

[0089] Sort by the information gain of each keyword and extract the first n keywords as the features of the text.

[0090] In this embodiment, the process of extracting audio features is as follows:

[0091] Cut the audio content into several segments of specified length, perform Fourier transform on each audio segment, and obtain the spectrum information of each audio segment. The specific formula for Fourier transform on each audio segment is:

[0092]

[0093] Among them, s(τ) is the audio signal, w(t-τ) is the window function, and f is the audio frequency;

[0094] The process of video feature extraction is as follows:

[0095] The video is decomposed into single-frame images, and features are extracted for the specified frame images based on a convolutional neural network model, wherein the convolutional neural network model is a 3D convolutional neural network model, which includes 3 convolutional layers, 1 fully connected layer and 2 pooling layers, at least two convolution kernels of the three convolutional layers are 3D convolutional layers, and the sizes of the convolution kernels of the two 3D convolutional layers are 7×6×3 and 7×7×3, each node of the fully connected layer is connected to all nodes of its previous layer, and the pooling layer calculates the average value of all values ​​in the window as the representative value of the specified image area.

[0096] By setting the feature extraction module to have text feature extraction submodule, audio feature extraction submodule, video feature extraction submodule and interactive feature extraction submodule, targeted feature extraction is performed on text features, audio features, video features and interactive features respectively, thereby effectively improving the accuracy of feature extraction, and combining with the subsequent similarity calculation module to perform similarity calculation on different features respectively, thereby further ensuring the accuracy of the similarity calculation results and further ensuring the accuracy of the recommendation results of the recommendation module.

[0097] Example 4

[0098] On the basis of Example 1 or Example 2 or Example 3, see Figure 2 The specific process of the retrieval enhancement generation matching model in step S5 performing retrieval enhancement generation matching on the feature vector of each modality is as follows:

[0099] S51: converting the video modality feature vector or the audio modality feature vector into a text feature vector, and inputting the text feature vector into the retrieval enhancement generation matching model;

[0100] S52: obtaining a specified text segment from the text feature vector, and performing block processing of a specified size on the specified text segment;

[0101] S53: Representing the text segments after block processing as a number of coordinate points in a continuous feature vector space, and matching similar objects by calculating the distances between the coordinate points corresponding to the specified feature vectors.

[0102] The video modality feature vector or the audio modality feature vector is converted into a text feature vector and input into the retrieval enhancement generation matching model; a specified text segment is obtained from the text feature vector, and the specified text segment is divided into blocks of specified sizes; the text segment after block processing is represented as a number of coordinate points in a continuous feature vector space, and similar objects are matched by calculating the distance between the coordinate points corresponding to the specified feature vector, which effectively improves the accuracy of matching similar objects.

[0103] An intelligent social media post generation system based on a multimodal architecture, used to implement any one of the intelligent social media post generation methods based on a multimodal architecture, comprising a data input module, a data identification module, a task decomposition module, a content generation large language model, a feature vector conversion module, a retrieval enhancement generation matching model, and an adaptation adjustment module, wherein the data input module is connected to the data identification module, the data identification module is connected to the task decomposition module, the task decomposition module is connected to the content generation large language model, the content generation large language model is connected to the feature vector conversion module, the feature vector conversion module is connected to the retrieval enhancement generation matching model, and the retrieval enhancement generation matching model is connected to the adaptation adjustment module;

[0104] The data input module is used to input user demand data;

[0105] The data identification module is used to identify the required data input by the user and determine whether the user input is text input;

[0106] The task decomposition module is used to decompose the input requirements into complex tasks based on the user's input requirements, obtain multiple subtasks, create a subtask list, construct a subtask thinking chain, and determine whether the subtask thinking chain captures all the user's requirements;

[0107] The content generates a large language model, performs multimodal content understanding and analysis on the user input requirements, and extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode;

[0108] The retrieval enhancement generation matching model performs retrieval enhancement generation matching on the feature vector of each modality, extracts content matching the user input requirements from social media as reference content, and semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles;

[0109] The adaptation and adjustment module is used to adapt the style and emotion of the logically structured content after multi-angle optimization, generate and output social media posts that meet user needs.

[0110] In summary, the intelligent social media post generation method and system based on multimodal architecture provided by the present invention formulates task decomposition standards based on user input requirements; based on the formulated task decomposition standards, the user's requirements are decomposed to obtain multiple subtasks, and the subtasks are tasks of different dimensions, which realizes the dimensionality reduction processing of the initial complex task and effectively reduces the difficulty of demand processing. Based on the specified order, all subtasks are constructed into subtask thinking chains, and there are logical connections and dependencies between adjacent subtasks on the subtask thinking chain. The response of the previous subtask in the subtask thinking chain is used as the input of the next subtask, avoiding errors when task decomposition leads to the final matching of similar objects. The feature extraction module is set to a text feature extraction submodule, an audio feature extraction submodule, a video feature extraction submodule and an interactive feature extraction submodule to perform targeted feature extraction on text features, audio features, video features and interactive features respectively, effectively improving the accuracy of feature extraction, and combining the subsequent similarity calculation module to perform similarity calculation on different features respectively, further ensuring the accuracy of the similarity calculation results, and further ensuring the accuracy of the recommendation results of the recommendation module.

[0111] The parameters of the multimodal analysis model are initialized; different types of known data are input into the initialized multimodal analysis model for training, and the parameters of the initialized multimodal analysis model are optimized to promote the improvement of the accuracy of the subsequent object matching process. The video modality feature vector or the audio modality feature vector is converted into a text feature vector and input into the retrieval enhancement generation matching model; the specified text segment is obtained from the text feature vector, and the specified text segment is divided into blocks of specified size; the text segment after block processing is represented as a number of coordinate points in a continuous feature vector space, and the distance between the coordinate points corresponding to the specified feature vector is calculated to match similar objects, which effectively improves the accuracy of matching similar objects.

Claims

1. An intelligent social media post generation method based on a multimodal architecture, characterized in that: The following steps are involved: S1: The user inputs the demand through the demand input module, and the data identification module identifies the demand data input by the user and determines whether the user input is text input. If so, execute step S2, if not, execute step S4; S2: Based on the user's input requirements, the input requirements are decomposed into complex tasks, all the complex tasks of the user's requirements are decomposed into multiple subtasks, and a subtask thinking chain is constructed, and it is determined whether the subtask thinking chain captures all the user's requirements. If so, step S3 is executed, if not, the subtask thinking chain is reconstructed until the subtask thinking chain captures all the user's requirements; S3: Decomposing all complex tasks required by users into multiple subtasks to create a subtask list, constructing and training a multimodal analysis model, and inputting the subtasks into the multimodal analysis model in sequence based on the subtask list; S4: the multimodal analysis model performs multimodal content understanding analysis on the user input requirements, extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode; S5: creating a retrieval enhancement generation matching model, and performing retrieval enhancement generation matching on the feature vector of each modality through the retrieval enhancement generation matching model, and extracting content matching the user input requirements from social media as reference content; S6: The content generation large language model semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles; S7: The social media post generation module adapts the style and emotion of the logically structured content optimized from multiple angles, generates and outputs social media posts that meet user needs.

2. The method for generating intelligent social media posts based on a multimodal architecture according to claim 1, characterized in that: The specific process in step S2 is as follows: S21: Develop task decomposition standards based on user input requirements; S22: Decomposing the user's needs based on the formulated task decomposition standard to obtain multiple subtasks, wherein the subtasks are tasks of different dimensions, thereby achieving dimensionality reduction processing of the initial complex task; S23: Construct a subtask thinking chain from all subtasks based on the specified order, where there are logical connections and dependencies between adjacent subtasks in the subtask thinking chain, and the response of the previous subtask in the subtask thinking chain serves as the input of the next subtask.

3. The method for generating intelligent social media posts based on a multimodal architecture according to claim 2, characterized in that: There are specified logical relationships between subtasks of different dimensions. After all subtasks are logically organized by induction or deduction, all requirements of the initial complex task are met.

4. The method for generating intelligent social media posts based on a multimodal architecture according to claim 1, characterized in that: Before step S4, the following specific process is also included: S411: Initializing parameters of the multimodal analysis model; S412: inputting known different types of data into the initialized multimodal analysis model to train it, and optimizing the parameters of the initialized multimodal analysis model; Among them, step S421 optimizes the parameters of the initialized multimodal analysis model by freezing the specified basic parameters of the multimodal analysis model, and after introducing the specified specific parameters, inputting known different types of data to perform multimodal analysis model training, thereby achieving parameter optimization of the multimodal analysis model.

5. The method for generating intelligent social media posts based on a multimodal architecture according to claim 4, characterized in that: Step S4 includes the following specific processes: S421: performing multimodal content understanding analysis on the user input demand based on the multimodal analysis model after parameter optimization, obtaining specific modal information of the user input demand, and dividing it into different modal data sets including a text modal data set, a video modal data set, or an audio modal data set; S422: performing text feature extraction on the text modality dataset, performing video feature extraction on the video modality dataset, or performing audio feature extraction on the audio modality dataset; S423: Perform feature conversion on the extracted text features, video features, or audio features to obtain a text modality feature vector, a video modality feature vector, or an audio modality feature vector.

6. The method for generating intelligent social media posts based on a multimodal architecture according to claim 5, characterized in that: The process of text feature extraction is: Count the keywords in the text and the frequencies corresponding to the keywords, and calculate the information entropy of each keyword. The information entropy calculation formula is: H(X)=E[I(X)]=E[-ln(P(X))]; Where I(X) is the information content of keyword X, E[x] is the expected function of variable x, and P(x) is the probability mass function of variable x; The information gain of each keyword is calculated based on the information entropy of each keyword, and the calculation formula of the information gain is: G=H(X)-H(X|a); Among them, H(X|a) represents the conditional entropy of keyword X under given condition a; Sort by the information gain of each keyword and extract the first n keywords as the features of the text.

7. The method for generating intelligent social media posts based on a multimodal architecture according to claim 5, characterized in that: The process of audio feature extraction is: Cut the audio content into several segments of specified length, perform Fourier transform on each audio segment, and obtain the spectrum information of each audio segment. The specific formula for Fourier transform on each audio segment is: Among them, s(τ) is the audio signal, w(t-τ) is the window function, and f is the audio frequency; The process of video feature extraction is: The video is decomposed into single-frame images, and features are extracted for the specified frame images based on a convolutional neural network model, wherein the convolutional neural network model is a 3D convolutional neural network model, which includes 3 convolutional layers, 1 fully connected layer and 2 pooling layers, at least two convolution kernels of the three convolutional layers are 3D convolutional layers, and the sizes of the convolution kernels of the two 3D convolutional layers are 7×6×3 and 7×7×3, each node of the fully connected layer is connected to all nodes of its previous layer, and the pooling layer calculates the average value of all values ​​in the window as the representative value of the specified image area.

8. The method for generating intelligent social media posts based on a multimodal architecture according to claim 5, characterized in that: The specific process of the retrieval enhancement generation matching model in step S5 performing retrieval enhancement generation matching on the feature vector of each modality is as follows: S51: converting the video modality feature vector or the audio modality feature vector into a text feature vector, and inputting the text feature vector into the retrieval enhancement generation matching model; S52: obtaining a specified text segment from the text feature vector, and performing block processing of a specified size on the specified text segment; S53: Representing the text segments after block processing as a number of coordinate points in a continuous feature vector space, and matching similar objects by calculating the distances between the coordinate points corresponding to the specified feature vectors.

9. An intelligent social media post generation system based on a multimodal architecture, used to implement the intelligent social media post generation method based on a multimodal architecture as described in any one of claims 1 to 8, characterized in that: It includes a data input module, a data identification module, a task decomposition module, a content generation large language model, a feature vector conversion module, a retrieval enhancement generation matching model, and an adaptation adjustment module. The data input module is connected to the data identification module, the data identification module is connected to the task decomposition module, the task decomposition module is connected to the content generation large language model, the content generation large language model is connected to the feature vector conversion module, the feature vector conversion module is connected to the retrieval enhancement generation matching model, and the retrieval enhancement generation matching model is connected to the adaptation adjustment module; The data input module is used to input user demand data; The data identification module is used to identify the required data input by the user and determine whether the user input is text input; The task decomposition module is used to decompose the input requirements into complex tasks based on the user's input requirements, obtain multiple subtasks, create a subtask list, construct a subtask thinking chain, and determine whether the subtask thinking chain captures all the user's requirements; The content generates a large language model, performs multimodal content understanding and analysis on the user input requirements, and extracts features of each mode of the multimodal content of the input requirements, converts the extracted features of each mode to obtain a feature vector, and obtains a feature vector of each mode; The retrieval enhancement generation matching model performs retrieval enhancement generation matching on the feature vector of each modality, extracts content matching the user input requirements from social media as reference content, and semantically expands the reference content through chain reasoning and organizes new logic for the expanded reference content, thereby optimizing the reference content from multiple angles; The adaptation and adjustment module is used to adapt the style and emotion of the logically structured content after multi-angle optimization, generate and output social media posts that meet user needs.

Images