Creation method and system based on multi-modal data processing and dynamic task planning

Abstract

The invention provides an intelligent content creation method and system based on multi-modal data processing and dynamic task planning. After a user submits social media information, the system automatically captures and collects historical social media data of the user, carries out deep analysis on the data through a multi-modal model, and constructs and stores a user portrait in a database. When a user creates contents, the system dynamically calls an adaptive large model according to a multi-mode identification result: processing video contents by using a video analysis model, converting audio data by using an audio large model, analyzing pictures by using an image analysis model and the like; and the large language model is accurately generated based on user problems and integrated analysis results. And the system can record the interactive dialogue and behavior data of the user. And the dynamic task planning module dynamically analyzes and calls a hotspot tracking tool or a network retrieval tool to analyze contents according to an analysis result of the multi-modal data uploaded by the user, so that personalized, intelligent and precise content creation is realized, and the responsivity and the precision of creation are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of multimodal data processing, and in particular to a creation method and system based on multimodal data processing and dynamic task planning. Background Art

[0002] In recent years, with the rapid development of social media and short video platforms, influencers have become digital content creators with great influence. However, the complexity and diversity of content creation have put forward higher requirements on the efficiency and effectiveness of influencers' creation. In the process of content creation, influencers need to process multiple data forms such as video, pictures, audio and text, and customize the content according to the needs of different social media platforms. This has led to an increase in the demand for intelligent tools to help influencers efficiently process multimodal data and generate targeted creation suggestions. With the maturity of large language model (LLM) technology, artificial intelligence has performed particularly well in text generation and natural language processing. At the same time, large models in fields such as video understanding, picture understanding and audio processing have also gradually matured. It provides new technical means for influencer content creation, which can intelligently process large amounts of multimodal data.

[0003] However, multimodal data processing in existing technologies usually lacks flexibility and can only process single or limited types of data. For example, it is difficult for existing systems to process video, pictures, audio and text at the same time, resulting in limited support for content creation, especially in scenarios that require cross-modal processing, where the system is prone to functional faults. Existing AI content creation systems usually adopt fixed task processes and lack flexible dynamic task planning capabilities. When faced with complex content generation tasks, fixed processes are prone to inefficient tool selection and execution, and cannot be optimized and adjusted according to the actual task status, limiting the creation efficiency. Although some existing systems can generate content suggestions, they lack personalized recommendations based on user-specific data (such as historical creation records, preferences, etc.), and the generated content is often generalized, making it difficult to truly meet the user's creative style or audience needs. Most existing systems do not have real-time feedback and self-optimization mechanisms during task execution. If inefficiency or unsatisfactory results are encountered during task execution, the system cannot automatically adjust the task execution process and tool combination, resulting in a decrease in overall creation quality and efficiency. Many existing content creation systems do not have real-time hotspot tracking capabilities. Even if they are partially implemented, they are not deeply integrated with user portrait data, making it difficult to generate targeted strategic recommendations. This makes it difficult for influencers to grasp hot topics in a timely manner, affecting traffic and user interaction.

[0004] Therefore, it is necessary to improve the multimodal data processing process in the existing technology in order to provide comprehensive multimodal data processing, realize dynamic task planning and intelligent tool selection, and provide personalized creation suggestions based on user portrait data, so as to achieve self-optimization and the combination of hotspot tracking and strategy generation, thereby ultimately improving the intelligence and personalization level of content creation, and solving the shortcomings of the existing technology in multimodal data processing, task planning, self-optimization and hotspot tracking, which is a technical problem that urgently needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a creation method and system based on multimodal data processing and dynamic task planning, so as to improve the multimodal data processing process in the prior art, so as to provide comprehensive multimodal data processing, realize dynamic task planning and intelligent tool selection, and provide personalized creation suggestions based on user portrait data, so as to realize self-optimization and the combination of hot spot tracking and strategy generation, and finally improve the intelligence and personalization level of content creation.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, a creation method based on multimodal data processing and dynamic task planning is provided, comprising the following steps:

[0008] S1: After the user submits social media information, the system automatically captures and collects the user's historical social media data, conducts in-depth analysis of the historical social media data through a multimodal model, and constructs and stores the user portrait in the database;

[0009] S2: When users create content, the system dynamically calls the adapted large model based on the multimodal recognition results: the video analysis model is used to process video content, the audio large model is used to convert audio data, and the image analysis model is used to analyze pictures. The large language model is based on user questions, combined with user portraits, and integrated with analysis results for accurate generation.

[0010] S3: The system records the user's interactive conversations and behavior data and stores them as historical data. The historical data provides a personalized basis for subsequent content generation and optimizes the system's dynamic task planning through user behavior analysis.

[0011] S4: The dynamic task planning module dynamically analyzes and calls hotspot tracking tools or network search tools to analyze content based on the analysis results of multimodal data uploaded by users, so as to generate accurate analysis content and realize personalized and intelligent content creation.

[0012] Preferably, the specific process of establishing a user's exclusive portrait database based on social media historical data in step S1 is as follows:

[0013] S11: Determine the target features and required dimensions for constructing user portraits. The required dimensions are the user's specific information dimensions, including the user's age, gender, region, and interests.

[0014] S12: Obtain the user's social media history data, and extract their behavioral characteristics, emotional preferences, and demographic information based on their social media history data pictures and video texts;

[0015] S13: Perform variable analysis on the extracted behavioral characteristics, emotional preferences, and demographic information to identify their differentiated behavioral patterns, perform role clustering, and form user portraits based on role clustering.

[0016] Preferably, it also includes a monitoring process, specifically: performing task execution monitoring in steps S1-S4, and judging whether the task execution result meets expectations or whether there is a better execution path; when the task execution result does not meet expectations or there is a better execution path, starting the self-optimization module, and the self-optimization module automatically adjusts the task process and tool combination.

[0017] Preferably, the exclusive portrait database in step S3 includes historical behavior data, content preferences and fan portraits, and the personalized content creation suggestions in step S4 include complete post copy, video analysis report or hot spot tracking report.

[0018] Preferably, the specific process of the data identification module in step S2 identifying the type of the multimodal data is as follows:

[0019] S21: inputting the data set to be identified into the data identification module, and sequentially calculating the Manhattan distance between the data set to be identified and the training set used by the data identification module during training;

[0020] S22: According to the calculated Manhattan distance, select the k neighbor data points closest to the data set to be identified, that is, select the k data points closest to the target point;

[0021] S23: Perform weighted calculation based on the Manhattan distances of the selected k neighbor data and the corresponding labels to obtain a predicted value, and determine the type of the data to be identified according to the predicted value.

[0022] Preferably, the process of calculating the Manhattan distance in step S21 is:

[0023] S211: preprocessing the dataset to be identified and the training set, removing noise data and invalid data in the dataset to be identified and the training set, and extracting designated key features representing the data from the dataset to be identified and the training set respectively;

[0024] S212: Convert the extracted specified key features of the to-be-recognized data set into an n-dimensional feature value vector V1=(x1, x2, ···, x n ), and convert the specified key features of the training set into an n-dimensional eigenvalue vector V2 = (y1, y2, ···, y n );

[0025] S213: Calculate the Manhattan distance between the dataset to be identified and the training set. The specific calculation formula of the Manhattan distance is as follows:

[0026]

[0027] Among them, dm represents the Manhattan distance value, x1,x2,···,x n Assign key feature values ​​to each item of the data set to be identified, y1,y2,···,y n Assign key feature values ​​to each item in the training set.

[0028] Preferably, the process of parsing the video data by the video parsing model in step S2 is:

[0029] S221: Convert the acquired binary data of the video data into a picture sequence in RGB format for each frame;

[0030] S222: inputting each frame of the image sequence in RGB format into the video parsing model in sequence for inference, and locking the target of interest from each frame of the image sequence in RGB format, including the target type, credibility and target position in the image;

[0031] S223: Track the continuous image sequence to track the target of interest in the continuous image sequence and obtain the analysis result.

[0032] Preferably, the process of performing image parsing by the image parsing model in step S2 is:

[0033] S231: Perform Fourier transformation on the pixel points of the image to be analyzed, enhance the specified features therein, and weaken other specified features therein;

[0034] S232: Parse and extract the underlying features and upper-level features of the image through the image parsing model based on the sparse representation decomposition method.

[0035] Preferably, the specific process of generating hotspot-based personalized content creation suggestions and generating personalized creation content according to the analyzed content in step S4 is as follows:

[0036] S41: obtaining the latest hot information based on news social media, extracting keywords from the latest hot information, and generating multiple related titles based on the keywords;

[0037] S42: Conduct online searches for keywords to obtain more accurate background information and enhance the accuracy of content analysis;

[0038] S43: Automatically generate personalized creative content based on the relevant titles, and perform personalized adjustments on the personalized creative content based on the user portrait.

[0039] In a second aspect, a creation system based on multimodal data processing and dynamic task planning is provided, which is used to implement any one of the creation methods based on multimodal data processing and dynamic task planning, including a data capture module, a data upload module, a data identification module, a data parsing module, and a dynamic task planning module, wherein the data upload module is connected to the data identification module, the data identification module is connected to the data parsing module, and the data parsing module is connected to the dynamic task planning module.

[0040] The data capture module is used to capture the user's past social media historical data, which includes video, picture, audio or text data, and analyzes the data using a multimodal model to establish a user's exclusive portrait database. The data upload module is used to upload multimodal data, which includes video, picture, audio or text data; the data identification module is used to identify the type of the multimodal data and call the corresponding large model based on the identification result of the data type; the data analysis module includes a video analysis model, a text analysis model, and an image analysis model, which are used to analyze video data, text data, and image data respectively; the dynamic task planning module is used to dynamically analyze and call hot spot tracking tools or network retrieval tools to analyze content based on the analysis results of multimodal data uploaded by users, so as to generate more accurate analysis content and realize personalized and intelligent content creation.

[0041] The beneficial effects of the present invention include:

[0042] The invention provides a creative method and system based on multimodal data processing and dynamic task planning. After the user submits social media information, the system automatically captures and collects the user's historical social media data (including pictures, videos, and texts), and analyzes these data in depth through multimodal models (such as video analysis models and image analysis models), and constructs and stores user portraits in a database; when the user creates content, the system dynamically calls the adapted large model based on the multimodal recognition results: such as using the video analysis model to process video content, the audio large model to convert audio data, and the image analysis model to analyze pictures, etc.; finally, the large language model integrates the analysis results based on user questions for accurate generation; the system records the user's interactive dialogue and behavior data and stores them as historical data. These data not only provide personalized basis for subsequent content generation, but also optimize the dynamic task planning of the system through user behavior analysis; the dynamic task planning module dynamically analyzes and calls the hotspot tracking tool or the network retrieval tool to analyze the content according to the analysis results of the multimodal data uploaded by the user, so as to generate more accurate analysis content, realize personalized and intelligent content creation, and solve the deficiencies in multimodal data processing, task planning, self-optimization and hotspot tracking in the prior art.

[0043] First, the system will capture social media historical data based on the social media information submitted by the user to establish a user-specific portrait database; by determining the target features and required dimensions for building user portraits, the required dimensions are the user's specific information dimensions; obtain the user's social media historical data and extract their behavioral characteristics, emotional preferences and demographic information; further perform variable analysis, identify their differentiated behavioral patterns, perform role clustering, and form user portraits based on role clustering. This achieves accurate user portraits and ensures the personalization of subsequent content creation.

[0044] Secondly, it provides task execution monitoring throughout the entire process to determine whether the task execution results meet expectations or whether there is a better execution path. When the task execution results do not meet expectations or there is a better execution path, the self-optimization module is activated, and the self-optimization module automatically adjusts the task process and tool combination. Through self-optimization, the system can improve execution efficiency and the quality of content generation, ensure that the system can optimize according to actual results during operation, and avoid the limitations of fixed processes. Through self-optimization, the system can ensure that each task step can be completed efficiently.

[0045] Thirdly, by successively calculating the Manhattan distance between the data set to be identified and the training set used by the data identification module during training; according to the calculated Manhattan distance, the k neighbor data points closest to the data set to be identified are selected, and the predicted value is obtained through weighted calculation, and then the type of the data to be identified is determined, thereby achieving accurate classification of the data to be identified and providing an accurate data basis for subsequent personalized creation.

[0046] Thirdly, by converting the binary data of the acquired video data into a picture sequence in RGB format for each frame; inputting each frame of the picture sequence in RGB format into the video parsing model in turn for inference and locking the target of interest; tracking the continuous picture sequence to achieve tracking of the target of interest in the continuous picture sequence to obtain the parsing result, and performing Fourier transform of the pixel points of the picture to be parsed to enhance the specified features and weaken other specified features; parsing and extracting the underlying features and upper-level features of the picture based on the sparse representation decomposition method through the picture parsing model, thereby achieving accurate parsing of the video data and picture data respectively.

[0047] Thirdly, the system will record the user's interactive conversations and behavior data and store them as historical data. These data not only provide a personalized basis for subsequent content generation, but also optimize the system's dynamic task planning through user behavior analysis;

[0048] Finally, the system will dynamically analyze the needs and call hot spot tracking tools or network search tools to analyze the content to generate more accurate analysis content and realize personalized and intelligent content creation.

[0049] Figure 1 It is a flowchart of the creation method based on multimodal data processing and dynamic task planning of the present invention.

[0050] Figure 2 A schematic diagram of a flow chart of the data identification module of the present invention identifying the type of the multimodal data.

[0051] Figure 3 It is a schematic diagram of the architecture of the creation system based on multimodal data processing and dynamic task planning of the present invention. DETAILED DESCRIPTION

[0052] The following is combined with Figure 1 to Figure 3 The present invention is further described in detail:

[0053] Example 1

[0054] See attached Figure 1 As shown, the creation method based on multimodal data processing and dynamic task planning includes the following steps:

[0055] S1: After the user submits social media information, the system automatically captures and collects the user's historical social media data (including pictures, videos, and texts), conducts in-depth analysis of these data through multimodal models (such as video analysis models and image analysis models), and constructs and stores user portraits in the database;

[0056] S2: When users create content, they upload multimodal data, including video, picture, audio or text data. The system dynamically calls the adapted large model based on the multimodal recognition results: for example, the video parsing model is used to process video content, the audio large model is used to convert audio data, and the image parsing model is used to analyze pictures. Finally, the large language model is accurately generated based on the user's questions and the analysis results are integrated.

[0057] S3: The system records the user's interactive conversations and behavior data and stores them as historical data. These data not only provide a personalized basis for subsequent content generation, but also optimize the system's dynamic task planning through user behavior analysis;

[0058] S4: The dynamic task planning module dynamically calls the hotspot tracking tool or network retrieval tool to generate analysis content based on the analysis results of the multimodal data uploaded by the user, and generates personalized content creation suggestions based on hotspots based on the analysis content, and generates personalized creation content based on personalized content creation suggestions. The dynamic task planning mechanism is adopted to automatically plan the task process according to the user's multimodal input and real-time task execution status, and intelligently select the optimal tool combination and execution order to ensure that the content creation task can be completed in the most efficient way. When the task execution results do not meet expectations, the system can automatically adjust the task process and tool selection to ensure the efficiency of task execution and the quality of the final generated content. Through the real-time hotspot tracking function, combined with the portrait data of Internet celebrities, it generates creation strategy suggestions based on hot topics, helping users to adjust the creation direction in time and improve the content flow and user interaction effect.

[0059] The above-mentioned system for realizing the creation method based on multimodal data processing and dynamic task planning integrates multiple large language models, including video parsing models, text parsing models, and image parsing models. It can intelligently process multimodal data such as video, pictures, audio, and text to ensure the efficiency and accuracy of content creation under different input forms.

[0060] Therefore, the above process solves the problem that multimodal data processing in the prior art is usually limited to a single modality or a limited type of data processing, and multiple data modalities cannot be effectively integrated for collaborative processing. The present invention integrates a large language model, a large video understanding model, a large image understanding model, a large audio-to-text model and other tools to efficiently process a variety of data input forms (video, picture, audio, text, etc.), and achieve comprehensive coverage and flexible processing of multimodal data. This significantly improves the system's ability to handle complex and diverse creative tasks, and solves the problem of inefficiency caused by insufficient processing power in the prior art.

[0061] In addition, the dynamic task planning module can automatically select the most appropriate tools and execution order based on the multimodal data input by the user and the real-time status of task execution, thereby realizing dynamic planning of tasks. This feature enables the system to flexibly adapt to different task scenarios, effectively improving the overall efficiency of content creation, especially when dealing with complex tasks, by intelligently selecting tool paths to reduce unnecessary steps and time consumption. By integrating exclusive portrait data and combining it with real-time hotspot tracking functions, it is able to generate highly personalized creative suggestions, significantly improving the relevance and quality of creative content, making the generated content more in line with the style of Internet celebrities and audience needs, and helping to improve the user's creative effect and popularity.

[0062] In this embodiment, a monitoring process is also included, specifically: in steps S1-S4, task execution monitoring is performed, and it is determined whether the task execution result meets expectations or whether there is a better execution path. When the task execution result does not meet expectations or there is a better execution path, the self-optimization module is started, and the self-optimization module automatically adjusts the task process and tool combination. The exclusive portrait database in step S1 includes social media historical data, content preferences, and fan portraits, and the personalized content creation suggestions in step S4 include complete post copywriting, video analysis reports, or hot spot tracking reports.

[0063] Example 2

[0064] In Example 1, the specific process of establishing a user's exclusive portrait database based on social media historical data in step S1 is as follows:

[0065] S11: Determine the target features and required dimensions for constructing user portraits. The required dimensions are the user's specific information dimensions, including the user's age, gender, region, and interests.

[0066] S12: Obtain the user's social media history data, and extract their behavioral characteristics, emotional preferences, and demographic information based on their social media history data pictures and video texts;

[0067] S13: Perform variable analysis on the extracted behavioral characteristics, emotional preferences, and demographic information to identify their differentiated behavioral patterns, perform role clustering, and form user portraits based on role clustering.

[0068] Example 3

[0069] On the basis of Example 1 or Example 2, the specific process of the data identification module in step S2 identifying the type of the multimodal data is as follows:

[0070] S21: inputting the data set to be identified into the data identification module, and sequentially calculating the Manhattan distance between the data set to be identified and the training set used by the data identification module during training;

[0071] S22: According to the calculated Manhattan distance, select the k neighbor data points closest to the data set to be identified, that is, select the k data points closest to the target point;

[0072] S23: Perform weighted calculation based on the Manhattan distance of the selected k neighbor data and the corresponding labels to obtain a predicted value, and determine the type of data to be identified based on the predicted value. The selection of the k value has an important impact on the final data recognition result. A smaller k value may make the model too complex and prone to overfitting; while a larger k value may make the model too simple and lead to underfitting. Therefore, it is necessary to select a suitable k value through cross-validation or other methods.

[0073] The process of calculating the Manhattan distance in step S21 is as follows:

[0074] S211: preprocessing the dataset to be identified and the training set, removing noise data and invalid data in the dataset to be identified and the training set, and extracting designated key features representing the data from the dataset to be identified and the training set respectively;

[0075] S212: Convert the extracted specified key features of the to-be-recognized data set into an n-dimensional feature value vector V1=(x1, x2, ···, x n ), and convert the specified key features of the training set into an n-dimensional eigenvalue vector V2 = (y1, y2, ···, y n );

[0076] S213: Calculate the Manhattan distance between the dataset to be identified and the training set. The specific calculation formula of the Manhattan distance is as follows:

[0077]

[0078] Among them, dm represents the Manhattan distance value, x1,x2,···,x n Assign key feature values ​​to each item of the data set to be identified, y1,y2,···,y n Assign key feature values ​​to each item in the training set.

[0079] The calculation of Manhattan distance can quantify the difference between two data samples, thereby accurately determining the data category.

[0080] In this embodiment, the process of parsing the video data by the video parsing model in step S2 is as follows:

[0081] S221: Convert the acquired binary data of the video data into a picture sequence in RGB format for each frame;

[0082] S222: inputting each frame of the image sequence in RGB format into the video parsing model in sequence for inference, and locking the target of interest from each frame of the image sequence in RGB format, including the target type, credibility and target position in the image;

[0083] S223: Track the continuous image sequence to track the target of interest in the continuous image sequence and obtain the analysis result.

[0084] The process of performing image parsing by the image parsing model in step S2 is as follows:

[0085] S231: Perform Fourier transformation on the pixel points of the image to be analyzed, enhance the specified features therein, and weaken other specified features therein;

[0086] S232: Parse and extract the underlying features and upper-level features of the image through the image parsing model based on the sparse representation decomposition method.

[0087] In this embodiment, the specific process of generating hotspot-based personalized content creation suggestions and generating personalized creation content according to the analyzed content in step S4 is as follows:

[0088] S41: obtaining the latest hot information based on news social media, extracting keywords from the latest hot information, and generating multiple related titles based on the keywords; S42: performing a network search for the keywords to obtain more accurate background information and enhance the accuracy of content analysis;

[0089] S43: Automatically generate personalized creative content based on the relevant titles, and perform personalized adjustments on the personalized creative content based on the user portrait.

[0090] A creation system based on multimodal data processing and dynamic task planning is used to implement any one of the creation methods based on multimodal data processing and dynamic task planning, including a data capture module, a data upload module, a data identification module, a data parsing module, and a dynamic task planning module, wherein the data upload module is connected to the data identification module, the data identification module is connected to the data parsing module, and the data parsing module is connected to the dynamic task planning module.

[0091] The data capture module is used to capture the user's past social media historical data, the data includes video, picture, audio or text data, and analyzes the data using a multimodal model to establish a user's exclusive portrait database; the data upload module is used to upload multimodal data, the multimodal data includes video, picture, audio or text data; the data identification module is used to identify the type of the multimodal data and call the corresponding large model based on the identification result of the data type; the data analysis module includes a video analysis model, a text analysis model, and an image analysis model, which are used to analyze video data, text data, and image data respectively; the dynamic task planning module is used to dynamically analyze and call hot spot tracking tools or network retrieval tools to analyze content according to the analysis results of multimodal data uploaded by the user, so as to generate more accurate analysis content and realize personalized and intelligent content creation;

[0092] In summary, the creation method and system based on multimodal data processing and dynamic task planning provided by the present invention, by determining the target features and required dimensions for constructing user portraits, the required dimensions are the specific information dimensions of users; obtaining the user's social media historical data set and extracting their behavioral characteristics, emotional preferences and demographic information; further performing variable analysis, identifying their differentiated behavioral patterns, and performing role clustering, and forming user portraits based on role clustering. Accurate portraits of users are achieved to ensure the personalization of subsequent creative content. And the system provides task execution monitoring throughout the process to determine whether the task execution results meet expectations or whether there is a better execution path. When the task execution results do not meet expectations or there is a better execution path, the self-optimization module is started, and the self-optimization module automatically adjusts the task process and tool combination. Through self-optimization, the system can improve the execution efficiency and the quality of content generation, ensure that the system can be optimized according to the actual effect during operation, and avoid the limitations of fixed processes. Through self-optimization, the system can ensure that each task step can be completed efficiently. By successively calculating the Manhattan distance between the data set to be identified and the training set used by the data identification module during training; according to the calculated Manhattan distance, the k neighbor data points closest to the data set to be identified are selected, and the prediction value is obtained through weighted calculation, and then the type of the data to be identified is determined, thereby achieving accurate classification of the data to be identified and providing an accurate data basis for subsequent personalized creation.

[0093] By converting the binary data of the acquired video data into a picture sequence in RGB format for each frame; inputting each RGB format picture sequence into the video parsing model in turn for inference and locking the target of interest; tracking the continuous picture sequence, tracking the target of interest in the continuous picture sequence, obtaining the parsing result, and performing Fourier transform of the pixel points of the picture to be parsed, enhancing the specified features in it, and weakening the other specified features in it; parsing and extracting the underlying features and upper features of the picture based on the sparse representation decomposition method through the picture parsing model, respectively realizing the accurate parsing of video data and picture data. By determining the target features and required dimensions for constructing user portraits, the required dimensions are the specific information dimensions of users; obtaining the user's behavior log data set and extracting their behavior characteristics, emotional preferences and demographic information; further performing variable analysis, identifying their differentiated behavior patterns, and performing role clustering, and forming user portraits based on role clustering. Accurate portraits of users are achieved to ensure the personalization of subsequent creative content.

Claims

1. A creation method based on multimodal data processing and dynamic task planning, characterized in that: The following steps are involved: S1: After the user submits social media information, the system automatically captures and collects the user's historical social media data, conducts in-depth analysis of the historical social media data through a multimodal model, and constructs and stores the user portrait in the database; S2: When users create content, the system dynamically calls the adapted large model based on the multimodal recognition results: the video analysis model is used to process video content, the audio large model is used to convert audio data, and the image analysis model is used to analyze pictures. The large language model is based on user questions, combined with user portraits, and integrated with analysis results for accurate generation. S3: The system records the user's interactive conversations and behavior data and stores them as historical data. The historical data provides a personalized basis for subsequent content generation and optimizes the system's dynamic task planning through user behavior analysis. S4: The dynamic task planning module dynamically analyzes and calls hotspot tracking tools or network search tools to analyze content based on the analysis results of multimodal data uploaded by users, so as to generate accurate analysis content and realize personalized and intelligent content creation.

2. The creation method based on multimodal data processing and dynamic task planning according to claim 1 is characterized in that: The specific process of establishing a user's exclusive portrait database based on social media historical data in step S1 is as follows: S11: Determine the target features and required dimensions for constructing user portraits. The required dimensions are the user's specific information dimensions, including the user's age, gender, region, and interests. S12: Obtain the user's social media history data, and extract their behavioral characteristics, emotional preferences, and demographic information based on their social media history data pictures and video texts; S13: Perform variable analysis on the extracted behavioral characteristics, emotional preferences, and demographic information to identify their differentiated behavioral patterns, perform role clustering, and form user portraits based on role clustering.

3. The creation method based on multimodal data processing and dynamic task planning according to claim 1, characterized in that: It also includes a monitoring process, specifically: monitoring the task execution in steps S1-S4, and judging whether the task execution result meets expectations or whether there is a better execution path. When the task execution result does not meet expectations or there is a better execution path, the self-optimization module is started, and the self-optimization module automatically adjusts the task process and tool combination.

4. The creation method based on multimodal data processing and dynamic task planning according to claim 1, characterized in that: The exclusive portrait database in step S1 includes social media historical data, content preferences and fan portraits, and the personalized content creation suggestions in step S4 include complete post copy, video analysis report or hot spot tracking report.

5. The creation method based on multimodal data processing and dynamic task planning according to claim 1, characterized in that: The specific process of the data identification module in step S2 identifying the type of the multimodal data is as follows: S21: inputting the data set to be identified into the data identification module, and sequentially calculating the Manhattan distance between the data set to be identified and the training set used by the data identification module during training; S22: According to the calculated Manhattan distance, select the k neighbor data points closest to the data set to be identified, that is, select the k data points closest to the target point; S23: Perform weighted calculation based on the Manhattan distances of the selected k neighbor data and the corresponding labels to obtain a predicted value, and determine the type of the data to be identified according to the predicted value.

6. The creation method based on multimodal data processing and dynamic task planning according to claim 4 is characterized in that: The process of calculating the Manhattan distance in step S21 is: S211: preprocessing the dataset to be identified and the training set, removing noise data and invalid data in the dataset to be identified and the training set, and extracting designated key features representing the data from the dataset to be identified and the training set respectively; S212: Convert the extracted specified key features of the to-be-recognized data set into an n-dimensional feature value vector V1=(x1, x2, ···, x n ), and convert the specified key features of the training set into an n-dimensional eigenvalue vector V2 = (y1, y2, ···, y n ); S213: Calculate the Manhattan distance between the dataset to be identified and the training set. The specific calculation formula of the Manhattan distance is as follows: Among them, dm represents the Manhattan distance value, x1,x2,···,x n Assign key feature values ​​to each item of the data set to be identified, y1,y2,···,y n Assign key feature values ​​to each item in the training set.

7. The creation method based on multimodal data processing and dynamic task planning according to claim 1 is characterized in that: The process of parsing the video data by the video parsing model in step S2 is as follows: S221: Convert the acquired binary data of the video data into a picture sequence in RGB format for each frame; S222: inputting each frame of the image sequence in RGB format into the video parsing model in sequence for inference, and locking the target of interest from each frame of the image sequence in RGB format, including the target type, credibility and target position in the image; S223: Track the continuous image sequence to track the target of interest in the continuous image sequence and obtain the analysis result.

8. The creation method based on multimodal data processing and dynamic task planning according to claim 1, characterized in that: The process of performing image parsing by the image parsing model in step S2 is as follows: S231: Perform Fourier transformation on the pixel points of the image to be analyzed, enhance the specified features therein, and weaken other specified features therein; S232: Parse and extract the underlying features and upper-level features of the image through the image parsing model based on the sparse representation decomposition method.

9. The creation method based on multimodal data processing and dynamic task planning according to claim 1, characterized in that: The specific process of generating hotspot-based personalized content creation suggestions and generating personalized creation content according to the analyzed content in step S4 is as follows: S41: obtaining the latest hot information based on news social media, extracting keywords from the latest hot information, and generating multiple related titles based on the keywords; S42: Conduct online searches for keywords to obtain more accurate background information and enhance the accuracy of content analysis; S43: Automatically generate personalized creative content based on the relevant titles, and perform personalized adjustments on the personalized creative content based on the user portrait.

10. A creation system based on multimodal data processing and dynamic task planning, used to implement the creation method based on multimodal data processing and dynamic task planning as described in any one of claims 1 to 9, characterized in that: It includes a data capture module, a data upload module, a data identification module, a data parsing module, and a dynamic task planning module. The data upload module is connected to the data identification module, the data identification module is connected to the data parsing module, and the data parsing module is connected to the dynamic task planning module; The data capture module is used to capture the user's past social media historical data, which includes video, picture, audio or text data, and analyze the data using a multimodal model to establish a user's exclusive portrait database; The data uploading module is used to upload multimodal data, wherein the multimodal data includes video, picture, audio or text data; The data identification module is used to identify the type of the multimodal data and call the corresponding large model based on the identification result of the data type; The data parsing module includes a video parsing model, a text parsing model, and an image parsing model, which are used to parse video data, text data, and image data, respectively; The dynamic task planning module is used to dynamically analyze and call hotspot tracking tools or network retrieval tools to analyze content based on the analysis results of multimodal data uploaded by users, so as to generate more accurate analysis content and realize personalized and intelligent content creation.

Images