Multimedia content analysis method, apparatus, device, and storage medium

Abstract

The application discloses a multimedia content analysis method, device, equipment and storage medium, which can analyze any one of articles, images and videos. First, the title and cover of the target multimedia content to be analyzed are extracted, and an image recognition algorithm is used to convert the cover into cover text information. On this basis, a large language model is called to analyze and process the title and cover text information in each set dimension to obtain title analysis results and cover analysis results. The application can fully utilize the understanding and summarizing capabilities of the large language model to analyze the title and cover of the target multimedia content to be analyzed, disassemble and analyze the title and cover of the target multimedia content from each set dimension, obtain the analysis results of the title and cover of the target multimedia content, and better help users understand and learn the characteristics of the target multimedia content from the perspective of the title and cover, thereby improving the creation ability of the users.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and more specifically, to a multimedia content analysis method, apparatus, device, and storage medium. Background Technology

[0002] With the development of internet technology, the quantity and types of multimedia content have experienced explosive growth. Creators of multimedia content can collect information of interest or current online trends to create content such as articles, videos, and posters, which they can then publish online for other users to browse.

[0003] However, multimedia content creators also experience creative blocks, especially beginners who often struggle with how to create content or how to create viral content. One solution is to learn from high-quality content created by others to find inspiration. However, understanding how to better learn from high-quality content to better understand and create high-quality content becomes a crucial step. Summary of the Invention

[0004] In view of the above problems, this application is made to provide a multimedia content analysis method, apparatus, device, and storage medium to help users analyze multimedia content, thereby better assisting users in learning and understanding the characteristics of multimedia content and improving users' creative abilities. The specific solution is as follows:

[0005] Firstly, a multimedia content analysis method is provided, including:

[0006] Acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos;

[0007] Extract the title and cover image of the target multimedia content;

[0008] The cover is converted into a text structure using an image recognition algorithm to obtain the cover text information;

[0009] The pre-configured large language model is invoked to perform analysis and processing on the title and cover text information according to each set dimension, so as to obtain the title analysis results and cover analysis results.

[0010] Preferably, the title analysis results include at least one of the following: title element point tags, title presentation form tags, title expression emotion tags, and title analysis description content;

[0011] The cover analysis results include at least one of the following: cover size label, cover element label, cover title label, cover color label, cover format label, and cover title layout label.

[0012] Preferably, when the target multimedia content is video, the method further includes:

[0013] Transcribe the audio in the video into recognized text;

[0014] The video frame is divided into frames, and the video frames are processed based on computer vision algorithms to obtain the text description information corresponding to the video.

[0015] The large language model is invoked to perform analysis and processing on the recognized text in various predetermined dimensions, and the analysis results are used as the analysis results corresponding to the audio.

[0016] The large language model is invoked to analyze and process the text description information in various defined dimensions, and the analysis results are used as the analysis results corresponding to the video images.

[0017] Preferably, the analysis results corresponding to the audio include:

[0018] At least one of the following: opening information, structure information, memorable quotes, closing information, character language, tone of voice, speaking speed, and background music.

[0019] The analysis results corresponding to the video footage include at least one of the following: information on the people appearing on camera, information on props, information on scene descriptions, and number of transitions.

[0020] Preferably, when the target multimedia content is an article, the method further includes:

[0021] The large language model is invoked to analyze and process the content of the article, and the analysis results of the article content are obtained.

[0022] Preferably, the analysis results of the article content include:

[0023] At least one of the following: article beginning type, article structure information, and article ending type.

[0024] Preferably, the step of calling a pre-configured large language model to perform analysis and processing on the title across various predetermined dimensions to obtain title analysis results includes:

[0025] Obtain a pre-configured first prompt format template, which includes a title information slot. The first prompt format template is used to instruct the large language model to extract labels of each set dimension from the title in the title information slot.

[0026] The title is filled into the title information slot to obtain the edited first prompt instruction, which is then input into the large language model to obtain the dimensional tags extracted from the title by the large language model.

[0027] Obtain a pre-configured second prompt format template, which includes a title information slot and is used to instruct the large language model to analyze the title in the title information slot.

[0028] The title is filled into the title information slot to obtain the edited second prompt instruction, which is then input into the large language model to obtain the analysis and description content of the title output by the large language model.

[0029] Preferably, it further includes:

[0030] Obtain users' creative needs;

[0031] The large language model is invoked, and based on the target multimedia content and / or the analysis results of the target multimedia content, as well as the user's creative needs, multimedia content that meets the user's creative needs and is identical to the target multimedia content is generated for the user's reference.

[0032] Preferably, it further includes:

[0033] Obtain users' creative needs;

[0034] The large language model is invoked to generate multimedia content that matches the user's personalized style, meets the user's creative needs, and is identical to the target multimedia content, based on the target multimedia content and / or the analysis results of the target multimedia content, the user's creative needs, and the pre-configured personalized knowledge of the user, for the user's reference.

[0035] Preferably, acquiring the target multimedia content to be analyzed includes:

[0036] Obtain users' creative needs;

[0037] The system retrieves multimedia content that meets the user's creative needs, selects the most popular multimedia content from the retrieved content according to the set criteria for viral content, and outputs and displays the most popular multimedia content for the user to choose from.

[0038] In response to the user's selection, the popular multimedia content selected by the user is used as the target multimedia content to be analyzed.

[0039] Preferably, the user's creative needs include at least one of the following: the track, platform, type of the content, and keywords that the user is interested in.

[0040] Secondly, a multimedia content analysis device is provided, comprising:

[0041] The content acquisition unit is used to acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos.

[0042] The title and cover extraction unit is used to extract the title and cover of the target multimedia content;

[0043] The cover conversion unit is used to convert the cover into a text structure using an image recognition algorithm to obtain cover text information;

[0044] The first model invocation unit is used to invoke a pre-configured large language model to perform analysis and processing on the title and cover text information in various preset dimensions, and obtain title analysis results and cover analysis results.

[0045] Thirdly, a multimedia content analysis device is provided, including: a memory and a processor;

[0046] The memory is used to store programs;

[0047] The processor is used to execute the program to implement the various steps of the multimedia content analysis method as described above.

[0048] Fourthly, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the various steps of the multimedia content analysis method as described above.

[0049] Using the above technical solution, this application can analyze any type of content, including articles, images, and videos. First, it extracts the title and cover of the target multimedia content to be analyzed. Then, to facilitate processing by the large language model, an image recognition algorithm can be used to convert the cover into a text structure, obtaining the cover text information. For example, OCR technology can be used to convert the cover into a text structure. Based on this, the large language model can be invoked to analyze the title and cover text information according to various predetermined dimensions, yielding title analysis results and cover analysis results. Clearly, by adopting the solution of this application, the understanding and summarizing capabilities of the large language model can be fully utilized to analyze the title and cover of the target multimedia content. By deconstructing and analyzing the title and cover of the target multimedia content from various predetermined dimensions, the analysis results can be obtained, which can better help users understand and learn the characteristics of the target multimedia content from the perspective of the title and cover, thereby improving users' creative abilities. Attached Figure Description

[0050] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0051] Figure 1 This application provides a schematic flowchart of a multimedia content analysis method.

[0052] Figure 2 An example of a search interface for users to input their creative needs is shown in the diagram.

[0053] Figure 3 An example of a display interface for retrieved trending content is shown below;

[0054] Figure 4 This example illustrates a schematic diagram of an interface for disassembling, analyzing, and displaying viral videos.

[0055] Figure 5 This example illustrates a schematic diagram of an interface for disassembling, analyzing, and displaying viral articles.

[0056] Figure 6 A schematic diagram of a multimedia content analysis device provided in this application embodiment;

[0057] Figure 7 This is a schematic diagram of the structure of a multimedia content analysis device provided in an embodiment of this application. Detailed Implementation

[0058] Before introducing the proposed solution, let's first explain the English terms used in this document:

[0059] Prompt: Instructions. When conversing with AI (such as a large language model), you need to send instructions to the AI. These can be a text description, such as "Please recommend a popular song for me" when you talk to the AI, or a parameter description in a certain format, such as describing the relevant drawing parameters to ask the AI ​​to draw a picture in a certain format.

[0060] Large language model (LLM) refers to a generative deep neural network model based on the Transformer structure. It has powerful semantic understanding and multi-turn dialogue capabilities. Common large language models include GT4 and other large language models developed by companies.

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] This application provides a multimedia content analysis scheme that can be applied to the decomposition and analysis of various types of multimedia content, such as article-type, image-type, and video-type multimedia content. Through analysis, the characteristics of multimedia content under different dimensions can be obtained, thereby helping users to better understand and learn multimedia content, and assisting users in creating higher-quality multimedia content.

[0063] The proposed solution can be implemented based on a terminal with data processing capabilities, such as a mobile phone, computer, learning machine, or intelligent robot.

[0064] Next, combined Figure 1 The multimedia content analysis method of this application may include the following steps:

[0065] Step S100: Obtain the target multimedia content to be analyzed.

[0066] The target multimedia content includes any one of the following types of content: articles, images, and videos.

[0067] Here, the target multimedia content to be analyzed in this step can be user-specified multimedia content. For example, a user might select high-quality multimedia content from those already published on a multimedia platform and want to analyze and learn from it to improve their own creative abilities. In this case, the user-specified high-quality multimedia content can be used as the target multimedia content to be analyzed.

[0068] Of course, there are other ways to obtain the target multimedia content, which will not be explained in this step.

[0069] Step S110: Extract the title and cover of the target multimedia content.

[0070] It should be noted that in order to be published on multimedia platforms, created multimedia content generally includes a title and a cover image, for example... Figure 3 , Figure 3This example demonstrates several multimedia content results retrieved from a multimedia platform, including videos and articles. Each piece of multimedia content includes a title and a cover image to visually present the content to the browsing user. Understandably, high-quality multimedia content should leave a good first impression on the browsing user through its title and cover image, attracting their attention.

[0071] Therefore, this embodiment analyzes the target multimedia content from the perspective of title and cover. First, the title and cover of the target multimedia content need to be extracted.

[0072] Step S120: Use an image recognition algorithm to convert the cover into a text structure to obtain the cover text information.

[0073] Specifically, considering that the cover is in image format, to facilitate processing by the large language model in subsequent steps, this step can use an image recognition algorithm to convert the cover into a text structure, obtaining the cover text information. Examples of image recognition algorithms include OCR technology and other image recognition algorithms.

[0074] Step S130: Call the pre-configured large language model to perform analysis and processing on the title and cover text information in each set dimension to obtain the title analysis result and the cover analysis result.

[0075] This application can pre-configure a large language model, leveraging its text understanding and summarization capabilities to analyze and process the title and cover text information across various defined dimensions, yielding title and cover analysis results. Furthermore, the analysis results can be aggregated to generate an analysis report of the target multimedia content, which is then provided back to the user.

[0076] Optionally, the title analysis results may include the following:

[0077] The title includes at least one of the following: title element tags, title presentation style tags, title emotional expression tags, and title analysis and description content.

[0078] Among them, the element tag refers to the key element information contained in the title, such as numbers, specific groups of people, specific behaviors, and specific scenarios. The presentation format tag refers to the presentation format of the title. This application can summarize and classify several different title presentation formats in advance, such as the informational type and the number + adjective type. The emotion expression tag refers to the emotion expressed in the title. This application can summarize and classify several different title emotion expressions in advance, such as joy, sadness, and humor.

[0079] The analysis results for the title can include any one or more of the three types of tags mentioned above, such as... Figure 4The example title analysis results contain the three types of tags mentioned above. Figure 4 The example shows the interface displaying the analysis results of a video, where video information such as... Figure 4 The section displayed at the top includes the video title, "In-depth reviews of 12 cars, from 100,000 to 1 million budget!", as well as other relevant information such as posting time, location, duration, number of likes, comments, favorites, related topic, and poster information. By analyzing the title using a large language model, the analysis results are obtained, including three tags: key elements, presentation style, and emotional expression. Key elements include "100,000 to 1 million," "car," and "in-depth review"; the presentation style is "number + adjective"; and the emotional expression is "joy." Based on these tags in the title analysis results, the key information of the title can be quickly and intuitively understood.

[0080] Furthermore, the title analysis results can also include title analysis descriptions. These descriptions refer to the specific analysis of the title, which can analyze the title's characteristics and advantages from different angles, allowing users to gain a deeper understanding of the title's in-depth information. Compared to the tags mentioned in the previous examples, this analysis is more in-depth, and the descriptions are more detailed and comprehensive. For example... Figure 4 As shown, the "Title Analysis" box on the right side of the title analysis results section contains the title analysis description. Figure 4 The example headline analysis includes the following: 1. Language: The headline uses concise and clear language, conforming to news headline conventions. 2. Presentation: The headline uses a combination of numbers and adjectives to highlight the number of vehicles and price range involved in the article… 3. Emotion: The headline conveys a positive and uplifting emotion…

[0081] Optionally, the cover analysis results may include any one or more of the following:

[0082] Cover size label, cover element label, cover title label, cover color label, cover format label, cover title layout label.

[0083] The following tags are used to describe the cover's dimensions: Cover Size Tag: Cover Elements Tag: Cover Elements Tag: Cover Title Tag: Cover Title Tag: Cover Title Tag: Cover Color Scheme ...

[0084] The analysis results for the cover can include any one or more of the above-mentioned types of tags, such as... Figure 4The example video title analysis results include three types of tags: cover size, elements contained in the cover, and cover title. Based on these tags, key information about the cover can be quickly and intuitively understood.

[0085] The multimedia content analysis method provided in this application can analyze any type of content, including articles, images, and videos. First, the title and cover of the target multimedia content to be analyzed are extracted. Further, to facilitate processing by a large language model, an image recognition algorithm can be used to convert the cover into a text structure, obtaining the cover text information. For example, OCR technology can be used to convert the cover into a text structure. Based on this, a large language model can be invoked to analyze the title and cover text information according to various predetermined dimensions, obtaining title analysis results and cover analysis results. Clearly, by adopting the solution of this application, the understanding and summarizing capabilities of the large language model can be fully utilized to analyze the title and cover of the target multimedia content. By deconstructing and analyzing the title and cover of the target multimedia content from various predetermined dimensions, the analysis results of the title and cover of the target multimedia content can be obtained. This can better help users understand and learn the characteristics of the target multimedia content from the perspective of the title and cover, improving users' creative abilities.

[0086] In some embodiments of this application, optional implementations of step S100 above, which obtains the target multimedia content to be analyzed, are described.

[0087] Specifically, users can provide the target multimedia content to be analyzed, such as the target multimedia content data itself, or the link, index number, etc. of the target multimedia content.

[0088] Furthermore, considering that in common scenarios, users typically have creative needs but lack corresponding reference materials or creative inspiration, this application can provide users with relevant target multimedia content. By deconstructing and analyzing the target multimedia content, it can be provided for users' reference and learning. Therefore, this embodiment provides an implementation of step S100, which specifically includes the following steps:

[0089] S1. Obtain the user's creative needs.

[0090] Specifically, in this embodiment, a search page can be provided to the user, offering multi-dimensional creative requirement settings options, as shown in the reference. Figure 2 As shown, creative requirements can include the category, platform, type of the content, and keywords that users are interested in.

[0091] S2. Retrieve multimedia content that meets the user's creative needs, and select the most popular multimedia content from the retrieved content according to the set criteria for trending content. Output and display the most popular multimedia content for the user to choose from.

[0092] Specifically, based on the creative requirements input by the user in the previous step, this application can combine the system's knowledge search engine to search for relevant content in the background knowledge base and existing online resources to obtain various multimedia content that meets the user's creative requirements.

[0093] Furthermore, this application can predefine the criteria for a viral hit. In simple terms, a viral hit refers to multimedia content that enjoys high popularity and receives a large number of views and shares. This application can define viral hit criteria from multiple dimensions for different platforms. For example, for content on the Xiaohongshu platform, for articles, videos, and images, the criteria require the highest total number of likes, favorites, and comments within a week, or reaching a set threshold. For content on the WeChat platform, for images and videos, the criteria require the highest number of likes, comments, and shares within a week, or reaching a set threshold; for articles, the criteria require the highest number of reads within a week, or reaching a set threshold, and so on.

[0094] Based on the aforementioned criteria for identifying trending content, trending multimedia content can be filtered from the retrieved content and displayed for users to choose from. For example... Figure 3 As shown, this example illustrates a schematic diagram of a trending content display interface. It can include the trending content selected above, such as videos, articles, and images. Each trending content display box can show its cover image, title, publication time, and some basic information, such as the number of likes, favorites, and comments. Additionally, each trending content display box can also display a control for users to break down and analyze, such as... Figure 3 The bottom right corner of each trending content display box is shown.

[0095] S3. Respond to the user's selection operation and use the popular multimedia content selected by the user as the target multimedia content to be analyzed.

[0096] Specifically, users can select their favorite popular multimedia content on the popular content display interface. In response to the user's selection, this application will use the selected popular multimedia content as the target multimedia content to be analyzed, and then perform the subsequent process of disassembling and analyzing the target multimedia content.

[0097] The method for obtaining target multimedia content to be analyzed provided in this embodiment can retrieve multimedia content that meets the user's creative needs based on the user's creative needs, and select popular multimedia content from it to display to the user. The user can then select the target multimedia content to be analyzed.

[0098] In some embodiments of this application, a multimedia content analysis method is further described.

[0099] When the target multimedia content to be analyzed is a video, in addition to the aforementioned analysis of the video title and cover, this embodiment can also analyze the video from other perspectives, such as analyzing the audio contained in the video, the visuals contained in the video, etc.

[0100] Specific analytical methods may include:

[0101] S1. Transcribe the audio in the video to be analyzed into recognized text.

[0102] Specifically, speech recognition technology can be used to transcribe the audio data in the video to be analyzed into recognized text.

[0103] S2. Divide the video frame into frames and process the video frames based on computer vision algorithms to obtain the text description information corresponding to the video.

[0104] Specifically, for video content, frame segmentation can be performed first to extract key frames, and computer vision algorithms can be used to analyze and process the key frames to obtain the corresponding text description information of the video.

[0105] S3. Call the large language model to perform analysis and processing on the recognized text in each set dimension, and use the analysis results as the analysis results corresponding to the audio.

[0106] Specifically, after obtaining the recognized text in the aforementioned step S1, a large language model can be called to analyze and process the recognized text, and the analysis result obtained is used as the analysis result corresponding to the audio.

[0107] The analysis results for the audio include, but are not limited to, the following dimensions:

[0108] Information included in the opening of the copy, the structure of the copy, key quotes in the copy, information at the end of the copy, information about the characters' language, information about the characters' tone of voice, information about the characters' speaking speed, and information about the background music.

[0109] The opening information of the copy can be of the type that appears in the first few seconds (e.g., 3 seconds), such as getting straight to the point, addressing pain points, highlighting benefits, setting a goal, creating suspense, posing a question, or incorporating current events.

[0110] The structure of copywriting information can be of many different types, such as posing a question + analyzing the reasons + guiding interaction + specific explanation, presenting the result first + explaining the method + positive case, pointing out the error + providing the solution + guiding interaction, etc.

[0111] A memorable quote in copywriting can be a standout sentence or phrase in the entire copy, along with a description of the effect it achieves.

[0112] The information at the end of a copy can refer to the type of ending, such as emotional appeal, summary, guidance, or product recommendation.

[0113] Language information refers to the type of language used by the person in the audio. Tone information refers to the tone of voice, such as low, high, slow, or sarcastic. Speech rate information refers to the speed at which the person speaks, such as steady, fast, or slow.

[0114] Background music information can refer to the type of background music in the audio, such as upbeat, heavy, or energetic.

[0115] S4. Call the large language model to analyze and process the text description information in each set dimension, and use the analysis results as the analysis results corresponding to the video screen.

[0116] Specifically, after obtaining the text description information in the aforementioned step S2, the large language model can be called to analyze and process the text description information, and the analysis result obtained is used as the analysis result corresponding to the video frame.

[0117] The analysis results corresponding to the video footage include, but are not limited to, the following dimensions:

[0118] Information on on-screen personnel, props, scene descriptions, and number of scene transitions.

[0119] Among them, the information of the people appearing on screen can refer to whether the people in the video are real or virtual, and other attribute information such as the gender, age, and occupation of the people.

[0120] Prop information can refer to the types of props appearing in the video footage, such as cars, clothing, mobile phones, computers, etc.

[0121] Scene description information can refer to information related to the shooting scene of the video footage, such as indoor scenes, outdoor scenes, etc.

[0122] The number of transitions can refer to the total number of transitions contained in the video frame, ranging from 1 to n.

[0123] In summary, for target multimedia content of the video type, the solution in this embodiment can disassemble and analyze the video from different angles such as title, cover, audio, and picture, and provide analysis results from the corresponding angles. This allows for a more comprehensive analysis of the video, and the results can be summarized into an analysis report and fed back to the user to help the user understand the video content more comprehensively and accurately.

[0124] In some embodiments of this application, a multimedia content analysis method is further described.

[0125] When the target multimedia content to be analyzed is an article, in addition to the aforementioned analysis of the article's title and cover, this embodiment can also analyze the article from the perspective of its content.

[0126] Specific analytical methods may include:

[0127] The large language model is invoked to analyze and process the content of the article, and the analysis results of the article content are obtained.

[0128] Specifically, in this embodiment, article content can be extracted from the article to be analyzed. The article content may include plain text content contained in the article, as well as text content identified from images contained in the article. Furthermore, a large language model can be invoked to analyze the article content and obtain the analysis results.

[0129] The analysis results corresponding to the article content include, but are not limited to, the following dimensions:

[0130] Article opening type, article structure information, article ending type, etc.

[0131] The type of article opening can refer to the expression style used in the beginning of the text, such as story-based, concise, thought-provoking, memorable quote-based, trending topic-based, or emotionally resonant.

[0132] Article structure information can refer to the expression structure used in the article, such as a list (listing 1, 2, 3, etc.), pain point + solution, story + insight, etc.

[0133] The type of article ending can refer to the type of expression used in the ending section of the article, such as summary, advertising placement, interactive, etc.

[0134] In summary, for target multimedia content of the article type, the solution in this embodiment can break down and analyze the article from different angles such as title, cover, and content, and provide analysis results from the corresponding angles. This allows for a more comprehensive analysis of the article, and the results can be summarized into an analysis report and fed back to the user to help the user understand the article content more comprehensively and accurately.

[0135] The preceding embodiments described the process of analyzing target multimedia content from different perspectives, such as the title, cover, audio of the video, video footage, and content of the article, to obtain corresponding analysis results. This embodiment details how to call a large language model to implement the above-mentioned analysis from various perspectives.

[0136] Regarding the process of title analysis:

[0137] refer to Figure 4 As shown, the title analysis results can include two parts: the tag analysis results on the left and the analysis description on the right.

[0138] Therefore, this application can configure different prompt format templates. The first prompt format template can include a title information slot, which instructs the large language model to extract tags for each defined dimension from the title within the title information slot. For example, the first prompt format template could be: "Please help me extract [tag1], [tag2], [tag3]... from [title information slot], where the corresponding tag definitions are: [tag1 definition], [tag2 definition], [tag3 definition]...".

[0139] Furthermore, the title is filled into the title information slot to obtain the edited first prompt instruction, which is then input into the large language model to obtain the dimensional tags extracted from the title by the large language model. For example... Figure 4 , Figure 5 As shown, the large language model can output element labels, expression form labels, and expression sentiment labels, etc.

[0140] The second prompt format template may include a title information slot. The second prompt format template is used to instruct the large language model to analyze the title in the title information slot. For example, the second prompt format template may be: "Please help me analyze [title information slot]".

[0141] Furthermore, the title is filled into the title information slot to obtain the edited second prompt instruction, which is then input into the large language model to obtain the analysis and description content of the title output by the large language model. For example... Figure 4 As shown, the large language model can output specific title analysis descriptions: "1. Language: The title uses concise and clear language, which conforms to the conventions of news headlines. 2. Presentation: The title uses a number + adjective format, highlighting the number of vehicles and price range involved in the article... 3. Emotion: The title conveys a positive and uplifting emotion..."

[0142] Regarding the process of cover analysis:

[0143] This application can pre-configure a third prompt format template, which may include a cover text information slot. The third prompt format template is used to instruct the large language model to analyze the cover text information in the cover text information slot. For example, the third prompt format template may be: "Please help me extract [Label 1], [Label 2], [Label 3]... from [Cover Text Information Slot], where the corresponding label definitions are: [Label 1 Definition], [Label 2 Definition], [Label 3 Definition]...".

[0144] Furthermore, the converted cover text information is filled into the cover text information slot to obtain the edited third prompt instruction, which is then input into the large language model to obtain the cover analysis results output by the large language model. For example... Figure 4 , Figure 5 As shown, the large language model can output size labels, element labels, cover title labels, etc.

[0145] The process of audio analysis in video:

[0146] refer to Figure 4 As shown, the audio analysis results (corresponding to) Figure 4 The video dialogue section can include two parts: the tag analysis results on the left and the analysis description on the right.

[0147] Therefore, this application can configure different prompt format templates. The fourth prompt format template can include recognition text information slots, and is used to instruct the large language model to extract labels for each set dimension from the recognition text within the recognition text information slots. For example, the fourth prompt format template could be: "Please help me extract [Label 1], [Label 2], [Label 3]... from [Recognition Text Information Slots], where the corresponding label definitions are: [Label 1 Definition], [Label 2 Definition], [Label 3 Definition]...".

[0148] Furthermore, the transcribed audio from the video is used to fill the recognized text information slots, resulting in an edited fourth prompt instruction. This prompt is then input into a large language model to obtain the dimensional labels extracted from the recognized text, as output by the large language model. For example... Figure 4 As shown, the large language model can output tags such as the first 3 seconds of the text analysis, the text structure analysis, the text ending analysis, and the text key phrase analysis.

[0149] The fifth prompt format template may include a recognition text information slot. The fifth prompt format template is used to instruct the large language model to analyze the recognition text in the recognition text information slot. For example, the fifth prompt format template may be: "Please help me analyze [recognition text information slot]".

[0150] Furthermore, the transcribed audio from the video is used to fill the recognized text information slots, resulting in an edited fifth prompt instruction. This prompt is then input into a large language model to obtain the analysis and description of the recognized text output by the large language model. For example... Figure 4 As shown, the large language model can output specific audio analysis descriptions: "1. Introduction: By introducing the task 'High Frost,' the topic of choosing a car for families with two children is brought up, highlighting that the recommended model is an MPV. 2. Conclusion: The reasons for the recommendation are given, emphasizing the importance of choosing a model that suits one's needs, while also leaving room for interaction in the comments section. 3. Structure: The video script is divided into three parts: an introduction to three models, a comparison and evaluation of models A and B, and suggestions for choosing an MPV. The first part summarizes the characteristics of the three models, the second part provides a detailed comparison and evaluation, and the third part gives selection suggestions. 4. Key phrases: 1. I recommend it to families with two children. 2. The third-row seating position in an MPV is comfortable even for normal users who sit for a long time. These key phrases highlight the characteristics of the models and enhance the persuasiveness of the script."

[0151] The process of analyzing images in a video:

[0152] This application can pre-configure a sixth prompt format template, which may include screen description information slots. The sixth prompt format template is used to instruct the large language model to analyze the text description information of the screen description information slots. For example, the sixth prompt format template may be: "Please help me extract [tag1], [tag2], [tag3]... from [screen description information slots], where the corresponding tag definitions are: [tag1 definition], [tag2 definition], [tag3 definition]...".

[0153] Furthermore, the text description information corresponding to the video is filled into the screen description information slot to obtain the edited sixth prompt instruction, which is then input into the large language model to obtain the video screen analysis results output by the large language model. For example... Figure 4 As shown, the large language model can output tags such as on-screen personnel, props, scene descriptions, and transition counts.

[0154] Regarding the process of analyzing the article content:

[0155] refer to Figure 5As shown, the results of the article content analysis (corresponding to) Figure 5 The article copy section can include two parts: the tag analysis results on the left and the analysis description on the right.

[0156] Therefore, this application can configure different prompt format templates. The seventh prompt format template can include article content slots, and is used to instruct the large language model to extract tags for each set dimension from the article content within the slots. For example, the seventh prompt format template could be: "Please help me extract [tag1], [tag2], [tag3]... from [article content slots], where the corresponding tag definitions are: [tag1 definition], [tag2 definition], [tag3 definition]...".

[0157] Furthermore, the article content is filled into the article content slots to obtain the edited seventh prompt instruction, which is then input into the large language model to obtain the dimensional labels extracted from the article content by the large language model. For example... Figure 5 As shown, the large language model can output tags such as beginning tags, structure tags, ending tags, and taglines for memorable quotes in copywriting.

[0158] The eighth prompt template can include an article content slot. The eighth prompt template is used to instruct the large language model to analyze the article content in the article content slot. For example, the eighth prompt template can be: "Please help me analyze [article content slot]".

[0159] Furthermore, the article content is filled into the article content slot to obtain the edited eighth prompt instruction, which is then input into the large language model to obtain the analysis and description of the article content output by the large language model. For example... Figure 5 As shown, the large language model can output specific analysis and descriptions of the article content: 1. Introduction: The article uses the sentence "Qingdao!! It's as beautiful as a cartoon..." as an introduction, directly expressing the author's admiration for the beautiful scenery of Qingdao and attracting the reader's attention. 2. Structure: It adopts an enumerative writing style, carefully introducing the beautiful views of attractions such as Badaguan, Xiaomaidao, and Signal Hill Park, and summarizing the main theme of the entire article at the end. Each attraction is accompanied by detailed textual descriptions and beautiful pictures. 3. Conclusion: The article summarizes and emphasizes the beautiful scenery and healing characteristics of Qingdao, encouraging readers to explore more beautiful sights. At the same time, the article provides some useful travel tips, enhancing the interactivity and practicality of the article. 4. Key Sentence: "Every scene is like a picture that has stepped out of a story!" This sentence vividly describes the beautiful scenery of Qingdao, making people think of scenes from fairy tales. At the same time, it also conveys the author's love and unique feelings for the city of Qingdao.

[0160] Optionally, for the first to eighth prompt format templates mentioned above, a data output requirement slot can be further added to allow users to specify their output format requirements. The large language model can then output the analysis results according to the user's specified output format. Furthermore, a role definition slot can be added to allow users to define the roles played by the large language model, instructing it to perform relevant task processing according to these user-defined roles. This improves the accuracy and professionalism of the large language model's output. For example, in the eighth prompt format template, the added role definition slot could be filled with a user-defined role: "You are a short article writing expert on the Xiaohongshu platform."

[0161] Based on the multimedia content analysis method described in the foregoing embodiments, this embodiment further provides a method for generating multimedia content. That is, in addition to analyzing the target multimedia content, it can also generate multimedia content that is identical to the target multimedia content. Here, "identical" can be understood as the two multimedia content sets having the same style, such as having the same or similar video content style, or the same or similar title and cover format.

[0162] This embodiment provides several different methods for generating multimedia content, which are described below:

[0163] The first type

[0164] S11. Obtain the user's creative needs.

[0165] The user's creative needs have already been introduced in the previous text, please refer to the previous text for details, and will not be repeated here.

[0166] S12. Call the large language model, based on the target multimedia content and / or the analysis results of the target multimedia content, as well as the user's creative needs, to generate multimedia content that meets the user's creative needs and is the same as the target multimedia content, for the user's reference.

[0167] In this embodiment, when calling the large language model to generate the same multimedia content, several different combinations of information can be provided to the large language model:

[0168] 1) Input the target multimedia content and user creation requirements into the large language model through a specific prompt.

[0169] The target multimedia content input to the large language model can be in text form. For example, for article-type target multimedia content, the article content can be directly input into the large language model. For video and image-type target multimedia content, it can be processed using image recognition algorithms, computer vision algorithms, etc., and then converted into text form before being input into the large language model.

[0170] 2) Input the analysis results of the target multimedia content and the user's creative needs into the large language model through a specific prompt.

[0171] 3) Input the target multimedia content, the analysis results of the target multimedia content, and the user's creative needs into the large language model through a specific prompt.

[0172] Let's take the third method above as an example:

[0173] This application can pre-configure an eighth prompt format template, which includes a target multimedia content slot, an analysis result slot, and a creation requirement slot. The eighth prompt format template is used to instruct the large language model to generate multimedia content that meets the user's creation requirements and is identical to the target multimedia content.

[0174] The second type

[0175] S21. Obtain the user's creative needs.

[0176] S22. Invoke the large language model, based on the target multimedia content and / or the analysis results of the target multimedia content, the user's creative needs, and the pre-configured user's personalized knowledge, generate multimedia content that conforms to the user's personalized style, meets the user's creative needs, and is the same as the target multimedia content, for the user's reference.

[0177] Compared to the previous implementation, this embodiment further inputs the user's personalized knowledge into the large language model when calling the large language model to generate the same multimedia content, so that the large language model can generate the same multimedia content that matches the user's personalized style.

[0178] The pre-configured personalized knowledge can be formed by combining the user's actions (collection, liking, disliking, number of clicks, comments, etc.) when browsing multimedia content (videos, images, text, etc.) provided by the platform.

[0179] Clearly, the second implementation method ensures that multimedia content identical to the target multimedia content can be generated, while also making the generated multimedia content conform to the user's personalized style.

[0180] This embodiment provides two different implementation methods for generating multimedia content identical to the target multimedia content. Combined with... Figure 4 and Figure 5As shown, users can trigger the "Generate Similar Content" control in the target multimedia section to enter the process of generating similar multimedia content. Of course, other interaction methods can be designed based on the interaction logic to achieve the goal of generating multimedia content identical to the target multimedia content.

[0181] The multimedia content analysis apparatus provided in the embodiments of this application is described below. The multimedia content analysis apparatus described below and the multimedia content analysis method described above can be referred to in correspondence.

[0182] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a multimedia content analysis device disclosed in an embodiment of this application.

[0183] like Figure 6 As shown, the device may include:

[0184] The content acquisition unit 11 is used to acquire the target multimedia content to be analyzed, wherein the target multimedia content includes any one of the following: articles, images, and videos.

[0185] Title and cover extraction unit 12 is used to extract the title and cover of the target multimedia content;

[0186] Cover conversion unit 13 is used to convert the cover into a text structure using an image recognition algorithm to obtain cover text information;

[0187] The first model calling unit 14 is used to call a pre-configured large language model to perform analysis and processing on the title and cover text information in various set dimensions to obtain title analysis results and cover analysis results.

[0188] Optionally, the title analysis results obtained by the first model calling unit may include at least one of the following: title element point tags, title presentation form tags, title expression sentiment tags, and title analysis description content.

[0189] The obtained cover analysis results may include at least one of the following: cover size label, cover element label, cover title label, cover color label, cover format label, and cover title layout label.

[0190] Optionally, when the target multimedia content is video, the device may further include:

[0191] An audio transcription unit is used to transcribe the audio in the video into recognizable text;

[0192] The video frame processing unit is used to divide the video frame into frames and process the video frames based on computer vision algorithms to obtain the text description information corresponding to the video.

[0193] The second model calling unit is used to call the large language model to perform analysis and processing on the recognized text in various set dimensions, and the analysis results are used as the analysis results corresponding to the audio.

[0194] The third model invocation unit is used to invoke the large language model to perform analysis and processing on the text description information in various set dimensions, and the analysis results are used as the analysis results corresponding to the video frame.

[0195] Optionally, the analysis results corresponding to the audio obtained by the second model calling unit mentioned above may include:

[0196] At least one of the following: opening information, structure information, memorable quotes, closing information, character's language, tone of voice, speaking speed, and background music.

[0197] The analysis results obtained from the video footage by the third model calling unit mentioned above may include:

[0198] Information on on-screen personnel, props, scene descriptions, and number of scene transitions.

[0199] Optionally, when the target multimedia content is an article, the device may further include:

[0200] The fourth model invocation unit is used to invoke the large language model to analyze and process the content of the article and obtain the analysis results of the article content.

[0201] Optionally, the analysis results of the article content obtained by the fourth model calling unit mentioned above may include:

[0202] At least one of the following: article beginning type, article structure information, and article ending type.

[0203] Optionally, the process by which the first model invocation unit invokes a pre-configured large language model to perform analysis and processing on the title across various defined dimensions to obtain the title analysis results may include:

[0204] Obtain a pre-configured first prompt format template, which includes a title information slot. The first prompt format template is used to instruct the large language model to extract labels of each set dimension from the title in the title information slot.

[0205] The title is filled into the title information slot to obtain the edited first prompt instruction, which is then input into the large language model to obtain the dimensional tags extracted from the title by the large language model.

[0206] Obtain a pre-configured second prompt format template, which includes a title information slot and is used to instruct the large language model to analyze the title in the title information slot.

[0207] The title is filled into the title information slot to obtain the edited second prompt instruction, which is then input into the large language model to obtain the analysis and description content of the title output by the large language model.

[0208] Optionally, the apparatus of this application may further include: a first multimedia content generation unit, used for:

[0209] Obtain the user's creative needs; invoke the large language model, based on the target multimedia content and / or the analysis results of the target multimedia content, and the user's creative needs, to generate multimedia content that meets the user's creative needs and is the same as the target multimedia content, for the user's reference.

[0210] Optionally, the apparatus of this application may further include: a second multimedia content generation unit, used for:

[0211] Obtain users' creative needs;

[0212] The large language model is invoked to generate multimedia content that matches the user's personalized style, meets the user's creative needs, and is identical to the target multimedia content, based on the target multimedia content and / or the analysis results of the target multimedia content, the user's creative needs, and the pre-configured personalized knowledge of the user, for the user's reference.

[0213] Optionally, the process by which the content acquisition unit acquires the target multimedia content to be analyzed may include:

[0214] Obtain users' creative needs;

[0215] The system retrieves multimedia content that meets the user's creative needs, selects the most popular multimedia content from the retrieved content according to the set criteria for viral content, and outputs and displays the most popular multimedia content for the user to choose from.

[0216] In response to the user's selection, the popular multimedia content selected by the user is used as the target multimedia content to be analyzed.

[0217] Optionally, the user's creative needs acquired by the content acquisition unit may include at least one of the following: the track, platform, type of the content, and keywords that the user is interested in.

[0218] The multimedia content analysis device provided in this application embodiment can be applied to multimedia content analysis devices, such as mobile phones, computers, learning machines, and intelligent robots. Optionally, Figure 7 The hardware structure block diagram of the multimedia content analysis device is shown below. Figure 7 The hardware structure of a multimedia content analysis device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0219] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4.

[0220] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0221] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0222] The memory stores a program, which the processor can call. The program is used for:

[0223] Acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos;

[0224] Extract the title and cover image of the target multimedia content;

[0225] The cover is converted into a text structure using an image recognition algorithm to obtain the cover text information;

[0226] The pre-configured large language model is invoked to perform analysis and processing on the title and cover text information according to each set dimension, so as to obtain the title analysis results and cover analysis results.

[0227] Optionally, the refined and extended functions of the program can be found in the description above.

[0228] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:

[0229] Acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos;

[0230] Extract the title and cover image of the target multimedia content;

[0231] The cover is converted into a text structure using an image recognition algorithm to obtain the cover text information;

[0232] The pre-configured large language model is invoked to perform analysis and processing on the title and cover text information according to each set dimension, so as to obtain the title analysis results and cover analysis results.

[0233] Optionally, the refined and extended functions of the program can be found in the description above.

[0234] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0235] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0236] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimedia content analysis method, characterized in that, include: Acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos; Extract the title and cover image of the target multimedia content; The cover is converted into a text structure using an image recognition algorithm to obtain the cover text information; The pre-configured large language model is invoked to perform analysis and processing on the title and cover text information according to each set dimension, so as to obtain the title analysis results and cover analysis results. The title analysis results include at least one of the following: title element tags, title presentation style tags, and title sentiment tags, as well as title analysis description content; the title element tags are the key element information contained in the title, the title presentation style tags are the presentation style to which the title belongs, and the title sentiment tags are the sentiment to which the title belongs; the title analysis description content is the specific analysis content of the title, analyzing the characteristics and advantages of the title from different perspectives; The cover analysis results include at least one of the following: cover size label, cover element label, cover title label, cover color label, cover format label, and cover title layout label.

2. The method according to claim 1, characterized in that, When the target multimedia content is video, the method further includes: Transcribe the audio in the video into recognized text; The video frame is divided into frames, and the video frames are processed based on computer vision algorithms to obtain the text description information corresponding to the video. The large language model is invoked to perform analysis and processing on the recognized text in various predetermined dimensions, and the analysis results are used as the analysis results corresponding to the audio. The large language model is invoked to analyze and process the text description information in various defined dimensions, and the analysis results are used as the analysis results corresponding to the video images.

3. The method according to claim 2, characterized in that, The analysis results corresponding to the audio include: At least one of the following: opening information, structure information, memorable quotes, closing information, character language, tone of voice, speaking speed, and background music. The analysis results corresponding to the video footage include at least one of the following: information on the people appearing on camera, information on props, information on scene descriptions, and number of transitions.

4. The method according to claim 1, characterized in that, When the target multimedia content is an article, the method further includes: The large language model is invoked to analyze and process the content of the article, and the analysis results of the article content are obtained.

5. The method according to claim 4, characterized in that, The analysis results of the article content include: At least one of the following: article beginning type, article structure information, and article ending type.

6. The method according to claim 1, characterized in that, The pre-configured large language model is invoked to perform analysis and processing on the title across various defined dimensions, resulting in title analysis results, including: Obtain a pre-configured first prompt format template, which includes a title information slot. The first prompt format template is used to instruct the large language model to extract labels of each set dimension from the title in the title information slot. The title is filled into the title information slot to obtain the edited first prompt instruction, which is then input into the large language model to obtain the dimensional tags extracted from the title by the large language model. Obtain a pre-configured second prompt format template, which includes a title information slot and is used to instruct the large language model to analyze the title in the title information slot. The title is filled into the title information slot to obtain the edited second prompt instruction, which is then input into the large language model to obtain the analysis and description content of the title output by the large language model.

7. The method according to any one of claims 1-6, characterized in that, Also includes: Obtain users' creative needs; The large language model is invoked, and based on the target multimedia content and / or the analysis results of the target multimedia content, as well as the user's creative needs, multimedia content that meets the user's creative needs and is identical to the target multimedia content is generated for the user's reference.

8. The method according to any one of claims 1-6, characterized in that, Also includes: Obtain users' creative needs; The large language model is invoked to generate multimedia content that matches the user's personalized style, meets the user's creative needs, and is identical to the target multimedia content, based on the target multimedia content and / or the analysis results of the target multimedia content, the user's creative needs, and the pre-configured personalized knowledge of the user, for the user's reference.

9. The method according to any one of claims 1-6, characterized in that, The acquisition of the target multimedia content to be analyzed includes: Obtain users' creative needs; The system retrieves multimedia content that meets the user's creative needs, selects the most popular multimedia content from the retrieved content according to the set criteria for viral content, and outputs and displays the most popular multimedia content for the user to choose from. In response to the user's selection, the popular multimedia content selected by the user is used as the target multimedia content to be analyzed.

10. The method according to claim 9, characterized in that, The user's creative needs include at least one of the following: the track, platform, type of the content, and keywords that the user is interested in.

11. A multimedia content analysis device, characterized in that, include: The content acquisition unit is used to acquire the target multimedia content to be analyzed, which includes any one of the following: articles, images, and videos. The title and cover extraction unit is used to extract the title and cover of the target multimedia content; The cover conversion unit is used to convert the cover into a text structure using an image recognition algorithm to obtain cover text information; The first model calling unit is used to call a pre-configured large language model to perform analysis and processing on the title and cover text information in various set dimensions to obtain title analysis results and cover analysis results; The title analysis results include at least one of the following: title element tags, title presentation style tags, and title sentiment tags, as well as title analysis description content; the title element tags are the key element information contained in the title, the title presentation style tags are the presentation style to which the title belongs, and the title sentiment tags are the sentiment to which the title belongs; the title analysis description content is the specific analysis content of the title, analyzing the characteristics and advantages of the title from different perspectives; The cover analysis results include at least one of the following: cover size label, cover element label, cover title label, cover color label, cover format label, and cover title layout label.

12. A multimedia content analysis device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the multimedia content analysis method as described in any one of claims 1 to 10.

13. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the multimedia content analysis method as described in any one of claims 1 to 10.

Images