Method and device for automatically making self-media short video cover

Abstract

The invention provides a method and a device for automatically making a self-media short video cover, and relates to the technical field of self-media short videos, the method comprises the following steps of: firstly extracting video frames and selecting key frames, and then converting characters, images or sounds in a video into numerical values by using a video content identification technology and a natural language processing or image identification technology; and determining the theme or keyword of the video. Then, multiple dimensions such as picture definition, explosion points, time factors and figure factors are comprehensively considered, and the key frame of the short video cover is determined through a complex formula including a sigmoid function, extraction operation, logarithm operation and square operation. Besides, according to the method, unique cover elements are generated by referring to hot stems and similar materials on the network and are added to the cover, so that the attraction of the cover is further improved. The method aims at automatically making the cover of the short video, improving the cover making efficiency and quality, and enhancing the attraction and click rate of the video at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of self-media short video, and in particular to a method and device for automatically making a self-media short video cover. Background Art

[0002] In the era of self-media, short videos have become an important channel for people to obtain information and entertainment. With the rise of short video platforms, users have extremely high requirements for the attractiveness and click-through rate of video content; the importance of a video cover as the first impression to attract viewers is self-evident. However, the existing short video cover production process mostly relies on manual design and selection, which is not only time-consuming and labor-intensive, but also inefficient and difficult to meet the rapidly changing market needs.

[0003] Specifically, the traditional cover production process includes manually selecting key frames of the video, matching copy materials, and designing cover elements. This process requires video editors to have certain design capabilities and aesthetic vision, as well as a deep understanding of the video content. In addition, manual selection of cover frames often lacks objectivity and systematicity, making it difficult to quantitatively evaluate the attractiveness and click-through rate potential of the cover. This method, which relies on personal experience and subjective judgment, limits the innovation and diversity of cover production.

[0004] In addition, with the development of technology, especially the advancement of artificial intelligence and machine learning technology, automated and intelligent cover production has become possible. The automated cover production system can analyze video content through algorithms, automatically identify key frames and match copy materials, thereby improving the efficiency and quality of cover production. However, existing automated systems often lack the ability to deeply understand video content, and there is a problem of being unable to accurately match video themes and user interests, resulting in insufficient attractiveness of the cover. Therefore, it is necessary to develop a new method for automatically producing self-media short video covers.

[0005] In view of this, this application is filed. Summary of the invention

[0006] The present invention provides a method and device for automatically producing a self-media short video cover, which can at least partially improve the above-mentioned problem.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for automatically making a self-media short video cover, comprising:

[0009] Obtain a short video to be processed, split the short video into multiple separate frames, and use a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result;

[0010] Based on the factor information, the video content recognition result is calculated using a key frame determination formula to obtain a key frame;

[0011] Obtain popular memes and similar materials on the current network, obtain cover elements based on the popular memes and similar materials, and add the cover elements to the key frame to obtain a cover corresponding to the short video.

[0012] The present invention also provides a device for automatically making a self-media short video cover, which comprises:

[0013] A video preprocessing unit is used to obtain a short video to be processed, split the short video into multiple separate frames, and use a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result;

[0014] A key frame determination unit, configured to calculate the video content recognition result based on the factor information using a key frame determination formula to obtain a key frame;

[0015] A cover generation unit is used to obtain popular memes and similar materials on the current network, obtain cover elements based on the popular memes and similar materials, and add the cover elements to the key frame to obtain a cover corresponding to the short video.

[0016] In summary, the self-media short video cover automatic production method automatically analyzes the short video content uploaded by users through technical means, matches the most accurate hot cover, and adds relevant copywriting materials. Specifically, video frame extraction and key frame selection are first performed, and then video content recognition technology is used to convert the text, image or sound in the video into numerical values ​​using natural language processing or image recognition technology to determine the theme or keywords of the video. Next, considering multiple dimensions such as picture clarity, hot spots, time factors and character factors, a complex formula including sigmoid function, square root operation, logarithm operation and square operation is used to determine the key frame of the short video cover. In addition, popular memes and similar materials on the Internet are referenced to generate unique cover elements and add them to the cover to further enhance the attractiveness of the cover.

[0017] The innovation of this method is that it can automatically evaluate and select the best cover frame from multiple perspectives, and combine popular online memes and similar materials to generate highly attractive covers. This method not only improves the efficiency of cover production, but also optimizes the cover design in a data-driven way to make it more in line with user interests and preferences. Ultimately, through automated and intelligent technical means, the automatic production of self-media short video covers can be achieved to increase video viewing rate and user engagement. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of the method for automatically making a self-media short video cover provided by the first embodiment of the present invention;

[0019] Figure 2 It is a module diagram of the self-media short video cover automatic production device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] refer to Figure 1 As shown, the first embodiment of the present invention discloses a method for automatically making a self-media short video cover, which can be executed by a self-media short video cover automatic making device (hereinafter referred to as a making device), and in particular, is executed by one or more processors in the making device to implement the following method:

[0022] S1, obtaining a short video to be processed, splitting the short video into multiple separate frames, and using a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result;

[0023] Specifically, in this embodiment, the short video to be processed is first obtained. This step is the starting point of the entire process of automatically making self-media short video covers. Among them, users upload their short video content through a user-friendly interface. These contents can be any form of self-media videos, such as education, entertainment, news, etc. Next, the short video is split into multiple separate frames; this process is achieved through video processing technology, in which the video is parsed into a series of image frames, each frame is a momentary picture of the video. This frame extraction technology is the basis in the field of video analysis, which allows the system to perform a more detailed analysis of the video content.

[0024] Subsequently, a multimodal large model is used to perform storyboard understanding on each frame of the short video to obtain the video content recognition result. This step is one of the core innovations of this patent. The multimodal large model can process the visual and auditory information of the video at the same time. Through deep learning technology, the model can recognize and understand the scenes, characters, objects and voice content in the video, so as to obtain the content recognition result of each frame. This result includes not only the numerical values ​​converted from the text, images or sounds in the video, but also the theme or keywords of the video. This information is crucial for the selection of subsequent cover frames. Through this storyboard understanding, the key information and hot spots in the video can be accurately captured, and this information will be used for the subsequent cover frame selection and matching of copywriting materials. For example, if a hot topic or event appears in the video, this will be automatically identified and highlighted in the cover design to attract the audience's attention.

[0025] In addition, the application of this multimodal large model also brings other beneficial effects. First, it improves the accuracy and relevance of cover production, because the selection of cover frames and the matching of copy are based on a deep understanding of the video content. Second, this method greatly reduces the need for manual intervention, thereby improving the efficiency of cover production. Finally, it can automatically identify and respond to changes in video content, which can adapt to the ever-changing social media trends and user preferences, so that the cover design always remains fresh and attractive.

[0026] S2, based on the factor information, using a key frame determination formula to calculate the video content recognition result to obtain a key frame;

[0027] Preferably, the factor information includes picture clarity, climax, time factor, and character factor, wherein the picture clarity is obtained by using image processing technology, the time factor is obtained by analyzing the position of the key frame in the short video, and the character factor is obtained by using face recognition technology or object recognition technology.

[0028] Preferably, the mathematical expression of the key frame determination formula is: Among them, S is the key frame, C is the video content recognition result, D is the picture clarity, E is the burst point, T is the time factor, P is the character factor, and α, e, γ, δ, ∈, ξ are all weight parameters.

[0029] In this embodiment, weight parameters α, e, γ, δ, ∈ and ξ are used to adjust the influence of each item, wherein the value range of the weight parameters is between 0 and 1.

[0030] Preferably, in the key frame determination formula is a sigmoid function operation, which is used to simulate the influence of the video content recognition result C on the video. When the picture clarity D reaches a preset level, the saturation occurs. The key frame determination formula is a square root operation, which is used to simulate that the impact of the burst point E on the video quality is nonlinear. The log(1+T) in the key frame determination formula is a logarithmic operation, which is used to simulate that as the time factor T increases, its impact on the video quality gradually decreases. The P in the key frame determination formula 2 is a square operation, which is used to simulate the nonlinear effect of the character factor P on the video quality. It is an interactive operation, which is used to simulate the impact of the interaction between the video content recognition result C and the character factor P on the video quality. When the burst point E reaches the preset level, saturation occurs.

[0031] In this embodiment, step S2 is the core link in the automatic generation process of the cover, which involves comprehensive consideration of multiple factors, including picture clarity, hot spots, time factors and character factors. Specifically, the video content recognition result C can be obtained through natural language processing or image recognition technology; for example, the theme or keywords of the video can be obtained by identifying the text, image or sound in the video, and then this information can be converted into a numerical value. The higher the value, the more consistent the recognized content is with the theme of the video. The picture clarity D is obtained through image processing technology, which reflects the visual quality of each frame of the image. The higher the value, the clearer the picture. This method calculates the clarity index of each frame, such as sharpness, contrast, etc., to ensure that the selected cover frame has a high visual impact. Hot spot E refers to the degree of fit between the picture content and the video copy. This factor is determined by analyzing the climax or eye-catching moment in the video. The higher the fit, the higher the score. The time factor T is obtained by analyzing the position of the key frame in the short video. Considering that the audience's attention is often most concentrated at the beginning of the video, the frames at the beginning of the video will obtain a higher time factor weight. As time increases, the time factor of T will gradually decrease. The character factor P is obtained by using face recognition technology or object recognition technology. It reflects the importance of the character or object in the video. It converts the character or object information in each frame into a numerical value; higher weights are given to the pictures where characters appear.

[0032] In this embodiment, the sigmoid function operation in the key frame determination formula is used to simulate the influence of the video content recognition result C on the video, and the saturation occurs after the picture clarity D reaches a preset degree. The square root operation is used to simulate the nonlinear influence of the burst point E on the video quality, that is, the influence of the burst point increases with the increase of its intensity, but the growth rate gradually slows down. The logarithmic operation is used to simulate the increase of the time factor T, and its influence on the video quality gradually decreases, reflecting the trend of the audience's attention decreasing over time. The square operation is used to simulate the influence of the character factor P on the video quality is nonlinear, that is, the influence of the character factor increases with the increase of the character's importance, and the growth rate gradually accelerates. The interaction term operation is used to simulate the influence of the interaction between the video content recognition result C and the character factor P on the video quality, and the saturation occurs after the burst point E reaches a preset degree.

[0033] Through this key frame determination formula that comprehensively considers multiple factors, the method can automatically select the key frame that best represents the video content and attracts the audience's attention as the cover.

[0034] S3, obtaining popular memes and similar materials on the current network, obtaining cover elements according to the popular memes and similar materials, and adding the cover elements to the key frame to obtain a cover corresponding to the short video.

[0035] Specifically, step S3 includes: obtaining hot memes on the Internet in real time by using a web crawler technology, and extracting key information of the hot memes by using a natural language processing technology;

[0036] Based on image recognition and deep learning technology, screen out covers similar to the content of the short video from a preset video cover database as cover references;

[0037] The key information and the cover reference are combined to generate a cover element unique to the short video.

[0038] Specifically, in this embodiment, the popular memes on the Internet are obtained in real time through web crawler technology. This technology can monitor major social media platforms, forums and video sharing websites, and quickly capture the current popular language, expressions, topics and other elements. Through natural language processing technology, this method can extract key information from these massive data and identify the core elements of popular memes, such as specific words, phrases or emotional colors. These key information will be used as part of the cover elements to increase the attractiveness and timeliness of the cover. Then, based on image recognition and deep learning technology, covers similar to the content of the short video are screened out from the preset video cover database as cover references. This step utilizes the image recognition capability of the deep learning model to find covers that are similar in style, theme or emotion by comparing the key frames of the short video with the visual features of the cover in the database. Such a cover reference can not only provide design inspiration, but also ensure the consistency of the cover with the video content.

[0039] Finally, by combining the key information of the popular meme and the cover reference, a unique cover element for the short video is generated. This process involves the integration of creativity and innovation of design. This method will automatically adjust and optimize the cover elements so that they are both in line with the video content and can catch the audience's attention. For example, if the popular meme is about a certain festival, festival-related icons or colors may be added to the cover; if the cover reference shows that a certain layout is more popular, the position of the copy and image may be adjusted to increase the attractiveness of the cover. Through this automated cover element generation and addition process, the efficiency of cover production is greatly improved, and the time and cost of manual design are reduced; secondly, by capturing network hot spots in real time, the cover can keep up with the trend and increase the click-through rate and viewing rate of the video; in addition, the generated cover elements are highly personalized and innovative, and can customize a unique cover for each video to enhance the recognition and attractiveness of the video.

[0040] In summary, the self-media short video cover automatic production method realizes a fully automated process from video content analysis to cover element generation through a series of technical means. Specifically, the method first splits the uploaded video into separate frames through video processing technology, and uses a multimodal large model to perform in-depth content recognition on each frame to extract the theme and keywords of the video. On this basis, the system comprehensively considers multiple dimensions such as picture clarity, burst point, time factor and character factor, and determines the key frame through a complex mathematical formula. The formula uses mathematical tools such as sigmoid function, square root operation, logarithm operation and square operation to simulate the nonlinear characteristics and saturation effects of different factors on video quality. In addition, the popular stalks on the Internet are obtained in real time through web crawler technology, and key information is extracted through natural language processing technology. At the same time, image recognition and deep learning technology are used to screen out covers similar to the short video content from the preset video cover database as reference. Finally, the key information of the popular stalks and the cover reference are combined to generate a unique cover element, which is added to the key frame to obtain the cover corresponding to the short video.

[0041] This method not only improves the efficiency and quality of cover production, but also significantly improves the attractiveness and viewing rate of videos by capturing online hot spots in real time and personalized design, providing self-media creators with a powerful tool to help them gain an advantage in a highly competitive market. Through automated and intelligent technical means, this method realizes the automatic production of self-media short video covers to increase video viewing rates and user engagement, demonstrating its importance in improving efficiency, reducing costs, and enhancing attractiveness and competitiveness.

[0042] See also Figure 2 The second embodiment of the present invention provides a self-media short video cover automatic production device, which includes:

[0043] The video preprocessing unit 201 is used to obtain a short video to be processed, split the short video into multiple separate frames, and use a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result;

[0044] A key frame determination unit 202, configured to calculate the video content recognition result based on the factor information using a key frame determination formula to obtain a key frame;

[0045] The cover generation unit 203 is used to obtain popular memes and similar materials on the current network, obtain cover elements based on the popular memes and similar materials, and add the cover elements to the key frame to obtain a cover corresponding to the short video.

[0046] Preferably, the factor information includes picture clarity, climax, time factor, and character factor, wherein the picture clarity is obtained by using image processing technology, the time factor is obtained by analyzing the position of the key frame in the short video, and the character factor is obtained by using face recognition technology or object recognition technology.

[0047] Preferably, the mathematical expression of the key frame determination formula is: Among them, S is the key frame, C is the video content recognition result, D is the picture clarity, E is the burst point, T is the time factor, P is the character factor, and α, e, γ, δ, ∈, ξ are all weight parameters.

[0048] Preferably, in the key frame determination formula is a sigmoid function operation, which is used to simulate the influence of the video content recognition result C on the video. When the picture clarity D reaches a preset level, the saturation occurs. The key frame determination formula is a square root operation, which is used to simulate that the impact of the burst point E on the video quality is nonlinear. The log(1+T) in the key frame determination formula is a logarithmic operation, which is used to simulate that as the time factor T increases, its impact on the video quality gradually decreases. The P in the key frame determination formula 2 is a square operation, which is used to simulate the nonlinear effect of the character factor P on the video quality. It is an interactive operation, which is used to simulate the impact of the interaction between the video content recognition result C and the character factor P on the video quality. When the burst point E reaches the preset level, saturation occurs.

[0049] Preferably, the hot memes and similar materials on the current network are obtained, and the copywriting data is obtained according to the hot memes and similar materials, specifically:

[0050] Using web crawler technology to obtain popular memes on the Internet in real time, and using natural language processing technology to extract key information of the popular memes;

[0051] Based on image recognition and deep learning technology, screen out covers similar to the content of the short video from a preset video cover database as cover references;

[0052] The key information and the cover reference are combined to generate a cover element unique to the short video.

[0053] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for automatically making a self-media short video cover, characterized in that: include: Obtain a short video to be processed, split the short video into multiple separate frames, and use a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result; Based on the factor information, the video content recognition result is calculated using a key frame determination formula to obtain a key frame; Obtain popular memes and similar materials on the current network, obtain cover elements based on the popular memes and similar materials, and add the cover elements to the key frame to obtain a cover corresponding to the short video.

2. The method for automatically making a self-media short video cover according to claim 1 is characterized in that: The factor information includes picture clarity, climax, time factor, and character factor, wherein the picture clarity is obtained by using image processing technology, the time factor is obtained by analyzing the position of the key frame in the short video, and the character factor is obtained by using face recognition technology or object recognition technology.

3. The method for automatically making a self-media short video cover according to claim 1 is characterized in that: The mathematical expression of the key frame determination formula is: Among them, S is the key frame, C is the video content recognition result, D is the picture clarity, E is the burst point, T is the time factor, P is the character factor, and α, e, γ, δ, ∈, ξ are all weight parameters.

4. The method for automatically making a self-media short video cover according to claim 3 is characterized in that: The key frame determination formula is a sigmoid function operation, which is used to simulate the influence of the video content recognition result C on the video. When the picture clarity D reaches a preset level, the saturation occurs. The key frame determination formula is a square root operation, which is used to simulate that the impact of the burst point E on the video quality is nonlinear. The log(1+T) in the key frame determination formula is a logarithmic operation, which is used to simulate that as the time factor T increases, its impact on the video quality gradually decreases. The P in the key frame determination formula 2 is a square operation, which is used to simulate the nonlinear effect of the character factor P on the video quality. It is an interactive operation, which is used to simulate the impact of the interaction between the video content recognition result C and the character factor P on the video quality. When the burst point E reaches the preset level, saturation occurs.

5. The method for automatically making a self-media short video cover according to claim 1 is characterized in that: Obtain the current popular memes and similar materials on the Internet, and obtain copywriting data based on the popular memes and similar materials, specifically: Using web crawler technology to obtain popular memes on the Internet in real time, and using natural language processing technology to extract key information of the popular memes; Based on image recognition and deep learning technology, screen out covers similar to the content of the short video from a preset video cover database as cover references; The key information and the cover reference are combined to generate a cover element unique to the short video.

6. A self-media short video cover automatic production device, characterized in that: include: A video preprocessing unit is used to obtain a short video to be processed, split the short video into multiple separate frames, and use a multimodal large model to perform storyboard understanding on each frame of the short video to obtain a video content recognition result; A key frame determination unit, configured to calculate the video content recognition result based on the factor information using a key frame determination formula to obtain a key frame; A cover generation unit is used to obtain popular memes and similar materials on the current network, obtain cover elements based on the popular memes and similar materials, and add the cover elements to the key frame to obtain a cover corresponding to the short video.

7. The self-media short video cover automatic production device according to claim 6 is characterized in that: The factor information includes picture clarity, climax, time factor, and character factor, wherein the picture clarity is obtained by using image processing technology, the time factor is obtained by analyzing the position of the key frame in the short video, and the character factor is obtained by using face recognition technology or object recognition technology.

8. The self-media short video cover automatic production device according to claim 6 is characterized in that: The mathematical expression of the key frame determination formula is: Among them, S is the key frame, C is the video content recognition result, D is the picture clarity, E is the burst point, T is the time factor, P is the character factor, and α, e, γ, δ, ∈, ξ are all weight parameters.

9. The self-media short video cover automatic production device according to claim 8 is characterized in that: The key frame determination formula is a sigmoid function operation, which is used to simulate the influence of the video content recognition result C on the video. When the picture clarity D reaches a preset level, the saturation occurs. The key frame determination formula is a square root operation, which is used to simulate that the impact of the burst point E on the video quality is nonlinear. The log(1+T) in the key frame determination formula is a logarithmic operation, which is used to simulate that as the time factor T increases, its impact on the video quality gradually decreases. The P in the key frame determination formula 2 is a square operation, which is used to simulate the nonlinear effect of the character factor P on the video quality. It is an interactive operation, which is used to simulate the impact of the interaction between the video content recognition result C and the character factor P on the video quality. When the burst point E reaches the preset level, saturation occurs.

10. The self-media short video cover automatic production device according to claim 6 is characterized in that: Obtain the current popular memes and similar materials on the Internet, and obtain copywriting data based on the popular memes and similar materials, specifically: Using web crawler technology to obtain popular memes on the Internet in real time, and using natural language processing technology to extract key information of the popular memes; Based on image recognition and deep learning technology, screen out covers similar to the content of the short video from a preset video cover database as cover references; The key information and the cover reference are combined to generate a cover element unique to the short video.

Images