Hot short video automatic generation method and device, computer equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a hotspot short video automatic generation method and apparatus, a computer device and a storage medium, and the method comprises the steps of obtaining hotspot information, and determining a hotspot picture and a hotspot text corresponding to the hotspot information by using a large language model; determining a hotspot voice corresponding to the hotspot text, performing emotion perception on the hotspot voice, and determining an emotion contained in the hotspot voice and background music corresponding to the emotion; and generating a hotspot short video corresponding to the hotspot information based on the hotspot picture, the hotspot voice and the background music. According to the technical scheme provided by one or more embodiments disclosed by the invention, the hotspot picture and the hotspot text corresponding to the hotspot information can be determined by adopting the large language model, so that the short video generation speed is increased; the hotspot short video is generated based on the hotspot picture, the hotspot voice and the background music, and the generated short video content is coordinated and matched with the background music, so that the quality of the short video can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for automatically generating hot short videos. Background Art

[0002] In the current era of mobile Internet, the rapid capture and dissemination of hot events and topics has become one of the core competitiveness of major media and social platforms; some short video automatic generation methods based on artificial intelligence (AI) have appeared in the existing technology. However, the short videos generated by AI in the existing technology have the problems of low speed and low quality, and a better solution is needed. Summary of the invention

[0003] In view of this, the present invention provides a method, device, computer equipment and storage medium for automatically generating hot short videos to solve the problem of how to provide users with hot information short video services quickly and with high quality.

[0004] On the one hand, the present invention provides a method for automatically generating hot short videos, which includes obtaining hot information, using a large language model to determine the hot pictures and hot texts corresponding to the hot information; determining the hot voice corresponding to the hot text, performing emotion perception on the hot voice, determining the emotions contained in the hot voice, and the background music corresponding to the emotions; and generating a hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music.

[0005] On the other hand, the present invention also provides a device for automatically generating hot short videos, which includes: an acquisition module, used to acquire hot information, and use a large language model to determine the hot pictures and hot texts corresponding to the hot information; an analysis module, used to determine the hot voice corresponding to the hot text, perform emotion perception on the hot voice, determine the emotions contained in the hot voice, and the background music corresponding to the emotions; a generation module, used to generate a hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music.

[0006] On the other hand, the present invention also provides a computer device, which includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned method for automatically generating hot short videos by executing the computer instructions.

[0007] On the other hand, the present invention also provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned method for automatically generating hot short videos.

[0008] In this process, on the one hand, a large language model is used to quickly generate hot images and hot texts corresponding to the hot information, thereby improving the efficiency of short video generation; on the other hand, emotional perception of hot voices is performed to determine background music that matches the emotions of the hot voices, so that the generated short video content can be coordinated with the background music, thereby improving the quality of short video generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 It is a flowchart of a method for automatically generating hot short videos provided by an embodiment of the present invention;

[0011] Figure 2 It is a specific flow chart of a method for automatically generating hot short videos provided by an embodiment of the present invention;

[0012] Figure 3 It is a structural schematic diagram of a hot short video automatic generation device provided by an embodiment of the present invention;

[0013] Figure 4 It is a structural schematic diagram of another hot short video automatic generation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In recent years, with the rapid development of Internet technology and short video platforms, we are currently in a hot era of the short video industry, and the amount of information generated every day is growing exponentially; how to quickly capture and disseminate hot events and topics has become one of the core competitiveness of major media and social platforms; the existing short video generation methods still have traditional manual screening and editing to determine hot information, and use short video generation software to generate short videos based on preset content.

[0015] However, the existing solutions have the following problems:

[0016] 1. The manual screening of hot information is time-consuming, labor-intensive, and inefficient. It lacks the ability to independently mine and transform hot information, resulting in slow short video generation.

[0017] 2. The text-only short videos generated by the AI ​​model are of low quality;

[0018] 3. The background music in the generated short video is inconsistent with the picture content and video content.

[0019] In order to solve at least one of the above problems, various embodiments of the present invention provide a method for automatically generating hot short videos, including: obtaining hot information, using a large language model to determine the hot pictures and hot texts corresponding to the hot information; determining the hot voice corresponding to the hot text, performing emotion perception on the hot voice, determining the emotions contained in the hot voice, and the background music corresponding to the emotions; generating a hot short video corresponding to the hot information based on the hot picture, hot voice and background music.

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] According to an embodiment of the present invention, a method for automatically generating hot short videos is provided. Figure 1 is a flow chart of a method for automatically generating hot short videos provided by an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0022] Step S101, obtaining hot spot information, and using a large language model to determine hot spot images and hot spot texts corresponding to the hot spot information;

[0023] Step S102, determining the hot voice corresponding to the hot text, performing emotion perception on the hot voice, determining the emotion contained in the hot voice, and the background music corresponding to the emotion;

[0024] Step S103, generating a hot short video corresponding to the hot information based on the hot picture, hot voice and background music.

[0025] In a possible implementation, hot information may refer to topics, events, or public opinion trends that have attracted widespread attention on multiple platforms such as social media and news websites.

[0026] Furthermore, the method of obtaining hotspot information may include:

[0027] By searching the Application Programming Interface (API) interface and using web crawler technology, hot information in designated links or network resources can be captured.

[0028] Here, the API interface may refer to a predefined function, through which certain services or data can be accessed; the API interface may include a search engine API and a social media API, through which hot information can be efficiently obtained from specified services without the need to manually retrieve and extract data.

[0029] Furthermore, web crawler technology can refer to an automated program that can browse the Internet and crawl web page content; by crawling information through a specific Uniform Resource Locator (URL) link or a set of predefined web pages, updates of specific websites can be quickly monitored; by crawling the entire network resources, hot information can be collected on a large scale; by searching for relevant API interfaces and using web crawler technology, efficient crawling of hot information in specified links or network resources can be achieved.

[0030] In one possible implementation, a large language model (LLM) may refer to a neural network model based on deep learning, which can be used to process and generate natural language text.

[0031] Furthermore, hot spot pictures and hot spot texts may refer to visual images related to hot spot information, and hot spot texts may refer to text content that describes or explains the hot spot information.

[0032] In a possible implementation, performing emotion perception on the hotspot speech and determining the emotion contained in the hotspot speech and the background music corresponding to the emotion may include:

[0033] Determine the emotion types corresponding to the hot speech in different time sequences, determine the background music clips corresponding to the hot speech in different time sequences according to the emotion categories, and splice the background music clips to determine the background music of the hot short video.

[0034] Specifically, emotion perception may refer to the use of a preset emotion perception model to analyze the features of hot speech (such as pitch, speaking speed, volume, etc.) and identify the emotion types of hot speech in different time series; the preset emotion perception model may refer to machine learning and deep learning models, such as recurrent neural networks (RNN) and long short-term memory networks (LSTM).

[0035] For example, the hotspot speech can be segmented into short time windows (i.e., time series) for frequency domain analysis through short-time Fourier Transform (STFT), and the corresponding emotion type (such as happiness, sadness, anger, etc.) can be output in each short time window through a trained preset emotion perception model.

[0036] Through the above method, hot short videos with unified background music, hot pictures and hot voice emotions are determined, which can improve the consistency of short video content and user viewing experience; by automatically capturing hot information on the Internet and generating hot short videos, a comprehensive and accurate presentation of hot information can be provided.

[0037] In one embodiment, obtaining hotspot information is implemented based on one of the following steps:

[0038] Obtain network information in real time across platforms, use hot information perception algorithms to identify keywords whose appearance times on multiple platforms are greater than the frequency threshold, and determine the keywords as hot information;

[0039] Obtain interactive data of network information on multiple platforms, use hotspot perception algorithm to analyze interactive data, and determine network information as hotspot information when interactive data meets interactive conditions; interactive data includes but is not limited to: click-through rate, number of reposts, number of comments, and number of favorites;

[0040] The heat increase speed of the network information is obtained, and when the heat increase speed is greater than a speed threshold, the network information is determined as hot spot information.

[0041] The platform may include but is not limited to: forum communities, news websites, social media, and search engine trends.

[0042] In a possible implementation, network information is obtained in real time across platforms, a hot information perception algorithm is used to identify keywords whose appearance times in network information on multiple platforms are greater than a frequency threshold, and the keywords are determined as hot information. This can be implemented based on the following steps:

[0043] Acquire network information in real time, use hot information perception algorithm to identify the term frequency and inverse document frequency (TF-IDF) value of each keyword in the network information, and determine the keywords whose number of appearances on multiple platforms is greater than the frequency threshold based on the TF-IDF value.

[0044] Among them, the TF-IDF value can be a statistical method used for text mining and information retrieval to evaluate the importance of a word in a document set or corpus; among them, TF can refer to the number of times a word appears in a document, and IDF can be a measure of the general importance of a word in the entire corpus.

[0045] Furthermore, a comprehensive analysis is performed on the keywords in the network information on multiple platforms. If the number of occurrences of a specific keyword on multiple platforms is greater than a frequency threshold, the specific keyword can be determined to be hot information.

[0046] In a possible implementation, the interactive data of network information on multiple platforms is obtained, and the interactive data is analyzed using a hotspot perception algorithm. When the interactive data meets the interactive conditions, the network information is determined as hotspot information; the interactive data includes but is not limited to: click rate, number of reposts, number of comments, and number of favorites, which can be implemented based on one of the following steps:

[0047] Obtain interactive data of network information on multiple platforms, and perform pre-processing and data cleaning on the interactive data;

[0048] Based on the interaction data, determine the interaction indicators corresponding to the network information, the interaction indicators include but are not limited to: interaction score;

[0049] When the interaction indicator satisfies the interaction condition, the network information is determined as information.

[0050] For example, the interaction score may be expressed as follows:

[0051] Interaction score = α·click rate + β·number of reposts + γ·number of comments + δ·number of favorites

[0052] Among them, α, β, γ and δ can be the weight values ​​corresponding to the click rate, the number of reposts, the number of comments and the number of favorites respectively, and the weight values ​​of different interactive data can be adjusted according to specific needs; when the interactive score meets the interactive score threshold in the interactive condition, the network information can be determined as hot information.

[0053] In a possible implementation, obtaining the heat increase speed of the network information, and determining the network information as hot spot information when the heat increase speed is greater than a speed threshold, can be implemented based on one of the following steps:

[0054] Determine the speed at which the network information increases in popularity on multiple platforms, and determine a comparison result between the speed at which the network information increases and a speed threshold; when the speed at which the network information increases is greater than the speed threshold, determine the network information as hot information.

[0055] For example, the heat rise rate can be expressed as follows:

[0056]

[0057] When the heat increase speed meets the speed threshold, the network information can be determined as hot spot information.

[0058] In a possible implementation, the method for obtaining hotspot information further includes:

[0059] The geographical location information and time information corresponding to each keyword in the network information are obtained. When in a specific time period and / or a specific geographical location, a specific keyword is more likely to become hot information.

[0060] For example, when the time information is in the Spring Festival period, keywords such as Spring Festival couplets and New Year's Eve dinner are more likely to become hot information; when the geographical location information is in the coastal area, keywords such as beach, seafood, and sunshine are more likely to become hot information.

[0061] Here, the location data corresponding to the geographical location information corresponding to each keyword in the network information can be converted into a coordinate format to display the popularity of keywords in different regions in the form of a heat map.

[0062] Through the above method, the latest hot information is obtained in real time to ensure the timeliness of the generated short video and ensure that the short video keeps up with current affairs; through web crawlers and API search methods, the manpower and time cost of manually determining hot information is reduced, thereby improving economy.

[0063] In one embodiment, a large language model is used to determine the hot image and hot text corresponding to the hot information, which is implemented based on the following steps:

[0064] Perform hot spot analysis on hot spot information, determine key information in the hot spot information, and determine corresponding hot spot images based on the key information; key information includes but is not limited to: people, places, events;

[0065] Use image recognition models to determine the image information in hot pictures, including but not limited to: picture scenes, picture objects, and content sentiment tendencies;

[0066] Based on key information and image information, a large language model is used to generate hot text that matches the hot image; the hot text is used to describe the hot image.

[0067] In a possible implementation, performing hotspot analysis on hotspot information may include:

[0068] De-duplicate and format the hot information to remove irrelevant content in the hot information; irrelevant content may include duplicate data and HTML tags;

[0069] Perform entity recognition on hot information, determine key information such as people, places, events mentioned in the hot information, and classify the hot information based on the key information;

[0070] Filter hot information and remove sensitive content from it.

[0071] Furthermore, crawling corresponding hot spot images on the network based on the determined key information belongs to the existing technology and will not be described in detail here.

[0072] In a possible implementation, using an image recognition model to determine image information in a hotspot image may include:

[0073] An image recognition model is used to determine the image scene and image object in the hotspot image, and the identified image scene and image object are compared with the key information in the hotspot information;

[0074] The emotion perception model is used to determine the emotional tendency of the content in the hot pictures for the generation of hot texts.

[0075] Among them, the image recognition model may include but is not limited to: convolutional neural network, target detection model; the picture scene in the hot picture may refer to the background environment or scene in the picture, the picture object may refer to the specific object or person in the picture, and the content emotional tendency may refer to the emotion expressed by the picture.

[0076] In a possible implementation, based on key information and image information, a large language model is used to generate hot text that matches the hot image, which may include:

[0077] A large language model is used to analyze and summarize hot information to make the analyzed hot content comprehensive and accurate.

[0078] Specifically, through the natural language understanding capabilities of the large language model, the most relevant and valuable hot texts can be accurately identified and refined, and redundant and irrelevant content can be removed.

[0079] Through the above method, by extracting key information, hot spot information can be accurately determined to provide accurate input for subsequent steps; using a large language model, hot spot text that matches the image content can be generated, which can ensure that the hot spot text is highly consistent with the image content, thereby enhancing the coherence and relevance of the hot spot information.

[0080] In one embodiment, determining the hotspot speech corresponding to the hotspot text, performing emotion perception on the hotspot speech, determining the emotion contained in the hotspot speech, and the background music corresponding to the emotion are implemented based on the following steps:

[0081] Use text-to-speech synthesis technology to generate hot speech corresponding to the hot text, and use audio signal processing technology to extract acoustic features in the hot speech;

[0082] Adopting a neural network-based emotion recognition model to determine the emotion type corresponding to the acoustic features;

[0083] A music index table corresponding to the emotion type is matched in the music library, and at least one music clip is selected from the music index table as background music.

[0084] Here, text-to-speech (TTS) technology may be an artificial intelligence technology for converting written text information into audible speech output.

[0085] Furthermore, acoustic features may include but are not limited to: fundamental frequency (Pitch), energy distribution, spectral envelope, and Mel-frequency cepstral coefficients (MFCCs); wherein, the fundamental frequency may be the frequency of vocal cord vibration in a speech signal, which may characterize the pitch of a hotspot speech, and by determining the pitch of the fundamental frequency, it may be possible to identify whether the emotion in the hotspot speech is excited or calm; energy distribution is used to characterize the energy distribution of a speech signal at different frequencies, and energy distribution may characterize changes in emotional state and intonation; the spectral envelope is a smooth curve of the speech signal spectrum, which represents the main contours of the spectrum and is used to analyze the vocal tract shape and the opening and closing state of the mouth of a speech signal; the Mel-frequency cepstral coefficient may be a coefficient representing the spectral characteristics of a speech signal, and is used to extract and represent the sound characteristics of a speech signal.

[0086] By adopting audio signal processing technology to extract the acoustic features in hot spot speech, the speech data of hot spot speech can be fully understood and processed.

[0087] In a possible implementation, a neural network-based emotion recognition model is used to determine the emotion type corresponding to the acoustic feature, which may include:

[0088] The extracted acoustic features are input into the neural network-based emotion recognition model to output the emotion type corresponding to the hot spot speech.

[0089] Among them, the emotion recognition model based on neural network can include but is not limited to: convolutional neural network, recursive neural network, long short-term memory network.

[0090] Through the above method, through text-to-speech technology, the generated hot speech can naturally reflect the hot text, ensuring the naturalness and comprehensibility of the hot speech; using the emotion recognition model to determine the emotion type corresponding to the hot speech and match the background music of the corresponding emotion type, the audiovisual presentation effect of the short video can be enhanced, and the automation and efficiency of content processing can be improved.

[0091] In one embodiment, matching and determining a music index table corresponding to an emotion type in a music library, and selecting at least one music clip in the music index table as background music, includes:

[0092] Based on the time series, at least one emotion type in the hot music is determined, and based on the at least one emotion type, a corresponding music style is determined by matching in a music library, and a music clip is selected from the at least one music style;

[0093] Based on the time sequence, at least one music clip is spliced ​​to determine the background music; the splicing of the music clips adopts a cross-fade technology.

[0094] In a possible implementation, the music library can be implemented based on the following steps:

[0095] Define the types of emotions that need to be mapped in the music library, which may include but are not limited to: happy, sad, angry, relaxed, excited, worried;

[0096] Obtain music clips of each music style, and use an automated emotion analysis tool to label the corresponding emotion type for each music clip;

[0097] Extracting audio features corresponding to each music clip, which may include but are not limited to: rhythm, melody, harmony, and timbre;

[0098] The extracted audio features are associated with the marked emotion types, the mapping relationship between each emotion type and the music style is determined, and a music library is established to store the mapping relationship between the emotion type and the music style.

[0099] For example, if the emotion type is happiness, the music style that can be retrieved from the music library is fast-tempo and high-pitched; if the emotion type is sadness, the music style that can be retrieved from the music library is slow-tempo and low-pitched.

[0100] In a possible implementation, determining at least one emotion type in hot music based on the time series, matching and determining a corresponding music style in a music library based on the at least one emotion type, and selecting a music clip in the at least one music style may include:

[0101] Based on the time series, the corresponding emotion type of each time window in the hot music is determined, and each emotion type is matched in the music library to determine the corresponding music clip.

[0102] Further, based on the time sequence, splicing the at least one music clip to determine the background music may include:

[0103] Each music clip is smoothly transitioned using cross-fade technology, and multiple music clips are spliced ​​into background music to make the background music continuous and natural.

[0104] Through the above method, it is possible to ensure that the background music is consistent with the emotional atmosphere of the video content, thereby enhancing the emotional expressiveness and appeal of the video; the video viewing can be made more attractive and emotionally resonant, thereby improving the user's viewing experience.

[0105] In one embodiment, generating a hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music includes:

[0106] The hot voices are divided into normal communication voices and picture introduction voices. Based on the normal communication voices, digital human generation technology is used to generate digital human communication videos;

[0107] Generate a first video based on the picture introduction voice and the hot spot pictures;

[0108] Generate a hot short video corresponding to the hot information based on the first video, the digital human communication video, and the background music.

[0109] In one possible implementation, normal communication voice may refer to voice content similar to daily human communication, and may refer to voice content for digital characters or virtual characters, so that they can communicate naturally in the video and create a more real and intimate communication experience with the audience.

[0110] Furthermore, based on normal communication voice, the digital human communication video is generated by using the digital human generation technology, which may include:

[0111] Digital human generation technology is used to synthesize normal communication voice with digital characters to generate digital human communication videos; wherein, the digital human communication videos may include the digital human's movements and expressions.

[0112] Here, digital human generation technology can refer to a technology that uses computer graphics, artificial intelligence, virtual reality, augmented reality and other related technologies to create realistic digital representations of humans, and is used to create a digital character that is close to a real person in appearance, movement, voice and emotional expression.

[0113] In a possible implementation, generating a hot short video corresponding to the hot information according to the first video, the digital human communication video, and the background music may include:

[0114] The hot short video corresponding to the hot information is generated by splicing the first video, the digital human communication video, the background music, the preset beginning video clip and the preset ending video clip.

[0115] Through the above method, the digital human communication video is integrated when generating hot short videos, which can make the presentation of hot information more intuitive and vivid; the emotional expression of the video is increased by combining voice, pictures and background music; through a variety of artificial intelligence methods, the generation and splicing of multiple media elements are realized, which improves the processing efficiency of short video generation and ensures the quality and consistency of the video.

[0116] In one embodiment, the picture information also includes the number of pictures. Based on the key information and the picture information, a large language model is used to generate hot texts that match the hot pictures, and the following is also included:

[0117] The prompt word constraint adjusted by the large language model is obtained, so that the large language model generates hot texts corresponding to each hot image based on the number of images in the image information.

[0118] Specifically, the prompt word constraint after adjustment of the large language model is obtained, and the large language model is guided to follow the preset segmentation rule when generating hot texts, so that the hot texts generated by the large language model meet the number of pictures in the picture information.

[0119] Exemplarily, when the number of pictures is 1, the prompt word constraint for adjusting the large language model can be "Please generate the following narrative structure: 1. Preface explanation, 2. Picture description, 3. Ending explanation"; when the number of pictures is 2, the prompt word constraint for adjusting the large language model can be "Please generate the following narrative structure: 1. Preface explanation, 2. First picture description, 3. Second picture description, 4. Ending explanation".

[0120] Furthermore, special identifiers with numbers are used in the generated hotspot text to clearly distinguish each part in the hotspot text, and the type of each part can be identified according to these identifiers or structured data when converting into hotspot speech.

[0121] For example, the special tags may be [1], [2], and [3] in “[1] previous explanation, [2] picture description, and [3] ending explanation”.

[0122] Through the above method, numbers and special symbols are added at the beginning of each part, so that the different parts of the text can be clearly seen at a glance, which is easy for subsequent processing and further improves the efficiency of short video generation.

[0123] In a specific embodiment, Figure 2 is a specific flow chart of a method for automatically generating hot short videos provided by an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0124] Step S201, determining hotspot information;

[0125] Here, a hotspot information perception algorithm is used to determine hotspot information based on network information;

[0126] Step S202, removing sensitive information;

[0127] Here, filter out sensitive information such as bad information in hot information;

[0128] Step S203, determining key information;

[0129] Here, key information in the hot information is determined;

[0130] Step S204, determining hot spot pictures;

[0131] Here, hot images are retrieved through key information;

[0132] Step S205, determining the picture information in the hotspot image;

[0133] Here, an image recognition model is used to determine the image information in the hotspot image;

[0134] Step S206, performing hot spot analysis on the hot spot information;

[0135] Step S207, generating hotspot text;

[0136] Here, based on the large language model, the hot texts that are highly consistent with the hot images are determined;

[0137] Step S208, generating hotspot voice;

[0138] Here, text-to-speech synthesis technology is used to generate hotspot speech corresponding to the hotspot text;

[0139] Step S209, generating normal communication voice;

[0140] Step S210, generating a picture introduction voice;

[0141] Step S211, generating a digital human communication video;

[0142] Step S212, generating a first video;

[0143] Here, the first video is generated based on the picture introduction voice and the hot spot picture;

[0144] Step S213, determining the emotion type of the hotspot speech;

[0145] Here, a neural network-based emotion recognition model is used to determine the emotion type corresponding to the acoustic features of the hotspot speech;

[0146] Step S214, determining a plurality of music clips;

[0147] Here, based on the time series, at least one emotion type in the hot music is determined, and based on the at least one emotion type, a corresponding music style is determined by matching in the music library, and a music clip is selected from the at least one music style;

[0148] Here, emotion perception is used to determine the emotion label of the hot voice; the music clip is determined by matching according to the preset music library;

[0149] Step S215, generating background music;

[0150] Step S216, generate hot short videos.

[0151] Figure 3 is a schematic diagram of the structure of a hot short video automatic generation device provided by an embodiment of the present invention, such as Figure 3 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: an acquisition module 301, an analysis module 302 and a generation module 303;

[0152] The acquisition module 301 is used to acquire hot spot information and use a large language model to determine the hot spot image and hot spot text corresponding to the hot spot information;

[0153] The analysis module 302 is used to determine the hot speech corresponding to the hot text, perform emotion perception on the hot speech, determine the emotion contained in the hot speech, and the background music corresponding to the emotion;

[0154] The generation module 303 is used to generate a hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music.

[0155] In one embodiment, the acquisition module 301 is used to acquire network information in real time across platforms, use a hot information perception algorithm to identify keywords of the network information whose occurrence times in multiple platforms are greater than a frequency threshold, and determine the keywords as the hot information;

[0156] Obtaining interactive data of the network information on multiple platforms, analyzing the interactive data using the hotspot sensing algorithm, and determining the network information as the hotspot information when the interactive data meets the interactive conditions; the interactive data includes but is not limited to: click rate, number of reposts, number of comments, and number of favorites;

[0157] The heat increase speed of the network information is obtained, and when the heat increase speed is greater than a speed threshold, the network information is determined as the hot spot information.

[0158] In one embodiment, the acquisition module 301 is used to perform hot spot analysis on the hot spot information, determine key information in the hot spot information, and determine corresponding hot spot images based on the key information; the key information includes but is not limited to: people, places, events;

[0159] Use image recognition models to determine the image information in hot pictures, including but not limited to: picture scenes, picture objects, and content sentiment tendencies;

[0160] Based on key information and image information, a large language model is used to generate hot text that matches the hot image; the hot text is used to describe the hot image.

[0161] In one embodiment, the analysis module 302 is used to generate a hotspot speech corresponding to the hotspot text by using text-to-speech synthesis technology, and extract acoustic features in the hotspot speech by using audio signal processing technology;

[0162] Adopting a neural network-based emotion recognition model to determine the emotion type corresponding to the acoustic features;

[0163] A music index table corresponding to the emotion type is matched in the music library, and at least one music clip is selected from the music index table as background music.

[0164] In one embodiment, the analysis module 302 is used to determine at least one emotion type in the hot music based on the time series, match and determine a corresponding music style in the music library based on the at least one emotion type, and select a music clip in the at least one music style;

[0165] Based on the time sequence, at least one music clip is spliced ​​to determine the background music; the splicing of the music clips adopts a cross-fade technology.

[0166] In one embodiment, the generating module 303 is used to divide the hot voice into normal communication voice and picture introduction voice, and generate a digital human communication video based on the normal communication voice using digital human generation technology;

[0167] Generate a first video based on the picture introduction voice and the hot spot pictures;

[0168] Generate a hot short video corresponding to the hot information based on the first video, the digital human communication video, and the background music.

[0169] In one embodiment, the analysis module 302 is further used to obtain the prompt word constraints adjusted by the large language model, so that the large language model generates hot texts corresponding to each hot picture based on the number of pictures in the picture information.

[0170] It should be noted that: the hot short video automatic generation device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding hot short video automatic generation method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-described processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0171] The embodiment of the present invention also provides a computer device having the above Figure 3 The hot short video automatic generation device shown.

[0172] See also Figure 4 , Figure 4 is a schematic diagram of the structure of another hot short video automatic generation device provided by an embodiment of the present invention. Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0174] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0178] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0179] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0180] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0181] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0182] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for automatically generating hot short videos, characterized in that: The method comprises: Acquire hot spot information, and use a large language model to determine hot spot images and hot spot texts corresponding to the hot spot information; Determine the hot voice corresponding to the hot text, perform emotion perception on the hot voice, determine the emotion contained in the hot voice, and the background music corresponding to the emotion; A hot short video corresponding to the hot information is generated based on the hot picture, the hot voice and the background music.

2. The method according to claim 1, characterized in that Get hotspot information based on one of the following steps: Acquire network information in real time across platforms, use a hot information perception algorithm to identify keywords whose appearance times of the network information on multiple platforms are greater than a frequency threshold, and determine the keywords as the hot information; Obtaining interactive data of the network information on multiple platforms, analyzing the interactive data using the hotspot sensing algorithm, and determining the network information as the hotspot information when the interactive data meets the interactive conditions; the interactive data includes but is not limited to: click rate, number of reposts, number of comments, and number of favorites; The heat increase speed of the network information is obtained, and when the heat increase speed is greater than a speed threshold, the network information is determined as the hot spot information.

3. The method according to claim 2, characterized in that The hot image and hot text corresponding to the hot information are determined using a large language model, based on the following steps: Perform hot spot analysis on the hot spot information to determine key information in the hot spot information, and determine corresponding hot spot pictures based on the key information; the key information includes but is not limited to: people, places, and events; Using an image recognition model to determine the image information in the hot image, the image information includes but is not limited to: image scene, image object and content emotional tendency; Based on the key information and the image information, a large language model is used to generate hot text that matches the hot image; the hot text is used to describe the hot image.

4. The method according to claim 3, characterized in that: Determining the hot voice corresponding to the hot text, performing emotion perception on the hot voice, determining the emotion contained in the hot voice, and the background music corresponding to the emotion, is achieved based on the following steps: Using text-to-speech synthesis technology to generate the hotspot speech corresponding to the hotspot text, and using audio signal processing technology to extract acoustic features in the hotspot speech; Using a neural network-based emotion recognition model to determine the emotion type corresponding to the acoustic feature; A music index table corresponding to the emotion type is matched in a music library, and at least one music clip is selected from the music index table as background music.

5. The method according to claim 4, characterized in that The step of matching and determining a music index table corresponding to the emotion type in a music library, and selecting at least one music clip as background music from the music index table includes: Based on the time series, at least one emotion type in the hot music is determined, and based on the at least one emotion type, a corresponding music style is determined in the music library, and the music clip is selected from at least one music style; Based on the time sequence, at least one of the music clips is spliced ​​to determine the background music; the splicing of the music clips adopts a cross-fade technology.

6. The method according to claim 5, characterized in that The generating of the hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music includes: The hot spot voice is divided into normal communication voice and picture introduction voice, and based on the normal communication voice, a digital human communication video is generated by using digital human generation technology; Generate a first video based on the picture introduction voice and the hot spot picture; A hot short video corresponding to the hot information is generated according to the first video, the digital human communication video, and the background music.

7. The method according to claim 3, characterized in that The picture information also includes the number of pictures, and the generating of hot texts matching the hot pictures by using a large language model based on the key information and the picture information also includes: The prompt word constraint adjusted by the large language model is obtained, so that the large language model generates hot texts corresponding to each hot picture based on the number of pictures in the picture information.

8. A device for automatically generating hot short videos, characterized in that: The device comprises: An acquisition module, used to acquire hot spot information and use a large language model to determine hot spot images and hot spot texts corresponding to the hot spot information; An analysis module, used to determine the hot speech corresponding to the hot text, perform emotion perception on the hot speech, determine the emotion contained in the hot speech, and the background music corresponding to the emotion; A generation module is used to generate a hot short video corresponding to the hot information based on the hot picture, the hot voice and the background music.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the method for automatically generating hot short videos according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for automatically generating hot short videos according to any one of claims 1 to 7.

Images