Video generation method and device, computer equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a video generation method and device, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of materials in a user image library, the materials comprising at least one of an image and a video; highlight recognition processing is carried out on a target material in the multiple materials, so that a highlight part is extracted from the target material in the multiple materials, and multiple processed materials are obtained; according to the shooting information of each material under at least one aggregation dimension, aggregating the plurality of processed materials to obtain a material set under each aggregation dimension; and generating a to-be-published video based on the materials in the material set. According to the embodiment of the invention, the materials with the same or similar characteristics in the user picture library are gathered based on the shooting information, and the published video with the theme is generated, so that the user does not need to manually screen the materials, the difficulty of publishing the content by the user is greatly reduced, and the enthusiasm of publishing the content by the user is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a video generation method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] When using social platforms, users are more willing to browse content posted by others, and are less likely to post content themselves. The reason why users post less content is that they have no inspiration. In addition, due to the rapid development of mobile photography technology and Internet technology, users have a large number of pictures and videos in their gallery. It is difficult for users to filter out suitable content for posting from a large number of images and videos, which makes it difficult for users to post content and their enthusiasm for posting content is low. Summary of the invention

[0003] The embodiments of the present application provide a video generation method, apparatus, computer device, and computer-readable storage medium, which can reduce the difficulty of users publishing content and increase the enthusiasm of users to publish content.

[0004] A video generation method provided in an embodiment of the present application includes:

[0005] Acquire multiple materials in a user gallery, wherein the materials include at least one of images and videos;

[0006] Performing highlight recognition processing on a target material among the multiple materials to extract a highlight portion of the target material among the multiple materials to obtain a plurality of processed materials;

[0007] Aggregating the processed multiple materials according to shooting information of each material in at least one aggregation dimension to obtain a material set in each aggregation dimension;

[0008] A video to be published is generated based on the material in the material set.

[0009] Accordingly, an embodiment of the present application further provides a video generating device, including:

[0010] An acquisition unit, used to acquire a plurality of materials in a user gallery, wherein the materials include at least one of images and videos;

[0011] an extraction unit, configured to perform highlight recognition processing on a target material among the plurality of materials, so as to extract a highlight portion of the target material among the plurality of materials, and obtain a plurality of processed materials;

[0012] an aggregation unit, configured to aggregate the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension;

[0013] A generating unit is used to generate a video to be published based on the material in the material set.

[0014] In one embodiment, the at least one aggregation dimension includes at least one of a time dimension, a location dimension, and a scene dimension, and the shooting information includes at least one of shooting time, shooting location, and shooting scene.

[0015] The polymerization unit comprises:

[0016] An information acquisition subunit, used to acquire at least one of the shooting information of the shooting time, shooting location and shooting scene of each material;

[0017] The first material aggregation subunit is configured to aggregate materials whose shooting time satisfies a preset time among the multiple materials if the aggregation dimension is a time dimension, so as to obtain a material set under the time dimension.

[0018] In one embodiment, the aggregation unit further comprises:

[0019] A second material aggregation subunit is configured to aggregate materials with the same shooting location among the multiple materials to obtain a candidate material set under the location dimension if the aggregation dimension is the location dimension;

[0020] A location determination subunit, configured to determine the user's permanent residence location according to the shooting location corresponding to each candidate material set under the location dimension and the number of materials included;

[0021] The filtering subunit is used to filter the candidate material set corresponding to the permanent residence from the candidate material set to obtain the candidate material set under the location dimension.

[0022] In one embodiment, the aggregation unit further comprises:

[0023] A third material aggregation subunit is configured to aggregate materials with the same shooting scene among the multiple materials to obtain a candidate material set under the scene dimension if the aggregation dimension is a scene dimension;

[0024] The sampling subunit is used to perform sampling processing on each candidate material set under the scene dimension based on the shooting time to obtain a material set under the scene dimension.

[0025] In one embodiment, the video generating device further includes:

[0026] A strategy determination unit, used to determine a text generation strategy according to the aggregation dimension and aggregation topic of the published video;

[0027] The text generation unit is used to generate the release text of the released video according to the text generation strategy, and the aggregation dimension and aggregation theme of the released video.

[0028] In one embodiment, the text generation strategy indicates that the release text is generated based on a text template, the text template includes at least a first component and a second component, the first component is a verb part, and the second component is related to an event recorded by the video.

[0029] The text generation unit comprises:

[0030] A first selection subunit is used to select a target verb text from a preset verb library as a first part of text corresponding to the first component;

[0031] A second selection subunit is used to select a second part of text corresponding to the second component from a text library corresponding to the aggregated topic;

[0032] A publishing text generating subunit is used to generate a publishing text for the published video based on the first part of text and the second part of text.

[0033] In one embodiment, the second component in the text template may include an event element part and an event part, and the second selection subunit includes:

[0034] A time text selection module, for selecting a target time text from a time text library corresponding to the aggregation topic as a partial text corresponding to the event element part if the aggregation dimension is a time dimension or a scene dimension;

[0035] A first event text selection module, configured to select a first target event text as a partial text corresponding to the event part from an event text library corresponding to the aggregated topic;

[0036] The first part of text generating module is used to generate the second text part based on the part of text corresponding to the event element part and the part of text corresponding to the event part.

[0037] In one embodiment, the second selection subunit further includes:

[0038] A location text determination subunit, configured to use the aggregation topic as a partial text corresponding to the event element part if the aggregation dimension is the location dimension;

[0039] A second event text selection subunit is used to select a second target event text as a partial text corresponding to the event part from the event text library corresponding to the location dimension;

[0040] The second text portion generating subunit is used to generate the second text portion based on the partial text corresponding to the event element portion and the partial text corresponding to the event portion.

[0041] Correspondingly, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any video generation method provided in the embodiment of the present application.

[0042] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any one of the video generation methods provided in the embodiments of the present application.

[0043] The embodiment of the present application obtains multiple materials in a user's gallery, where the materials include at least one of images and videos; performs highlight recognition processing on a target material among the multiple materials to extract the highlight part of the target material among the multiple materials to obtain multiple processed materials; aggregates the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension; and generates a video to be published based on the materials in the material set.

[0044] The embodiment of the present application performs highlight recognition processing on multiple materials in the user's gallery to obtain multiple processed materials, and based on the shooting information under the aggregation dimension, the materials with the same or similar characteristics are grouped together to obtain a material collection. Based on the material collection, a published video with a theme can be generated. The user does not need to determine the theme based on the materials in the gallery and manually filter the materials that meet the theme from the large amount of materials in the gallery, which greatly reduces the difficulty of users publishing content and can thereby increase the user's enthusiasm for publishing content. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 is a flow chart of a video generation method provided in an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of a video generating device provided in an embodiment of the present application;

[0048] Figure 3It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0050] The embodiment of the present application provides a video generation method, device, computer equipment and computer readable storage medium. The video generation device can be integrated in a computer equipment, which can be a server or a terminal.

[0051] The terminal may include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.

[0052] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0053] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0054] This embodiment will be described from the perspective of a video generating device. The video generating device may be integrated into a computer device, which may be a server or a terminal.

[0055] The embodiment of the present application provides a video generation method, such as Figure 1 As shown, the specific process of the video generation method can be as follows:

[0056] 101. Acquire multiple materials in a user gallery, where the materials include at least one of images and videos.

[0057] The video publishing request may include a request to generate a video for publishing, and the material may include materials of types such as pictures, videos, GIF animations, and dynamic pictures (Live). The gallery may be a gallery on a terminal used by the user, and the gallery may store pictures and videos taken by the user, as well as pictures and videos saved by the user from a website or other location.

[0058] For example, the server obtains multiple materials in the user's gallery, and may obtain materials shot within a preset time period, or may obtain a preset number of materials.

[0059] Optionally, the user can trigger a video publishing request on the client, and the server receives the video publishing request sent by the client and obtains multiple materials in the user's gallery, and can obtain materials shot within a preset time period or a preset number of materials.

[0060] Exemplarily, the materials in the gallery within the past year can be counted. If the number of materials in the past year is greater than or equal to 3,000, multiple materials in the gallery within the past year are obtained, and the number of materials obtained does not exceed 10,000; if the number of materials in the past year is less than 3,000, the more recent 3,000 materials within the past three years are taken.

[0061] Optionally, after obtaining multiple materials, the materials can be deduplicated to filter out materials with duplicate content.

[0062] 102. Perform highlight recognition processing on a target material among the multiple materials to extract a highlight portion of the target material among the multiple materials, thereby obtaining multiple processed materials.

[0063] Among them, highlight recognition processing can be used to find highlight parts from the target material, such as highlight clips in a video, or highlight areas in an image.

[0064] The target material may be a video material among multiple materials, or may be a picture material and a video material among the material videos.

[0065] For example, highlight detection may be performed on a video material among multiple materials to obtain highlight segments in the video material, and multiple processed materials may be obtained based on the highlight segments of a target material and other materials among the multiple materials.

[0066] Optionally, the target material may also include image material. For example, if a face is detected from an image, the area ratio of the face in the image material is determined. If the area ratio is less than a preset ratio threshold, the image material can be cropped with the position of the face as a reference point, so that the area ratio of the face in the cropped image is greater than or equal to the preset ratio threshold and is located at the target position (for example, the middle of the image).

[0067] Optionally, human body detection or pedestrian detection can be used to determine whether the image contains a human body. If a human body is contained, the area ratio of the human body region in the image is calculated. If the area ratio is less than a preset ratio threshold, the image material can be cropped based on the location of the human body region, so that the area ratio of the face in the cropped image is greater than or equal to the preset ratio threshold and is located at the target position.

[0068] A highlight portion is extracted from a target material among multiple materials to obtain multiple processed materials.

[0069] 103. Aggregate the processed multiple materials according to shooting information of each material in at least one aggregation dimension to obtain a material set in each aggregation dimension.

[0070] The aggregation dimensions may include time, location, and scene dimensions, etc. The shooting information of the material may include shooting time, shooting location, and shooting scene, etc. The shooting time may be the date when the user shoots the material, such as 2023.12.20, and the shooting location may be the geographical location where the user is when shooting the material, such as Shenzhen City, Guangdong Province. The shooting scene may be related to the content shot by the user, such as selfies, group photos, sunsets, seas, food, etc.

[0071] Among them, the shooting scene can be obtained by marking the material through a marking model (DeepDanbooru), and the material can also be classified and processed to determine the shooting scene of the material according to the classification result. For example, image feature extraction can be performed on each material to obtain image feature information of each material, and the material can be classified based on the image feature information through a classification network to obtain the classification result of the material. Multi-label classification can be used, and each material can correspond to multiple scene labels, such as sunrise, group photo, sea, etc.

[0072] For at least one aggregation dimension, the processed multiple materials may be aggregated according to shooting information related to the materials and the aggregation dimension to obtain a material set with the same or similar shooting information.

[0073] For example, a plurality of materials are aggregated based on a time dimension, a plurality of materials are aggregated based on a location dimension, and a plurality of materials are aggregated based on a scene dimension. That is, in one embodiment, at least one aggregation dimension includes at least one of a time dimension, a location dimension, and a scene dimension, and the shooting information includes at least one of shooting time, shooting location, and shooting scene. The step of “aggregating a plurality of materials according to the shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension” includes:

[0074] Obtain at least one of the shooting information of the shooting time, shooting location and shooting scene of each material;

[0075] If the aggregation dimension is the time dimension, then the materials whose shooting time meets the preset time among the multiple materials are aggregated to obtain a material set under the time dimension.

[0076] You can aggregate multiple materials based on the time dimension to obtain material collections corresponding to different aggregation themes under the time dimension. For example, you can filter out materials from this day last year from multiple materials based on the shooting time and aggregate these materials into a material collection; you can also filter out materials from the past week from multiple materials based on the shooting time and aggregate them into a material collection; you can also filter out materials within a month from multiple materials based on the shooting time and aggregate them into a material collection, such as materials in May; or you can filter out materials on holidays from multiple materials based on the shooting time and aggregate them into a material collection, such as the National Day holiday, May Day holiday, Winter Solstice, Spring Festival, etc.

[0077] If the aggregation dimension is location, the materials can be aggregated according to the shooting location. That is, in one embodiment, the video generation method provided in the embodiment of the present application may further include:

[0078] If the aggregation dimension is the location dimension, then the materials with the same shooting location among multiple materials are aggregated to obtain a candidate material set under the location dimension;

[0079] Determine the user's permanent residence location based on the shooting location and the number of materials included in each candidate material set under the location dimension;

[0080] A candidate material set corresponding to the permanent residence is filtered from the candidate material set to obtain a candidate material set under the location dimension.

[0081] For example, based on the shooting location of each material, the materials with the same shooting location can be aggregated to obtain a candidate material set under the location dimension. Since users have permanent residences, the content shot by users without leaving their permanent residences is usually diverse. If the material set aggregated by permanent residence is messy and has no clear theme, the candidate material set corresponding to the permanent residence can be filtered out to obtain a material set under the location dimension.

[0082] The shooting location corresponding to the candidate material set containing the largest number of materials can be determined as the user's permanent residence, and the candidate material set can be removed. Optionally, the number of materials contained in each material set can also be set. For example, the number of materials contained in each material set can be set to 5-15. Then, the same shooting location can be aggregated into multiple candidate material sets, and the shooting location corresponding to the largest number of candidate material sets can be determined as the user's permanent residence.

[0083] For example, the smallest aggregation unit may be a prefecture-level city, and based on the materials in the gallery, a material set corresponding to Shenzhen City, a material set corresponding to Beijing City, and so on may be obtained.

[0084] If the aggregation dimension is a scene dimension, materials with the same shooting scene can be aggregated. That is, in one embodiment, the video generation method provided in the embodiment of the present application may further include:

[0085] If the aggregation dimension is the scene dimension, then the materials with the same shooting scene among multiple materials are aggregated to obtain a set of candidate materials under the scene dimension;

[0086] Each candidate material set under the scene dimension is sampled based on the shooting time to obtain a material set under the scene dimension.

[0087] For example, based on the shooting scene of each material, materials with the same shooting scene can be aggregated to obtain multiple candidate material sets. The shooting scene of the materials in each candidate material set is the same. The candidate material sets can be sampled according to the shooting time to obtain a material set with evenly distributed shooting time.

[0088] For example, the materials in each month may be sampled from the candidate material set according to the shooting time to obtain a material set in which the materials of each month are evenly distributed.

[0089] Optionally, the number of materials contained in each material set can be set, and the candidate material set can be sampled based on the number of materials. For example, the candidate material set can be collected from fine granularity to coarse granularity until the number of material sets meets the requirement. For example, a candidate material set with a shooting scene at the seaside can be aggregated, and materials within a month can be selected based on the shooting information. If the number is insufficient, the granularity is expanded to a quarter. If the number of materials within a quarter is still insufficient, the granularity is expanded to a year.

[0090] In one embodiment, the aggregation themes under each aggregation dimension can be preset. For example, the time dimension can include themes such as this day last year, holidays, weekly reports, and monthly reports. The aggregation themes under the location dimension are different shooting locations. The scene dimension can include portraits, outdoors, objects, sports, indoors and other themes. There is a certain number of material sets under each aggregation theme, such as 5-15 materials, 3-5 materials, etc., and each aggregation theme only aggregates one material set.

[0091] For each aggregation dimension, the candidate material set under each aggregation theme can be sampled based on the shooting information corresponding to other aggregation dimensions other than the aggregation dimension to obtain a material set with a quantity that meets the requirements. For example, for the candidate material set under the time dimension, the candidate material set can be sampled according to the shooting scene and shooting location, so that different shooting scenes and shooting locations are evenly distributed in the obtained material set; for the candidate material set under the location dimension, the candidate material set can be sampled according to the shooting scene and shooting time, so that different shooting scenes and shooting times are evenly distributed in the obtained material set; for the candidate material set under the scene dimension, the candidate material set can be sampled according to the shooting time, so that different shooting times are evenly distributed in the obtained material set.

[0092] Aggregating the materials in the user gallery based on the aggregation dimension may be to select a target aggregation dimension from multiple aggregation dimensions, the number of the target aggregation dimensions being equal to the number of the multiple aggregation dimensions, and aggregating the materials based on the target aggregation dimension may reduce the amount of data processing, improve the efficiency of generating the video to be published, and reduce the occupation of terminal computing resources. That is, in one embodiment, the step of "aggregating the processed multiple materials based on the shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension" may include:

[0093] Obtaining video information of a video published under a publishing account, where video features are determined based on account feature information of the publishing account, user interaction data of the published video, or video content of the published video;

[0094] According to the video information, a target aggregation dimension is selected from multiple aggregation dimensions;

[0095] The processed multiple materials are aggregated based on the target aggregation dimension to obtain a material set under the target aggregation dimension.

[0096] The publishing account may be a user account registered by the user on the client. The video information may indicate information such as the video type and / or the adopted aggregation dimension of the published video under the publishing account.

[0097] For example, account characteristic information may include information such as the preferences or fields of the publishing account. Account characteristic information may indicate that the publisher of the video is a travel blogger, a lifestyle blogger, or a beauty blogger, etc. The content posted by users is usually related to their own positioning or preferences, so the video information of the published videos under the publishing account can be determined based on the account characteristic information of the publishing account. The account characteristic information can be determined based on the account information of the publishing account, such as gender, introduction, content of interest, and other information.

[0098] The video information can also be determined based on the video content of the published videos under the publishing account. For example, video features of the published videos can be extracted, and the video classification of the video can be determined based on the extracted video feature information. The video classification can correspond to the aggregation dimension, such as scenery, or characters.

[0099] The video information may also be determined based on the aggregation dimension corresponding to the published video. The video information may indicate an aggregation dimension that is used more frequently by the published videos under the publishing account.

[0100] Among them, user interaction data can be record data of users' operations such as likes, browsing and collection of the video after the video is released. Generally, the more likes, browsing and collections, the more popular the video is and the higher the quality of the video.

[0101] High-quality videos among the published videos under the publishing account can be determined based on the user interaction data, and the target aggregation dimension can be determined based on the aggregation dimension adopted by the high-quality videos.

[0102] After determining the video information of the videos published under the publishing account, at least one target aggregation dimension can be selected from multiple aggregation dimensions based on the video information, and the materials in the user gallery can be aggregated according to the target aggregation dimension to obtain a material set under the target aggregation dimension.

[0103] By aggregating materials through target aggregation dimensions, it is possible to pre-filter aggregation dimensions that users are not interested in, reduce the number of material sets, reduce the difficulty for users to select videos to be published, and improve the efficiency of users publishing videos.

[0104] 104. Generate a video to be published based on the material in the material collection.

[0105] For example, you can combine the materials in the material collection into a published video. Optionally, you can also add transitions, special effects, music, filters and other video effects to make the generated published video more exquisite and creative.

[0106] In one embodiment, corresponding text may also be generated for the generated published video so that the user can publish the video with one click, that is, after the step of "generating a video to be published based on the material in the material set", the video generation method provided in the embodiment of the present application may further include:

[0107] Determine the text generation strategy based on the aggregation dimensions and aggregation topics of the published videos;

[0108] Generate the release text of the video to be released according to the text generation strategy, as well as the aggregation dimensions and aggregation topics of the released video.

[0109] For example, at least two text generation strategies may be provided. The first text generation strategy may be to select a target published text from a text set corresponding to an aggregated topic according to the aggregated topic.

[0110] For example, the text sets corresponding to the monthly and weekly report themes under the time dimension include "The joy of inventory, send it today!", "The memories of inventory, send it today!", and "Share the fragments of life in inventory", etc. The text sets under the theme of that day may include "That day", "That day under the lens", and "Where is that day in your photo album?", etc. The text sets corresponding to the mountain and river theme under the scene dimension may include "Become an outdoor life lover", "Close contact with nature", and "The moment of being healed by nature", etc.

[0111] The second text generation strategy may be to generate a release text according to a text template and an aggregated theme. The text template may include at least two components, a first component and a second component. The first component may be a verb part, and the second component is related to an event. That is, in one embodiment, the text generation strategy indicates to generate a release text based on a text template. The text template includes at least a first component and a second component. The first component is a verb part, and the second component is related to an event recorded by the video. According to the text generation strategy, and the aggregated dimension and aggregated theme of the released video, the release text of the video to be released is generated, including:

[0112] Select a target verb text from a preset verb library as the first part of the text corresponding to the first component;

[0113] Selecting the second part of text corresponding to the second component from the text library corresponding to the aggregated topic;

[0114] A publishing text for the video to be published is generated according to the first part of the text and the second part of the text.

[0115] For example, the text template may include a first component and a second component, the first component being a verb part, and the second component being related to an event recorded in the video, such as at least one of time, place, and event, and the target verb text is selected from a preset verb library as the partial text corresponding to the first component, that is, the first partial text; the second component may include an event part, and different aggregated topics may correspond to an event text library. From the event text library corresponding to the aggregated topic, the target text is selected as the second partial text corresponding to the second component, and the first partial text and the second partial text are combined to obtain the published text of the published video.

[0116] Exemplarily, the preset verb library may include "show off", "show off", "record", "recall", "reminisce", "share", "review", etc. If the aggregation theme is the location dimension, the corresponding event text library may include "travel time", "daily travel", "travel fragments" and "travel reality", etc.

[0117] By randomly selecting "show off" from the preset verb library as the first part of the text, and selecting "travel fragments" from the event text library corresponding to the aggregated topic as the second part of the text, we can get the published text "show off travel fragments".

[0118] In addition to the event part, the second component may also include an event element part, such as a time part or a location part, etc. That is, in one implementation, the step of "selecting and determining the second part of the text corresponding to the second component from the text library corresponding to the aggregated topic" may include:

[0119] If the aggregation dimension is the time dimension or the scene dimension, the target time text is selected from the time text library corresponding to the aggregation theme as the partial text corresponding to the event element part;

[0120] According to the event text library corresponding to the aggregated topic, the first target event text is selected as the partial text corresponding to the event part;

[0121] A second text portion is generated based on the partial text corresponding to the event element portion and the partial text corresponding to the event portion.

[0122] For example, the text template may include an event element part that corresponds to time or location, so the text template may be verb+location+event or verb+time+event. If the aggregation dimension is the time dimension or the scene dimension, the event element part is the time element, and the target time text is selected from the time text library corresponding to the aggregation theme as the partial text corresponding to the event element part, and the first target event text is selected from the event text library corresponding to the aggregation theme as the partial text corresponding to the event part, and the partial text corresponding to the event element part and the partial text corresponding to the event part are merged to obtain the second text part.

[0123] Exemplarily, the aggregated theme is the group photo theme under the scene dimension. The corresponding time text library may include "recent", "most recent", etc., from which "recent" is selected, that is, the target time text. The corresponding event text library may include "group photo moments", "friends' daily life", "happy moments", "casual photos", etc., from which "friends' daily life" is selected, that is, the first target event text. After merging, we can get "recent friends' daily life".

[0124] If the aggregation dimension is the location dimension, the event element part is the location, and the aggregation topic can be used as the partial text corresponding to the event element part. That is, in one embodiment, the video generation method provided in the embodiment of the present application can also include:

[0125] If the aggregation dimension is the location dimension, the aggregation topic is taken as the partial text corresponding to the event element part;

[0126] According to the event text library corresponding to the location dimension, a second target event text is selected as a partial text corresponding to the event part;

[0127] A second text portion is generated based on the partial text corresponding to the event element portion and the partial text corresponding to the event portion.

[0128] For example, if the aggregation dimension is the location dimension, the event element part is the location, and the aggregation theme is the shooting location, the aggregation theme can be used as the partial text corresponding to the event element part, and the second target event text can be selected from the event text library corresponding to the aggregation theme as the partial text corresponding to the event part, and the partial text corresponding to the event element part and the partial text corresponding to the event part are merged to obtain the second text part.

[0129] Exemplarily, the aggregation theme is Beijing, and the corresponding event text library may include "travel time", "travel daily life", "travel fragments" and "travel reality", etc., from which "travel daily life" is selected, that is, the second target event text, and merged to obtain "travel fragments in Beijing".

[0130] The text generation strategy used can be determined according to the aggregation dimension and the aggregation theme. For example, the aggregation theme under the location dimension is the location where the event occurred, so the video collection under the location dimension can use the second text generation strategy to generate the release text. Under the time dimension, if the aggregation theme is a certain period of time, such as January, February, and the past week, the second text generation strategy can be used, and the corresponding time texts can include January, February, spring, spring, winter, winter, etc. If the aggregation theme is this day last year or that day, the first text generation strategy is used. If it is a scene dimension, the first text generation strategy can be used because the scene dimension can also use the second text generation strategy.

[0131] After the to-be-published video and publication text are generated, the generated publication video and publication text can be displayed to the user. The user can edit the publication text and publication video, or directly publish them with one click.

[0132] It is understandable that multiple release videos can be generated, each release video has a corresponding release text, and the user can switch the release video, and the release text in the text editing box will switch accordingly.

[0133] Exemplarily, multiple video collections can be aggregated for each aggregation dimension. For example, four material collections can be aggregated under the scene dimension, three material collections can be aggregated under the time dimension, and three material collections can be aggregated under the location dimension, and the aggregation themes of the material collections do not overlap. A video to be published is generated for each material collection, so as to obtain 10 published videos. A corresponding published text can be generated for each published video, and the 10 published videos and the corresponding published texts can be displayed to the user so that the user can select one of the published videos for publication, thereby improving the efficiency of video publishing for the user.

[0134] Generating a video from the materials in the material collection may be performed by splicing the materials in the material collection to obtain a video composed of multiple materials, that is, publishing the video.

[0135] Optionally, the materials in the material collection can be synthesized into a video by applying video templates. For example, multiple video templates can be preset, and each video template can include a sample video, a title, an aggregated theme, and a template name, etc. Feature extraction is performed on the video and image content in each video template to obtain visual feature information of each video template, and feature extraction is performed on the text content to obtain text feature information of each video template. The visual feature information and the text feature information are fused to obtain template feature information of each video template, thereby forming a template feature library to be retrieved.

[0136] Feature extraction is performed on the materials in the material set to obtain visual feature information of the material set. Based on the visual feature information of the material set and the template feature information of the video template, a target video template with the greatest similarity is retrieved from the template feature library. Then, video effects such as transitions, special effects, filters, and music in the target video template are added to the video generated based on the material set to obtain a published video. Optionally, the materials in the material set can be replaced with the materials in the target material template to obtain a published video.

[0137] Optionally, feature extraction of video templates and material collections can be achieved through a clip model. The clip model extracts features from the video and image content in each video template to obtain visual feature information of each video template, and extracts features from the text content to obtain text feature information of each video template. The visual feature information and text feature information are fused to obtain template feature information of each video template, thereby forming a template feature library to be retrieved.

[0138] Feature extraction is performed on the materials in the material set to obtain visual feature information of each material, the visual feature information of multiple materials is fused to form a user material query, the user material query is compared with the template feature library for similarity, templates with a similarity greater than a certain threshold are retrieved as candidate video templates, and at least one published video is generated based on the candidate video template for user selection.

[0139] Optionally, video editing effects such as audio, filters, and special effects may be added to the video. Step 104 of "generating a video to be published based on the material in the material collection" may include:

[0140] 4101. Extract material features from a plurality of clipping materials to obtain video content feature information of a video to be generated corresponding to the plurality of clipping materials;

[0141] 4102. Selecting a video editing effect that matches the video to be generated according to video content feature information of the video to be generated;

[0142] 4103. Generate a published video with added video editing effects based on multiple editing materials and video editing effects.

[0143] 4101. Perform material feature extraction on a plurality of editing materials to obtain video content feature information of a to-be-generated video corresponding to the plurality of editing materials.

[0144] For example, extracting material features from multiple clip materials may include extracting the aggregate theme and material quantity of the material set, and obtaining the video strength of the video to be generated, that is, the video feature information includes content motion information. The step of "extracting content features from each clip material to obtain material content feature information of each clip material" may include:

[0145] Extract motion features from each clip material to obtain content motion information of each clip material;

[0146] The video strength of the video to be generated is determined according to the content motion information of each clip material. The video strength is used to describe the degree of change of the content of the video to be generated.

[0147] For example, for video materials in editing materials, the optical flow method can be used to analyze the movement of pixels in two adjacent frames of the video material, and the motion vector of the object in the video material can be calculated, thereby obtaining the motion intensity of the video material. For image materials, a motion intensity corresponding to an image material can be preset, and the average motion intensity can be calculated based on the number of materials of multiple editing materials and the motion intensity of each editing material. The average motion intensity is the video intensity of the video to be synthesized. The greater the video intensity, the faster the content of the video generated based on multiple editing materials changes, and the smaller the video intensity, the slower the content of the video generated based on multiple editing materials changes. Therefore, the video intensity can indicate the drastic degree of change in the video content.

[0148] 4102. Select a video editing effect that matches the video to be generated based on the video content feature information of the video to be generated.

[0149] The video editing effect may include at least one of a visual effect and an auditory effect, and may also include a dynamic effect. For example, the vibration of a mobile phone may be controlled to convey a dynamic effect to the user.

[0150] For example, audio that matches the music rhythm can be selected according to the video intensity, so that in the generated video, the rhythm of the music and the rhythm of the video are matched, and there will be no dynamic video with soothing music, or a slow-paced soothing video with fast-paced music. That is, in one embodiment, the video editing effect includes an auditory effect, and the step of "selecting a video editing effect that matches the video to be generated according to the video content feature information" may include:

[0151] Determining an audio speed that matches the video intensity according to a preset correspondence between the video intensity and the audio speed;

[0152] Based on the audio speed that matches the video intensity, a target audio that matches the degree of content change of the video to be generated is obtained, and the target audio is enhanced to add an auditory effect.

[0153] The audio speed may be related to the number of beats per minute (BPM) of the audio. The larger the BPM, the faster the audio tempo, and the smaller the BPM, the slower the audio tempo.

[0154] For example, a correspondence between video intensity and audio speed can be preset. In the correspondence, the greater the video intensity, the faster the corresponding audio speed. Multiple video intensity intervals and multiple audio speed intervals can be set, and the video intensity intervals and the audio speed intervals correspond one to one. Optionally, the video intensity can have three levels: strong, medium, and weak, and the audio speed can have three levels: fast, medium, and slow, and the video intensity levels and the audio speed levels correspond one to one.

[0155] The audio speed matching the video intensity can be determined according to the corresponding relationship, and the target audio is obtained according to the audio speed.

[0156] Combine multiple clips to generate a video, and add target audio to the video to obtain a published video with music.

[0157] Optionally, the playing time of each clip material in the generated published video may also be determined according to the beat of the target audio. That is, in one embodiment, the step of “generating a published video with added video clip effects based on multiple clip materials and video clip effects” includes:

[0158] Obtaining a rebeat point of the target audio, where the rebeat point includes a time point at which the rebeat occurs in the target audio;

[0159] Place at least one clip material in the time slots corresponding to every two adjacent retake points in the target audio;

[0160] Generate a video to be released based on the target audio and the editing materials placed based on the key shots of the target audio.

[0161] The beats of music can be divided into strong beats and weak beats. The strong beats can also be called the beats, and the beat points can be the time points at which the beats of the audio occur.

[0162] For example, every two adjacent strong beats in the target audio can form a slot, and one slot can correspond to a clip material. For example, the first retake point is 00:24 in the audio, and the second retake point is 00:35 in the audio. 00:24-00:29 is a time slot, and the length of the time slot is 5 seconds. Then the display time of the clip material inserted into the time slot is 5s. Place a clip material in each slot, and the generated published video switches the clip material when playing each retake. The video has a sense of tapping and a good editing effect.

[0163] Since the audio also has highlight points, the highlight points can be considered as the time point where the climax of the audio begins. The highlight points are usually before or at the first retake point. The editing material can be displayed at the highlight point, and the editing material displayed can be switched at each key shot, so that the editing material in the video and the audio can be "in step", and the video effect is better. If the time interval between the highlight point and the first retake point is long, the time slot formed by the highlight point and the first retake point can be used as the first slot. If the time interval between the highlight point and the first retake point is short, the time slot formed by the highlight point and the second retake point can be used as the first slot. That is, in one embodiment, the step of "placing at least one editing material in the time slots corresponding to two adjacent retake points in the target audio" can include:

[0164] Get the highlight point of the target audio. The highlight point is the time point where the highlight audio segment in the target audio appears. The highlight point should not be later than the re-beat point of the target audio.

[0165] If the time interval between the first re-beat point and the highlight point is greater than a preset threshold, the first time slot of the target audio corresponds to the highlight point and the first re-beat point;

[0166] If the time interval between the first re-beat point and the highlight point is not greater than a preset threshold, the first time slot of the target audio corresponds to the highlight point and the second re-beat point of the target audio.

[0167] Among them, the highlight points can be obtained by manually marking the audio in advance, or by pre-recognizing and processing the audio. Specifically, the highlight points in the audio can be automatically identified through a neural network model.

[0168] For example, the highlight point of the target audio can be obtained. If the time interval between the first retake point and the highlight point of the target audio is greater than a preset threshold, the time slot formed by the highlight point and the first retake point is the first time slot of the video to be generated.

[0169] If the time interval between the highlight point and the first retake point of the target audio is not greater than a preset threshold, the time slot formed by the highlight point and the second retake point is the first time slot of the video to be generated.

[0170] Exemplarily, the preset threshold is 1s. If the time interval between the highlight point and the first retake point is less than 1s, the time slot formed by the highlight point and the second retake point is the first time slot, and the time slot formed by the second retake point and the third retake point is the second time slot, and so on. Multiple time slots can be obtained, and a clip material is placed in each time slot, and a video to be released is generated. The display effect of the released video is that when the video plays to the highlight point of the target audio, the first clip material comes in, and when it plays to the second retake point, it switches to display the second clip material, and when it plays to the third retake point, it switches to display the third clip material, and so on.

[0171] The selected audio speed may be related not only to the video strength, but also to the audio with the same amount of material and the same duration. For audio with a fast rhythm, there are more retake points. For videos with the same video strength, the video length of videos with fewer materials is very short. Therefore, the audio speed may be determined in combination with the video strength and the amount of material. That is, in one embodiment, the step of "determining the audio speed matching the video strength according to the correspondence between the preset video strength and the audio speed" may include:

[0172] Get the number of materials contained in the video to be generated;

[0173] According to the corresponding relationship between the preset material quantity, video strength and audio speed, the audio speed matching the material quantity and video strength of the video to be generated is determined.

[0174] Among them, the corresponding relationship between the preset material quantity, video intensity and audio speed may include the audio speed corresponding to each combination of material quantity and video intensity. Exemplarily, the corresponding relationship between the preset material quantity, video intensity and audio speed is shown in the following table.

[0175]

[0176] For example, the number of materials contained in the video to be generated may be obtained, and the corresponding audio speed may be determined according to the number of materials and the video strength of the video to be generated.

[0177] In addition to selecting the audio speed, the music type can also be selected according to the video content characteristics. For example, the music type can be determined according to the video theme of the video to be generated. That is, in one embodiment, the video feature information includes the video theme. Based on the audio speed that matches the video intensity, the step of "obtaining the target audio that matches the content change degree of the video to be generated" can include:

[0178] Determine the target music type that matches the video to be generated according to the correspondence between the preset video theme and the music type;

[0179] Select target audio from the audios under target music type based on the audio speed that matches the video intensity.

[0180] Among them, the video theme can be an aggregate theme of multiple clip materials corresponding to the video to be generated. For example, a certain shooting scene can be used as the aggregate theme to obtain multiple clip materials. The shooting scene of the multiple clip materials is the same. The shooting scene is the video theme of the video to be generated. Time can also be used as the aggregate theme to obtain multiple clip materials of a certain time or a certain period of time.

[0181] The correspondence between the video theme and the music type can be preset. For example, the birthday theme is relatively relaxed and happy, and the corresponding music types are R&B, Lo-Fi, and houses. The time theme is usually a review and summary of the past, and the corresponding music type can be relatively soothing and healing, for example, it can correspond to light music, future-bass, etc. After determining the target music type according to the video theme of the video to be generated, the target audio that can be based on the audio speed matching the video intensity is obtained from the target music type.

[0182] In addition to adding music to the video, filters can also be added to the video to enhance the visual effect of the video. That is, in one embodiment, the video feature information includes the video theme, and the video editing effect includes the visual effect. The step of "selecting a video editing effect that matches the video to be generated according to the video content feature information of the video to be generated" can include:

[0183] According to the correspondence between the preset video theme and the filter, a target filter matching the video theme of the video to be generated is determined, and the target filter is used to enhance the visual effect.

[0184] Among them, filters can be used to achieve various special visual effects of images or videos. Filters can adjust properties such as color, contrast, brightness, etc. by changing pixel values, or add special visual effects to produce different visual effects. Filters can include: black and white filters, hue filters, and comic effect filters. Black and white filters are used to convert images into black and white tones and remove color information; hue filters are used to change the overall hue of an image, such as warm or cold tones; comic effect filters are used to give images comic-like visual effects, emphasizing edges and colors. There are many types of hue filters, such as filters that adjust the image to a reddish tone and filters that adjust the image to a greenish tone.

[0185] Some filters can be preset, and the relationship between filters and video themes can be set so as to determine the filters to be used for the video based on the video theme.

[0186] Exemplarily, a "rosy" filter, a "native" filter, a "wishful" filter, a "leisure" filter, and a "clear" filter can be pre-set, and the "portrait" theme can be set to correspond to the "rosy" filter or the "native" filter, and the "mountains", "sea", "sunset" and "sky" themes can be set to correspond to the "leisure" filter or the "clear" filter.

[0187] According to the correspondence between the preset video theme and the filter, the target filter to be used is determined, and the target filter is added to the video synthesized based on multiple clip materials to obtain the published video.

[0188] Special effects may also be added to the video. That is, in one embodiment, the video feature information includes the video theme, and the video editing effect includes a visual effect. The video editing effect matching the video to be generated is selected according to the video content feature information of the video to be generated, including:

[0189] According to the correspondence between the preset video theme and the special effects, the published video special effects that match the video theme of the video to be generated are determined, and the target special effects are used to enhance the visual effects.

[0190] Among them, special effects can be used to realize various dynamic visual effects in videos, such as halo, falling fireworks, shining stars and other effects.

[0191] You can preset some special effects and set the relationship between the special effects and the video theme, so as to determine the special effects to be used in the video based on the video theme.

[0192] Exemplarily, the “bubble zoom” special effect, “backlight focus” special effect, “color change” special effect, “halo” special effect, “fireworks gun” special effect and “starry sky” special effect can be pre-set. The “this day last year” theme corresponds to the “bubble zoom” special effect, “backlight focus” special effect or “color change” special effect, and the “birthday” theme corresponds to the “halo” special effect, “fireworks gun” special effect and “starry sky” special effect.

[0193] According to the correspondence between the preset video theme and the special effects, the target special effects to be used are determined, and the target special effects are added to the video synthesized based on multiple editing materials to obtain the published video.

[0194] Optionally, you can also add transition effects to the video. You can determine whether to use transitions based on the number of materials. If the number of materials is small, you can add one transition effect. If the number of materials is large, you can add multiple transition effects, or add a transition effect to each material, or add a transition effect for every two materials.

[0195] Exemplarily, when the number of materials is 2-5, a transition effect is randomly set, such as overlay or blur, and the video effect is that each material is transitioned by overlay or blur. When the number of materials is 6-15, a transition effect can be set for every two editing materials, and the video effect is that every two videos are transitioned by using a transition effect.

[0196] 4103. Generate a published video with added video editing effects based on multiple editing materials and video editing effects.

[0197] For example, after determining the audio, filters, special effects and other video editing effects used, a video to be published can be generated based on the video editing effects. When the published video is played, it is accompanied by the target audio, and the video has video effects such as filters and special effects added to it.

[0198] In one embodiment, a video to be published may be generated for a material set based on videos selected by a user in history, that is, the step of “generating a video to be published based on materials in a material set” includes:

[0199] Based on user interaction data of the published videos of the publishing account, select target published videos from the published videos;

[0200] A video to be released is generated according to the video effects of the target released video and the material set.

[0201] The target published video may be a relatively high-quality published video under the publishing account.

[0202] The target published video can be determined based on the user interaction data, and the video to be published can be generated based on the video effects applied to the published video and the material collection. The video effects of the video to be published are the same as those of the target published video, and the video effects may include filters, soundtracks, special effects, etc.

[0203] As can be seen from the above, the embodiment of the present application obtains multiple materials in the user's gallery, and the materials include at least one of images and videos; performs highlight recognition processing on a target material among the multiple materials to extract the highlight part of the target material among the multiple materials to obtain multiple processed materials; aggregates the processed multiple materials according to the shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension; and generates a video to be published based on the material in the material set.

[0204] The embodiment of the present application performs highlight recognition processing on multiple materials in the user's gallery to obtain multiple processed materials, and based on the shooting information under the aggregation dimension, the materials with the same or similar characteristics are grouped together to obtain a material collection. Based on the material collection, a published video with a theme can be generated. The user does not need to determine the theme based on the materials in the gallery and manually filter the materials that meet the theme from the large amount of materials in the gallery, which greatly reduces the difficulty of users publishing content and can thereby increase the user's enthusiasm for publishing content.

[0205] In order to better implement the video generation method provided in the embodiment of the present application, a video generation device is also provided in one embodiment. The meanings of the terms are the same as those in the above-mentioned video generation method, and the specific implementation details can refer to the description in the method embodiment.

[0206] The video generation device can be integrated into a computer device, such as Figure 2 As shown, the video generation device may include: an acquisition unit 301, an extraction unit 302, an aggregation unit 303 and a generation unit 304, which are specifically as follows:

[0207] (1) An acquisition unit 301 is used to acquire multiple materials in a user gallery, where the materials include at least one of images and videos.

[0208] (2) An extraction unit 302 is used to perform highlight recognition processing on a target material among the multiple materials, so as to extract a highlight portion of the target material among the multiple materials, thereby obtaining multiple processed materials.

[0209] (3) An aggregation unit 303, configured to aggregate the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension.

[0210] In one embodiment, the at least one aggregation dimension includes at least one of a time dimension, a location dimension, and a scene dimension, and the shooting information includes at least one of shooting time, shooting location, and shooting scene.

[0211] Aggregation unit 303, including:

[0212] An information acquisition subunit, used to acquire at least one of the shooting information of the shooting time, shooting location and shooting scene of each material;

[0213] The first material aggregation subunit is used to aggregate the materials whose shooting time meets the preset time among the multiple materials if the aggregation dimension is the time dimension, so as to obtain a material set under the time dimension.

[0214] In one embodiment, the aggregation unit 303 further includes:

[0215] The second material aggregation subunit is configured to aggregate materials with the same shooting location among the multiple materials to obtain a candidate material set under the location dimension if the aggregation dimension is the location dimension;

[0216] A location determination subunit, used to determine the user's permanent residence location according to the shooting location and the number of materials included in each candidate material set under the location dimension;

[0217] The filtering subunit is used to filter the candidate material set corresponding to the permanent residence from the candidate material set to obtain the candidate material set under the location dimension.

[0218] In one embodiment, the aggregation unit 303 further includes:

[0219] A third material aggregation subunit is configured to aggregate materials with the same shooting scene among multiple materials to obtain a candidate material set under the scene dimension if the aggregation dimension is the scene dimension;

[0220] The sampling subunit is used to perform sampling processing on each candidate material set under the scene dimension based on the shooting time to obtain the material set under the scene dimension.

[0221] (4) A generating unit 304, configured to generate a video to be published based on the material in the material set.

[0222] In one embodiment, the video generating device further includes:

[0223] A strategy determination unit, used to determine a text generation strategy according to the aggregation dimension and aggregation topic of the released video;

[0224] The text generation unit is used to generate the release text of the video to be released according to the text generation strategy, and the aggregation dimension and aggregation theme of the released video.

[0225] In one embodiment, the text generation strategy indicates that the release text is generated based on a text template, the text template includes at least a first component and a second component, the first component is a verb part, and the second component is related to an event recorded by the video.

[0226] Text generation unit, including:

[0227] A first selection subunit is used to select a target verb text from a preset verb library as a first part of text corresponding to the first component;

[0228] A second selection subunit is used to select a second part of text corresponding to a second component from a text library corresponding to the aggregated topic;

[0229] The publishing text generation subunit is used to generate the publishing text of the video to be published based on the first part of the text and the second part of the text.

[0230] In one embodiment, the second component in the text template may include an event element part and an event part, and the second selection subunit includes:

[0231] A time text selection module is used to select a target time text from a time text library corresponding to the aggregation theme as a partial text corresponding to the event element part if the aggregation dimension is a time dimension or a scene dimension;

[0232] A first event text selection module, configured to select a first target event text as a partial text corresponding to the event part from an event text library corresponding to the aggregated topic;

[0233] The first part text generation module is used to generate the second text part based on the partial text corresponding to the event element part and the partial text corresponding to the event part.

[0234] In one embodiment, the second selection subunit further includes:

[0235] A location text determination subunit is used to determine the aggregation subject as a partial text corresponding to the event element part if the aggregation dimension is the location dimension;

[0236] A second event text selection subunit is used to select a second target event text as a partial text corresponding to the event part from the event text library corresponding to the location dimension;

[0237] The second part text generation subunit is used to generate a second text part based on the partial text corresponding to the event element part and the partial text corresponding to the event part.

[0238] As can be seen from the above, the video generation device of the embodiment of the present application obtains multiple materials in the user's gallery through the acquisition unit 301, and the materials include at least one of images and videos; the extraction unit 302 performs highlight recognition processing on the target material in the multiple materials to extract the highlight part of the target material in the multiple materials, and obtain the processed multiple materials; the aggregation unit 303 aggregates the processed multiple materials according to the shooting information of each material under at least one aggregation dimension, and obtains a material set under each aggregation dimension; the generation unit 304 generates a video to be released based on the material in the material set.

[0239] The embodiment of the present application performs highlight recognition processing on multiple materials in the user's gallery to obtain multiple processed materials, and based on the shooting information under the aggregation dimension, the materials with the same or similar characteristics are grouped together to obtain a material collection. Based on the material collection, a published video with a theme can be generated. The user does not need to determine the theme based on the materials in the gallery and manually filter the materials that meet the theme from the large amount of materials in the gallery, which greatly reduces the difficulty of users publishing content and can thereby increase the user's enthusiasm for publishing content.

[0240] The present application also provides a computer device, which may be a terminal or a server. Figure 3 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0241] The computer device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Figure 3 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0242] The processor 1001 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and computer programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0243] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0244] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0245] The computer device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0246] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:

[0247] Acquire multiple materials in the user's gallery, where the materials include at least one of images and videos;

[0248] Performing highlight recognition processing on a target material among the multiple materials to extract a highlight portion of the target material among the multiple materials to obtain multiple processed materials;

[0249] Aggregating the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension;

[0250] Generate a video to be published based on the materials in the material collection.

[0251] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0252] As can be seen from the above, the computer device of the embodiment of the present application can obtain multiple materials in the user's gallery, where the materials include at least one of images and videos; perform highlight recognition processing on a target material among the multiple materials to extract the highlight part of the target material among the multiple materials to obtain multiple processed materials; aggregate the processed multiple materials according to the shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension; and generate a video to be released based on the material in the material set.

[0253] The embodiment of the present application performs highlight recognition processing on multiple materials in the user's gallery to obtain multiple processed materials, and based on the shooting information under the aggregation dimension, the materials with the same or similar characteristics are grouped together to obtain a material collection. Based on the material collection, a published video with a theme can be generated. The user does not need to determine the theme based on the materials in the gallery and manually filter the materials that meet the theme from the large amount of materials in the gallery, which greatly reduces the difficulty of users publishing content and can thereby increase the user's enthusiasm for publishing content.

[0254] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations in the above-mentioned embodiments.

[0255] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0256] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any video generation method provided in the embodiment of the present application.

[0257] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0258] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0259] Since the computer program stored in the computer-readable storage medium can execute any one of the video generation methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the video generation methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0260] The above is a detailed introduction to a video generation method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video generation method, characterized in that: include: Acquire multiple materials in a user gallery, wherein the materials include at least one of images and videos; Performing highlight recognition processing on a target material among the multiple materials to extract a highlight portion of the target material among the multiple materials to obtain a plurality of processed materials; Aggregating the processed multiple materials according to shooting information of each material in at least one aggregation dimension to obtain a material set in each aggregation dimension; A video to be published is generated based on the material in the material set.

2. The method according to claim 1, characterized in that The at least one aggregation dimension includes at least one of a time dimension, a location dimension, and a scene dimension, and the shooting information includes at least one of a shooting time, a shooting location, and a shooting scene. The step of aggregating the multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension includes: Obtain at least one of the shooting information of the shooting time, shooting location and shooting scene of each material; If the aggregation dimension is a time dimension, then the materials whose shooting time satisfies a preset time among the multiple materials are aggregated to obtain a material set under the time dimension.

3. The method according to claim 2, characterized in that The method further comprises: If the aggregation dimension is a location dimension, the materials shot at the same location among the multiple materials are aggregated to obtain a candidate material set under the location dimension; Determine the user's permanent residence location according to the shooting location corresponding to each candidate material set under the location dimension and the number of materials included; The candidate material set corresponding to the permanent residence location is filtered from the candidate material set to obtain a candidate material set under the location dimension.

4. The method according to claim 3, characterized in that The method further comprises: If the aggregation dimension is a scene dimension, the materials with the same shooting scene among the multiple materials are aggregated to obtain a candidate material set under the scene dimension; Sampling processing is performed on each candidate material set under the scene dimension based on the shooting time to obtain a material set under the scene dimension.

5. The method according to claim 1, characterized in that After generating the to-be-published video based on the material in the material set, the method further includes: Determining a text generation strategy according to the aggregation dimensions and aggregation topics of the published videos; The published text of the published video is generated according to the text generation strategy, and the aggregation dimension and aggregation theme of the published video.

6. The method according to claim 5, characterized in that The text generation strategy indicates that a published text is generated based on a text template, wherein the text template includes at least a first component and a second component, wherein the first component is a verb part, and the second component is related to an event recorded by the video. Generating the published text of the published video according to the text generation strategy, and the aggregation dimension and aggregation theme of the published video includes: Selecting a target verb text from a preset verb library as the first part of text corresponding to the first component; Selecting a second part of text corresponding to the second component from a text library corresponding to the aggregated topic; A publishing text for the published video is generated based on the first part of text and the second part of text.

7. The method according to claim 6, characterized in that The second component in the text template may include an event element part and an event part, and the selecting a second part of text corresponding to the second component from the text library corresponding to the aggregated topic includes: If the aggregation dimension is a time dimension or a scene dimension, a target time text is selected from a time text library corresponding to the aggregation topic as a partial text corresponding to the event element part; According to the event text library corresponding to the aggregated topic, a first target event text is selected as the partial text corresponding to the event part; The second text portion is generated based on the partial text corresponding to the event element portion and the partial text corresponding to the event portion.

8. The method according to claim 7, characterized in that The method further comprises: If the aggregation dimension is the location dimension, the aggregation topic is used as the partial text corresponding to the event element part; Selecting a second target event text as a partial text corresponding to the event part from an event text library corresponding to the location dimension; The second text portion is generated based on the partial text corresponding to the event element portion and the partial text corresponding to the event portion.

9. The method according to claim 1, characterized in that: The step of aggregating the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension includes: Obtaining video information of a video published under a publishing account, wherein the video feature is determined based on account feature information of the publishing account, user interaction data of the published video, or video content of the published video; Selecting a target aggregation dimension from multiple aggregation dimensions according to the video information; The processed multiple materials are aggregated based on the target aggregation dimension to obtain a material set under the target aggregation dimension.

10. The method according to claim 1, characterized in that The generating a video to be published based on the material in the material set includes: Based on user interaction data of published videos of the publishing account, filter a target published video from the published videos; A video to be published is generated according to the video effect of the target published video and the material set.

11. A video generating device, characterized in that: include: An acquisition unit, used to acquire a plurality of materials in a user gallery, wherein the materials include at least one of images and videos; an extraction unit, configured to perform highlight recognition processing on a target material among the plurality of materials, so as to extract a highlight portion of the target material among the plurality of materials, and obtain a plurality of processed materials; an aggregation unit, configured to aggregate the processed multiple materials according to shooting information of each material under at least one aggregation dimension to obtain a material set under each aggregation dimension; A generating unit is used to generate a video to be published based on the material in the material set.

12. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the video generation method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the video generation method according to any one of claims 1 to 10.

Images