A short video creation method, system, terminal, and medium based on AI image

Abstract

The present invention provides a short video creation method and system based on AI images, and relates to the technical field of AI short video production. The method includes: receiving image materials input by a user and basic style data selected by a user, and establishing preliminary comic image data based on the basic style data and the image materials; generating preliminary short video data based on the preliminary comic image data and short video content information input by the user; receiving voice revision information input by the user, and revising the preliminary short video data based on the voice revision information to generate draft short video data. After the user uploads the image material, the present application can also convert the image material into an animation image by selecting basic style data and short video content information, and produce preliminary short video data in combination with the short video content information, thereby enhancing the content of the short video while increasing the degree of free adjustment of the animation image in the short video, thereby enriching the content of the short video and enhancing its creativity.

Description

Technical Field

[0001] The present invention relates to the field of AI short video production technology, and specifically, to a short video creation method, system, terminal, and medium based on AI images. Background Art

[0002] The development of short videos has shown a rapid growth trend in recent years. Especially with the popularization of mobile Internet, short videos have become an important way for people to obtain information and entertainment. At the same time, short videos have also become a way for people to record their lives, by shooting videos and images, and drawing short videos to record every bit of life.

[0003] Short videos are used to record life and are also shared on the Internet, which has led to various ways of shooting short videos. Images, as an important carrier for recording travel and life, are also widely published on short video platforms.

[0004] But usually the images are only released with multiple images and music, making the short video content monotonous. Summary of the invention

[0005] In order to increase the content richness of short videos composed of images, the present application provides a short video creation method and system based on AI images.

[0006] In the first aspect, the present application provides a short video creation method based on AI images, which adopts the following technical solutions:

[0007] A short video creation method based on AI image, the method comprising:

[0008] Receiving image material input by a user and basic painting style data selected by the user, and establishing preliminary comic image data according to the basic painting style data and the image material;

[0009] Generate preliminary short video data according to the preliminary comic image data and the short video content information input by the user;

[0010] Receive voice revision information input by the user, and revise the preliminary short video data according to the voice revision information to generate revised short video data.

[0011] Optionally, before receiving the image material input by the user and the basic painting style data selected by the user, the method further includes:

[0012] Receiving the clipping dynamic comic information input by the user, and generating character construction basic information and object construction basic information according to the clipping dynamic comic information;

[0013] Receive the designated character portrayal information and the designated object portrayal information input by the user, and generate basic painting style data according to the designated character portrayal information, the designated object portrayal information and the clip dynamic comic information, wherein the basic painting style data includes preliminary character construction information and preliminary object construction information.

[0014] Optionally, before establishing preliminary comic image data according to the basic painting style data and the image material, the method further includes:

[0015] Obtain multiple historical short video data corresponding to the current user account under the same target basic style;

[0016] An animation production database is constructed based on a plurality of the historical draft review short video data, and the animation production database includes a character identity information data set and an object image data set.

[0017] Optionally, the step of constructing an animation production database based on a plurality of historical draft review short video data includes:

[0018] Acquire historical image materials and historical short video content information corresponding to the plurality of historical draft review short video data;

[0019] According to the plurality of historical image materials and the corresponding historical short video content information, character identity information and object image information are captured, wherein the character identity information includes character social relationship information and character facial feature information;

[0020] Capturing a plurality of historical mimicry character data and a plurality of historical mimicry object data according to the historical draft review short video data;

[0021] Establishing a one-to-one mapping relationship between the plurality of historical simulated character data and the plurality of character identity information, and aggregating and generating a character identity information data set;

[0022] A one-to-one mapping relationship between the plurality of historical mimicry object data and the plurality of object image information is established, and the object image data set is generated by aggregation.

[0023] Optionally, the establishing preliminary comic image data according to the basic painting style data and the image material includes:

[0024] Capturing the person image and the object image in the currently received image material;

[0025] Generate dynamic comic character model data according to the character image, preliminary character construction information and the character identity information data set;

[0026] Generate dynamic comic object model data according to the object image, the preliminary object construction information and the object image data set;

[0027] Based on the dynamic comic character model data and the dynamic comic object model data, preliminary comic image data is generated.

[0028] Optionally, generating preliminary short video data according to the preliminary comic image data and the short video content information input by the user includes:

[0029] Generating a plurality of virtual scene construction images according to the preliminary comic image data and the short video content information;

[0030] Matching one or more corresponding voice package data in a preset voice package database according to the short video content information, and generating matching voice monologue data according to the multiple voice package data;

[0031] Preliminary short video data is generated based on the preliminary comic image data, the virtual scene construction image and the matching voice monologue data.

[0032] Optionally, generating a plurality of virtual scene construction images according to the preliminary comic image data and the short video theme information includes:

[0033] Divide the short video content information into multiple storyline sections according to the storyline;

[0034] A virtual scene construction image is generated according to the target storyline segment, the basic painting style data and the animation production database.

[0035] In the second aspect, the present application provides a short video creation system based on AI images, which adopts the following technical solutions:

[0036] A short video creation system based on AI images, the short video creation system comprising:

[0037] A comic image creation module, used to receive image material input by a user and basic painting style data selected by the user, and to create preliminary comic image data according to the basic painting style data and the image material;

[0038] A short video construction module, used to generate preliminary short video data according to the preliminary comic image data and the short video content information input by the user;

[0039] The short video revision module is used to receive voice revision information input by the user, and revise the preliminary short video data according to the voice revision information to generate the reviewed short video data.

[0040] In a third aspect, the present application provides a smart terminal, which adopts the following technical solution:

[0041] A smart terminal, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the processing of the smart terminal in the AI ​​image-based short video creation method as described in the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0043] A computer-readable storage medium, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the processing of the intelligent terminal in the AI ​​image-based short video creation method as described in the first aspect.

[0044] In summary, the present application includes at least one of the following beneficial technical effects:

[0045] After users upload image materials, they can also convert image materials into cartoon images by selecting basic painting style data and short video content information, and generate preliminary short video data by combining short video content information, thereby enhancing the content of short videos and increasing the degree of free adjustment of cartoon images in short videos, thereby enriching the content of short videos and enhancing their creativity;

[0046] By directly intercepting and editing dynamic comic information, the difficulty of producing comic images of different styles is reduced, giving users more choices;

[0047] By establishing a corresponding relationship between the character and the dynamic comic character model data, the system can directly match the dynamic comic character model data after recognizing the facial features, reducing the system data processing intensity. At the same time, it makes the comic character image corresponding to the same character under the user account unified, enhancing the professionalism and recognizability of the short video data under the user account. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a flowchart of a short video creation method based on AI images provided in an embodiment of the present application.

[0050] Figure 2 It is a schematic diagram of a process for generating basic painting style data provided in an embodiment of the present application.

[0051] Figure 3 It is a flowchart of constructing an animation production database provided in an embodiment of the present application.

[0052] Figure 4 This is another flowchart of constructing an animation production database provided in an embodiment of the present application.

[0053] Figure 5 It is a schematic diagram of a process for establishing preliminary comic image data provided by an embodiment of the present application.

[0054] Figure 6 It is a flow chart of generating preliminary short video data provided by an embodiment of the present application.

[0055] Figure 7 This is a system block diagram of the AI ​​image-based short video creation system provided in an embodiment of the present application.

[0056] Explanation of the accompanying drawings: 701, comic image creation module; 702, short video construction module; 703, short video revision module; 704, comic image basic construction module; 705, historical data collection module; 706, information capture module; 707, voice matching module. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below in conjunction with Figures 1-7.

[0058] The embodiment of the present application provides a short video creation method based on AI images, which can be applied to a short video creation system based on AI images, and the execution subject of the method can be a smart terminal in the short video creation system. The embodiment of the present application takes the conversion between real-scene images and cartoon images in the process of short video creation as an example to illustrate, and other situations are similar and will not be described one by one.

[0059] The following will be combined with specific implementation methods. Figure 1 The processing flow shown is described in detail, and the content can be as follows:

[0060] S101, receiving image material input by a user and basic painting style data selected by the user, and establishing preliminary comic image data according to the basic painting style data and the image material.

[0061] During implementation, the intelligent terminal in the short video creation system receives image materials input by the user and basic painting style data selected by the user.

[0062] At the same time, multiple basic style data are stored in the short video creation system. The basic style data here include preliminary character construction information and preliminary object construction information. The style here can be pre-stored in the memory, or obtained through deep learning after intercepting clips in the animation.

[0063] Then the intelligent terminal converts the image material from the selected basic style data into comic image data, so as to enhance the fun of the image material in the short video data and enrich the content of the short video data.

[0064] S102, generating preliminary short video data according to the preliminary comic image data and the short video content information input by the user.

[0065] During implementation, the intelligent terminal of the short video creation system needs to receive the short video content information input by the user before completing the production of the short video data.

[0066] The short video content information includes the storyline, character positions and scene environment of the short video. After generating the preliminary comic image data, the intelligent terminal divides the storyline into multiple paragraphs, and generates a virtual scene construction image corresponding to each storyline paragraph in combination with the existing preliminary comic image data. The character image in the virtual scene construction image can be derived from the preliminary comic image data or from a character model constructed from basic painting style data. The object scene image in the virtual scene construction image can be derived from the preliminary comic image data or from an object model constructed from basic painting style data, and the scene image is composed of multiple object models.

[0067] S103, receiving the voice revision information input by the user, and revising the preliminary short video data according to the voice revision information to generate the reviewed short video data.

[0068] During implementation, the short video creation system receives voice revision information input by the user, where the voice revision information may be information for revising an image or preliminary short video data constructed for a certain virtual scene.

[0069] The specific processing of revising the preliminary short video data of smart terminals is as follows:

[0070] Receiving voice revision information input by a user, and matching a corresponding modified virtual scene construction image or preliminary comic image data according to the voice revision information;

[0071] The revised content of the voice revision information is identified, and the virtual scene construction image or the preliminary comic image data is modified according to the revised content.

[0072] In implementation, when the smart terminal receives voice revision information and makes revisions, it can first match the corresponding virtual scene construction image or preliminary comic image data according to the voice revision information. The match here can be specified by the user, or it can be the virtual scene construction image or preliminary comic image data displayed correspondingly when the user's voice revision information is received. The virtual scene construction image matched here is called a revised virtual scene construction image.

[0073] Then the intelligent terminal recognizes the revised content of the voice revision and modifies the position corresponding to the revised virtual scene construction image or the preliminary comic image data, for example, the hair color of female A is revised to yellow.

[0074] Furthermore, the intelligent terminal can also have the following processing:

[0075] A speech deep learning module is established for deep learning of user voice data to facilitate fast and accurate identification of revision locations and revision contents.

[0076] During implementation, the smart terminal uses a large amount of historical voice revision information corresponding to the current user account as training samples, and enhances recognition accuracy through a voice deep learning module to facilitate fast and accurate identification of revision locations and revision contents.

[0077] Optionally, before step S101, there is also the following Figure 2 The specific operations are as follows:

[0078] S201, receiving the editing dynamic comic information input by the user, and generating character construction basic information and object construction basic information according to the editing dynamic comic information.

[0079] In the implementation, the smart terminal of the short video creation system receives the clipping dynamic comic information input by the user, and the clipping dynamic comic information here can come from a part of the animation clips of a certain authorized animation, or from the user's own hand-drawn animation. Then the smart terminal performs deep learning on the animation characters and animation scenes based on the obtained clipping dynamic comic information to generate basic character construction information and object construction basic information.

[0080] For example, the basic information for character construction is to obtain the method of drawing the facial features of the comic character, facial contour composition, object construction, etc. It should be noted that due to the limited amount of user input samples, the learning effect is different. For example, when the given samples only contain male and female cartoon characters with relatively symmetrical figures, then in the later process of constructing comics based on the real picture, only after identifying the gender of the character in the real picture, the corresponding male or female cartoon character with a relatively symmetrical figure is matched.

[0081] S202, receiving the designated character characterization information and the designated object characterization information input by the user, and generating basic painting style data according to the designated character characterization information, the designated object characterization information and the clipped dynamic animation information, wherein the basic painting style data includes preliminary character construction information and preliminary object construction information.

[0082] In implementation, after generating basic character construction information and object construction basic information, the intelligent terminal needs to receive the specified character depiction information and the specified object depiction information input by the user. Then the intelligent terminal combines the specified character depiction information and the specified object depiction information with the clip dynamic comic information to generate basic painting style data corresponding to the current user account.

[0083] For example, by specifying character portrayal information, such as defining the size of the cartoon character's eyes in the basic painting style data, the short video produced can have its originality and richness.

[0084] Optionally, after receiving the basic style information selected by the user in step S103, before establishing the preliminary comic image data, there is also the following Figure 3 The specific operations are as follows:

[0085] S301, obtaining multiple historical short video data corresponding to the current user account under the same target basic style.

[0086] During implementation, the smart terminal obtains multiple draft review short video data corresponding to the current user account, and filters out the draft review short video data with the same basic style data currently selected by the user. The basic style data selected by the user here is called target basic style data, and the filtered draft review short video data can be called historical draft review short video data.

[0087] S302, constructing an animation production database based on the plurality of historical draft review short video data, wherein the animation production database includes a character identity information data set and an object image data set.

[0088] During implementation, the smart terminal extracts character animated images and object animated images from the historical short video data, and then summarizes multiple character animated images and multiple object animated images into a character identity information data set and an object image data set, respectively, so as to reduce the difficulty of the smart terminal needing to call the same character animated image or the same object animated image again.

[0089] Optionally, in step S302, there is also Figure 4 The specific operations are as follows:

[0090] S401, obtaining historical image materials and historical short video content information corresponding to the plurality of historical draft review short video data.

[0091] During implementation, the intelligent terminal of the short video creation system obtains image materials and short video content information corresponding to multiple historical draft short video data in a specified network address or its own memory. The image materials here are called historical image materials, and the short video content information here is called historical short video content information.

[0092] S402, capturing character identity information and object image information based on the plurality of historical image materials and corresponding historical short video content information, wherein the character identity information includes character social relationship information and character facial feature information.

[0093] In the implementation, the smart terminal captures the identity information of each person in the historical image material based on the short video content information. The identity information here includes the facial feature information of the person and the corresponding social relationship information of the person. At the same time, the smart terminal also captures the image information of objects, such as houses and trees, from the historical image material.

[0094] S403, capturing a plurality of historical simulated character data and a plurality of historical simulated object data according to the historical manuscript review short video data.

[0095] During implementation, the intelligent terminal captures multiple simulated character data and multiple simulated object data in multiple historical draft short-sighted data, where the simulated character data are characters in specific cartoon forms, and the simulated character data here are called historical simulated character data, and the object simulated data here are objects in cartoon forms, and the simulated object data here are called historical simulated data.

[0096] S404, establishing a one-to-one mapping relationship between the plurality of historical simulated character data and the plurality of character identity information, and aggregating and generating a character identity information data set.

[0097] During implementation, the intelligent terminal establishes a one-to-one mapping relationship between multiple historical simulated character data and multiple character identity information by comparing historical image materials and historical draft short video data, and at the same time aggregates multiple character identity information and multiple historical simulated character data to generate a character identity information data set.

[0098] S405, establishing a one-to-one mapping relationship between the plurality of historical mimicry object data and the plurality of object image information, and summarizing and generating an object image data set.

[0099] During implementation, the intelligent terminal establishes a one-to-one mapping relationship between multiple object image information and multiple historical mimetic object data by comparing historical image materials and historical draft short video data, and at the same time aggregates multiple object image information and multiple historical mimetic object data to generate an object image data set.

[0100] Optionally, in step S101, there is also Figure 5 The specific operations are as follows:

[0101] S501, capturing the person image and the object image in the currently received image material.

[0102] In implementation, the intelligent terminal captures the human image and the object image in the currently received image material, so as to convert the human image and the object image into the animation image respectively.

[0103] S502, generating dynamic comic character model data according to the character image, preliminary character construction information and the character identity information data set.

[0104] In the implementation, the smart terminal first matches the currently captured person image with the corresponding person identity information in the person identity information dataset, wherein the matching method can be based on the facial features of the person in the person image, or based on the social relationship of the person in the person image. There are two situations here:

[0105] Case 1: The character identity data set contains character identity information corresponding to the character image, then a corresponding relationship between the character image and the character identity information and the corresponding historical mimicry character data is established, and dynamic comic character model data is established from the historical mimicry character data;

[0106] Case 2: If the character identity data set does not contain the character identity information corresponding to the character image, the dynamic comic character model data is established based on the current character image and preliminary character construction information.

[0107] S503, generating dynamic comic object model data according to the object image, the preliminary object construction information and the object image data set.

[0108] In the implementation, the intelligent terminal first matches the currently captured object image with the corresponding object image information in the object image dataset. The matching method can be based on the object name in the object image or the structural image in the object image itself. There are two cases:

[0109] Case 1: The object image data set contains object image information corresponding to the object image, then a corresponding relationship between the object image and the object image information and the corresponding historical mimicry object data is established, and dynamic comic object model data is established from the historical mimicry object data;

[0110] Case 2: If the object image data set does not contain object image information corresponding to the object image, the dynamic comic object model data is established based on the current object image and preliminary object construction information.

[0111] S504, generating preliminary comic image data based on the dynamic comic character model data and the dynamic comic object model data.

[0112] During implementation, the intelligent terminal combines the dynamic comic character model data and the dynamic comic object model data based on the positional relationship between the character image and the object image in the image material to generate preliminary comic image data.

[0113] Optionally, in step S102, there is also Figure 6 The specific operations are as follows:

[0114] S601, generating a plurality of virtual scene construction images according to the preliminary comic image data and the short video content information.

[0115] In implementation, after receiving the short video content information, the smart terminal divides the short video content information into multiple story segments, where the story segments can be divided according to changes in scenes, or numbers or paragraph distribution in the short video content information.

[0116] Then the intelligent terminal constructs a virtual scene construction image according to the corresponding story content of each story segment. The virtual scene construction image here is mainly composed of dynamic comic object model data and dynamic comic object model data, and the specific position distribution can come from the intelligent arrangement after AI deep learning, or AI's migration or deep learning of historical draft short video data, or the basic object arrangement rules input by the user.

[0117] S602, matching one or more corresponding voice package data in a preset voice package database according to the short video content information, and generating matching voice monologue data according to the multiple voice package data.

[0118] In implementation, the smart terminal stores a voice package database, which is composed of multiple voice package data. The smart terminal matches the voice monologue content in the short video content information with the corresponding voice package data, where the voice package data can be matched based on age, gender, etc.

[0119] Then, the intelligent terminal generates matching voice monologue data matching the current short video content information from the matched multiple voice package data.

[0120] S603, generating preliminary short video data according to the preliminary comic image data, the virtual scene construction image and the matching voice monologue data.

[0121] During implementation, the smart terminal sorts the preliminary comic image data and multiple virtual scene construction images according to the arrangement order of multiple storyline segments in the short video content information, and generates preliminary short video data in combination with matching voice monologue data.

[0122] Optionally, in step S601, there is also the following processing method, and the specific operation flow is as follows:

[0123] Divide the short video content information into multiple storyline sections according to the storyline;

[0124] A virtual scene construction image is generated according to the target storyline segment, the basic painting style data and the animation production database.

[0125] In implementation, the smart terminal divides the storyline in the short video content information into multiple storyline short circuits, where the storyline segments can be divided according to the changes in the scenes, or the labels or paragraph distribution in the short video content information.

[0126] Here, the processing of one of the storyline segments is taken as an example for explanation, which is called target storyline short-circuiting. The intelligent terminal uses the target storyline short-circuiting to capture the names of characters and objects appearing in the storyline, and preferentially compares the character identity information and object image information in the animation production database.

[0127] When the animation production database contains the same character names and object names as those in the target storyline, the corresponding target dynamic comic character model data and target dynamic comic object model data are first screened out. Then, when the character names and object names in the target storyline do not exist in the animation production database, the remaining animation character images and animation object images are constructed from the target storyline and the basic painting style data. Since fewer references can be designed here, the remaining animation character images and animation object images here can come from the pre-designed basic character model database and basic model design database, or from the deep learning obtained from the clipped dynamic comic information.

[0128] The embodiment of the present application also discloses a short video creation system based on AI images, characterized in that the short video creation system includes:

[0129] The comic image creation module 701 is used to receive the image material input by the user and the basic painting style data selected by the user, and to create preliminary comic image data according to the basic painting style data and the image material;

[0130] A short video construction module 702, configured to generate preliminary short video data according to the preliminary comic image data and the short video content information input by the user;

[0131] The short video revision module 703 is used to receive the voice revision information input by the user, and revise the preliminary short video data according to the voice revision information to generate the reviewed short video data.

[0132] Optionally, the short video creation system can also be used for:

[0133] A comic image basic construction module 704 is used to receive the clipping dynamic comic information input by the user, and generate character construction basic information and object construction basic information according to the clipping dynamic comic information;

[0134] The comic image basic construction module 704 is used to receive the specified character portrayal information and the specified object portrayal information input by the user, and generate basic style data based on the specified character portrayal information, the specified object portrayal information and the edited dynamic comic information. The basic style data includes preliminary character construction information and preliminary object construction information.

[0135] Optionally, the short video creation system can also be used for:

[0136] The historical data collection module 705 is used to obtain multiple historical proofreading short video data corresponding to the current user account under the same target basic painting style;

[0137] The cartoon image basic construction module 704 is used to construct an animation production database based on a plurality of the historical draft short video data, and the animation production database includes a character identity information data set and an object image data set.

[0138] Optionally, the short video creation system is specifically used for:

[0139] The historical data collection module 705 is used to obtain historical image materials and historical short video content information corresponding to the historical draft review short video data;

[0140] The information capture module 706 is used to capture character identity information and object image information based on the plurality of historical image materials and the corresponding historical short video content information, wherein the character identity information includes character social relationship information and character facial feature information;

[0141] An information capture module 706 is used to capture a plurality of historical mimicry character data and a plurality of historical mimicry object data according to the historical draft review short video data;

[0142] A comic image basic construction module 704 is used to establish a one-to-one mapping relationship between a plurality of the historical mimicry character data and a plurality of the character identity information, and to generate a character identity information data set in aggregating manner;

[0143] The comic image basic construction module 704 is used to establish a one-to-one mapping relationship between the plurality of historical mimicry object data and the plurality of object image information, and to generate an object image data set by aggregation.

[0144] Optionally, the short video creation system is specifically used for:

[0145] An information capturing module 706 is used to capture the person image and the object image in the currently received image material;

[0146] A comic image basic construction module 704 is used to generate dynamic comic character model data according to the character image, preliminary character construction information and the character identity information data set;

[0147] A comic image basic construction module 704 is used to generate dynamic comic object model data according to the object image, the preliminary object construction information and the object image data set;

[0148] The comic image creation module 701 generates preliminary comic image data based on the dynamic comic character model data and the dynamic comic object model data.

[0149] Optionally, the short video creation system is specifically used for:

[0150] A comic image creation module 701 is used to generate a plurality of virtual scene construction images according to the preliminary comic image data and the short video content information;

[0151] A voice matching module 707 is used to match one or more corresponding voice package data in a preset voice package database according to the short video content information, and generate matching voice monologue data according to the multiple voice package data;

[0152] The short video construction module 702 is used to generate preliminary short video data according to the preliminary comic image data, the virtual scene construction image and the matching voice monologue data.

[0153] Optionally, the short video creation system is specifically used for:

[0154] A comic image creation module 701 is used to divide the storyline in the short video content information into multiple storyline sections;

[0155] The comic image creation module 701 is used to generate a virtual scene construction image according to the target storyline paragraph, the basic painting style data and the animation production database.

[0156] In an embodiment of the present application, a smart terminal is provided, which may have relatively large differences due to different configurations or performances, and may include one or more central processing units (e.g., one or more processors) and memories, and one or more storage media (e.g., one or more mass storage devices) for storing applications or data. Among them, the memories and storage media may be short-term storage or permanent storage. The program stored in the storage medium may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the smart terminal.

[0157] The intelligent terminal may also include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, one or more keyboards, and / or one or more operating systems.

[0158] The smart terminal may include a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, including the processing of the smart terminal in the above-mentioned AI image-based short video creation method.

[0159] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program. The program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, etc.

[0160] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

[0161] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A short video creation method based on AI image, characterized in that: The method comprises: Receiving image material input by a user and basic painting style data selected by the user, and establishing preliminary comic image data according to the basic painting style data and the image material; Generate preliminary short video data according to the preliminary comic image data and the short video content information input by the user; Receiving voice revision information input by a user, and revising the preliminary short video data according to the voice revision information to generate a revised short video data; Before receiving the image material input by the user and the basic painting style data selected by the user, the method further includes: Receiving the clipping dynamic comic information input by the user, and generating character construction basic information and object construction basic information according to the clipping dynamic comic information; Receiving designated character characterization information and designated object characterization information input by a user, and generating basic painting style data according to the designated character characterization information, the designated object characterization information and the clip dynamic comic information, wherein the basic painting style data includes preliminary character construction information and preliminary object construction information; Before establishing preliminary comic image data according to the basic painting style data and the image material, the method further includes: Obtain multiple historical short video data corresponding to the current user account under the same target basic style; Constructing an animation production database based on the plurality of historical draft review short video data, wherein the animation production database includes a character identity information data set and an object image data set; The method of constructing an animation production database based on a plurality of historical draft review short video data includes: Acquire historical image materials and historical short video content information corresponding to the plurality of historical draft review short video data; According to the plurality of historical image materials and the corresponding historical short video content information, character identity information and object image information are captured, wherein the character identity information includes character social relationship information and character facial feature information; Capturing a plurality of historical mimicry character data and a plurality of historical mimicry object data according to the historical draft review short video data; Establishing a one-to-one mapping relationship between the plurality of historical simulated character data and the plurality of character identity information, and aggregating and generating a character identity information data set; Establishing a one-to-one mapping relationship between the plurality of historical mimicry object data and the plurality of object image information, and aggregating and generating an object image data set; The step of establishing preliminary comic image data according to the basic painting style data and the image material comprises: Capturing the person image and the object image in the currently received image material; Generate dynamic comic character model data according to the character image, preliminary character construction information and the character identity information data set; Generate dynamic comic object model data according to the object image, the preliminary object construction information and the object image data set; Based on the dynamic comic character model data and the dynamic comic object model data, preliminary comic image data is generated.

2. The method according to claim 1, characterized in that The generating preliminary short video data according to the preliminary comic image data and the short video content information input by the user comprises: Generating a plurality of virtual scene construction images according to the preliminary comic image data and the short video content information; Matching one or more corresponding voice package data in a preset voice package database according to the short video content information, and generating matching voice monologue data according to the multiple voice package data; Preliminary short video data is generated based on the preliminary comic image data, the virtual scene construction image and the matching voice monologue data.

3. The method according to claim 2, characterized in that The step of generating a plurality of virtual scene construction images according to the preliminary comic image data and the short video theme information includes: Divide the short video content information into multiple storyline sections according to the storyline; A virtual scene construction image is generated according to the target storyline segment, the basic painting style data and the animation production database.

4. A short video creation system based on AI image, characterized in that: The short video creation system comprises: A comic image creation module (701) is used to receive image material input by a user and basic painting style data selected by a user, and to create preliminary comic image data based on the basic painting style data and the image material; A short video construction module (702), configured to generate preliminary short video data according to the preliminary comic image data and the short video content information input by the user; A short video revision module (703) is used to receive voice revision information input by a user, and revise the preliminary short video data according to the voice revision information to generate a proofread short video data; A comic image basic construction module (704) is used to receive the clipping dynamic comic information input by the user, and generate character construction basic information and object construction basic information according to the clipping dynamic comic information; A comic image basic construction module (704) is used to receive the designated character characterization information and the designated object characterization information input by the user, and generate basic painting style data according to the designated character characterization information, the designated object characterization information and the clip dynamic comic information, wherein the basic painting style data includes preliminary character construction information and preliminary object construction information; A historical data collection module (705) is used to obtain multiple historical proofreading short video data corresponding to the current user account under the same target basic painting style; A cartoon image basic construction module (704) is used to construct an animation production database based on a plurality of the historical draft short video data, wherein the animation production database includes a character identity information data set and an object image data set; A historical data collection module (705) is used to obtain historical image materials and historical short video content information corresponding to a plurality of historical draft review short video data; An information capture module (706) is used to capture character identity information and object image information based on the plurality of historical image materials and corresponding historical short video content information, wherein the character identity information includes character social relationship information and character facial feature information; An information capture module (706) is used to capture a plurality of historical mimicry character data and a plurality of historical mimicry object data according to the historical draft review short video data; A comic image basic construction module (704) is used to establish a one-to-one mapping relationship between a plurality of the historical mimicry character data and a plurality of the character identity information, and to generate a character identity information data set by aggregating the data; A comic image basic construction module (704) is used to establish a one-to-one mapping relationship between a plurality of the historical mimicry object data and a plurality of the object image information, and to generate an object image data set in aggregating manner; An information capturing module (706) is used to capture the person image and the object image in the currently received image material; A comic image basic construction module (704) is used to generate dynamic comic character model data based on the character image, preliminary character construction information and the character identity information data set; A comic image basic construction module (704), used to generate dynamic comic object model data according to the object image, the preliminary object construction information and the object image data set; The comic image creation module (701) generates preliminary comic image data based on the dynamic comic character model data and the dynamic comic object model data.

5. An intelligent terminal, characterized in that: The smart terminal includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the processing of the smart terminal in the AI ​​image-based short video creation method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the processing of the intelligent terminal in the AI ​​image-based short video creation method as described in any one of claims 1 to 3.

Images