Video generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video generation method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface; performing key information extraction on the insurance video description data to obtain insurance video description key sentences; performing video generation on the insurance video description key sentence based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data; based on the insurance video script, screening out target video material segments from the plurality of candidate video material segments, and combining the target video material segments into an initial insurance video; editing the initial insurance video based on editing instruction data in the insurance video script to obtain an edited insurance video; and performing subtitle generation on the edited insurance video to obtain a target insurance video. According to the embodiment of the invention, the insurance video generation efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a video generation method and device, an electronic device and a storage medium. Background Art

[0002] At present, in various business fields, videos are often used to popularize business knowledge for various objects so that the objects can fully understand different businesses. For example, in the insurance field, insurance agents often use insurance videos to introduce different types of insurance to insured objects and popularize various insurance-related knowledge.

[0003] However, since the interests, preferences and actual needs of each insured person are often different, if a universal insurance video is used to introduce insurance to each insured person, the video content of the insurance video will often be inconsistent with the actual explanation. In this regard, the relevant technology considers that insurance agents produce and operate different insurance videos for different insured persons. This method often relies heavily on the video production experience of insurance agents, and there is a problem of low efficiency in the production and generation of insurance videos. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a video generation method and device, an electronic device and a storage medium, aiming to improve the generation efficiency of insurance videos.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides a video generation method, the method comprising:

[0006] In response to an insurance video generation request initiated by a target object on an interactive interface, obtaining insurance video description data;

[0007] Extracting key information from the insurance video description data to obtain key sentences of the insurance video description;

[0008] Generate a video of the key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data;

[0009] Based on the insurance video script, select target video material segments from a plurality of candidate video material segments, and combine the target video material segments into an initial insurance video;

[0010] Performing editing processing on the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video;

[0011] Subtitles are generated for the edited insurance video to obtain a target insurance video.

[0012] In some embodiments, the step of generating a video based on a preset video generation model for the key sentences of the insurance video description to obtain an insurance video script corresponding to the insurance video description data includes:

[0013] Based on the video generation model, embedding processing is performed on the key sentences of the insurance video description to obtain video description embedding features;

[0014] Performing feature encoding on the video description embedding feature to obtain a video description encoding feature;

[0015] Performing self-attention calculation on the video description encoding features to obtain a video frame feature representation vector and a clip instruction feature vector;

[0016] Feature decoding is performed on the video frame feature representation vector and the editing instruction feature vector to obtain the insurance video script.

[0017] In some embodiments, the step of selecting a target video material segment from a plurality of candidate video material segments based on the insurance video script includes:

[0018] Extracting video scene description information from the insurance video script;

[0019] Extracting content from the candidate video material segment to obtain video segment content of the candidate video material segment;

[0020] For each of the candidate video material segments, performing content similarity calculation on the video scene description information and the video segment content to obtain video content similarity;

[0021] Among the plurality of candidate video material segments, the candidate video material segments whose video content similarity is greater than a preset threshold are screened out as the target video material segments.

[0022] In some embodiments, the editing of the initial insurance video based on the editing instruction data in the insurance video script to obtain the edited insurance video includes:

[0023] If the editing instruction in the editing instruction data is to replace the reference object in the initial insurance video with the target object, acquiring a target object image of the target object;

[0024] Extracting object features from the target object image to obtain hidden facial features of the target object;

[0025] Segmenting each video frame of the initial insurance video to obtain a video background image corresponding to each video frame;

[0026] The video is generated based on the video background image and the hidden features of the object's face to obtain the edited insurance video.

[0027] In some embodiments, after the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain an edited insurance video, the method further includes:

[0028] Receiving video adjustment data input by the target object on the interactive interface;

[0029] Parsing the video adjustment data to obtain a clip adjustment sequence, additional animation materials, and rendering information of the additional animation materials;

[0030] Rearranging the multiple video frames of the edited insurance video based on the editing adjustment order to obtain a rearranged insurance video;

[0031] The additional animation material is added to a target video frame of the rearranged insurance video based on the rendering information.

[0032] In some embodiments, after generating subtitles for the edited insurance video to obtain a target insurance video, the method further includes:

[0033] In response to a video export request initiated by a target object on an interactive interface, obtaining a video export format and a video export resolution;

[0034] Based on the video export format and the video export resolution, the target insurance video is exported to a local database.

[0035] In some embodiments, the step of obtaining insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface includes:

[0036] In response to the insurance video generation request, obtaining video description voice data input by the target object on the interactive interface, wherein the video description voice data is used to describe the video content of the insurance video to be generated;

[0037] Extracting features of the video description voice data to obtain voice Mel-spectrogram features of the video description voice data;

[0038] Speech recognition is performed based on the speech mel-spectrogram feature to obtain insurance video description data corresponding to the video description speech data.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a video generation device, the device comprising:

[0040] An acquisition unit, configured to acquire insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface;

[0041] An extraction unit, used to extract key information from the insurance video description data to obtain key sentences of the insurance video description;

[0042] A first generating unit, configured to generate a video for the key sentences of the insurance video description based on a preset video generation model, and obtain an insurance video script corresponding to the insurance video description data;

[0043] A screening unit, configured to screen out target video material segments from a plurality of candidate video material segments based on the insurance video script, and combine the target video material segments into an initial insurance video;

[0044] An editing unit, configured to perform editing processing on the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video;

[0045] The second generating unit is used to generate subtitles for the edited insurance video to obtain a target insurance video.

[0046] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the video generation method described in the first aspect is implemented.

[0047] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video generation method described in the first aspect is implemented.

[0048] The video generation method and device, electronic device and storage medium proposed in the present application obtain insurance video description data by responding to the insurance video generation request initiated by the target object on the interactive interface, and extract key information from the insurance video description data to obtain insurance video description key sentences, which can simplify the effective content of the insurance video description data and eliminate the influence of irrelevant content on video generation. Further, based on the preset video generation model, the insurance video description key sentences are generated to obtain the insurance video script corresponding to the insurance video description data, and based on the insurance video script, the target video material segments are screened out from multiple candidate video material segments, and the target video material segments are combined into an initial insurance video, which can automatically generate the initial insurance video using the video generation model and the collected candidate video material segments, thereby improving the generation efficiency of the initial insurance video. Further, the initial insurance video is used as a template video, and the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain the edited insurance video, which can realize flexible adjustment of the template video so that the edited insurance video meets personalized needs. Finally, subtitles are generated for the edited insurance video to obtain the target insurance video, which can improve the generation efficiency of the insurance video and reduce the complexity and cost of producing personalized target insurance videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a video generation method provided in an embodiment of the present application;

[0050] Figure 2 yes Figure 1 Flow chart of step S101 in FIG.

[0051] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0052] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0053] Figure 5 yes Figure 1 Flow chart of step S105 in FIG.

[0054] Figure 6 is a flow chart of video adjustment of an insurance video after editing according to an embodiment of the present application;

[0055] Figure 7 is a flow chart of storing a target insurance video according to an embodiment of the present application;

[0056] Figure 8 is a structural schematic diagram of a video generating device provided in an embodiment of the present application;

[0057] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] The video generation method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the video generation method in the embodiments of the present application is described.

[0062] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0064] The video generation method provided in the embodiment of the present application relates to the fields of financial technology and artificial intelligence technology. The video generation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video generation method, etc., but is not limited to the above forms.

[0065] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0066] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to the identity or characteristics of the object, such as object information, object behavior data, object historical data, and object location information, the permission or consent of the object will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the personal information of the object, the separate permission or separate consent of the object will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the object, the necessary object-related data for the normal operation of the embodiment of the present application will be obtained.

[0067] Figure 1 is an optional flowchart of the video generation method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0068] Step S101, in response to an insurance video generation request initiated by a target object on an interactive interface, obtaining insurance video description data;

[0069] Step S102, extracting key information from the insurance video description data to obtain key sentences of the insurance video description;

[0070] Step S103, generating a video of the key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data;

[0071] Step S104, based on the insurance video script, selecting target video material segments from a plurality of candidate video material segments, and combining the target video material segments into an initial insurance video;

[0072] Step S105, editing the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video;

[0073] Step S106, generating subtitles for the edited insurance video to obtain a target insurance video.

[0074] In the steps S101 to S106 shown in the embodiment of the present application, by responding to the insurance video generation request initiated by the target object on the interactive interface, the insurance video description data is obtained, and the key information of the insurance video description data is extracted to obtain the insurance video description key sentence, which can simplify the effective content of the insurance video description data and eliminate the influence of irrelevant content on video generation. Further, based on the preset video generation model, the insurance video description key sentence is generated to obtain the insurance video script corresponding to the insurance video description data, and based on the insurance video script, the target video material segment is screened out from multiple candidate video material segments, and the target video material segment is combined into an initial insurance video, which can automatically generate the initial insurance video using the video generation model and the collected candidate video material segments, thereby improving the generation efficiency of the initial insurance video. Further, the initial insurance video is used as a template video, and the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain the edited insurance video, which can realize the flexible adjustment of the template video so that the edited insurance video meets the personalized needs. Finally, subtitles are generated for the edited insurance video to obtain the target insurance video, which can improve the generation efficiency of the insurance video and reduce the complexity and cost of producing personalized target insurance videos.

[0075] In step S101 of some embodiments, in response to an insurance video generation request initiated by a target object on an interactive interface, insurance video description data is acquired.

[0076] The target object refers to the object for which the insurance video is to be generated, for example, the target object may be an insurance agent, etc.

[0077] The interactive interface refers to the visual interface provided to the insured or insurance agent for interactive operations.

[0078] An insurance video generation request refers to a request initiated by a target object when generating a specific insurance video.

[0079] The insurance video description data is used to indicate information such as the specific video content, video scene description, and video plot development of the insurance video to be generated.

[0080] For example, the insurance video description data may be: generating a video related to vehicle accident insurance, where the scene where the vehicle accident occurred is xxxxx.

[0081] See also Figure 2 In some embodiments, step S101 may include but is not limited to steps S201 to S203:

[0082] Step S201, in response to an insurance video generation request, obtaining video description voice data input by a target object on an interactive interface;

[0083] Step S202, extracting features of the video description speech data to obtain speech Mel-spectrogram features of the video description speech data;

[0084] Step S203, performing speech recognition based on the speech Mel-spectrogram feature to obtain insurance video description data corresponding to the video description speech data.

[0085] Steps S201 to S203 are described in detail below.

[0086] In step S201 of some embodiments, the video description voice data is used to describe the video content of the insurance video to be generated.

[0087] In the specific implementation of this embodiment, when an insurance video is to be generated, the target object can describe the specific information of the insurance video to be generated in the form of voice input on the interactive interface and initiate an insurance video generation request. Then, the server can receive the insurance video generation request and obtain the video description voice data input by the target object.

[0088] In step S202 of some embodiments, when the server parses the video description voice data from the insurance video generation request, it performs noise reduction and frame processing on the video description voice data to obtain voice frame data to improve voice quality. Then, it extracts key feature information from the voice frame data to obtain the Mel frequency cepstral coefficients corresponding to the video description voice data, and combines the Mel frequency cepstral coefficients into voice Mel frequency spectrum features.

[0089] In step S203 of some embodiments, the speech mel-spectrogram feature is input into the acoustic model to extract the phoneme feature to obtain the phoneme feature corresponding to the video description speech data, and the phoneme feature is decoded into a word sequence, and each word in the word sequence is combined into the insurance video description data corresponding to the video description speech data.

[0090] The advantage of this embodiment is that when generating an insurance video, the target object can provide video description voice data through voice input on the interactive interface. For the video description voice data, the server can convert the video description voice data into text form by means of feature extraction and voice recognition to obtain insurance video description data, which is conducive to improving the input diversity and processing accuracy of video description information.

[0091] In step S102 of some embodiments, key information is extracted from the insurance video description data to obtain key sentences of the insurance video description.

[0092] The insurance video description key sentence refers to a key sentence in the insurance video description data that describes the video content of the insurance video to be generated.

[0093] When this embodiment is implemented specifically, first, the insurance video description data is divided into sentences, and the sentence boundaries are identified according to the punctuation marks (such as periods, question marks, exclamation marks, etc.) at the end of each sentence in the insurance video description data and the subsequent spaces or paragraph marks, so as to divide the insurance video description data into multiple initial description sentences. Then, for each initial description sentence, the initial description sentence is input into a preset text encoding model, and the text encoding model encodes the initial description sentence, converts the initial description sentence from the data space to the latent vector space, and obtains the sentence encoding features of the initial description sentence, wherein the sentence encoding features are used to indicate the semantic content representation of the initial description sentence in the latent vector space. Furthermore, based on the text encoding model, the sentence encoding features of each initial description sentence are pooled, and the key semantic content information in the sentence encoding features is extracted to obtain the sentence representation vector of the initial description sentence. Furthermore, based on the random forest algorithm, the sentence representation vector of each initial description sentence is scored for importance, and the importance score of the sentence representation vector of each initial description sentence for determining the video content is calculated. According to the importance score, a predetermined number of initial description sentences with larger importance scores are screened out from multiple initial description sentences as target description sentences, and the target description sentences are combined into key sentences for insurance video description.

[0094] This method realizes the sentence segmentation of insurance video description data, filters out sentence information irrelevant to the determined video content at the sentence level, and uses sentences with a high degree of relevance to the determined video content as key sentences for insurance video description, thereby realizing the accurate extraction of key information of insurance video description data.

[0095] In step S103 of some embodiments, video generation is performed on key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data.

[0096] The video generation model refers to a neural network model built based on the transformer model, which can generate corresponding video script data based on the insurance video description text.

[0097] The insurance video script is used to indicate the scene description and plot development information of the video required to generate the insurance video that meets the video generation request. The insurance video script is also used to indicate what the editing process is to be done after the preliminary insurance video is generated.

[0098] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S304:

[0099] Step S301, embedding the key sentences of the insurance video description based on the video generation model to obtain the video description embedding features;

[0100] Step S302, feature encoding is performed on the video description embedded feature to obtain the video description encoding feature;

[0101] Step S303, performing self-attention calculation on the video description coding features to obtain a video frame feature representation vector and a clip instruction feature vector;

[0102] Step S304, feature decoding is performed on the video frame feature representation vector and the editing instruction feature vector to obtain the insurance video script.

[0103] Steps S301 to S304 are described in detail below.

[0104] In step S301 of some embodiments, each insurance video description key sentence is word-embedded based on the video generation model, and each description word of the insurance video description key sentence is converted from the data space to the latent vector space to obtain a video description embedding feature corresponding to each insurance video description key sentence. Each description word of the insurance video description key sentence is a word embedding feature in the video description embedding feature of the insurance video description key sentence.

[0105] In step S302 of some embodiments, when the video description embedding feature is feature encoded, first, a preset word list is obtained, wherein the preset word list is used to indicate the candidate feature vectors corresponding to each candidate word embedding feature in the latent vector space. Further, for each insurance video description key sentence, the preset word list is searched for the candidate word embedding feature that is consistent with the word embedding feature of each description word of the insurance video description key sentence in the video description embedding feature, and the candidate feature vector corresponding to the candidate word embedding feature that is consistent with the word embedding feature is used as the description encoding feature of the description word. Finally, the description encoding features of all description words are spliced ​​into the video description encoding feature of the insurance video description key sentence.

[0106] In step S303 of some embodiments, for each key sentence of the insurance video description, the self-attention layer of the video generation model linearly projects the video description encoding features of the key sentence of the insurance video description on the query channel, the key channel, and the value channel, respectively, to obtain the first query vector output by the query channel, the first key vector output by the key channel, and the first value vector output by the value channel. Next, a dot product operation is performed on the first key vector and the first query vector to obtain a first attention score, and the square root of the feature dimension of the first key vector is used as the first scaling factor, and the first attention score is divided by the first scaling factor to achieve a scaling operation on the first attention score to obtain the scaled first attention score. Furthermore, the scaled first attention score is normalized using a softmax function to obtain a first attention weight, and the first value vector is weighted using the first attention weight to obtain the attention calculation result of the key sentence of the insurance video description.

[0107] Furthermore, a classification function is used to perform a binary classification calculation on the attention calculation results of the key sentences describing the insurance video, generating a first probability that the key sentences describing the insurance video indicate the content of the video frame and a second probability that the key sentences describing the insurance video indicate editing instructions. Then, when the first probability is greater than the second probability, the attention calculation results of the key sentences describing the insurance video are divided into a first set, and when the first probability is less than the second probability, the attention calculation results of the key sentences describing the insurance video are divided into a second set. Finally, the attention calculation results of the first set are spliced ​​into a video frame feature representation vector. The attention calculation results of the second set are spliced ​​into an editing instruction feature vector. Among them, the editing instruction feature vector is used to indicate the editing operation to be performed on the generated initial insurance video, and the video frame feature representation vector is used to indicate the video frame content of each video frame of the initial insurance video to be generated.

[0108] In step S304 of some embodiments, first, the video frame feature representation vector and the editing instruction feature vector are spliced ​​into a target video splicing feature. Then, the target video splicing feature is input into the linear layer of the video generation model, and the linear layer performs feature mapping on the target video splicing feature, mapping the target video splicing feature from a high-dimensional space to a low-dimensional space to obtain a target video mapping feature. Then, the target video mapping feature is input into the decoding layer of the video generation model, and the decoding layer converts the target video mapping feature into a sequence form to obtain an insurance video script.

[0109] The benefit of this embodiment is that it introduces a video generation model based on the self-attention mechanism to perform video frame content extraction and clip content extraction on key sentences in the insurance video description, so as to obtain the video script of the insurance video to be generated from the key sentences in the insurance video description. Using the insurance video script for video generation can realize the automatic generation and editing of insurance videos, which is beneficial to improving the generation efficiency of insurance videos and reducing the generation complexity and production cost of insurance videos.

[0110] In step S104 of some embodiments, based on the insurance video script, target video material segments are screened out from a plurality of candidate video material segments, and the target video material segments are combined into an initial insurance video.

[0111] The candidate video material segments are used to indicate pre-collected video segments used to generate various types of videos related to insurance knowledge explanation.

[0112] The target video material segment is used to indicate a video segment of a video related to insurance knowledge explanation that matches the current insurance video generation request.

[0113] The initial insurance video is used to indicate a complete insurance video generated according to a video script and satisfying an insurance video generation request.

[0114] See also Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S404:

[0115] Step S401, extracting video scene description information from the insurance video script;

[0116] Step S402, extracting content from the candidate video material segment to obtain video segment content of the candidate video material segment;

[0117] Step S403, for each candidate video material segment, performing content similarity calculation on the video scene description information and the video segment content to obtain the video content similarity;

[0118] Step S404 , selecting candidate video material segments whose video content similarity is greater than a preset threshold from among the multiple candidate video material segments as target video material segments.

[0119] Steps S301 to S304 are described in detail below.

[0120] In step S401 of some embodiments, since the insurance video script contains multiple types of script data, for example, the insurance video script contains script data representing the editing operation, script data representing the video content of each video frame, etc. Based on this, in the insurance video script, according to the specific type of each script data, the video script data whose script type is scene description is obtained, and the obtained video script data is used as the video scene description information.

[0121] In step S402 of some embodiments, a named entity algorithm is used to extract content from the candidate video material segments, obtain video entity features in each candidate video material segment, and decode the video entity features into video segment content of the candidate video material segment.

[0122] In step S403 of some embodiments, first, the video scene description information and the video segment content of each candidate video material segment are mapped to a vector space of the same dimension to obtain a scene description feature vector corresponding to the video scene description information and a video segment content feature vector of each candidate video material segment. Next, for each candidate video material segment, a cosine similarity algorithm is used to calculate the vector similarity of the video segment content feature vector and the scene description feature vector to obtain cosine similarity, and the cosine similarity is used as the video content similarity of the candidate video material segment and the video scene description information.

[0123] In step S404 of some embodiments, since the greater the video content similarity, the more the video content of the candidate video material segment conforms to the video scene required by the insurance video script. Based on this, the video content similarity of each candidate video material segment is compared with a preset threshold. Then, among the multiple candidate video material segments, the candidate video material segment with a video content similarity greater than the preset threshold is used as the target video material segment.

[0124] The advantage of this embodiment is that when screening candidate video material segments for constructing a template video, the similarity calculation between the video segment content of the candidate video material segments and the video scene description information in the insurance video script is taken into consideration, and the video content similarity can be used to quantify the degree of adaptation between the video segment content of the candidate video material segments and the video scene description information in the insurance video script, thereby improving the accuracy of screening out the target video material segments.

[0125] In step S105 of some embodiments, the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain an edited insurance video.

[0126] The editing instruction data is used to personalize the initial insurance video after it is assembled. The personalization adjustment includes but is not limited to adding specific video material, adjusting the order of video frames, replacing objects in the initial insurance video, etc. Objects refer to people, animals, objects, etc.

[0127] The edited insurance video is used to indicate the insurance video generated after personalized adjustment.

[0128] See also Figure 5 In some embodiments, step S105 may include but is not limited to steps S501 to S504:

[0129] Step S501, if the editing instruction in the editing instruction data is to replace the reference object in the initial insurance video with the target object, then obtaining a target object image of the target object;

[0130] Step S502, extracting object features from the target object image to obtain the target object's facial latent features;

[0131] Step S503, segmenting each video frame of the initial insurance video to obtain a video background image corresponding to each video frame;

[0132] Step S504, generating a video based on the video background image and the hidden features of the object's face to obtain an edited insurance video.

[0133] Steps S501 to S504 are described in detail below.

[0134] In step S501 of some embodiments, the editing instruction is used to instruct to replace the reference object in the initial insurance video with the target object, and to instruct the specific image path for obtaining the object image of the target object and the image name of the object image.

[0135] When this embodiment is specifically implemented, first, the editing instruction data is parsed to obtain the editing instruction pre-input by the target object. Then, if the editing instruction in the editing instruction data is to replace the reference object in the initial insurance video with the target object, it indicates that the reference object of each video frame of the initial insurance video needs to be replaced with the target object to achieve object replacement. Further, the specific image path and image name are parsed in the editing instruction. Finally, according to the specific image path and image name, an image search is performed in the preset image database, and the object image indicated by the specific image path and whose image name is consistent with the image name indicated in the editing instruction is called as the target object image.

[0136] In step S502 of some embodiments, when extracting object features from the target object image, the target object image is first convolved to capture object feature information in the target object image and obtain object image features. Next, attention calculation is performed on the object image features to obtain feature information related to the facial features of the target object in the object image features and obtain the object facial latent features of the target object, wherein the specific process of performing attention calculation on the object image features is similar to the above step S303. To save space, it will not be described in detail.

[0137] In step S503 of some embodiments, when segmenting each video frame of the initial insurance video, first, for each video frame of the initial insurance video, the video frame image corresponding to the video frame is gray-processed to obtain a gray-scale image corresponding to the video frame. Next, for the gray-scale image, the gray-scale value of each pixel is compared with a preset gray-scale threshold, and the pixel whose gray-scale value is greater than or equal to the preset gray-scale threshold is taken as the foreground pixel, and the pixel whose gray-scale value is less than the preset gray-scale threshold is taken as the background pixel. Finally, for each video frame, the foreground pixel is filtered out, and the background pixel is retained to obtain the video background image corresponding to each video frame.

[0138] In step S504 of some embodiments, for each video frame, the object's facial hidden features are embedded as foreground features into the video background image of the video frame to embed the target object into the video frame, obtain the replaced video frame images corresponding to each video frame, and splice all the replaced video frame images into an edited insurance video.

[0139] The benefit of this embodiment is that, according to the editing instruction of replacing the reference object in the initial insurance video with the target object, the target object's object facial latent features are extracted from the target object image of the target object, and the foreground image of each video frame is replaced with the target object's object facial latent features, thereby realizing the object replacement operation of each video frame, so that the target object gives an insurance explanation in the generated edited insurance video, thereby improving the flexibility of insurance video generation and meeting the personalized needs of insurance agents when making insurance videos.

[0140] See also Figure 6 In some embodiments, after step S105, the video generation method further includes but is not limited to steps S601 to S604:

[0141] Step S601, receiving video adjustment data input by a target object in an interactive interface;

[0142] Step S602, parsing the video adjustment data to obtain the editing adjustment sequence, additional animation material, and rendering information of the additional animation material;

[0143] Step S603, rearranging the multiple video frames of the edited insurance video based on the editing adjustment order to obtain a rearranged insurance video;

[0144] Step S604: adding the additional animation material to the target video frame of the rearranged insurance video based on the rendering information.

[0145] Steps S601 to S604 are described in detail below.

[0146] In step S601 of some embodiments, after the target object views the video effect of the edited insurance video on the interactive interface, if the target object wants to make video adjustments to the edited insurance video, the target object can input adjustment parameters for the edited insurance video on the interactive interface. Based on this, the server will receive the video adjustment data input by the target object on the interactive interface.

[0147] In step S602 of some embodiments, since the video adjustment data includes multiple types of adjustment parameters. Based on this, the video adjustment data is first parsed, and the editing adjustment order, additional animation material, and rendering information of the additional animation material are parsed from the video adjustment data. Among them, the editing adjustment order is used to indicate the sequence of each video frame of the target insurance video to be generated in the end. The additional animation material refers to the animation effects to be added to the video frame of the edited insurance video. The rendering information of the additional animation material is used to indicate the video frame identifier of the video frame to which the additional animation material is specifically to be added, and the specific rendering position on the video frame.

[0148] In step S603 of some embodiments, multiple video frames of the edited insurance video are rearranged based on the editing adjustment order so that the order of the rearranged video frames is consistent with the editing adjustment order, thereby obtaining a rearranged insurance video.

[0149] In step S604 of some embodiments, first, based on the rendering information, additional animation material is added to the video frame specified by the video frame identifier in the rendering information, and the additional animation material is embedded in the specified rendering position on the video frame to achieve content update of each video frame of the rearranged insurance video.

[0150] The advantage of this embodiment is that when generating insurance videos, visualization and real-time interaction are adopted. The target object can obtain the generation progress and generation effect of the insurance video on the interactive interface, and adjust the insurance video at any time. The editing order of the insurance video can be freely modified, and various animation effects can be added to the insurance video. This method can improve the generation flexibility of insurance videos and the richness of the content of insurance videos, and can meet the personalized needs of insurance agents when making insurance videos.

[0151] In step S106 of some embodiments, subtitles are generated for the edited insurance video to obtain a target insurance video.

[0152] The target insurance video refers to video data with subtitles that complies with the insurance video generation request and is generated by adjusting according to the video adjustment data.

[0153] In the specific implementation of this embodiment, first, for each video frame of the edited insurance video, the voice data corresponding to the video frame is obtained. Then, the text description corresponding to the voice data is recognized using voice recognition technology, and the specific process is similar to the above step S101. Furthermore, according to the preset font type, preset font size and preset font color, the text description corresponding to each video frame is added to a fixed position of the video frame for display, and a video frame with subtitles is obtained. Finally, the video frames with subtitles are combined into the target insurance video.

[0154] See also Figure 7 In some embodiments, after step S106, the video generation method may include but is not limited to steps S701 to S702:

[0155] Step S701, in response to a video export request initiated by a target object on an interactive interface, obtaining a video export format and a video export resolution;

[0156] Step S702: export the target insurance video to a local database based on the video export format and the video export resolution.

[0157] Steps S701 to S702 are described in detail below.

[0158] In step S701 of some embodiments, when the target object wants to export the target insurance video to a predetermined location, the target object can trigger a video export control on the interactive interface and set the video format and video resolution to be followed when the target insurance video is exported. Based on this, the server can obtain the video export format and video export resolution in response to the video export request initiated by the target object on the interactive interface.

[0159] In step S702 of some embodiments, the target insurance video is exported to a local device or uploaded to a cloud storage platform in a video export format and video export resolution; or the target insurance video is shared to a social media platform or a video sharing platform in a video export format and video export resolution.

[0160] The benefit of this embodiment is that a channel for exporting target insurance videos is provided for the target object on the interactive interface, so that the target object can easily export the target insurance video to a local device or upload it to a cloud storage platform in the required video format and video resolution, thereby enabling convenient storage, editing and sharing of the target insurance video, and improving the target object's flexibility in handling the target insurance video.

[0161] See also Figure 8 The embodiment of the present application further provides a video generating device, which can implement the above-mentioned video generating method, and the device includes:

[0162] The acquisition unit 801 is used to acquire insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface;

[0163] An extraction unit 802 is used to extract key information from the insurance video description data to obtain key sentences of the insurance video description;

[0164] The first generating unit 803 is used to generate a video for the key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data;

[0165] A screening unit 804 is used to screen out target video material segments from a plurality of candidate video material segments based on the insurance video script, and combine the target video material segments into an initial insurance video;

[0166] The editing unit 805 is used to edit the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video;

[0167] The second generating unit 806 is used to generate subtitles for the edited insurance video to obtain a target insurance video.

[0168] In some embodiments, the acquisition unit 801 is used to:

[0169] In response to the insurance video generation request, obtaining video description voice data input by the target object on the interactive interface, wherein the video description voice data is used to describe the video content of the insurance video to be generated;

[0170] Extracting features of the video description speech data to obtain speech Mel-spectrogram features of the video description speech data;

[0171] Speech recognition is performed based on the speech Mel-spectrogram features to obtain the insurance video description data corresponding to the video description speech data.

[0172] In some embodiments, the first generating unit 803 is used to:

[0173] Based on the video generation model, the key sentences of the insurance video description are embedded to obtain the video description embedding features;

[0174] Performing feature encoding on the video description embedding feature to obtain the video description encoding feature;

[0175] Perform self-attention calculation on the video description encoding features to obtain the video frame feature representation vector and the editing instruction feature vector;

[0176] Feature decoding is performed on the video frame feature representation vector and the editing instruction feature vector to obtain the insurance video script.

[0177] In some embodiments, the screening unit 804 is used to:

[0178] Extracting video scene description information from insurance video scripts;

[0179] Extracting content from the candidate video material segments to obtain video segment content of the candidate video material segments;

[0180] For each candidate video material segment, the video scene description information and the video segment content are calculated to obtain the video content similarity;

[0181] Among the multiple candidate video material segments, the candidate video material segments with video content similarity greater than a preset threshold are screened out as target video material segments.

[0182] In some embodiments, the editing unit 805 is used to:

[0183] If the editing instruction in the editing instruction data is to replace the reference object in the initial insurance video with the target object, then obtaining a target object image of the target object;

[0184] Extracting object features from the target object image to obtain the target object's facial latent features;

[0185] Segment each video frame of the initial insurance video to obtain a video background image corresponding to each video frame;

[0186] Video generation is performed based on the video background image and the hidden facial features of the object to obtain an edited insurance video.

[0187] In some embodiments, the video generating device further includes an adjustment unit (not shown), the adjustment unit (not shown) being configured to:

[0188] Receiving video adjustment data input by the target object in the interactive interface;

[0189] Parsing the video adjustment data to obtain the editing adjustment order, the additional animation material, and the rendering information of the additional animation material;

[0190] Rearranging multiple video frames of the edited insurance video based on the editing adjustment order to obtain a rearranged insurance video;

[0191] Additional animation material is added to a target video frame of the rearranged insurance video based on the rendering information.

[0192] In some embodiments, the video generating device further includes a storage unit (not shown), the storage unit (not shown):

[0193] In response to a video export request initiated by a target object on an interactive interface, obtaining a video export format and a video export resolution;

[0194] Export the target insurance video to the local database based on the video export format and video export resolution.

[0195] The specific implementation of the video generating device is basically the same as the specific implementation of the above-mentioned video generating method, and will not be repeated here.

[0196] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned video generation method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0197] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0198] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0199] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the video generation method of the embodiment of this application;

[0200] Input / output interface 903, used to implement information input and output;

[0201] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0202] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0203] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0204] An embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the above-mentioned video generation method.

[0205] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0206] The video generation method, video generation device, electronic device and computer-readable storage medium provided in the embodiment of the present application obtain insurance video description data by responding to the insurance video generation request initiated by the target object on the interactive interface, and extracting key information from the insurance video description data to obtain insurance video description key sentences, which can simplify the effective content of the insurance video description data and eliminate the influence of irrelevant content on video generation. Further, based on the preset video generation model, the insurance video description key sentences are generated to obtain the insurance video script corresponding to the insurance video description data, and based on the insurance video script, the target video material segments are screened out from multiple candidate video material segments, and the target video material segments are combined into an initial insurance video, which can automatically generate the initial insurance video using the video generation model and the collected candidate video material segments, thereby improving the generation efficiency of the initial insurance video. Further, the initial insurance video is used as a template video, and the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain the edited insurance video, which can realize flexible adjustment of the template video so that the edited insurance video meets personalized needs. Finally, subtitles are generated for the edited insurance video to obtain the target insurance video, which can improve the generation efficiency of the insurance video and reduce the complexity and cost of producing personalized target insurance videos.

[0207] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0208] It can be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0209] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0211] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0212] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0213] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0214] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0216] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0217] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A video generation method, characterized in that: The video generation method comprises: In response to an insurance video generation request initiated by a target object on an interactive interface, obtaining insurance video description data; Extracting key information from the insurance video description data to obtain key sentences of the insurance video description; Generate a video of the key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data; Based on the insurance video script, select target video material segments from a plurality of candidate video material segments, and combine the target video material segments into an initial insurance video; Performing editing processing on the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video; Subtitles are generated for the edited insurance video to obtain a target insurance video.

2. The method according to claim 1, characterized in that The step of generating a video of the key sentences of the insurance video description based on a preset video generation model to obtain an insurance video script corresponding to the insurance video description data includes: Based on the video generation model, embedding processing is performed on the key sentences of the insurance video description to obtain video description embedding features; Performing feature encoding on the video description embedding feature to obtain a video description encoding feature; Performing self-attention calculation on the video description encoding features to obtain a video frame feature representation vector and a clip instruction feature vector; Feature decoding is performed on the video frame feature representation vector and the editing instruction feature vector to obtain the insurance video script.

3. The method according to claim 1, characterized in that The step of selecting a target video material segment from a plurality of candidate video material segments based on the insurance video script includes: Extracting video scene description information from the insurance video script; Extracting content from the candidate video material segment to obtain video segment content of the candidate video material segment; For each of the candidate video material segments, performing content similarity calculation on the video scene description information and the video segment content to obtain video content similarity; Among the plurality of candidate video material segments, the candidate video material segments whose video content similarity is greater than a preset threshold are screened out as the target video material segments.

4. The method according to claim 1, characterized in that The step of editing the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video includes: If the editing instruction in the editing instruction data is to replace the reference object in the initial insurance video with the target object, acquiring a target object image of the target object; Extracting object features from the target object image to obtain hidden facial features of the target object; Segmenting each video frame of the initial insurance video to obtain a video background image corresponding to each video frame; The video is generated based on the video background image and the hidden features of the object's face to obtain the edited insurance video.

5. The method according to claim 1, characterized in that After the initial insurance video is edited based on the editing instruction data in the insurance video script to obtain the edited insurance video, the method further includes: Receiving video adjustment data input by the target object on the interactive interface; Parsing the video adjustment data to obtain a clip adjustment sequence, additional animation materials, and rendering information of the additional animation materials; Rearranging the multiple video frames of the edited insurance video based on the editing adjustment order to obtain a rearranged insurance video; The additional animation material is added to a target video frame of the rearranged insurance video based on the rendering information.

6. The method according to claim 1, characterized in that After generating subtitles for the edited insurance video to obtain a target insurance video, the method further includes: In response to a video export request initiated by a target object on an interactive interface, obtaining a video export format and a video export resolution; Based on the video export format and the video export resolution, the target insurance video is exported to a local database.

7. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface includes: In response to the insurance video generation request, obtaining video description voice data input by the target object on the interactive interface, wherein the video description voice data is used to describe the video content of the insurance video to be generated; Extracting features of the video description voice data to obtain voice Mel-spectrogram features of the video description voice data; Speech recognition is performed based on the speech mel-spectrogram feature to obtain insurance video description data corresponding to the video description speech data.

8. A video generating device, characterized in that: The video generating device comprises: An acquisition unit, configured to acquire insurance video description data in response to an insurance video generation request initiated by a target object on an interactive interface; An extraction unit, used to extract key information from the insurance video description data to obtain key sentences of the insurance video description; A first generating unit, configured to generate a video for the key sentences of the insurance video description based on a preset video generation model, and obtain an insurance video script corresponding to the insurance video description data; A screening unit, configured to screen out target video material segments from a plurality of candidate video material segments based on the insurance video script, and combine the target video material segments into an initial insurance video; An editing unit, configured to perform editing processing on the initial insurance video based on the editing instruction data in the insurance video script to obtain an edited insurance video; The second generating unit is used to generate subtitles for the edited insurance video to obtain a target insurance video.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the video generation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video generating method according to any one of claims 1 to 7 is implemented.

Images