Intelligent explanation method and device for short video copywriting

Abstract

The embodiment of the invention provides a short video copywriting intelligent explanation method and device, and the method comprises the steps: obtaining a product oral broadcast copywriting and a product video according to a product selling point and a preset product video generation model, and enabling a sentence in the product oral broadcast copywriting to correspond to the product selling point, performing slicing operation on the product video according to each sentence in the product oral playing copywriting to obtain a corresponding video slice; according to the text length of the sentence corresponding to each video slice and a user group portrait sent by a developer, adjusting the playing speed of each video slice by setting a user portrait personalized model, and adding the product oral playing copywriting as a video subtitle into the product video to obtain an adjusted product video; the quality and the propagation effect of the video content can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to a method and device for intelligent explanation of short video copy. Background Art

[0002] With the rapid development of mobile Internet and short videos, short videos have become one of the most popular forms of information dissemination and marketing. Compared with traditional static text or image advertisements, short videos can more vividly display product features, attract consumers' attention, and enhance product experience and memory. Therefore, more and more companies are beginning to use short videos in product marketing and promotion.

[0003] However, it is not easy to produce high-quality product marketing short videos. First, it is necessary to generate appealing video content and copy based on the characteristics and selling points of the product. This requires marketers to have certain creativity and professional skills, and invest a lot of manpower and time costs. Secondly, for different target user groups, it is also necessary to personalize and optimize the video content to improve the dissemination effect of the video among various audiences. This personalized adjustment usually relies on in-depth analysis of user portraits and requires a lot of data support. Summary of the invention

[0004] In response to the problems in the prior art, the present application provides a method and device for intelligent explanation of short video copy, which can effectively improve the quality and dissemination effect of video content.

[0005] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0006] In a first aspect, the present application provides a short video copy intelligent explanation method, comprising:

[0007] Receive product multi-source heterogeneous data sent by developers, perform data preprocessing operations on the product multi-source heterogeneous data, including at least one of text segmentation, image enhancement, and video frame segmentation, and extract the underlying product feature representations in the product multi-source heterogeneous data, unify the extracted underlying product feature representations into a multimodal feature set and input them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, perform feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, perform semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determine the corresponding product selling points;

[0008] According to the product selling point and the preset product video generation model, a product oral copy and a product video are obtained, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices;

[0009] The playback speed of each video slice is adjusted according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and the product oral copy is added to the product video as video subtitles to obtain an adjusted product video.

[0010] Furthermore, the extracting of the underlying product feature representations from the multi-source heterogeneous data of the product, unifying the extracted underlying product feature representations into a multimodal feature set and inputting the extracted features into a set deep learning multi-layer perceptron to obtain the corresponding high-order product feature representations, includes:

[0011] The text data in the multi-source heterogeneous data of the product is converted into a text vector representation by a word embedding algorithm, the visual features of the image data in the multi-source heterogeneous data of the product are extracted by a preset convolutional neural network, the spatiotemporal features of the video data in the multi-source heterogeneous data of the product are extracted by a frame division operation, and the underlying product feature representation is obtained according to the text vector representation and / or the visual features and / or the spatiotemporal features;

[0012] The underlying product feature representations are unified into a multimodal feature set through weighted fusion and input into a set deep learning multi-layer perceptron for complex interactive relationship learning to obtain the corresponding high-order product feature representation.

[0013] Furthermore, the high-order product feature representations are weighted and fused through the attention mechanism to obtain product multimodal features, including:

[0014] Calculate the correlation between each of the high-order product feature representations and the feature set consisting of all high-order product feature representations through a self-attention mechanism to obtain an attention weight for each of the high-order product feature representations;

[0015] The high-order product feature representations are weightedly fused using the attention weights to obtain product multimodal features.

[0016] Furthermore, the semantic recognition of the multimodal features of the product by a preset knowledge graph reasoning algorithm to determine the corresponding product selling points includes:

[0017] Performing semantic matching and association reasoning on the multimodal features of the product through a preset knowledge graph reasoning algorithm to obtain a concept entity in the preset knowledge graph that is most relevant to the multimodal features of the product, wherein the concept entity includes at least one of product function, product performance, and product value;

[0018] The concept entity is determined as a corresponding product selling point.

[0019] Furthermore, obtaining the product oral copy and product video according to the product selling point and the preset product video generation model includes:

[0020] Inputting historical product selling points as a model training set into a preset autoregressive generation model for model training, thereby obtaining a product video generation model after the model training and a product oral copy output by the product video generation model;

[0021] The historical product selling points and the corresponding historical product oral copy are input into a preset generative adversarial network model as a model training set for model training, so as to obtain a product video generation model after the model training and a product video output by the product video generation model.

[0022] Furthermore, the slicing operation of the product video according to each sentence in the product oral copy to obtain corresponding video slices includes:

[0023] Determine the corresponding slice label contained in the product video according to each sentence in the product oral copy;

[0024] The product video is sliced ​​according to the slice tags to obtain corresponding video slices.

[0025] Furthermore, adjusting the playback speed of each video slice by setting a user portrait personalized model according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer includes:

[0026] Constructing a corresponding playback speed adjustment function according to the text length of the sentence corresponding to each of the video slices, and updating the playback speed adjustment function according to the user characteristics in the user group portrait sent by the developer;

[0027] The playback speed of each of the video slices is adjusted according to the playback speed adjustment function.

[0028] In a second aspect, the present application provides a short video copy intelligent explanation device, comprising:

[0029] A product selling point determination module is used to receive multi-source heterogeneous product data sent by developers, perform data preprocessing operations on the multi-source heterogeneous product data, including at least one of text segmentation, image enhancement, and video frame segmentation, and extract the underlying product feature representations in the multi-source heterogeneous product data, unify the extracted underlying product feature representations into a multimodal feature set and input them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, perform feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, perform semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determine the corresponding product selling points;

[0030] A copy determination module, used to obtain a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices;

[0031] The video playback module is used to adjust the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and add the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0032] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the short video copy intelligent explanation method when executing the program.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the short video copy intelligent explanation method.

[0034] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the short video copy intelligent explanation method.

[0035] It can be seen from the above technical solution that the present application provides a method and device for intelligent explanation of short video copywriting, which obtains product oral copywriting and product video according to product selling points and a preset product video generation model, wherein the sentences in the product oral copywriting correspond to the product selling points, and the product video is sliced ​​according to each sentence in the product oral copywriting to obtain corresponding video slices; the playback speed of each video slice is adjusted by setting a user portrait personalized model according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer, and the product oral copywriting is added to the product video as video subtitles to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 This is one of the flow charts of the short video copy intelligent explanation method in the embodiment of the present application;

[0038] Figure 2 This is the second flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0039] Figure 3 This is the third flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0040] Figure 4 This is a fourth flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0041] Figure 5 This is the fifth flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0042] Figure 6 This is the sixth flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0043] Figure 7 This is the seventh flow chart of the short video copy intelligent explanation method in the embodiment of the present application;

[0044] Figure 8 This is a structural diagram of a short video text intelligent explanation device in an embodiment of the present application;

[0045] Fig. 9It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0046] Reference numerals:

[0047] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0050] In view of the problems existing in the prior art, the present application provides a method and device for intelligent explanation of short video copywriting, which obtains product oral copywriting and product video according to product selling points and a preset product video generation model, wherein the sentences in the product oral copywriting correspond to the product selling points, and the product video is sliced ​​according to each sentence in the product oral copywriting to obtain corresponding video slices; the playback speed of each video slice is adjusted by setting a user portrait personalized model according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer, and the product oral copywriting is added to the product video as video subtitles to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0051] In order to effectively improve the quality and dissemination effect of video content, this application provides an embodiment of a short video copywriting intelligent explanation method, see Figure 1 The short video copywriting intelligent explanation method specifically includes the following contents:

[0052] Step S101: receiving product multi-source heterogeneous data sent by a developer, performing data preprocessing operations including at least one of text segmentation, image enhancement and video frame segmentation on the product multi-source heterogeneous data, extracting underlying product feature representations from the product multi-source heterogeneous data, unifying the extracted underlying product feature representations into a multimodal feature set and inputting them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, performing feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, performing semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determining corresponding product selling points;

[0053] Optionally, in this embodiment, in step S101, the system first receives multi-source heterogeneous product data sent by the developer. These data may include text descriptions, product pictures, and demonstration videos. In order to effectively process different types of data, the system performs data preprocessing operations on them. Text data is processed by word segmentation to extract keywords and semantic information. Image data is improved by image enhancement technology, such as adjusting brightness, contrast, and noise removal to improve image quality and feature extraction effects. For video data, the system performs frame segmentation operations to decompose the video into a series of static frames and extract useful information from them.

[0054] Next, the system extracts the underlying product feature representations from these preprocessed data. For example, keywords in text data, visual features in images, and dynamic scenes in videos are all extracted. These feature representations are standardized and unified into a multimodal feature set. The construction of a multimodal feature set enables the system to integrate and analyze data from different sources in subsequent processing.

[0055] The system inputs the multimodal feature set into a set of deep learning multi-layer perceptrons (MLP). The multi-layer perceptron is a neural network model that can handle complex nonlinear relationships. Through the multi-layer perceptron, the system generates high-order product feature representations that are more abstract and semantically valuable, and can better reflect the potential characteristics and advantages of the product.

[0056] Then, the system uses the attention mechanism to perform feature weighted fusion on the high-order product feature representation. The attention mechanism highlights the most important features for product identification by assigning different weights. This process can improve the accuracy of feature fusion and ensure that the final generated product multimodal features are the most comprehensive and accurate description of the product.

[0057] Finally, the system uses a preset knowledge graph reasoning algorithm to semantically identify the multimodal features of the product. The knowledge graph provides a structured semantic network that helps the system map features to higher-level concepts, such as the core selling points of the product. Through the reasoning algorithm, the system is able to identify the unique advantages and market highlights of the product.

[0058] The technical problem solved by this solution is mainly how to efficiently process and integrate multi-source heterogeneous data. Traditional methods often find it difficult to effectively combine text, image and video data, but through multimodal learning and knowledge graph reasoning, the system can comprehensively analyze product features and improve recognition accuracy.

[0059] In terms of technical effects, through the above steps, the system has significantly improved the depth and breadth of product feature recognition. Preprocessing and multimodal feature fusion ensure efficient data integration, while knowledge graph reasoning enhances the intelligence of product selling point recognition. This not only helps product development and optimization, but also provides strong support for marketing.

[0060] For example, on an e-commerce platform, the system receives multi-source data about a new mobile phone, including technical specifications (text), product images, and usage demonstration videos. Through the above steps, the system extracts and integrates the multimodal features of the mobile phone, and finally identifies the selling points of the mobile phone, such as "high-definition camera" and "long-lasting battery." This information can be used for product recommendations and advertising, improving users' willingness to buy and satisfaction.

[0061] In summary, step S101 realizes intelligent identification and presentation of product selling points through preprocessing, feature fusion and semantic recognition of multi-source data. This solution not only improves the automation level of data processing, but also provides important support for the market competitiveness of products, and has wide practical application value.

[0062] Step S102: obtaining a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and slicing the product video according to each sentence in the product oral copy to obtain corresponding video slices;

[0063] Optionally, in this embodiment, the first step is to generate product oral copy based on a generative model. This technology aims to automatically generate corresponding oral copy content based on the product selling point features identified in the above steps. To this end, we can design a copy generation module based on a generative language model. This module can use a Transformer-based autoregressive generative network architecture, such as GPT. Use the product selling point features as input conditions to guide the language model to generate infectious oral copy.

[0064] Specifically, we can build a Transformer-based generative language model. The input of this model includes:

[0065] 1) Product selling points determined in the previous steps, such as functional advantages, innovations, etc.;

[0066] 2) Optional product background information, such as industry knowledge, competitive product analysis, etc.

[0067] These input conditions will provide guidance for the copywriting generation of the language model.

[0068] During the model training phase, we can use a large number of high-quality product marketing copywriting as supervision signals to allow the model to learn and master the language style and expression skills of product descriptions. During the training process, we can also introduce knowledge graph enhancement to integrate background knowledge of the product field into the language model to further enhance the professionalism and pertinence of the generated copywriting.

[0069] After the training is completed, in actual application, we only need to input the identified product selling point features into the trained generative language model to automatically generate contagious product oral copywriting. These copywriting can not only fully reflect the core highlights of the product, but also follow the writing style of high-quality marketing copywriting, providing high-quality copywriting materials for subsequent video content generation.

[0070] Another core technology is the product video slicing operation based on text-video alignment. After obtaining the product oral copy, we need to accurately align it with the product promotion video to achieve the structured expression of the video content. This requires the design of a text-video alignment module to accurately match each sentence in the oral copy with the corresponding segment in the video.

[0071] Specifically, we can learn from the attention mechanism to achieve text-video association learning. First, we will obtain the product oral copy and the corresponding product promotion video. Then, we design an attention-based multimodal association network, with inputs including text sequences and video frame sequences. By adaptively assigning relevance weights to each element of text and video through the attention mechanism, the network can learn a fine text-video alignment relationship.

[0072] Finally, we cut and extract each sentence in the spoken copy with the corresponding clip in the video to obtain a series of closely corresponding video slices. Each video slice has a corresponding copy content label, which lays the foundation for subsequent personalized video adjustment. This video slicing method based on text-video alignment can achieve structured expression of product promotion video content and provide important support for intelligent generation and optimization of videos.

[0073] In general, the two core technologies involved in step S102, namely the product oral copy generation based on the generative model and the product video slicing operation based on text-video alignment, together constitute the key innovations of this invention patent. The former can automatically generate infectious product marketing copy, while the latter realizes the structured expression of video content. The organic combination of these two technologies has laid a solid technical foundation for the intelligent creation of product marketing short videos.

[0074] Step S103: adjusting the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and adding the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0075] Optionally, in this embodiment, the first step is to adjust the playback speed of the video slice based on the length of the text. In the above steps, we have obtained a series of video slices that closely correspond to the product oral copy. In order to further optimize the viewing experience of the video content, we can automatically adjust the playback speed of each video slice according to the length of the copy sentence corresponding to it.

[0076] Specifically, we can design a text length analysis module to calculate the number of words in each spoken copy sentence. Then, we build a playback speed adjustment strategy to determine the optimal playback speed by analyzing the text length. For sentences with longer text, we can appropriately reduce the playback speed of its video slice to ensure that users can read the copy content completely; for sentences with shorter text, we can appropriately increase the playback speed to avoid excessive blank time when watching.

[0077] This automatic playback speed adjustment based on text length can effectively alleviate the problem of video content being too fast or too slow, maximizing the user's viewing experience. At the same time, it also lays the foundation for subsequent personalized optimization and provides targeted video presentation for different user groups.

[0078] On the other hand, we also need to combine the user group portraits provided by developers to further optimize the personalized performance of video content. User portraits can describe the target audience's age, gender, interests and hobbies, and provide a basis for personalized adjustment of video content.

[0079] Specifically, we can design a personalized adjustment module based on user portraits. First, we convert the user group data provided by developers into structured user feature vectors, including multiple dimensions such as age, gender, and interests. Then, we build a user portrait-video feature association model and preset the optimal video playback parameters for different user groups.

[0080] In actual applications, we associate and match the video slice playback speed calculated above with the user portrait features. Based on the matching results, we dynamically adjust the playback speed of each video slice. For example, for young user groups, we can appropriately increase the video playback speed to cater to their short attention span; while for elderly users, we can reduce the playback speed to ensure that they can better understand and appreciate the video content.

[0081] Finally, we reassembled the adjusted video slices and added the product voice-over copy as video subtitles to obtain the final product video that has been personalized and optimized. This video personalization adjustment method that combines text length and user portrait can effectively improve the viewing experience and communication effect of product marketing videos among different audience groups.

[0082] In general, the two core technologies involved in step S103 are video slice playback speed adjustment based on text length and personalized video optimization combined with user portraits. The former can automatically optimize the presentation rhythm of video content, while the latter can further enhance the appeal of videos among various user groups based on the characteristics of the target audience. The integration of these two technologies lays an important foundation for the personalized optimization of product marketing short videos, which can significantly improve the conversion effect of video content.

[0083] From the above description, it can be seen that the short video copy intelligent explanation method provided in the embodiment of the present application can obtain product oral copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices; according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer, the playback speed of each video slice is adjusted by setting the user portrait personalized model, and the product oral copy is added to the product video as a video subtitle to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0084] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 2 , and can also include the following:

[0085] Step S201: converting text data in the product multi-source heterogeneous data into a text vector representation through a word embedding algorithm, extracting visual features of image data in the product multi-source heterogeneous data through a preset convolutional neural network, extracting spatiotemporal features of video data in the product multi-source heterogeneous data through a frame division operation, and obtaining underlying product feature representation according to the text vector representation and / or visual features and / or spatiotemporal features;

[0086] Step S202: unify the underlying product feature representation into a multimodal feature set through weighted fusion and input it into a set deep learning multi-layer perceptron to perform complex interactive relationship learning to obtain the corresponding high-order product feature representation.

[0087] Optionally, in this embodiment, the first step is step S201, that is, extracting the underlying feature representation from the multi-source heterogeneous product data. This step aims to extract the underlying feature representation that can fully describe the product characteristics from multiple data sources such as product text, images, and videos.

[0088] Specifically, for product text data, we can use word embedding technology to convert text into numerical vector representation. Commonly used word embedding algorithms include Word2Vec, GloVe, etc., which can map text words into dense semantic vector space and capture the implicit relationship between words. In this way, we can convert product text data into computable text vector features.

[0089] For product image data, we can use a pre-trained convolutional neural network (CNN) model to extract visual features. Convolutional neural networks are good at discovering low-level visual features in images, such as lines, textures, etc., and combining them layer by layer into higher-level semantic features. By fine-tuning the pre-trained CNN model, we can obtain visual feature representations specifically for product images.

[0090] For product video data, we can convert the video into a series of image frames through frame segmentation, and then extract the spatiotemporal features of each frame. These spatiotemporal features can include motion information, scene changes, etc., which can reflect the presentation of the product in the video.

[0091] Through the above feature extraction methods, we can obtain the underlying feature representations of product text, image, and video data, including text vectors, visual features, and spatiotemporal features. These underlying features lay the foundation for subsequent multimodal fusion.

[0092] Next is step S202, which is to fuse the multimodal features of the product through deep learning. This step aims to automatically learn and fuse the various underlying features extracted above into a high-level multimodal feature representation of the product.

[0093] Specifically, we can design a deep learning fusion network based on a multi-layer perceptron (MLP) or Transformer. First, we weightedly fuse different types of underlying features (text vectors, visual features, spatiotemporal features, etc.) to build a unified multimodal feature set. Then, we input this multimodal feature set into a preset deep learning network to let the network automatically learn the interactive relationship between each feature.

[0094] Through multi-layer feature transformation and combination, the network can extract high-order product feature representations that contain rich semantic information. This multimodal fusion method based on deep learning can effectively mine the complex associations between multi-source product information and generate more representative product feature vectors.

[0095] Compared with traditional manual feature engineering, the deep learning technology used in these two steps has several key advantages:

[0096] Automatic feature extraction. Without relying on the experience and knowledge of domain experts, the network can automatically learn to extract high-level features from raw data.

[0097] Cross-modal association learning. It can model the complex interaction between different types of data sources and extract feature representations containing multi-dimensional semantic information.

[0098] End-to-end optimization. Integrate feature extraction and fusion into a unified deep learning framework to achieve end-to-end automated learning.

[0099] In general, these two steps construct a product multimodal feature representation learning framework based on deep learning. It can automatically extract underlying features from multi-source heterogeneous product data and generate high-order product feature vectors through multimodal fusion. This feature representation is not only rich in multidimensional semantic information of the product, but also provides high-quality input for subsequent semantic analysis and video generation.

[0100] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 3 , and can also include the following:

[0101] Step S301: Calculate the correlation between each of the high-order product feature representations and the feature set consisting of all high-order product feature representations through a self-attention mechanism to obtain the attention weight of each of the high-order product feature representations;

[0102] Step S302: performing feature weighted fusion on each of the high-order product feature representations using the attention weights to obtain product multimodal features.

[0103] Optionally, in this embodiment, first, step S301 is performed, that is, the correlation between each high-order product feature and the entire feature set is calculated through the self-attention mechanism to obtain the attention weight of each feature. This step aims to adaptively highlight those components that are more important to the shaping of product features.

[0104] Specifically, we can use the self-attention mechanism to achieve this goal. The self-attention mechanism is a core component of deep learning models such as Transformer, which can capture the interdependencies between elements in the input sequence. In our scenario, we take the above-generated set of high-order product feature representations as input, and calculate the correlation between each feature vector and the entire feature set through self-attention.

[0105] The calculation process of self-attention is as follows: First, we transform the high-order product feature representation X through three linear transformations to obtain the Query (Q), Key (K), and Value (V) matrices. Then, we calculate the dot product of Query and Key and divide it by the scaling factor to obtain the attention weight matrix. Finally, we multiply the attention weight matrix with the Value matrix to obtain the final self-attention output, that is, the attention weight of each high-order feature.

[0106] This self-attention-based feature importance modeling can adaptively enhance those components that are more important to product feature shaping and improve the expressiveness of key features. Compared with simple weighted average or concatenation, the self-attention mechanism can capture the interaction between features more finely, thereby generating more representative feature representations.

[0107] Next is step S302, which is to perform weighted fusion on the high-order product features through the obtained attention weights to obtain the final product multimodal features.

[0108] Specifically, we perform weighted summation of the attention weights calculated in step S301 and the original high-order product feature vector. Such a weighted fusion process can adaptively enhance those feature components that are more important for shaping product features and generate more representative product multimodal features.

[0109] In order to further enhance the expressive power of multimodal features, we can also introduce some technical means, such as residual connection or gating mechanism.

[0110] Residual connections can effectively alleviate the gradient vanishing problem of deep networks and improve the quality of feature fusion. By adding the original features to the transformed features, residual connections can better preserve the original useful information.

[0111] The gating mechanism can dynamically adjust the contribution of each modal feature in the final fusion vector. We can design a gating-based fusion unit to dynamically adjust the fusion weights of each modal feature according to its importance, thereby obtaining a more optimized multimodal feature representation of the product.

[0112] In general, these two steps realize an adaptive fusion method of product multimodal features based on the attention mechanism. First, the importance weights of each high-order feature are calculated through self-attention; then, these weights are used to perform weighted fusion on the features to generate the final product multimodal features. This feature fusion mechanism can effectively give play to the advantages of each feature source, enhance the expressiveness of key features, and provide high-quality input for subsequent semantic analysis and video generation.

[0113] Compared with simple feature concatenation or average fusion, the attention mechanism can adaptively highlight those components that are more critical to the description of product features, greatly improving the quality and pertinence of feature representation. At the same time, the introduction of technical means such as residual connection and gating mechanism also further enhances the expression ability of product multimodal features. In general, these two steps reflect the innovation and technical advantages of this invention patent in feature fusion.

[0114] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 4 , and can also include the following:

[0115] Step S401: performing semantic matching and association reasoning on the multimodal features of the product through a preset knowledge graph reasoning algorithm to obtain the most relevant concept entity to the multimodal features of the product in the preset knowledge graph, wherein the concept entity includes at least one of product function, product performance and product value;

[0116] Step S402: Determine the concept entity as the corresponding product selling point.

[0117] Optionally, in this embodiment, the first step is step S401, that is, semantic matching and association reasoning are performed on the multimodal features of the product through a preset knowledge graph reasoning algorithm. This step aims to mine the most relevant conceptual entities with the multimodal features of the product from the existing knowledge graph.

[0118] Specifically, we first need to build a preset knowledge graph covering product knowledge, industry background, marketing strategy and other fields. The knowledge graph organizes various related knowledge in the form of entity-relationship and establishes rich semantic connections. The knowledge graph can be automatically constructed through knowledge extraction, ontology learning, etc., or manually modeled by domain experts.

[0119] Then, we inject the product multimodal features generated above as input information into the knowledge graph reasoning module. Through semantic matching and association reasoning, we can retrieve the concept entities most relevant to the product multimodal features from the knowledge graph. These highly relevant concept entities often represent the core selling points of the product, such as function, performance, value, etc.

[0120] The specific algorithms of knowledge graph reasoning can include rule-based reasoning and semantic-based reasoning. Rule-based reasoning mainly relies on preset ontology rules and mines implicit knowledge through logical deduction. Semantic reasoning focuses more on the similarity and correlation between concepts, and uses indicators such as semantic distance to identify semantically related entities.

[0121] This semantic analysis method based on knowledge graph can effectively make up for the limitations of single natural language processing and improve the depth and comprehensiveness of understanding of product connotations. Compared with semantic recognition that only relies on text features, knowledge graph reasoning can introduce rich domain knowledge and more accurately extract the core selling features of products.

[0122] Next is step S402, which is to determine the conceptual entity obtained through knowledge graph reasoning as the corresponding product selling point.

[0123] This step is very simple and direct. We only need to directly identify the concept entities identified in step S401 as the core selling features of the product. These concept entities include the functional advantages, performance parameters, use value and other aspects of the product, which can comprehensively describe the main highlights of the product.

[0124] Identifying these concept entities as product selling points not only provides an important basis for subsequent video generation, but also lays the foundation for subsequent personalized adjustments. Because each video slice will correspond to a specific product selling point feature, it can more accurately meet the needs of different user groups.

[0125] In general, these two steps reflect the innovation of using knowledge graph technology to conduct in-depth semantic analysis and selling point extraction of product multimodal features. First, we built a preset knowledge graph covering product domain knowledge; then, through the knowledge graph reasoning algorithm, we mined the most relevant conceptual entities to the product multimodal features and determined them as the core selling points of the product.

[0126] This knowledge-driven semantic analysis method can make up for the limitations of single natural language processing and enhance the depth of understanding of product connotations. Compared with methods that rely solely on text features, knowledge graph reasoning can introduce rich background knowledge and more accurately extract the key selling features of products. This lays a solid technical foundation for subsequent video generation and personalized optimization.

[0127] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 5 , and can also include the following:

[0128] Step S501: inputting historical product selling points as a model training set into a preset autoregressive generation model for model training, thereby obtaining a product video generation model after the model training and a product oral copy output by the product video generation model;

[0129] Step S502: Input the historical product selling points and the corresponding historical product oral copy as a model training set into a preset generative adversarial network model for model training, and obtain a product video generation model after the model training and a product video output by the product video generation model.

[0130] Optionally, in this embodiment, the first step is step S501, that is, using the autoregressive generation model to generate product oral copy. The goal of this step is to automatically generate the corresponding oral copy content according to the aforementioned determined product selling point features.

[0131] To this end, we can design a Transformer-based autoregressive generative network. This generative language model is good at automatically inferring and generating coherent and fluent text based on context. We use historical product selling points as the training set input of the model to let the model learn the language style and expression skills of product descriptions.

[0132] During the training process, we can further introduce knowledge graph enhancement. That is, we can integrate the background knowledge of the product field, such as functional characteristics, performance parameters, etc., into the training of the language model. This can enhance the professionalism and pertinence of the generated copy, making it closer to the actual situation of the product.

[0133] After the training is completed, we only need to input the aforementioned product selling point features into the trained generative language model to automatically generate contagious product oral copywriting. These copywriting can not only fully reflect the core highlights of the product, but also follow the writing style of high-quality marketing copywriting, providing high-quality text materials for subsequent video generation.

[0134] Next is step S502, which is to use a generative adversarial network (GAN) to generate a product promotion video. The goal of this step is to automatically generate a product promotion video that meets user needs based on the product selling points and oral copy.

[0135] We can design a video generation model based on the GAN architecture. The generator is responsible for generating preliminary product promotion videos based on product selling point features and oral copywriting, while the discriminator provides feedback optimization for the generator by evaluating indicators such as the authenticity and attractiveness of the video.

[0136] Through the adversarial training mechanism of GAN, the generator can continuously optimize the video content and generate product promotion videos that better meet user needs. During the training process, we can also introduce historical high-quality product oral copywriting and corresponding videos as supervision signals to further guide the learning of the generator.

[0137] After the training is completed, we can input the product selling point features and oral copywriting determined above into the trained video generation model to automatically generate appealing product promotion videos. These videos can not only fully display the core highlights of the product, but also follow the visual style and content rhythm preferred by users.

[0138] In general, these two steps reflect the use of generative modeling technology to achieve automatic creation of product marketing content. Specifically include:

[0139] Product oral copy generation based on the autoregressive generative model can automatically generate infectious product description copy.

[0140] Product video generation based on generative adversarial networks can automatically generate product promotion videos that fit user aesthetics and needs through adversarial training mechanisms.

[0141] This generative modeling method has greatly improved the ability to automatically create product marketing content, saving companies a lot of labor costs. At the same time, it can also ensure that the final output of copywriting and video content is more in line with user preferences and marketing demands, improving marketing effectiveness.

[0142] In general, these two steps build an automatic creation framework for product marketing content based on generative modeling, which fully reflects the innovation and practicality of this invention patent in intelligent marketing creation.

[0143] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 6 , and can also include the following:

[0144] Step S601: determining the corresponding slice label contained in the product video according to each sentence in the product oral copy;

[0145] Step S602: Slice the product video according to the slice tags to obtain corresponding video slices.

[0146] Optionally, in this embodiment, the first step is step S601, that is, according to each sentence in the product oral copy, determining the slice label contained in the corresponding product video. This step is intended to establish an accurate mapping relationship between the text content and the video clip.

[0147] Specifically, we can adopt a multimodal association learning method based on the attention mechanism. First, we obtain the aforementioned generated product oral copy and the corresponding product promotion video. Then, we design an attention-based text-video alignment network module.

[0148] The input of this module includes text sequences (product oral copy) and video frame sequences. Through the attention mechanism, the network can adaptively assign relevance weights to various elements of text and video and establish a fine text-video alignment relationship.

[0149] Finally, we can accurately match each sentence in the spoken copy with the corresponding clip in the video and determine the label information of each video slice. This attention-based text-video alignment method can effectively capture the fine correlation between the two modal data and lay the foundation for subsequent video slicing operations.

[0150] Next is step S602, which is to slice the product video according to the slice tags determined above to obtain corresponding video slices.

[0151] This step is very straightforward. We only need to apply each video slice label determined in step S601 to the original product promotion video and accurately cut it into a series of short video slices.

[0152] The benefit of this is that each video slice corresponds to a specific sentence in the product's spoken copy, reflecting the structured expression of the video content. This not only helps with subsequent personalized video adjustments, but also provides an important basis for the intelligent generation and optimization of video content.

[0153] For example, in personalized optimization, we can adjust the playback speed of different video slices according to the characteristics of target users. Or, when generating new product videos, we can organize and edit these video slices in a targeted manner according to the product selling points to meet the marketing needs of different scenarios.

[0154] In general, these two steps reflect the technical innovation of this invention patent in accurately aligning product oral copy with product video. First, we use the attention mechanism to build a multimodal learning model of text-video association and achieve a fine mapping between the two types of data. Then, based on this alignment relationship, we perform structured slicing operations on the original product video.

[0155] This text-video structured expression method not only helps to improve the comprehensibility and editability of video content, but also creates a good technical foundation for subsequent personalized optimization and intelligent generation. Compared with the traditional single video presentation method, it can better meet the needs of different usage scenarios and user groups, and enhance the flexibility and pertinence of product marketing videos.

[0156] In general, these two steps have made important contributions to the structured expression of product marketing videos, further demonstrating the technical innovation and practicality of this invention patent.

[0157] In one embodiment of the short video copy intelligent explanation method of the present application, see Figure 7 , and can also include the following:

[0158] Step S701: constructing a corresponding playback speed adjustment function according to the text length of the sentence corresponding to each of the video slices, and updating the playback speed adjustment function according to the user characteristics in the user group portrait sent by the developer;

[0159] Step S702: adjusting the playback speed of each of the video slices according to the playback speed adjustment function.

[0160] Optionally, in this embodiment, the first step is step S701, that is, constructing a playback speed adjustment function according to the text length of the sentence corresponding to the video slice, and dynamically updating it in combination with the user group portrait. This step is intended to lay the foundation for subsequent personalized video optimization.

[0161] Specifically, we first need to calculate the text length of the spoken copy sentence corresponding to each video slice. This can be achieved through simple character statistics. Then, we design a playback speed adjustment function based on the text length.

[0162] This function can be a simple linear or nonlinear mapping relationship that maps sentence length to an appropriate playback speed. For example, for longer sentences, we can reduce the playback speed of its video slice to ensure that users can read it completely; for shorter sentences, we can increase the playback speed appropriately to avoid excessive blank time.

[0163] In addition to considering the length of text, we also need to introduce the target user group portrait information provided by the developer. The user portrait can describe the audience's age, gender, interests and hobbies. We can dynamically update the aforementioned playback speed adjustment function based on these user characteristics.

[0164] For example, for young groups, we can appropriately increase the video playback speed to cater to their shorter attention span; while for the elderly, we can reduce the playback speed to ensure that they can better understand and appreciate the video content.

[0165] Through this personalized playback speed adjustment strategy based on text length and user portrait, we can maximize the viewing experience of different audience groups.

[0166] Next is step S702, which is to actually adjust the playback speed of each video slice according to the playback speed adjustment function constructed above.

[0167] The implementation of this step is very simple and direct. We only need to apply the playback speed adjustment function determined in step S701 to each video slice to automatically adjust their playback speed.

[0168] Through this personalized video adjustment based on text length and user portrait, we can significantly improve the appeal and conversion rate of product promotion videos among different audience groups.

[0169] First of all, considering the text length factor can ensure that users can fully read and understand the copy content of each video slice, optimizing the viewing experience of the video content.

[0170] Secondly, combined with the dynamic adjustment of user portraits, it can provide the best video playback rhythm according to the characteristics of different audience groups and meet the viewing needs of various users. This is very important for improving the dissemination effect of videos in the target market.

[0171] In general, these two steps reflect the innovation of this invention patent in the personalized optimization of product marketing videos. First, we combined two important factors, text length and user portrait, to build a multi-dimensional video adjustment strategy. Then, we applied this strategy to each video slice to achieve targeted playback speed adjustment.

[0172] This personalized adjustment mechanism can not only maximize the viewing experience of different user groups, but also provide strong support for the conversion effect of product marketing videos in the target market. Compared with the traditional one-size-fits-all video presentation, it embodies a higher level of intelligence and differentiation, injecting new vitality into product marketing.

[0173] In general, these two steps provide an effective solution for the personalized optimization of product marketing videos, with strong technical innovation and practical value.

[0174] In order to effectively improve the quality and dissemination effect of video content, the present application provides an embodiment of a short video copy intelligent explanation device for realizing all or part of the content of the short video copy intelligent explanation method, see Figure 8 The short video copywriting intelligent explanation device specifically includes the following contents:

[0175] The product selling point determination module 10 is used to receive the product multi-source heterogeneous data sent by the developer, perform data preprocessing operations on the product multi-source heterogeneous data, including at least one of text segmentation, image enhancement and video frame segmentation, and extract the underlying product feature representation in the product multi-source heterogeneous data, unify the extracted underlying product feature representation into a multimodal feature set and input it into a set deep learning multi-layer perceptron to obtain the corresponding high-order product feature representation, perform feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, perform semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determine the corresponding product selling point;

[0176] The copy determination module 20 is used to obtain the product oral copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices;

[0177] The video playback module 30 is used to adjust the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and add the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0178] From the above description, it can be seen that the short video copy intelligent explanation device provided in the embodiment of the present application can obtain product oral copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices; according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer, the playback speed of each video slice is adjusted by setting the user portrait personalized model, and the product oral copy is added to the product video as a video subtitle to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0179] From the hardware level, in order to effectively improve the quality and dissemination effect of video content, the present application provides an embodiment of an electronic device for implementing all or part of the content of the short video copy intelligent explanation method, and the electronic device specifically includes the following content:

[0180] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the short video copy intelligent explanation device and the core business system, user terminal and related databases and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In this embodiment, the logic controller can be implemented with reference to the embodiment of the short video copy intelligent explanation method and the embodiment of the short video copy intelligent explanation device in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0181] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0182] In practical applications, part of the short video copy intelligent explanation method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0183] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0184] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0185] In one embodiment, the short video copy intelligent explanation method function can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0186] Step S101: receiving product multi-source heterogeneous data sent by a developer, performing data preprocessing operations including at least one of text segmentation, image enhancement and video frame segmentation on the product multi-source heterogeneous data, extracting underlying product feature representations from the product multi-source heterogeneous data, unifying the extracted underlying product feature representations into a multimodal feature set and inputting them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, performing feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, performing semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determining corresponding product selling points;

[0187] Step S102: obtaining a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and slicing the product video according to each sentence in the product oral copy to obtain corresponding video slices;

[0188] Step S103: adjusting the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and adding the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0189] From the above description, it can be seen that the electronic device provided in the embodiment of the present application obtains product spoken copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product spoken copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product spoken copy to obtain corresponding video slices; the playback speed of each video slice is adjusted according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting the user portrait personalization model, and the product spoken copy is added to the product video as a video subtitle to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0190] In another embodiment, the short video copy intelligent explanation device can be configured separately from the central processing unit 9100. For example, the short video copy intelligent explanation device can be configured as a chip connected to the central processing unit 9100, and the short video copy intelligent explanation method function can be realized through the control of the central processing unit.

[0191] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0192] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0193] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0194] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0195] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0196] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0197] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0198] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0199] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the short video copy intelligent explanation method in the above embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the short video copy intelligent explanation method in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0200] Step S101: receiving product multi-source heterogeneous data sent by a developer, performing data preprocessing operations including at least one of text segmentation, image enhancement and video frame segmentation on the product multi-source heterogeneous data, extracting underlying product feature representations from the product multi-source heterogeneous data, unifying the extracted underlying product feature representations into a multimodal feature set and inputting them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, performing feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, performing semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determining corresponding product selling points;

[0201] Step S102: obtaining a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and slicing the product video according to each sentence in the product oral copy to obtain corresponding video slices;

[0202] Step S103: adjusting the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and adding the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0203] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application obtains product oral copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices; the playback speed of each video slice is adjusted by setting a user portrait personalized model according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer, and the product oral copy is added to the product video as a video subtitle to obtain an adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0204] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the short video copy intelligent explanation method in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the short video copy intelligent explanation method are implemented. For example, the computer program / instruction implements the following steps:

[0205] Step S101: receiving product multi-source heterogeneous data sent by a developer, performing data preprocessing operations including at least one of text segmentation, image enhancement and video frame segmentation on the product multi-source heterogeneous data, extracting underlying product feature representations from the product multi-source heterogeneous data, unifying the extracted underlying product feature representations into a multimodal feature set and inputting them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, performing feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, performing semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determining corresponding product selling points;

[0206] Step S102: obtaining a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and slicing the product video according to each sentence in the product oral copy to obtain corresponding video slices;

[0207] Step S103: adjusting the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and adding the product oral copy as video subtitles to the product video to obtain an adjusted product video.

[0208] From the above description, it can be seen that the computer program product provided in the embodiment of the present application obtains product spoken copy and product video according to the product selling point and the preset product video generation model, wherein the sentences in the product spoken copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product spoken copy to obtain corresponding video slices; the playback speed of each video slice is adjusted according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting the user portrait personalized model, and the product spoken copy is added to the product video as a video subtitle to obtain the adjusted product video, thereby effectively improving the quality of the video content and the dissemination effect.

[0209] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0210] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A short video copy intelligent explanation method, characterized in that: The method comprises: Receive product multi-source heterogeneous data sent by developers, perform data preprocessing operations on the product multi-source heterogeneous data, including at least one of text segmentation, image enhancement, and video frame segmentation, and extract the underlying product feature representations in the product multi-source heterogeneous data, unify the extracted underlying product feature representations into a multimodal feature set and input them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, perform feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, perform semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determine the corresponding product selling points; According to the product selling point and the preset product video generation model, a product oral copy and a product video are obtained, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices; The playback speed of each video slice is adjusted according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and the product oral copy is added to the product video as video subtitles to obtain an adjusted product video.

2. The short video copy intelligent explanation method according to claim 1 is characterized in that: The step of extracting the underlying product feature representation from the multi-source heterogeneous data of the product, unifying the extracted underlying product feature representation into a multimodal feature set and inputting the extracted feature representation into a set deep learning multi-layer perceptron to obtain the corresponding high-order product feature representation includes: The text data in the multi-source heterogeneous data of the product is converted into a text vector representation by a word embedding algorithm, the visual features of the image data in the multi-source heterogeneous data of the product are extracted by a preset convolutional neural network, the spatiotemporal features of the video data in the multi-source heterogeneous data of the product are extracted by a frame division operation, and the underlying product feature representation is obtained according to the text vector representation and / or the visual features and / or the spatiotemporal features; The underlying product feature representations are unified into a multimodal feature set through weighted fusion and input into a set deep learning multi-layer perceptron for complex interactive relationship learning to obtain the corresponding high-order product feature representation.

3. The short video copy intelligent explanation method according to claim 1 is characterized in that: The weighted fusion of the high-level product feature representations is performed through the attention mechanism to obtain product multimodal features, including: Calculate the correlation between each of the high-order product feature representations and the feature set consisting of all high-order product feature representations through a self-attention mechanism to obtain an attention weight for each of the high-order product feature representations; The high-order product feature representations are weightedly fused using the attention weights to obtain product multimodal features.

4. The short video copy intelligent explanation method according to claim 1 is characterized in that: The semantic recognition of the multimodal features of the product by using a preset knowledge graph inference algorithm to determine the corresponding product selling points includes: Performing semantic matching and association reasoning on the multimodal features of the product through a preset knowledge graph reasoning algorithm to obtain a concept entity in the preset knowledge graph that is most relevant to the multimodal features of the product, wherein the concept entity includes at least one of product function, product performance, and product value; The concept entity is determined as a corresponding product selling point.

5. The short video copy intelligent explanation method according to claim 1 is characterized in that: The method of obtaining the product oral copy and product video according to the product selling point and the preset product video generation model includes: Inputting historical product selling points as a model training set into a preset autoregressive generation model for model training, thereby obtaining a product video generation model after the model training and a product oral copy output by the product video generation model; The historical product selling points and the corresponding historical product oral copy are input into a preset generative adversarial network model as a model training set for model training, so as to obtain a product video generation model after the model training and a product video output by the product video generation model.

6. The short video copy intelligent explanation method according to claim 1 is characterized in that: The slicing operation of the product video according to each sentence in the product oral copy to obtain corresponding video slices includes: Determine the corresponding slice label contained in the product video according to each sentence in the product oral copy; The product video is sliced ​​according to the slice tags to obtain corresponding video slices.

7. The short video copy intelligent explanation method according to claim 1 is characterized in that: The adjusting the playback speed of each video slice by setting a user portrait personalized model according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer includes: Constructing a corresponding playback speed adjustment function according to the text length of the sentence corresponding to each of the video slices, and updating the playback speed adjustment function according to the user characteristics in the user group portrait sent by the developer; The playback speed of each of the video slices is adjusted according to the playback speed adjustment function.

8. A short video copy intelligent explanation device, characterized in that: The device comprises: A product selling point determination module is used to receive multi-source heterogeneous product data sent by developers, perform data preprocessing operations on the multi-source heterogeneous product data, including at least one of text segmentation, image enhancement, and video frame segmentation, and extract the underlying product feature representations in the multi-source heterogeneous product data, unify the extracted underlying product feature representations into a multimodal feature set and input them into a set deep learning multi-layer perceptron to obtain corresponding high-order product feature representations, perform feature weighted fusion on each of the high-order product feature representations through an attention mechanism to obtain product multimodal features, perform semantic recognition on the product multimodal features through a preset knowledge graph inference algorithm, and determine the corresponding product selling points; A copy determination module, used to obtain a product oral copy and a product video according to the product selling point and a preset product video generation model, wherein the sentences in the product oral copy correspond to the product selling point, and the product video is sliced ​​according to each sentence in the product oral copy to obtain corresponding video slices; The video playback module is used to adjust the playback speed of each video slice according to the text length of the sentence corresponding to each video slice and the user group portrait sent by the developer by setting a user portrait personalization model, and add the product oral copy as video subtitles to the product video to obtain an adjusted product video.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the short video copy intelligent explanation method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the short video copy intelligent explanation method described in any one of claims 1 to 7 are implemented.

Images