Artificial intelligence-based copywriting data generation method and device, equipment and medium

Abstract

The invention belongs to the field of artificial intelligence and the field of financial science and technology, and relates to an artificial intelligence-based copywriting data generation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed video; extracting an image sequence from the video, and generating a comprehensive image vector corresponding to the image sequence based on an image encoder; extracting a target audio from the video, and generating a comprehensive audio vector corresponding to the target audio based on an audio encoder; extracting a target text from the video, and generating a comprehensive text vector corresponding to the target text based on a text encoder; performing combination processing on the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain a target vector; decoding the target vector based on a decoder to obtain word data; and generating a target copywriting corresponding to the video based on the word data. In addition, the target copywriting may be stored in the blockchain. The copywriting generation efficiency and the copywriting generation accuracy are improved through the encoder and the decoder based on the three modes.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence development technology and financial technology, and in particular to methods, devices, computer equipment and storage media for generating copywriting data based on artificial intelligence. Background Art

[0002] Amid the rapid growth of the insurance industry, video content, as a visual and engaging means of disseminating information, has been widely adopted across all aspects of the insurance business, including but not limited to company background presentations, business process introductions, insurance product explanations, and sales training. These video resources not only enrich information delivery but also enhance the understanding and awareness of the insurance business among both customers and internal staff.

[0003] Currently, when using these videos for knowledge absorption and transformation, it is necessary to generate copy based on the video content. The existing method of generating copy corresponding to videos requires business personnel to watch the entire video one by one, and then manually summarize the key information in the video to write insurance copy that meets the needs, such as marketing language, product promotional copy, etc. However, video content is often lengthy and contains a lot of information. Watching it from beginning to end and manually summarizing it is time-consuming and laborious, resulting in low processing efficiency. Moreover, during the manual summarization process, key information can be easily omitted or misunderstood due to individual differences in understanding or distraction, resulting in low accuracy of the generated copy. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for generating text data based on artificial intelligence, so as to solve the technical problems that the existing method of manually generating text corresponding to videos has low processing efficiency and low accuracy of the generated text.

[0005] In order to solve the above technical problems, the present application provides an artificial intelligence-based copywriting data generation method, which adopts the following technical solutions:

[0006] Get the video to be processed;

[0007] Extracting an image sequence from the video based on a preset image clustering algorithm, and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder;

[0008] Extracting target audio from the video, and generating a synthetic audio vector corresponding to the target audio based on the trained audio encoder;

[0009] Extracting target text from the video, and generating a comprehensive text vector corresponding to the target text based on the trained text encoder;

[0010] Combining the integrated image vector, the integrated audio vector, and the integrated text vector to obtain a corresponding target vector;

[0011] Decoding the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality;

[0012] A target text corresponding to the video is generated based on the word data.

[0013] Furthermore, the step of extracting an image sequence from the video based on a preset image clustering algorithm specifically includes:

[0014] Read the video file in the video based on a preset video processing library;

[0015] Performing frame extraction processing on the video file based on a preset time interval to obtain a corresponding first image;

[0016] Performing image clustering processing on the first image based on a preset image fingerprint algorithm, and obtaining fingerprint similarity of the first image;

[0017] Filtering images whose fingerprint similarity is greater than a preset similarity threshold from the first images to obtain corresponding second images;

[0018] A corresponding image sequence is constructed based on the second image.

[0019] Furthermore, the step of extracting target audio from the video specifically includes:

[0020] Call the preset audio extraction tool;

[0021] Performing audio extraction processing on the video based on the audio extraction tool to obtain a corresponding first audio;

[0022] performing speech separation processing on the first audio to obtain a corresponding second audio;

[0023] The second audio is used as the target audio.

[0024] Furthermore, the step of extracting target text from the video specifically includes:

[0025] Get multiple preset text extraction strategies;

[0026] Filtering a target text extraction strategy that matches the video from all the text extraction strategies;

[0027] Performing text extraction processing on the video based on the text extraction strategy to obtain corresponding text data;

[0028] The text data is used as the target text.

[0029] Furthermore, the step of combining the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector specifically includes:

[0030] Get the preset splicing strategy;

[0031] Performing splicing processing on the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector based on the splicing strategy to obtain a corresponding splicing vector;

[0032] The concatenated vector is used as the target vector.

[0033] Furthermore, the step of generating a target text corresponding to the video based on the word data specifically includes:

[0034] Performing splicing processing on all the word data to obtain corresponding splicing data;

[0035] Get the preset cleaning rules;

[0036] Performing text cleaning on the spliced ​​data based on the cleaning rules to obtain corresponding target data;

[0037] The target data is used as the target text.

[0038] Furthermore, before the step of extracting an image sequence from the video based on a preset image clustering algorithm and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder, the method further includes:

[0039] Get pre-built video sample data;

[0040] Obtaining a preset initial encoder and a feature cross-alignment training strategy corresponding to the initial encoder; wherein the initial encoder is an initial image encoder, an initial audio encoder, or an initial text encoder;

[0041] Determining a target optimization algorithm corresponding to the initial encoder;

[0042] Based on the feature cross-alignment training strategy, the target optimization algorithm, and the preset InfoNCE loss function, the initial encoder is trained using the video sample data to obtain a trained designated encoder;

[0043] The designated encoder is stored.

[0044] In order to solve the above technical problems, the present application also provides an artificial intelligence-based copywriting data generation device, which adopts the following technical solutions:

[0045] A first acquisition module is used to acquire the video to be processed;

[0046] a first processing module, configured to extract an image sequence from the video based on a preset image clustering algorithm, and generate a comprehensive image vector corresponding to the image sequence based on a trained image encoder;

[0047] a second processing module, configured to extract target audio from the video and generate a synthetic audio vector corresponding to the target audio based on a trained audio encoder;

[0048] a third processing module, configured to extract target text from the video and generate a comprehensive text vector corresponding to the target text based on the trained text encoder;

[0049] a combining module, configured to combine the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector;

[0050] A decoding module, configured to decode the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality;

[0051] A generation module is used to generate a target text corresponding to the video based on the word data.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0053] Get the video to be processed;

[0054] Extracting an image sequence from the video based on a preset image clustering algorithm, and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder;

[0055] Extracting target audio from the video, and generating a synthetic audio vector corresponding to the target audio based on the trained audio encoder;

[0056] Extracting target text from the video, and generating a comprehensive text vector corresponding to the target text based on the trained text encoder;

[0057] Combining the integrated image vector, the integrated audio vector, and the integrated text vector to obtain a corresponding target vector;

[0058] Decoding the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality;

[0059] A target text corresponding to the video is generated based on the word data.

[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0061] Get the video to be processed;

[0062] Extracting an image sequence from the video based on a preset image clustering algorithm, and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder;

[0063] Extracting target audio from the video, and generating a synthetic audio vector corresponding to the target audio based on the trained audio encoder;

[0064] Extracting target text from the video, and generating a comprehensive text vector corresponding to the target text based on the trained text encoder;

[0065] Combining the integrated image vector, the integrated audio vector, and the integrated text vector to obtain a corresponding target vector;

[0066] Decoding the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality;

[0067] A target text corresponding to the video is generated based on the word data.

[0068] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0069] The present application first obtains a video to be processed; then extracts an image sequence from the video based on a preset image clustering algorithm, and generates a comprehensive image vector corresponding to the image sequence based on a trained image encoder; and extracts target audio from the video, and generates a comprehensive audio vector corresponding to the target audio based on a trained audio encoder; and extracts target text from the video, and generates a comprehensive text vector corresponding to the target text based on a trained text encoder; then combines the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain a corresponding target vector; subsequently, decodes the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes multiple; finally, generates a target text corresponding to the video based on the word data. This application uses an image encoder, an audio encoder, and a text encoder to perform feature learning and information mapping of three modalities on the video to be processed to obtain a corresponding comprehensive image vector, a comprehensive audio vector, and a comprehensive text vector, and combines the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector, and then uses a trained decoder to decode the target vector to obtain the corresponding word data, and automatically and accurately generate the target text corresponding to the video based on the word data. Different from the existing method of manually generating text corresponding to the video, this application uses encoders and decoders based on three modalities to automatically and accurately generate corresponding text according to the video content, thereby improving the efficiency of text generation and ensuring the accuracy of the generated text. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0072] Figure 2 A flowchart of an embodiment of a method for generating copywriting data based on artificial intelligence according to the present application;

[0073] Figure 3 This is a structural diagram of an embodiment of an artificial intelligence-based copywriting data generation device according to the present application;

[0074] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0076] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0077] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0078] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0079] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0080] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0081] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0082] It should be noted that the artificial intelligence-based copywriting data generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based copywriting data generation device is generally set in the server / terminal device.

[0083] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0084] Continue to refer Figure 2 , shows a flowchart of an embodiment of the method for generating copy data based on artificial intelligence according to the present application. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted. The method for generating copy data based on artificial intelligence provided in the embodiment of the present application can be applied to any scenario that requires copy data, and the method for generating copy data based on artificial intelligence can be applied to products in these scenarios, for example, insurance copy data in the field of finance and insurance. The method for generating copy data based on artificial intelligence includes the following steps:

[0085] Step S201: Obtain the video to be processed.

[0086] In this embodiment, the method for generating text data based on artificial intelligence is executed on the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the video through a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. In the business scenario of insurance product marketing in financial insurance, the above-mentioned video may be a video containing content such as background introduction, business introduction, product introduction, and speech training of the insurance industry.

[0087] Step S202 : extracting an image sequence from the video based on a preset image clustering algorithm, and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder.

[0088] In this embodiment, the image sequence is input into a trained image encoder for feature encoding, and then the image encoder outputs the feature vector (ff) of each key frame contained in the image sequence. These vectors usually contain the depth representation of the image. Subsequently, the feature vectors of all key frames are aggregated to obtain the corresponding comprehensive image vector (fv). Among them, the aggregation process can adopt any one of the processing methods of averaging, weighted averaging, and pooling. In addition, the specific implementation process of extracting the image sequence from the video based on the preset image clustering algorithm will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here. In addition, the training process of the above-mentioned image encoder can refer to the training process of the specified encoder, and will not be described in detail here.

[0089] Step S203: extract target audio from the video, and generate a comprehensive audio vector corresponding to the target audio based on the trained audio encoder.

[0090] In this embodiment, the target audio is input into a trained audio encoder for feature encoding, and then the audio encoder outputs the feature vectors (fas) of each audio segment contained in the target audio. These vectors contain a deep representation of the audio. Subsequently, the feature vectors of all audio segments are aggregated to obtain the corresponding comprehensive audio vector (fa). The aggregation process can adopt any one of the processing methods of averaging, weighted averaging, and pooling. In addition, the specific implementation process of extracting the target audio from the video will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here. In addition, the training process of the above-mentioned audio encoder can refer to the training process of the specified encoder, and will not be described in detail here.

[0091] Step S204: extract target text from the video, and generate a comprehensive text vector corresponding to the target text based on the trained text encoder.

[0092] In this embodiment, feature encoding is performed by inputting the target text into a trained text encoder, and then the text encoder outputs the feature vectors (fw) of each token contained in the target text. These vectors contain the deep semantic information of the text. Subsequently, the feature vectors of all tokens are aggregated to obtain the corresponding comprehensive text vector (fs). Among them, the aggregation processing can adopt any one of the processing methods of averaging, weighted averaging, and pooling. In addition, the specific implementation process of extracting the target text from the video will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated on here. In addition, the training process of the above-mentioned text encoder can refer to the training process of the specified encoder, and will not be described in detail here.

[0093] Step S205 , combining the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector.

[0094] In this embodiment, the above-mentioned specific implementation process of combining the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain the corresponding target vector will be further described in detail in subsequent specific embodiments of this application and will not be elaborated on here.

[0095] Step S206: decoding the target vector based on the trained decoder to obtain corresponding word data; wherein the number of the word data includes multiple.

[0096] In this embodiment, the decoder specifically employs an autoregressive model with a transformer structure. After the target vector is input to the trained decoder, the decoder gradually generates each word in the text based on the target vector. At the first moment, the decoder generates the first word based on the encoder's target vector. At the second moment, the decoder uses the first word's vector as input to generate the second word, and this continues until a stop symbol is generated, thereby generating the corresponding word data. The words generated at all moments can be concatenated to form the complete text content.

[0097] Step S207: Generate a target text corresponding to the video based on the word data.

[0098] In this embodiment, the specific implementation process of generating the target text corresponding to the video based on the word data will be further described in detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0099] The present application first obtains a video to be processed; then extracts an image sequence from the video based on a preset image clustering algorithm, and generates a comprehensive image vector corresponding to the image sequence based on a trained image encoder; and extracts target audio from the video, and generates a comprehensive audio vector corresponding to the target audio based on a trained audio encoder; and extracts target text from the video, and generates a comprehensive text vector corresponding to the target text based on a trained text encoder; then combines the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain a corresponding target vector; subsequently, decodes the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes multiple; finally, generates a target text corresponding to the video based on the word data. This application uses an image encoder, an audio encoder, and a text encoder to perform feature learning and information mapping of three modalities on the video to be processed to obtain a corresponding comprehensive image vector, a comprehensive audio vector, and a comprehensive text vector, and combines the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector, and then uses a trained decoder to decode the target vector to obtain the corresponding word data, and automatically and accurately generate the target text corresponding to the video based on the word data. Different from the existing method of manually generating text corresponding to the video, this application uses encoders and decoders based on three modalities to automatically and accurately generate corresponding text according to the video content, thereby improving the efficiency of text generation and ensuring the accuracy of the generated text.

[0100] In some optional implementations, the step S202 of extracting an image sequence from the video based on a preset image clustering algorithm includes the following steps:

[0101] The video file in the video is read based on a preset video processing library.

[0102] In this embodiment, the video processing library may specifically be OpenCV, Ffmpeg, etc., and the video file in the video may be read by using the video processing library.

[0103] The video file is subjected to frame extraction processing based on a preset time interval to obtain a corresponding first image.

[0104] In this embodiment, the value of the time interval is not specifically limited and can be set according to actual usage requirements. For example, it can be set to extract one frame per second. Frames can be extracted from the video file at this time interval and saved as image files (e.g., JPEG or PNG format) to obtain the first image.

[0105] Perform image clustering processing on the first image based on a preset image fingerprint algorithm, and obtain fingerprint similarity of the first image.

[0106] In this embodiment, because the video contains many repeated and similar images, image clustering is used to extract only key, significant images in the video that may represent the plot. This application uses image fingerprints to perform image clustering on the first image to obtain fingerprint similarity for the first image. Image fingerprints are representations used to uniquely identify images generated based on visual features.

[0107] Images having fingerprint similarities greater than a preset similarity threshold are filtered from the first images to obtain corresponding second images.

[0108] In this embodiment, the value of the similarity threshold is not specifically limited and can be set according to actual business requirements, preferably 0.5. The corresponding second image is obtained by screening out images with fingerprint similarity greater than the similarity threshold from all first images and then deleting the images from the first images.

[0109] A corresponding image sequence is constructed based on the second image.

[0110] In this embodiment, the center point images (cluster centers) after clustering processing can be screened out as key frames, and the key frames can be tiled according to the video playback time to string together the corresponding image sequences. By using the image clustering algorithm to construct the image sequence, fewer static frames and more dynamic scene frames in the video can be retained. Even if the obtained image sequence does not include all frames, the extracted image sequence can also represent the entire video by capturing significant scenes with significant changes. By utilizing this method, the basic image sequence of the video can be created by focusing on scenes with meaningful changes. These processed image sequences provide a new training sample as the input of the image encoder, which highlights the storyline with fewer frames and distinguishes it from the traditional input of all frames of the video.

[0111] The present application reads the video file in the video based on a preset video processing library; then extracts frames from the video file based on a preset time interval to obtain a corresponding first image; then performs image clustering processing on the first image based on a preset image fingerprint algorithm and obtains the fingerprint similarity of the first image; subsequently, images with fingerprint similarity greater than a preset similarity threshold are filtered from the first image to obtain a corresponding second image; and finally, a corresponding image sequence is constructed based on the second image. The present application performs video extraction and image clustering processing on the video based on the use of a video processing library and an image fingerprint algorithm, which can quickly and accurately capture significant scenes with significant changes to represent the image sequence of the entire video, thereby improving the intelligence and accuracy of image sequence generation.

[0112] In some optional implementations of this embodiment, extracting the target audio from the video in step S203 includes the following steps:

[0113] Call the preset audio extraction tool.

[0114] In this embodiment, there is no specific limitation on the selection of the above-mentioned audio extraction tool. For example, the ffmpeg tool can be used.

[0115] Perform audio extraction processing on the video based on the audio extraction tool to obtain a corresponding first audio.

[0116] In this embodiment, the audio extraction process is performed on the video by using the above-mentioned audio extraction tool to ensure that the length of the extracted audio matches the length of the original video, thereby obtaining the corresponding first audio.

[0117] Perform speech separation processing on the first audio to obtain a corresponding second audio.

[0118] In this embodiment, because the subtitles of the video already contain information about the language sound, the separated speech does not require speech information, and only retains the pitch intensity information. The bytesep framework can be used to perform speech separation on the first audio, retain the pitch intensity information, and remove the speech content to obtain the corresponding second audio. Specifically, the bytesep framework is used to decouple the speech of the first audio from the mixed audio containing various sounds. The result of the decoupled sound can be considered as audio data that shares the content plot with the clip video in the original video in the absence of any language information. This embodiment performs speech decoupling because the speech itself includes the content plot, and the model can learn to focus on the content plot of the speech, rather than the desired audio (text content).

[0119] The second audio is used as the target audio.

[0120] This application calls a preset audio extraction tool; then, based on the audio extraction tool, performs audio extraction processing on the video to obtain a corresponding first audio; then, performs speech separation processing on the first audio to obtain a corresponding second audio; and subsequently uses the second audio as the target audio. This application, based on the use of the audio extraction tool, performs audio extraction processing and speech classification processing on the video to quickly and accurately obtain a target audio that does not contain speech information but only contains pitch intensity information, thereby improving the extraction intelligence of the target audio and ensuring the accuracy of the obtained target audio.

[0121] In some optional implementations, extracting the target text from the video in step S204 includes the following steps:

[0122] Get multiple preset text extraction strategies.

[0123] In this embodiment, the above-mentioned text extraction strategy may specifically include a text extraction strategy based on ASR (automatic speech recognition), a text extraction strategy based on OCR (optical character recognition), and a text extraction strategy based on the fusion of CMD and LSMDC. The ASR-based text extraction strategy includes: if the video contains human voice, use ASR services or libraries (such as Google Speech-to-Text, IBM Watson Speech to Text, or open source DeepSpeech, Kaldi, etc.) to recognize the human voice in the video. Specifically, the OCR-based text extraction strategy includes: if the video contains subtitles or text information, use OCR libraries (such as Tesseract, EasyOCR, etc.) to recognize the text in the video frame. Post-process the recognized text, such as removing noise, correcting OCR errors, etc. Save the recognized subtitle text as a string or file. The text extraction strategy based on the fusion of CMD and LSMDC involves generating text information using a CMD and LSMDC fusion method if the video lacks subtitles and vocalization. There are slight differences in clip length between CMD and LSMDC, although both provide a short description for each video segment. Because LSMDC video segments need to be concatenated to match the length of CMD video segments, the descriptions must be formatted by concatenating the same number of descriptions to the video. Consequently, the concatenated descriptions exhibit variation in sentence length. Furthermore, there are inherent differences between the two datasets in how text descriptions are generated. CMD uses user-generated descriptions from YouTube, while LSMDC uses a descriptive video service. Due to the differences in text variation between the two datasets, text fusion is further performed by integrating reading capabilities using chatGPT. The CMD and LSMDC descriptions are labeled [CMD] and [LSMDC], and a chatGPT prompt is constructed: the contents of [CMD] and [LSMDC] are combined into a single description. The chatGPT output then yields a text description for videos without subtitles and vocalization.

[0124] A target text extraction strategy matching the video is selected from all the text extraction strategies.

[0125] In this embodiment, by analyzing whether the video contains human voice or subtitles, a matching target text extraction strategy can be selected from all the text extraction strategies. Specifically, if the video contains human voice, an ASR-based text extraction strategy is adopted as the target text extraction strategy. If the video contains subtitles, an OCR-based text extraction strategy is adopted as the target text extraction strategy. If the video does not contain subtitles or human voice, a CMD and LSMDC fusion-based text extraction strategy is adopted as the target text extraction strategy.

[0126] The video is subjected to text extraction processing based on the text extraction strategy to obtain corresponding text data.

[0127] In this embodiment, text extraction processing may be performed on the video according to the strategy steps in the above text extraction strategy, thereby extracting corresponding text data.

[0128] The text data is used as the target text.

[0129] The present application obtains a plurality of preset text extraction strategies; then selects a target text extraction strategy that matches the video from all the text extraction strategies; then performs text extraction processing on the video based on the text extraction strategy to obtain corresponding text data; and subsequently uses the text data as the target text. The present application selects a target text extraction strategy that matches the video from a plurality of preset text extraction strategies, and then uses the text extraction strategy to perform text extraction processing on the video, thereby achieving rapid and accurate extraction of the corresponding target text from the video, improving the extraction intelligence of the target text and ensuring the accuracy of the obtained target audio.

[0130] In some optional implementations, step S205 includes the following steps:

[0131] Get the preset stitching strategy.

[0132] In this embodiment, the above-mentioned splicing strategy may include concatenation, or may also include more complex fusion methods, such as weighted sum, attention mechanism and other strategies.

[0133] The integrated image vector, the integrated audio vector and the integrated text vector are spliced ​​based on the splicing strategy to obtain a corresponding splicing vector.

[0134] In this embodiment, the integrated image vector, the integrated audio vector, and the integrated text vector may be spliced ​​according to the strategy content of the above-mentioned splicing strategy, so as to obtain a corresponding splicing vector.

[0135] The concatenated vector is used as the target vector.

[0136] The present application obtains a preset splicing strategy; then, based on the splicing strategy, splices the integrated image vector, the integrated audio vector, and the integrated text vector to obtain a corresponding splicing vector; and subsequently uses the splicing vector as the target vector. By using the splicing strategy, the present application can quickly and intelligently combine the integrated image vector, the integrated audio vector, and the integrated text vector to obtain a corresponding target vector, thereby improving the efficiency and intelligence of generating the target vector.

[0137] In some optional implementations of this embodiment, step S207 includes the following steps:

[0138] All the word data are spliced ​​together to obtain corresponding spliced ​​data.

[0139] In this embodiment, the word data refers to the words generated in all time steps. The splicing process refers to splicing all word data into a complete sentence or text to obtain corresponding splicing data.

[0140] Get the preset cleanup rules.

[0141] In this embodiment, the above-mentioned cleaning rules refer to rules for removing duplicate words, correcting grammatical errors, removing special marks, formatting output, and other processing on the text.

[0142] The spliced ​​data is subjected to text cleaning processing based on the cleaning rules to obtain corresponding target data.

[0143] In this embodiment, the spliced ​​data may be subjected to text cleaning processing according to the rule content of the cleaning rule, thereby obtaining cleaned spliced ​​data, that is, the target data.

[0144] The target data is used as the target text.

[0145] The present application performs splicing processing on all the word data to obtain corresponding spliced ​​data; then obtains a preset cleaning rule; then performs text cleaning processing on the spliced ​​data based on the cleaning rule to obtain corresponding target data; and subsequently uses the target data as the target copy. After decoding the target vector based on the trained decoder to obtain the corresponding word data, the present application performs splicing processing on all the word data to obtain corresponding spliced ​​data, and intelligently uses the cleaning rule to perform text cleaning processing on the spliced ​​data to generate the final target copy, effectively improving the data accuracy of the target copy.

[0146] In some optional implementations of this embodiment, before step S202, the electronic device may further perform the following steps:

[0147] Get pre-built video sample data.

[0148] In this embodiment, the video sample data may be a certain amount of pre-collected insurance video data. The insurance video data may include content such as an introduction to the insurance industry, business introductions, product introductions, and sales pitch training. The selection of the first amount is not specifically limited and can be determined based on actual encoder training requirements.

[0149] Obtaining a preset initial encoder and a feature cross-alignment training strategy corresponding to the initial encoder;

[0150] In this embodiment, the initial encoder is an initial image encoder, an initial audio encoder, or an initial text encoder. The feature cross-alignment training strategy refers to a strategy of cross-aligning features of three modalities (image, audio, and text) and using the InfoNCE loss function for training.

[0151] A target optimization algorithm corresponding to the initial encoder is determined.

[0152] In this embodiment, the target optimization algorithm may specifically adopt a gradient descent algorithm, such as SGD or Adam algorithm.

[0153] Based on the feature cross-alignment training strategy, the target optimization algorithm and the preset InfoNCE loss function, the initial encoder is trained using the video sample data to obtain a trained designated encoder.

[0154] In this embodiment, the process of training the initial encoder includes: 1. By inputting the video sample data into the initial encoder, the features of three modalities (image, audio, and text) are obtained, and the features of the three modalities are cross-aligned. Among them, the features of the three modalities include the feature vector (ff) of each key frame contained in the video sample data, the comprehensive image vector (fv), the feature vector (fas) of each audio clip contained in the video sample data, the comprehensive audio vector (fa), the feature vector (fw) of each token contained in the video sample data, and the comprehensive text vector (fs). 2. Calculate the similarity matrix. Initialize the similarity matrix Sa_v (similarity between fa and fv), Sa_f (similarity between fa and ff), Sas_v (similarity between fas and fv), and Sas_f (similarity between fas and ff). Among them, these matrices can be calculated using cosine similarity or other similarity measurement methods. For example, Sa_v = cosine_similarity(fa,fvT), where fvT is the transpose of fv. 3. Integrate Instance-Level similarity. For each audio segment (each element in fas), calculate its similarity with the entire video (fv) to obtain the audio-level similarity Saud′. This can be achieved by averaging the rows of Sfas_fv. Similarly, for the key frames of the video (each element in ff), calculate its similarity with the comprehensive audio vector (fa) to obtain the video-level similarity Svid′. This can be achieved by averaging the columns of Sfa_ff (transposition processing is required here to match the dimensions). 4. Calculate the similarity between Audio and Video. Add and average Saud′ and Svid′ to obtain Sasf′, which is the overall similarity simAV between audio and video. 5. Calculate the InfoNCE loss and backpropagate. Use simAV and positive and negative sample pairs to calculate the InfoNCE loss. The loss gradient is backpropagated to the encoder of the corresponding audio mode and the encoder of the corresponding video mode to update the parameters to align the features between speech and image, thereby completing the training of the specified encoder. This application uses speech as anchor information, aligns features pairwise, and iteratively trains until the model converges to obtain the corresponding trained encoder. In addition, since the network has a symmetrical structure, based on the above training method, the encoder corresponding to the audio modality and the encoder corresponding to the text modality can be trained to align the features between speech (audio) and text (description), and the InfoNCE loss function can also be used to train the encoder corresponding to the audio modality and the encoder corresponding to the text modality.

[0155] The designated encoder is stored.

[0156] In this embodiment, there is no specific limitation on the storage method of the above-mentioned designated encoder, and it can be set according to actual storage requirements, for example, it may include blockchain storage, database storage, cloud server storage, etc.

[0157] The present application obtains pre-constructed video sample data; then obtains a preset initial encoder, and obtains a feature cross-alignment training strategy corresponding to the initial encoder; wherein the initial encoder is an initial image encoder or an initial audio encoder or an initial text encoder; then determines the target optimization algorithm corresponding to the initial encoder; further based on the feature cross-alignment training strategy, the target optimization algorithm and the preset InfoNCE loss function, uses the video sample data to train the initial encoder to obtain a trained designated encoder; and subsequently stores the designated encoder. After obtaining the pre-constructed video sample data and the initial encoder, the present application uses the video sample data to train the initial encoder according to the feature cross-alignment training strategy corresponding to the initial encoder, the target optimization algorithm corresponding to the initial encoder and the InfoNCE loss function, so as to quickly and intelligently complete the construction process of the designated encoder, improve the construction intelligence of the designated encoder, and ensure the effect of the obtained designated encoder. In addition, the designated encoder will also be intelligently stored to ensure the security of the generated designated encoder.

[0158] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0159] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target document, the above-mentioned target document can also be stored in a node of a blockchain.

[0160] The blockchain referred to in this application is a new application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0161] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0162] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0163] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0164] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0165] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for generating text data based on artificial intelligence. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0166] like Figure 3As shown, the artificial intelligence-based copywriting data generation device 300 of this embodiment includes: a first acquisition module 301, a first processing module 302, a second processing module 303, a third processing module 304, a combination module 305, a decoding module 306 and a generation module 307. Among them:

[0167] A first acquisition module 301 is used to acquire a video to be processed;

[0168] A first processing module 302 is configured to extract an image sequence from the video based on a preset image clustering algorithm, and generate a comprehensive image vector corresponding to the image sequence based on a trained image encoder;

[0169] A second processing module 303 is configured to extract target audio from the video and generate a comprehensive audio vector corresponding to the target audio based on the trained audio encoder;

[0170] A third processing module 304 is configured to extract target text from the video and generate a comprehensive text vector corresponding to the target text based on the trained text encoder;

[0171] A combining module 305 is configured to combine the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector;

[0172] A decoding module 306 is configured to decode the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes multiple;

[0173] The generation module 307 is used to generate a target text corresponding to the video based on the word data.

[0174] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0175] In some optional implementations of this embodiment, the first processing module 302 includes:

[0176] A reading submodule, configured to read the video file in the video based on a preset video processing library;

[0177] a processing submodule, configured to perform frame extraction processing on the video file based on a preset time interval to obtain a corresponding first image;

[0178] a clustering submodule, configured to perform image clustering processing on the first image based on a preset image fingerprint algorithm, and obtain fingerprint similarity of the first image;

[0179] a filtering submodule, configured to filter images having fingerprint similarities greater than a preset similarity threshold from the first images to obtain corresponding second images;

[0180] A construction submodule is used to construct a corresponding image sequence based on the second image.

[0181] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0182] In some optional implementations of this embodiment, the second processing module 303 includes:

[0183] Call submodule, used to call the preset audio extraction tool;

[0184] A first extraction submodule, configured to perform audio extraction processing on the video based on the audio extraction tool to obtain a corresponding first audio;

[0185] a separation submodule, configured to perform speech separation processing on the first audio to obtain a corresponding second audio;

[0186] The second determining submodule is configured to use the second audio as the target audio.

[0187] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and will not be repeated here.

[0188] In some optional implementations of this embodiment, the third processing module 304 includes:

[0189] The first acquisition submodule is used to obtain multiple preset text extraction strategies;

[0190] A screening submodule, configured to screen out a target text extraction strategy that matches the video from all the text extraction strategies;

[0191] A second extraction submodule is used to perform text extraction processing on the video based on the text extraction strategy to obtain corresponding text data;

[0192] The third determining submodule is configured to use the text data as the target text.

[0193] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0194] In some optional implementations of this embodiment, the combining module 305 includes:

[0195] The second acquisition submodule is used to obtain a preset splicing strategy;

[0196] A first splicing submodule is configured to perform splicing processing on the integrated image vector, the integrated audio vector, and the integrated text vector based on the splicing strategy to obtain a corresponding splicing vector;

[0197] The fourth determining submodule is configured to use the splicing vector as the target vector.

[0198] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0199] In some optional implementations of this embodiment, the generating module 307 includes:

[0200] A second splicing submodule is used to splice all the word data to obtain corresponding spliced ​​data;

[0201] The third acquisition submodule is used to obtain preset cleaning rules;

[0202] A cleaning submodule, configured to perform text cleaning on the spliced ​​data based on the cleaning rules to obtain corresponding target data;

[0203] The fifth determining submodule is configured to use the target data as the target text.

[0204] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0205] In some optional implementations of this embodiment, the artificial intelligence-based copywriting data generation device further includes:

[0206] A first acquisition module is used to acquire pre-built video sample data;

[0207] A second acquisition module is configured to acquire a preset initial encoder and a feature cross-alignment training strategy corresponding to the initial encoder; wherein the initial encoder is an initial image encoder, an initial audio encoder, or an initial text encoder;

[0208] A determination module, configured to determine a target optimization algorithm corresponding to the initial encoder;

[0209] A training module, configured to train the initial encoder using the video sample data based on the feature cross-alignment training strategy, the target optimization algorithm, and a preset InfoNCE loss function to obtain a trained designated encoder;

[0210] A storage module is used to store the designated encoder.

[0211] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based copywriting data generation method in the aforementioned embodiment, and are not repeated here.

[0212] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0213] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0214] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0215] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the artificial intelligence-based copywriting data generation method. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0216] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for executing the artificial intelligence-based copywriting data generation method.

[0217] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0218] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0219] In an embodiment of the present application, the image encoder, audio encoder and text encoder are used to perform feature learning and information mapping of three modalities on the video to be processed to obtain a corresponding comprehensive image vector, a comprehensive audio vector and a comprehensive text vector, and the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector are combined and processed to obtain a corresponding target vector, and then the trained decoder is used to decode the target vector to obtain the corresponding word data, and based on the word data, the target text corresponding to the video is automatically and accurately generated. Different from the existing method of manually generating text corresponding to the video, the use of encoders and decoders based on the three modalities of the present application can automatically and accurately generate the corresponding text according to the video content, thereby improving the efficiency of text generation and ensuring the accuracy of the generated text.

[0220] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based copywriting data generation method.

[0221] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0222] In an embodiment of the present application, the image encoder, audio encoder and text encoder are used to perform feature learning and information mapping of three modalities on the video to be processed to obtain a corresponding comprehensive image vector, a comprehensive audio vector and a comprehensive text vector, and the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector are combined and processed to obtain a corresponding target vector, and then the trained decoder is used to decode the target vector to obtain the corresponding word data, and based on the word data, the target text corresponding to the video is automatically and accurately generated. Different from the existing method of manually generating text corresponding to the video, the use of encoders and decoders based on the three modalities of the present application can automatically and accurately generate the corresponding text according to the video content, thereby improving the efficiency of text generation and ensuring the accuracy of the generated text.

[0223] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0224] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for generating copywriting data based on artificial intelligence, characterized in that: The steps include: Get the video to be processed; Extracting an image sequence from the video based on a preset image clustering algorithm, and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder; Extracting target audio from the video, and generating a comprehensive audio vector corresponding to the target audio based on the trained audio encoder; Extracting target text from the video, and generating a comprehensive text vector corresponding to the target text based on the trained text encoder; Combining and processing the comprehensive image vector, the comprehensive audio vector, and the comprehensive text vector to obtain a corresponding target vector; Decoding the target vector based on the trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality; A target text corresponding to the video is generated based on the word data.

2. The method for generating copywriting data based on artificial intelligence according to claim 1, characterized in that: The step of extracting an image sequence from the video based on a preset image clustering algorithm specifically includes: Read the video file in the video based on a preset video processing library; Performing frame extraction processing on the video file based on a preset time interval to obtain a corresponding first image; Performing image clustering processing on the first image based on a preset image fingerprint algorithm, and obtaining fingerprint similarity of the first image; Filtering images whose fingerprint similarity is greater than a preset similarity threshold from the first images to obtain corresponding second images; A corresponding image sequence is constructed based on the second image.

3. The method for generating copywriting data based on artificial intelligence according to claim 1, characterized in that: The step of extracting target audio from the video specifically includes: Call the preset audio extraction tool; Performing audio extraction processing on the video based on the audio extraction tool to obtain a corresponding first audio; Performing speech separation processing on the first audio to obtain a corresponding second audio; The second audio is used as the target audio.

4. The method for generating copywriting data based on artificial intelligence according to claim 1, characterized in that: The step of extracting the target text from the video specifically includes: Get multiple preset text extraction strategies; Filter out a target text extraction strategy that matches the video from all the text extraction strategies; Performing text extraction processing on the video based on the text extraction strategy to obtain corresponding text data; The text data is used as the target text.

5. The method for generating copywriting data based on artificial intelligence according to claim 1, characterized in that: The step of combining and processing the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain a corresponding target vector specifically includes: Get the preset splicing strategy; Based on the splicing strategy, the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector are spliced ​​to obtain a corresponding splicing vector; The concatenated vector is used as the target vector.

6. The method for generating text data based on artificial intelligence according to claim 1, characterized in that: The step of generating a target text corresponding to the video based on the word data specifically includes: Performing concatenation processing on all the word data to obtain corresponding concatenated data; Get the preset cleaning rules; Performing text cleaning on the spliced ​​data based on the cleaning rules to obtain corresponding target data; The target data is used as the target text.

7. The method for generating text data based on artificial intelligence according to claim 1, characterized in that: Before the step of extracting an image sequence from the video based on a preset image clustering algorithm and generating a comprehensive image vector corresponding to the image sequence based on a trained image encoder, the method further includes: Get pre-built video sample data; Obtaining a preset initial encoder, and obtaining a feature cross-alignment training strategy corresponding to the initial encoder; wherein the initial encoder is an initial image encoder or an initial audio encoder or an initial text encoder; Determining a target optimization algorithm corresponding to the initial encoder; Based on the feature cross-alignment training strategy, the target optimization algorithm and the preset InfoNCE loss function, the initial encoder is trained using the video sample data to obtain a trained designated encoder; The designated encoder is stored.

8. A copywriting data generation device based on artificial intelligence, characterized in that: include: A first acquisition module, used to acquire a video to be processed; A first processing module, configured to extract an image sequence from the video based on a preset image clustering algorithm, and generate a comprehensive image vector corresponding to the image sequence based on a trained image encoder; A second processing module, configured to extract target audio from the video and generate a comprehensive audio vector corresponding to the target audio based on a trained audio encoder; A third processing module, configured to extract a target text from the video and generate a comprehensive text vector corresponding to the target text based on a trained text encoder; A combination module, used for combining the comprehensive image vector, the comprehensive audio vector and the comprehensive text vector to obtain a corresponding target vector; A decoding module, used for decoding the target vector based on a trained decoder to obtain corresponding word data; wherein the number of the word data includes a plurality; A generation module is used to generate a target text corresponding to the video based on the word data.

9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for generating copywriting data based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method for generating copywriting data based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

Images