Creative data generation method, delivery method, system, server and storage medium

Abstract

The embodiment of the application provides a creative data generation method, a delivery method, a system, a server and a storage medium, wherein the method comprises: acquiring multi-modal material data, the multi-modal material data comprising at least two modes among images, videos, voices and texts; for any mode of material data, pre-processing the material data to determine a material element set of the material data; for any mode of material data, processing the material elements in the material element set of the material data to obtain a material element set of the processed material data, so as to form the processed material data; the material element set of the material data after processing is different from that before processing; and obtaining creative data according to the processed multi-modal material data. The technical scheme provided by the embodiment of the application can automatically adjust the material elements of the multi-modal material data and generate creative data, so as to improve the generation efficiency of the creative data and individually deliver the creative data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a creative data generation method, delivery method, system, server, and storage medium. Background Technology

[0002] Creative data can be used by creative data distributors to promote and advertise products, services, and other objects. Creative data can be understood as scripted content primarily disseminated through text, audio, images, and video. Against this backdrop, how to provide technical solutions to improve the efficiency of creative data generation has become a pressing technical problem for those skilled in the art. Summary of the Invention

[0003] In view of this, embodiments of this application provide a creative data generation method, a delivery method, a system, a server, and a storage medium to improve the generation efficiency of creative data.

[0004] To achieve the above objectives, the embodiments of this application provide the following technical solutions.

[0005] In a first aspect, embodiments of this application provide a method for generating creative data, including:

[0006] Acquire multimodal material data, wherein the multimodal material data includes at least two modalities among images, videos, audio, and text;

[0007] For any modality of material data, preprocess the material data to determine the set of material elements in the material data;

[0008] For any modality of material data, the material elements in the material element set of the material data are processed to obtain the material element set after processing, thus forming the processed material data; wherein, the material element set after processing is different from the material element set before processing;

[0009] Creative data is obtained based on the processed multimodal material data.

[0010] Secondly, embodiments of this application provide a creative data delivery method, including:

[0011] Acquire creative data, wherein the creative data is generated according to the creative data generation method described in the first aspect above;

[0012] Using a pre-trained scoring model, the creative data is scored to obtain a score for the creative data;

[0013] Based on the scores of the creative data, determine the creative data to be deployed.

[0014] Thirdly, embodiments of this application provide a creative data delivery system, including: a client, a creative data generation platform, and a creative data delivery platform;

[0015] The client displays a creative data input interface; the creative data input interface is used by creative data providers to input multimodal material data; and the multimodal material data is sent to the creative data generation platform.

[0016] The creative data generation platform is configured to perform the creative data generation method as described in the first aspect above.

[0017] The creative data delivery platform is configured to execute the creative data delivery method as described in the second aspect above.

[0018] The creative data generation platform and the creative data distribution platform can be integrated or set up independently.

[0019] Fourthly, embodiments of this application provide a server, including a memory and a processor. The memory stores a program, and the processor calls the program stored in the memory to implement the creative data generation method as described in the first aspect, or to implement the creative data delivery method as described in the second aspect.

[0020] Fifthly, embodiments of this application provide a storage medium storing a computer program, which, when executed, implements the creative data generation method as described in the first aspect above, or the creative data delivery method as described in the second aspect above.

[0021] Sixthly, embodiments of this application provide a computer program that, when executed, implements the creative data generation method as described in the first aspect above, or the creative data delivery method as described in the second aspect above.

[0022] The creative data generation method provided in this application supports adjusting material elements in material data, thereby generating new creative data by combining multimodal material data with adjusted material elements. Based on this, this application can acquire multimodal material data; then, for any modality of material data, preprocess the material data to determine the set of material elements; for any modality of material data, process the material elements in the set of material elements to obtain a processed set of material elements, thus forming processed material data; wherein, the processed set of material elements differs from the unprocessed set of material elements, meaning that the material elements in the processed material data have been adjusted. Therefore, this application can generate new creative data based on the processed multimodal material data with adjusted material elements, achieving automatic generation of new creative data.

[0023] As can be seen, the technical solution provided in this application can automatically generate new creative data by adjusting the material elements of each modality in the multimodal material data, and then combining the adjusted multimodal material data with the adjusted material elements. This allows for the rapid generation of different creative data. Since the material data used to generate creative data is multimodal, after adjusting the material elements of the multimodal material data, this application can obtain creative data combining different types of multimodal material data based on the combination of material data of different modalities, thus diversifying the material modalities of the generated creative data. Furthermore, by changing the modality of the material data used to generate the creative data, this application can flexibly generate creative data combining materials of different modalities. Therefore, the solution provided in this application has the advantage of improving the efficiency of creative data generation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a creative data generation method provided in an embodiment of this application.

[0026] Figure 2 This is a schematic diagram of a creative data generation apparatus provided in an embodiment of this application.

[0027] Figure 3 This is another flowchart illustrating the creative data generation method provided in the embodiments of this application.

[0028] Figure 4 This is another schematic diagram of the creative data generation method provided in the embodiments of this application.

[0029] Figure 5 This is another schematic diagram of the creative data generation method provided in the embodiments of this application.

[0030] Figure 6 This is a flowchart illustrating the creative data delivery method provided in the embodiments of this application.

[0031] Figure 7 This is a schematic diagram of the framework of the creative data generation system provided in the embodiments of this application.

[0032] Figure 8This is an architecture diagram of a server provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] To increase the visibility of goods and services, providers of goods and services can become creative data advertisers. Creative data advertisers can then use creative data advertising platforms to distribute creative data for goods and services. For example, a goods provider can act as an advertiser, placing ads for their goods on an advertising platform to increase product visibility.

[0035] To obtain creative data for advertising and promotion, advertisers often utilize creative data designers (such as advertising designers) to design their creative data. Advertisers provide materials to the designers, who then design the data and continuously coordinate with the advertisers to meet their design requirements, ultimately resulting in creative data that satisfies the advertiser and is ready for campaign deployment. This method, which relies on creative data designers for design, increases the time and manpower required for manual design.

[0036] Based on this, this application provides a creative data generation method, which adjusts the material elements of the provided multimodal material data, thereby automatically generating new creative data by combining the adjusted multimodal material data, thus improving the generation efficiency of creative data.

[0037] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a creative data generation method provided in an embodiment of this application. This method can be applied to a server, which can be a server corresponding to a creative data generation platform.

[0038] As shown in the figure, the process may include the following steps.

[0039] Step S100: Obtain multimodal material data, wherein the multimodal material data includes at least two modalities among images, videos, audio, and text.

[0040] Optionally, the creative data provider can provide the multimodal material data, or the creative data provider can select the multimodal material data from the creative data material library on the server.

[0041] The multimodal material data may include: any single modality of image modality, text modality, video modality, and audio modality; it may also be a combination of image modality and text modality material data, a combination of video modality and audio modality material data, a combination of video modality and audio modality material data, a combination of image modality, text modality, and audio modality material data, a combination of image modality, text modality, and video modality material data, or a combination of image modality, video, audio, and text material data, as long as it can be used to generate creative data.

[0042] In an optional implementation, when the creative data provider offers a multimodal combination of content data, this embodiment of the application can utilize a feature extraction model (e.g., a multimodal extraction model) to perform modality recognition and extraction on the content data formed by the combination of various modalities when acquiring the multimodal content data, thereby obtaining individual content data categorized by modality for subsequent processing. For example, when the acquired multimodal content data is a combination of image and text modal content data, a multimodal extraction model can be used to identify and extract the text and image features contained in the multimodal content data to obtain image modal content data and text modal content data. Here, multimodality refers to data with multiple different forms of existence or information sources. Each form of existence or each information source can be called a modality. Data composed of two or more modalities is called multimodal data (e.g., multimodal content data combining text, images, audio, and video as described above).

[0043] When selecting text features, the corresponding text feature terms should be able to accurately identify the text content; have the ability to distinguish the target text from other texts; and the number of selected feature terms should not be too large; feature term separation should be relatively easy to achieve. For example, in Chinese text, characters, words, or phrases can be used as text features to represent the text.

[0044] Image features can include geometric features of the source data of the image modality (e.g., position and orientation, perimeter, area, major and minor axes, distance (including Euclidean distance, Manhattan distance, Chebyshev distance, cosine distance)); or shape features (geometric morphology analysis: rectangularity, circularity, invariant moments, eccentricity, polygon description, curve description); magnitude features (moments, projections); histogram features (statistical features): mean, variance, energy, entropy, etc.; color features (color histogram, color moments), and many other features as image features to choose from.

[0045] Of course, in other implementations, when the acquired multimodal material data is a combination of video modal and audio modal material data, a multimodal extraction model can be used to identify and extract the video modal material data and the audio modal material data based on the video and audio features contained in the multimodal material data.

[0046] In the extraction of speech features from speech modality source data, the speech features can be obtained by using the feature parameters that represent the essence of the speech modality source data. The speech signal contained in the speech modality source data can be divided into the time domain, frequency domain, and cepstral domain to obtain the feature parameters.

[0047] Video feature extraction for video modal source data can be divided into audio feature extraction and image feature extraction (keyframe extraction).

[0048] Of course, if the creative data provider offers single-modal creative data, then the single-modal creative data can be directly processed.

[0049] Step S101: For any modality of material data, preprocess the material data to determine the set of material elements of the material data.

[0050] In an optional implementation, for any modality of material data, embodiments of this application can extract the material elements of the material data based on the constituent elements of the material data (referred to as material elements), thereby using the set of material elements of the material data as the material element set of the material data to achieve preprocessing of the material data.

[0051] After performing modality recognition on the multimodal material data to obtain material data for each modality, the material data for each modality is preprocessed to facilitate subsequent processing. The material data for each modality is then split to obtain a set of material elements that make up the material data for each modality.

[0052] In one example, the material elements of the text modality's material data are text vectors. This embodiment of the application can preprocess the text modality's material data to obtain a set of text vectors. The material elements of the image modality's material data are image elements. This embodiment of the application can preprocess the image modality's material data to obtain a set of image elements. The material elements of the speech modality's material data are speech segments. This embodiment of the application can preprocess the speech modality's material data to obtain a set of speech segments. The material elements of the video modality's material data are video segments. This embodiment of the application can preprocess the video modality's material data to obtain a set of video segments.

[0053] By preprocessing the material data of each modality, a set of material elements is obtained, which can then be used for subsequent processing based on each material element in the set.

[0054] Step S102: For any modality of material data, process the material elements in the material element set of the material data to obtain the material element set after processing, so as to form the processed material data; wherein, the material element set after processing is different from the material element set before processing.

[0055] In the preceding steps, the material data for each modality has been preprocessed, resulting in a set of material elements for each modality. Therefore, the set of material elements for each modality can be further processed, that is, some or all of the material elements in the set can be adjusted to make the processed set of material elements significantly different from the unprocessed set, thereby altering the expressiveness of the creative data. For example, after preprocessing the text modality material data, a set of text vectors is obtained. Multiplying one of the text vectors in this set will amplify the text content represented by that single text vector within the overall text modality material data, thus highlighting the text content represented by that particular text vector.

[0056] In one implementation, material elements can be processed according to their importance in the overall material data, and the importance can be represented by the weight value set for each material element in the material element set.

[0057] Step S103: Based on the processed multimodal material data, obtain creative data.

[0058] After processing the set of material elements in the material data for each modality, the set of material elements after processing differs significantly from the set of material elements before processing for any given modality. Therefore, the processed material data is significantly different from the original data. Thus, based on the combination of the processed multimodal material data, embodiments of this application can generate new creative data, the content of which possesses new expressive effects, expressiveness, and appeal.

[0059] As can be seen, the technical solution provided in this application can automatically generate new creative data by adjusting the material elements of each modality in the multimodal material data, and then combining the adjusted multimodal material data with the adjusted material elements. This allows for the rapid generation of different creative data. Since the material data used to generate creative data is multimodal, after adjusting the material elements of the multimodal material data, this application can obtain creative data combining different multimodal materials based on the combination of different modal material data, making the generated creative data have diverse material modalities. Furthermore, by changing the modality of the material data used to generate creative data, this application can flexibly generate creative data combining materials of different modalities. Therefore, the solution provided in this application has the advantage of improving the efficiency of creative data generation.

[0060] To better understand the implementation of the above creative data generation method, please refer to... Figure 2 , Figure 2 This is a schematic diagram of a creative data generation apparatus provided in an embodiment of this application.

[0061] As shown in the figure, the device includes:

[0062] Material data acquisition module 200 is used to acquire multimodal material data;

[0063] The preprocessing module 201 is used to preprocess the material data for any modality and determine the set of material elements in the material data;

[0064] The processing module 202 is used to process the material elements in the material element set of the material data for any modality, so as to obtain the material element set after processing the material data, thus forming the processed material data; wherein, the material element set after processing the material data is different from the material element set before processing;

[0065] The creative data generation module 203 is used to obtain creative data based on the processed multimodal material data.

[0066] To enable modal recognition and segmentation of multimodal material data, the device may further include a multimodal extraction model 210, used to extract the material data of each modality acquired by the material data acquisition module 200, facilitating subsequent preprocessing. Of course, when the material data provided by the creative data provider is single-modal, subsequent preprocessing can be performed directly on the single-modal material data, without the need for the multimodal extraction model 210.

[0067] Optionally, the preprocessing module 201 described above can be a model built on the framework of a Transformer model to process material data of various modalities.

[0068] As can be seen, the technical solution provided in this application can automatically generate new creative data by adjusting the material elements of each modality in the multimodal material data, and then combining the adjusted multimodal material data with the adjusted material elements. This allows for the rapid generation of different creative data. Since the material data used to generate creative data is multimodal, after adjusting the material elements of the multimodal material data, this application can obtain creative data combining different multimodal materials based on the combination of material data from different modalities, thus diversifying the material modalities of the generated creative data. Furthermore, by changing the modality of the material data used to generate the creative data, this application can flexibly generate creative data combining materials from different modalities. Therefore, the solution provided in this application has the advantage of improving the efficiency of creative data generation.

[0069] Based on the foregoing, it can be understood that the creative data generation method provided in this application, in order to improve the diversity of creative data formats and the personalization of creative data content, generates creative data based on material data of any one modality or at least two modalities among video, text, image, and audio. In one embodiment, when the material data includes text modal material data and image modal material data, the text modal material data and the image modal material data can be processed separately to generate creative data.

[0070] Alternatively, please refer to Figure 3 , Figure 3 This is another flowchart illustrating the creative data generation method provided in the embodiments of this application.

[0071] As shown in the figure, the process may include the following steps:

[0072] Step S300: Obtain multimodal material data, which includes image modal material data and text modal material data.

[0073] Step S301: For the text modality material data, vectorize the text in the text modality material data to obtain a set of text vectors;

[0074] Step S302: For the material data of the image modality, perform image segmentation on the material data of the image modality to obtain a set of image elements.

[0075] Optionally, step S300 can be performed by Figure 2 In the creative data generation device shown, the material data acquisition module 200 is implemented.

[0076] Preprocessing the material data of text modality and image modality can facilitate the subsequent processing of each element in the element set, thereby obtaining creative data with different expressive power.

[0077] Optionally, step S301 can be performed by Figure 2 In the creative data generation device shown, the text model 2011 included in the preprocessing module 201 is implemented, and step S302 can be achieved through... Figure 2 In the creative data generation device shown, the image model 2012 is implemented in the preprocessing module 201.

[0078] After obtaining the preprocessed set of elements, further processing of the set is required. Please refer to [link / reference needed]. Figure 3 Creative data generation methods may also include the following steps:

[0079] Step S303: For the material data of the text modality, the set of text vectors is decoded, and the text description is regenerated based on the decoding result. The regenerated text description forms the processed material data of the text modality.

[0080] Step S304: For the image modality material data, perform at least one of the following processing operations on the image element set: rearrangement, modification, or addition of image elements to obtain the processed image element set, thereby forming the processed image modality material data.

[0081] As an optional implementation, step S303 can utilize Figure 2 In the creative data generation device shown, the text processing model 2021 included in the processing module 202 is implemented, and step S304 can utilize... Figure 2 The creative data generation device shown includes an image processing model 2022 implemented in the processing module 202.

[0082] Step S305: Combine the processed image modal material data and the processed text modal material data to obtain creative data combining images and text.

[0083] As an optional implementation, step S305 can utilize Figure 2 The creative data generation module 203 in the creative data generation device shown implements the graphic multimodal model 2031.

[0084] In one implementation, each text vector in the text vector set is decoded to obtain a processed text description. For example, a column of text vectors may be deleted, or a column of text vectors may be multiplied, so that a certain character in the text modality material data is deleted or enlarged, in order to achieve different creative data expression methods and enhance the personalization of creative data.

[0085] For image modal material data, the image elements in the image element set are processed to obtain processed image modal material data. For example, the outline of an image element or the color expression of an image element can be changed, such as changing the shape of an image element from a circle to a triangle, or changing the color of an image element from blue to red, thus achieving a change in visual effect.

[0086] Therefore, based on the processed image and text modal data, multiple image and text data sets are synthesized to form creative data. For example, an image data set with a red triangle could display a prompt sign magnified three times. This enhances the expressiveness of the creative data.

[0087] By decoding each text vector in the text modality material data set after preprocessing to form a new text description, and by rearranging, modifying, or adding any image element in the image element set obtained after preprocessing the image modality material data, the final generated creative data has outstanding characteristics, different expressive power, and can attract the audience of creative data, thereby meeting the distribution needs of creative data distributors.

[0088] In other embodiments, the modality of the material data may further include audio modal material data and video modal material data; thereby generating creative data based on the audio modal material data and the video modal material data. Please refer to [reference needed]. Figure 4 , Figure 4 This is another schematic diagram of the creative data generation method provided in the embodiments of this application.

[0089] As shown in the figure, the process may include the following steps:

[0090] Step S400: Obtain multimodal material data, which includes audio modal material data and video modal material data.

[0091] Step S401: For the material data of the speech modality, the speech segments in the material data of the speech modality are split to obtain a set of speech segments;

[0092] Step S402: For the video modality material data, the video segments in the video modality material data are split to obtain a video segment set.

[0093] As an optional implementation, step S400 above can be achieved by... Figure 2 In the creative data generation device shown, the material data acquisition module 200 is implemented; the above step S401 can be utilized Figure 2 In the creative data generation device shown, the preprocessing module 201 includes a speech model 2013; step S402 can utilize... Figure 2 The creative data generation device shown includes a video model 2014 in the preprocessing module 201.

[0094] Preprocessing the audio and video source data facilitates subsequent processing of individual elements within the set, resulting in creative data with varying expressive power. Please continue to refer to [link / reference needed]. Figure 4 .

[0095] Step S403: For the speech modality material data, perform at least one of the following processing operations on the speech segments in the speech segment set to obtain processed speech segments, which form the processed speech modality material data:

[0096] The intonation of the speech segments in the speech segment set is identified, and the intonation of the speech segments in the speech segment set is adjusted according to the intonation identification results;

[0097] The volume of the speech segments in the speech segment set is identified, and the volume of the speech segments in the speech segment set is adjusted according to the volume identification result;

[0098] Identify key speech segments in the speech segment set and highlight them.

[0099] In the process of processing each speech segment described above, identifying key speech segments in the speech segment set and highlighting these key speech segments can be done based on either intonation recognition results or volume recognition results, highlighting key intonations or key volumes within the speech segment. Alternatively, it can be done simultaneously based on both intonation and volume recognition results, highlighting key intonations and key volumes within the speech segment. For example, a sudden rise in intonation within a speech segment indicates a key speech segment and can be highlighted, or a sudden increase in volume within a speech segment indicates a key speech segment and can be highlighted. In other embodiments, the speech segment can also be highlighted based on speech content with emphasis descriptions. For example, if the speech content contains prompts such as "mainly" or "focused on," indicating a key point, then the corresponding speech segment can be highlighted.

[0100] Step S404: For the video modality's material data, perform at least one of the following processing operations on the video segments in the video segment set to obtain processed video segments, which form the processed video modality's material data:

[0101] The order of the video segments in the video segment set is rearranged;

[0102] Identify key video segments in the video segment set and highlight them.

[0103] The above-mentioned rearrangement of the order of video segments in the video segment set can be a rearrangement of the playback order of each video segment, or a rearrangement of the order of video frames in each video segment. In other words, the rearrangement of video segments can refer to the reordering of the video frame order within a video segment, or it can refer to the reordering of all video segments in the video segment element set.

[0104] The process of identifying key video segments in the video segment set and highlighting them can be based on recognizing the voice volume and intonation of the video segments, highlighting video segments with obvious changes in tone or volume, or it can be based on the content of the video and audio within the video segments to identify key video segments and then highlight them. For example, a video segment can be identified as a key video segment based on an upward inflection of the voice, and then its brightness parameter can be adjusted to increase its brightness, thereby highlighting the key video segment.

[0105] As an optional implementation, step S403 can utilize Figure 2 In the creative data generation device shown, the speech processing model 2023 included in the processing module 202 is implemented; step S404 can utilize... Figure 2 The creative data generation device shown includes a video processing model 2024 implemented in the processing module 202.

[0106] After obtaining the processed video clip elements and the processed audio clip elements, creative data can be formed. Please continue to refer to... Figure 4 The process may also include the following steps:

[0107] Step S405: Align the audio segments of the processed audio modality material data with the video segments of the processed video modality material data to obtain creative data combining audio and video.

[0108] As an optional implementation, step S405 above can utilize... Figure 2 In the creative data generation device shown, the audio-visual multimodal model 2032 included in the creative data generation module 203 is implemented.

[0109] By adjusting and optimizing each audio segment in the audio modality's preprocessed audio data set, and adjusting and optimizing each video segment in the video modality's preprocessed video data set, the final generated creative data has outstanding characteristics, different expressive power, and is attractive to the audience, thus meeting the distribution needs of creative data advertisers.

[0110] In other embodiments, the modality of the material data may further include image modality material data, text modality material data, audio modality material data, and video modality material data; thereby generating creative data based on the material data of these four modalities. Please refer to [reference needed]. Figure 5 , Figure 5 This is another schematic diagram of the creative data generation method provided in the embodiments of this application.

[0111] As shown in the figure, the process may include the following steps:

[0112] Step S500: Obtain multimodal material data, which includes image modal material data, text modal material data, audio modal material data, and video modal material data.

[0113] Step S501: For the text modality material data, vectorize the text in the text modality material data to obtain a set of text vectors;

[0114] Step S502: For the material data of the image modality, perform image segmentation on the material data of the image modality to obtain a set of image elements;

[0115] Step S503: For the material data of the speech modality, the speech segments in the material data of the speech modality are split to obtain a set of speech segments;

[0116] Step S504: For the video modality material data, the video segments in the video modality material data are split to obtain a video segment set.

[0117] As an optional implementation, step S500 above can be achieved by... Figure 2 In the creative data generation device shown, the material data acquisition module 200 is implemented; the above step S501 can be achieved through... Figure 2 In the creative data generation device shown, the text model 2011 included in the preprocessing module 201 is implemented; the above step S502 can be achieved through... Figure 2 In the creative data generation device shown, the image model 2012 in the preprocessing module 201 is implemented; the above step S503 can utilize Figure 2 In the creative data generation device shown, the preprocessing module 201 includes a speech model 2013; step S504 can utilize... Figure 2 The creative data generation device shown includes a video model 2014 in the preprocessing module 201.

[0118] Preprocessing the audio and video source data facilitates subsequent processing of individual elements within the set, resulting in creative data with varying expressive power. Please continue to refer to [link / reference needed]. Figure 5 .

[0119] like Figure 5 As shown, the process also includes the following steps:

[0120] Step S505: For the material data of the text modality, the set of text vectors is decoded, and the text description is regenerated based on the decoding result. The regenerated text description forms the processed material data of the text modality.

[0121] Step S506: For the material data of the image modality, perform at least one of the following processing operations on the image element set: rearrangement, modification, or addition of image elements to obtain the processed image element set, so as to form the material data of the processed image modality.

[0122] Step S507: For the speech modality material data, perform at least one of the following processing operations on the speech segments in the speech segment set to obtain processed speech segments, which form the processed speech modality material data:

[0123] The intonation of the speech segments in the speech segment set is identified, and the intonation of the speech segments in the speech segment set is adjusted according to the intonation identification results;

[0124] The volume of the speech segments in the speech segment set is identified, and the volume of the speech segments in the speech segment set is adjusted according to the volume identification result;

[0125] Identify key speech segments in the speech segment set and highlight the key speech segments;

[0126] Step S508: For the video modality's material data, perform at least one of the following processing operations on the video segments in the video segment set to obtain processed video segments, which form the processed video modality's material data:

[0127] The order of the video segments in the video segment set is rearranged;

[0128] Identify key video segments in the video segment set and highlight them.

[0129] As an optional implementation, step S505 can utilize Figure 2 In the creative data generation device shown, the text processing model 2021 included in the processing module 202 is implemented; step S506 can utilize... Figure 2 In the creative data generation device shown, the image processing model 2022 included in the processing module 202 is implemented; step S507 can utilize... Figure 2 In the creative data generation device shown, the speech processing model 2023 included in the processing module 202 is implemented; step S508 can utilize... Figure 2 The creative data generation device shown includes a video processing model 2024 implemented in the processing module 202.

[0130] Step S509: Combine the processed image modal material data and the processed text modal material data to obtain combined image and text material data;

[0131] Step S510: Align the audio segments of the processed speech modal material data with the video segments of the processed video modal material data to obtain material data combining audio and video.

[0132] Step S511: Combine the combined text and image data with the combined audio and video data, and adjust the layout and arrangement of the materials in the combined result to obtain creative data.

[0133] As an optional implementation, the combined text and image data obtained in step S509 above can be utilized... Figure 2In the creative data generation device shown, the creative data generation module 203 includes a graphic multimodal model 2031. The audio and video combined material data obtained in step S510 can be used... Figure 2 In the creative data generation device shown, the audio-visual multimodal model 2032 included in the creative data generation module 203 is implemented, and the above step S511 can be achieved using... Figure 2 The creative data generation device shown includes a unified multimodal model 2033 implemented in the creative data generation module 203.

[0134] In some implementations, at least one of the following operations is performed on each speech segment in the speech segment set: intonation adjustment, volume adjustment, and highlighting of key audio content, to obtain a processed speech segment. For example, this could be changing the order of appearance of two speech segments, increasing the volume of a speech segment, or highlighting key audio content in a speech segment. This would enable the processed speech segment to convey new speech information in the processed speech modality material data, or to highlight key audio content by increasing the corresponding intonation.

[0135] For video modal source data, the video segments in the video segment set are processed to obtain processed video modal source data. For example, this could involve changing the frame order of a video segment, highlighting key video content in a video segment, altering the order in which video content appears (e.g., changing "open door" to "close door"), or magnifying key video content in a video segment (e.g., enlarging fish in a river). This achieves changes in the visual effects and intuitive experience during video playback, adjusting the persuasiveness of the video modal source data.

[0136] Therefore, based on the processed audio modal and video modal material data, multiple audio and video material data are synthesized; further, text and image material data are combined to form creative data. The creative data generated based on the diverse and processed material data adjusts the overall expressiveness of the creative data and enhances the personalization of the generated creative data.

[0137] Optionally, the above steps for adjusting the layout and arrangement of materials in the composite result can be utilized... Figure 2 The creative data generation device shown includes an enhancement model 204, which is implemented to obtain the final creative data.

[0138] By adjusting the composite data of combined text and images with combined audio and video data, the final result can be optimized. For example, audio and video data can be embedded into blank areas within combined text and images, or placed side-by-side and blended to achieve a natural integration of audio and video data into the text and images. This ensures that the final creative data meets basic format requirements and that the content is presented more appropriately.

[0139] In other implementations, the processed image modality material data and the processed text modality material data can be synthesized to obtain combined image-text material data, which is then combined with the speech modality material data. The speech modality material data is embedded into the image-text material data, so that the speech playback content of the speech modality material data is consistent with the processed text content in the combined image-text material data, thereby improving the listening effect of the combined image-text material data.

[0140] Alternatively, the processed image modal material data and the processed text modal material data can be synthesized to obtain image-text combined material data, which is then combined with video modal material data. The video modal material data is embedded into the image-text combined material data, so that the audio playback content and video display content of the video modal material data are consistent with the processed text content in the image-text combined material data, thereby further improving the listening effect of the image-text combined material data.

[0141] The above-mentioned image, text, audio, and video materials can be combined arbitrarily. Based on the different creative data advertisers' needs and the material data provided, creative data of different modalities can be generated quickly, improving the efficiency of creative data generation.

[0142] To further meet the needs of creative data advertisers, namely, that the final creative data can be directly used for advertising, this application also provides a creative data advertising method.

[0143] Please refer to Figure 6 , Figure 6 This is a flowchart illustrating a creative data delivery method provided in an embodiment of this application. The method can be applied to a server, which can be a server corresponding to a creative data delivery platform. In optional implementations, the server corresponding to the creative data generation platform and the server corresponding to the creative data delivery platform can be integrated (e.g., servers belonging to the same server cluster can perform both creative data generation and creative data delivery), or they can be set up independently.

[0144] As shown in the figure, the process may include the following steps:

[0145] Step S600: Obtain creative data.

[0146] The creative data is generated according to the creative data generation method described in the foregoing embodiments. The rapid generation of creative data can provide a reliable data foundation for subsequent creative data deployment, meeting the basic requirements for deployment.

[0147] Step S601: Use the pre-trained scoring model to score the creative data and obtain the score of the creative data.

[0148] As an optional implementation, embodiments of this application can utilize the distribution channels of creative data and the click-through rate / conversion rate of the target audience of the creative data to form platform historical data, thereby training a scoring model through the platform historical data, scoring the creative data according to the scoring model, and obtaining the score of the creative data.

[0149] To achieve personalized delivery in different ways and meet the diverse needs of creative data delivery providers, the aforementioned scoring model can be used to score the creative data obtained using the creative data generation method provided in this application embodiment. In an optional implementation, the scoring model provided in this application embodiment can include a first type of scoring model and a second type of scoring model. The first type of scoring model can score the creative data, and the second type of scoring model can score both the creative data and the users to whom the creative data is to be delivered.

[0150] Optionally, for the first type of scoring model, the first type of scoring model can score each creative data point, thereby automatically delivering creative data that conforms to a set sorting based on the score obtained by each creative data point. For example, if there are 5 creative data points obtained based on the creative data generation method provided in this application embodiment, namely creative data 01, creative data 02, creative data 03, creative data 04, and creative data 05, after scoring each creative data point, the creative data can be sorted in descending order of score, and the creative data in the set sorting (e.g., the top three creative data points) can be pushed. Thus, the first type of scoring model can score each of the 5 creative data points and automatically deliver the top three creative data points. Of course, in other embodiments, the set sorting can be to push the creative data ranked first in descending order of score.

[0151] In other implementations, the second type of scoring model can score both creative data and the users to whom the creative data is to be pushed. For example, the second type of scoring model can score different users (e.g., different user types) to whom the same creative data is to be pushed, thus obtaining different scores for the same creative data for different users (e.g., different scores for the same creative data for different user types). Then, based on different users (e.g., different user types), the creative data delivery platform can select creative data with high scores for users and automatically deliver it to them. For example, for the user type to be targeted (e.g., the population to whom creative data is to be delivered), the creative data delivery platform can deliver creative data to the user type based on the score of each creative data point. In this case, the second type of scoring model can be used to determine the score of each creative data point for the user type to be targeted, and based on the scores of each creative data point, automatically select creative data that meets the set sorting criteria and deliver it to the user type to be targeted. For example, creative data can be sorted from highest to lowest score based on the scores of the target audience, and the top-ranked creative data can be delivered. The creative data delivery platform can then select the top-scoring creative data based on the scores from the second type of scoring model and deliver it to the target audience.

[0152] In a further example, suppose that five creative data points are obtained based on the creative data generation method provided in this application embodiment: Creative Data 01, Creative Data 02, Creative Data 03, Creative Data 04, and Creative Data 05. The users targeted by the creative data are divided into type A users and type B users. Then, when targeting type A users, the second type of scoring model can score each of the five creative data points separately. For example, if the score for Creative Data 01 is 8, the score for Creative Data 02 is 6, the score for Creative Data 03 is 7, the score for Creative Data 04 is 5, and the score for Creative Data 05 is 8.5, then Creative Data 05 is selected and pushed to type A users. When targeting type B users, the five creative data points can be scored separately. For example, if the score for Creative Data 01 is 7, the score for Creative Data 02 is 8, the score for Creative Data 03 is 6, the score for Creative Data 04 is 4, and the score for Creative Data 05 is 3, then Creative Data 02 is selected and pushed to type B users. Therefore, the second type of scoring model can give different scores to different users for the same creative data, enabling personalized delivery of creative data to users.

[0153] Step S602: Determine the creative data to be deployed based on the scores of the creative data.

[0154] The creative data to be deployed is determined based on the scores of the creative data. This can be done by using the first type of scoring model to score each creative data and then automatically deploying multiple creative data that meet the set sorting range according to the scores; or by using the second type of scoring model to score each creative data for different types of users, and then automatically deploying multiple creative data that meet the set sorting range according to the scores of each creative data for each user type.

[0155] As can be seen, the creative data delivery method provided in this application acquires multiple creative data points generated using a creative data generation method, then uses a scoring model to score each creative data point, and automatically delivers creative data within a set sorting order based on the scores (e.g., automatically delivering creative data with higher scores). This application embodiment can automatically deliver data according to the sorting order corresponding to the scores of each creative data point. Although the material data input by the creative data delivery person is the same, the type and presentation of the generated creative data are different after processing the material data based on the creative data generation method provided in this application embodiment.

[0156] It should be noted that in the optional implementation, since the click-through rate and conversion rate of each piece of creative data differ for different users (users refer to the audience of the creative data), the scoring model can determine the score for each piece of creative data for different users (e.g., different user types). Therefore, for any user type, this embodiment can determine the automatic delivery of creative data for that user type based on the scoring results of the creative data for that user type. Thus, the creative data delivery method provided by this embodiment can achieve personalized delivery of creative data to different users, making the delivery of creative data targeted.

[0157] In order to enable creative data to receive different scores from different users, in one implementation, the step of using a pre-trained scoring model to score the creative data and obtain the score of the creative data may include:

[0158] Using a pre-trained scoring model (such as the second type of scoring model mentioned above), for any creative data, scores are given based on the characteristics of the target user group, channel characteristics, and creative data characteristics, thus obtaining the score of the creative data for the target user group.

[0159] Optionally, a scoring model can be trained based on the click-through rate and conversion rate of the target user groups for the creative data. This embodiment of the application can then use the trained scoring model to score the creative data according to the group characteristics, channel characteristics, and creative data characteristics of the target user groups. This ensures that the scoring model assigns different scores to the same creative data when it is targeted at different user groups. This embodiment of the application can automatically deliver appropriate creative data to different users (e.g., automatically delivering the creative data with the highest score to different users), making the delivery of creative data targeted.

[0160] Based on the foregoing, it can be understood that creative data providers can offer source material data used to generate creative data, and the creative data delivery platform can automatically generate the creative data for delivery based on the source material data input by the creative data provider. For easier understanding, please refer to... Figure 7 , Figure 7 This is a schematic diagram of the framework of the creative data generation system provided in the embodiments of this application.

[0161] like Figure 7 As shown, the creative data delivery system for generating creative data may include: client 10, creative data generation platform 20, and creative data delivery platform 30.

[0162] The client 10 can be considered as the terminal device used by the creative data delivery provider, such as a laptop, smartphone, or tablet. The client 10 can interact with the creative data delivery platform 30 by accessing the platform's webpage or through an application used to deliver creative data.

[0163] The creative data delivery platform 30 can be a server platform that provides creative data delivery services. The creative data delivery platform 30 can be implemented by a single server or a server system composed of multiple servers.

[0164] In the creative data delivery system provided in this application embodiment, the client 10 displays a creative data material input interface; the creative data material input interface is used for creative data delivery users to input multimodal material data; and the multimodal material data is sent to the creative data generation platform 20;

[0165] The creative data generation platform 20 is configured to perform the creative data generation method as described in any of the foregoing embodiments;

[0166] The creative data delivery platform 30 is configured to execute the creative data delivery method as described in any of the foregoing embodiments;

[0167] The creative data generation platform 20 and the creative data delivery platform 30 provided in this application embodiment can be integrated or set up independently. Based on different service needs, the creative data generation platform 20 and the creative data delivery platform 30 can be integrated or set up independently to cater to different users.

[0168] As can be seen, the technical solution provided in this application can quickly obtain multiple different creative data by directly generating creative data based on the acquired multimodal material data. Since the multimodal material data can include at least two modalities such as images, audio, text, and video, different modal creative data can be obtained based on material data with different modal combinations, making the generated creative data modally diverse and flexibly generating different creative data according to changes in the modality of the material data. Furthermore, when generating creative data based on multiple material data of different modalities, the material data of various modalities is further preprocessed, and the material elements in the material element sets of various modalities obtained after preprocessing are processed to change the material elements in the original acquired multimodal material data, further enhancing the personalization of the generated creative data. At the same time, since different users may have different click-through rates and conversion rates for the same creative data, the scoring of the same creative data for different users can also be different. Therefore, based on the scoring of creative data for different users, the corresponding creative data can be automatically selected for different users, making the delivery of creative data more targeted.

[0169] This application embodiment also provides a server, including a memory and a processor. The memory stores a program for the creative data generation method or the creative data delivery method described in the above embodiments. The processor calls the program stored in the memory to implement the creative data generation method or the creative data delivery method described in the above embodiments.

[0170] Optionally, Figure 8 This is an optional hardware device architecture diagram provided in an embodiment of this application. This hardware device architecture can be the server architecture described above; see reference... Figure 8 The hardware architecture of the electronic device may include: at least one memory 3 and at least one processor 1; the memory stores a program, and the processor calls the program to execute the aforementioned creative data generation method or creative data delivery method; in addition, at least one communication interface 2 and at least one communication bus 4; the processor 1 and the memory 3 may be located in the same electronic device, for example, the processor 1 and the memory 3 may be located in a server device or a terminal device; the processor 1 and the memory 3 may also be located in different electronic devices.

[0171] As an optional implementation of the content disclosed in the embodiments of this application, the memory 3 can store a program, and the processor 1 can call the program to execute the creative data generation method or creative data delivery method provided in the above embodiments of this application.

[0172] In this embodiment of the application, the hardware device may be a tablet computer, a laptop computer, a smart speaker, or other device capable of executing a creative data generation method or a creative data delivery method.

[0173] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4; obviously, Figure 8 The communication connection diagram of processor 1, communication interface 2, memory 3 and communication bus 4 shown is only one optional method;

[0174] Optionally, communication interface 2 can be an interface of a communication module, such as the interface of a GSM module;

[0175] Processor 1 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0176] Memory 3 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0177] It should be noted that the aforementioned terminal device may also include other devices (not shown) that may not be essential to understanding the content disclosed in the embodiments of this application; given that these other devices may not be essential for understanding the content disclosed in the embodiments of this application, they will not be described one by one in the embodiments of this application.

[0178] This application also provides a storage medium storing a computer program, which, when executed, implements the creative data generation method as described in the above embodiments, or the creative data delivery method as described in the above embodiments.

[0179] This application also provides a computer program that, when executed, implements the creative data generation method or the creative data delivery method provided in this application.

[0180] While the embodiments disclosed above are described in this application, this application is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A method for generating creative data, wherein, include: Acquire multimodal material data, wherein the multimodal material data includes at least two modalities among images, videos, audio, and text; For any modality of material data, preprocess the material data to determine the set of material elements in the material data; For any modality of source data, the source elements in the source element set are processed to obtain a processed source element set, forming the processed source data. The processed source element set differs from the unprocessed source element set. The processing of source elements is based on their importance within the overall source data, including: For the text modality material data, the text in the text modality material data is vectorized to obtain a set of text vectors; one of the text vectors in the set of text vectors is multiplied to highlight the text content represented by that text vector; the set of text vectors is decoded, and the text description is regenerated based on the decoding result. The regenerated text description forms the processed text modality material data. Furthermore, for the image modality material data, the image modality material data is split into images to obtain an image element set; the image element set is subjected to at least one of the following processing operations: rearrangement, modification, or addition of image elements to obtain a processed image element set, thereby forming the processed image modality material data; For the source data of the speech modality, identify the key speech segments in the set of speech segments obtained after splitting the source data of the speech modality, and highlight the key speech segments; For video modal material data, identify key video segments in the set of video segments obtained after splitting the video modal material data, and highlight the key video segments; Based on the processed multimodal material data, creative data is obtained, including: By combining the processed image modal data and the processed text modal data, we obtain combined image and text data. Additionally, the audio segments of the processed speech modal material data are aligned with the video segments of the processed video modal material data to obtain material data combining audio and video. Creative data is obtained by combining text and image data with audio and video data, and by adjusting the layout and arrangement of the combined data.

2. The creative data generation method as described in claim 1, By combining the processed image modal data and the processed text modal data, creative data combining images and text is obtained.

3. The creative data generation method as described in claim 1, wherein the preprocessing of the material data for any modality to determine the set of material elements includes: For the source data of the aforementioned speech modality, the speech segments in the source data of the speech modality are split to obtain a set of speech segments; Furthermore, for the material data of the video modality, the video segments in the material data of the video modality are split to obtain a set of video segments.

4. The creative data generation method as described in claim 3, wherein, For any modality of material data, the material elements in the material element set of the material data are processed to obtain a processed material element set, thus forming the processed material data, including: For the audio modality source data, the audio segments in the audio segment set are processed as follows to obtain processed audio segments, which form the processed audio modality source data: The intonation of the speech segments in the speech segment set is identified, and the intonation of the speech segments in the speech segment set is adjusted according to the intonation identification results; The volume of the speech segments in the speech segment set is identified, and the volume of the speech segments in the speech segment set is adjusted according to the volume identification result; Furthermore, for the video modality's source data, the video segments in the video segment set are processed as follows to obtain processed video segments, which form the processed video modality's source data: The order of the video segments in the video segment set is rearranged; The audio segments of the processed audio modality material data are aligned with the video segments of the processed video modality material data to obtain creative data that combines audio and video.

5. A creative data delivery method, wherein, include: Acquire creative data, wherein the creative data is generated according to the creative data generation method as described in any one of claims 1-4; Using a pre-trained scoring model, the creative data is scored to obtain a score for the creative data; Based on the scores of the creative data, determine the creative data to be deployed.

6. A creative data delivery system, wherein, include: Client-side application, creative data generation platform, and creative data delivery platform; The client displays a creative data input interface; The creative data input interface is used by creative data distributors to input multimodal material data; The multimodal material data is sent to the creative data generation platform; The creative data generation platform is configured to perform the creative data generation method as described in any one of claims 1-4; The creative data delivery platform is configured to execute the creative data delivery method as described in claim 5; The creative data generation platform and the creative data distribution platform can be integrated or set up independently.

7. A server, wherein, The method includes a memory and a processor, wherein the memory stores a program, and the processor calls the program stored in the memory to implement the creative data generation method as described in any one of claims 1-4, or to implement the creative data delivery method as described in claim 5.

8. A storage medium, wherein, The device contains a computer program that, when executed, implements the creative data generation method as described in any one of claims 1-4, or the creative data delivery method as described in claim 5.

Images