Method for generating multi-modal marketing copywriting based on image-text

Abstract

The invention discloses a multi-modal marketing copywriting generation method based on images and texts, and the method specifically comprises the following steps: 1, judging whether historical data contains user preference information or not, and extracting the user preference information and keywords in the preference information from the historical data; 2, projecting information of different modes such as language and visual and auditory sense into the mutually associated common subspace, realizing collaborative representation of multi-mode data, and providing a shared semantic subspace for marketing copywriting generation; and 3, generating a marketing document by utilizing the picture feature vector of the commodity, the character description feature vector of the commodity and the shared semantic subspace of the customer preference information. According to the method, the features of the commodity picture and the description text are extracted, and the image-text sharing semantic subspace is constructed, so that interests and preferences of the user can be accurately identified, marketing documents conforming to different user preferences are generated, and the internal and external feature display effect of the commodity is improved.

Description

Technical Field

[0001] The present invention relates to the field of copywriting processing, and in particular to a method for generating marketing copywriting based on image-text multimodal. Background Art

[0002] In the era of big data, multimodal data is widely used, including various copywriting generation. Automatic copywriting generation can not only improve the efficiency of copywriting creation, but also improve the conversion effect of copywriting. There are many cutting-edge researches on automatic copywriting generation in the fields of natural language processing and artificial intelligence. Now there are a large number of copywriting generation patents, and the generation of copywriting mainly includes planning, extraction and generation.

[0003] Prior art 1, Publication (Announcement) No.: CN112287685A, the disclosed invention patent: copy generation method, device, electronic device and storage medium. This invention limits modifiers when generating copy, converts entity indicators and the threshold into calculable event conditions, then generates events based on event information and the event conditions, and generates copy based on the events. Through the coordination of entity indicators, thresholds, events and event conditions, the copy generation process is abstracted into modules such as event generation, event relations and copy generation, so that copy generation can be applied to a wider range of scenarios and meet the needs of copy generation with complex decision logic.

[0004] This scheme is a planned copywriting generation method. The copywriting is planned and generated according to the structured information of the copywriting, grammatical rules, etc. It adopts a copywriting generation method with a planned method at the granularity of grammatical sentences, which solves the problems of "what to say" and "how to say it" in copywriting generation. First of all, the advantage of this copywriting generation method is that it has strong control and high accuracy, and is suitable for copywriting generation of news types with standard formats. However, this copywriting generation method does not have good generalization ability and has a narrow scope of application; secondly, it is difficult to achieve end-to-end optimization; finally, this copywriting generation method only considers copywriting generation under a single mode, so the generated copy is not conducive to fully and comprehensively expressing semantic information.

[0005] Prior art 2, Publication (Announcement) No.: CN109840273A, the disclosed invention patent: A method and device for generating copy. The invention obtains the copy information of the target application, searches for the target copy corresponding to the copy identifier in a predetermined copy library according to the copy information, and determines the incremental copy information corresponding to the copy identifier. The copy library contains historical copy in different language scenarios. The incremental copy information represents the copy information corresponding to the copy identifier when the target copy is not found. The incremental copy is obtained based on the copy content contained in the incremental copy information. The incremental copy corresponds to the language scenarios one by one. The copy of the target application is obtained based on the target copy and the incremental copy.

[0006] This scheme is an extractive copywriting generation method. The target copywriting corresponding to the copywriting identifier is searched in the predetermined copywriting library, and finally the historical text data is extracted to generate the copywriting. First, this method searches for historical copywriting information in the historical data, and then extracts a part of it as output. Because of the data size limit of the search target library, it is easy to cause the generated text to not express the semantic information well; secondly, the generated text is easily constrained by the original text, and the generalization ability is not strong, resulting in the same generated copywriting in multiple situations; finally, this copywriting generation method only considers the copywriting generation under a single modality, so the generated copywriting is not conducive to fully and comprehensively expressing the semantic information.

[0007] Prior art 3, Publication (Announcement) No.: CN110765273B, the disclosed invention patent: recommendation copy generation method, device, electronic device and readable storage medium. Based on the graph entity information corresponding to the recommended object and the commodity knowledge graph entity information, the user demand vector of the target user is obtained through the encoding module based on the multi-head attention mechanism, and the decoding module of the multi-head attention mechanism obtains the recommendation copy of the recommended object.

[0008] This scheme is a generative copywriting method. By representing the code on the encoding side and completing the sequence generation task on the decoding side, complete end-to-end optimization can be achieved. First, this method has advantages in generalization ability, but it is difficult to control the generated text. Without the support of a large amount of text training corpus, it is easy to generate copywriting with incoherent sentences. Secondly, the encoding and decoding modeling complexity of the multi-head attention mechanism is also very high. Finally, this copywriting generation method only considers the generation of copywriting under a single modality, so the generated copywriting is not conducive to the full and comprehensive expression of semantic information.

[0009] Therefore, the present invention proposes a method for generating marketing copy based on multi-modality of images and texts. The present invention adopts a multi-modality copy generation method to solve the problems of lack of logic and poor accuracy of copy generated under a single mode in the above-mentioned prior art. Summary of the invention

[0010] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method for generating multimodal marketing copy based on images and texts.

[0011] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0012] The present invention provides a method for generating marketing copy based on multi-modal images and texts, which specifically comprises the following steps:

[0013] Step 1: Determine whether the historical data contains user preference information, and extract the user preference information and keywords in the preference information from the historical data;

[0014] Step 2: Different modal data such as pictures and texts have different levels of knowledge expression effects to different degrees. Through feature extraction and vectorization processing, information of different modalities such as language and visual and auditory senses is projected into a mutually related common subspace, realizing the collaborative representation of multimodal data and providing a shared semantic subspace for marketing copy generation;

[0015] Step 3: Generate marketing copy using the shared semantic subspace of the product image feature vector, the product text description feature vector, and customer preference information.

[0016] As a preferred technical solution of the present invention, the specific steps of step 1 are:

[0017] S101: Perform word segmentation on the customer's transaction product details and user comment text data, and use a binary classification algorithm to identify text information of interest and preference;

[0018] S102: Classify the user's products and define a product category set C, where C = {C1, C2, C3...Ci...Cn}, where Ci represents the i-th category of products and n represents the total number of product categories;

[0019] S103: Establish a keyword set B of interest and preference of all users for each category in the commodity category set C, and define B = {B1, B2, B3...Bj...Bn}, where Bj represents the interest and preference word list for commodity category Cj;

[0020] S104: extract the content features of the texts of interest and preference identified in S101 to establish a relationship matrix A between the customer and the interest preference keyword set B of each category of goods in S103, A includes A1, A2, ...An, where An represents the preference vector of the nth product, An = [Wn1, Wn2, ...Wnk], where Wnk represents the kth interest and preference keyword of the nth category of goods;

[0021] S105: If the database does not have the user's preference information for a certain type of product, or if the database has the user's preference information for this type of product but the preference vocabulary is not in the preference vocabulary set B, then based on the user's age, gender and other information, by calculating the similarity of the user's feature vectors, select the user with the highest similarity to the user in the database, extract the keywords of interest in this type of product, and then output the customer's interest and preference relationship keyword matrix A for each type of product, so as to extract the user's preference information.

[0022] As a preferred technical solution of the present invention, the specific steps of step 2 are:

[0023] S201: Extract and establish the external feature set of material, color, and style of each category of goods in the product category set C, and define Outside = {O1, O2, O3...Oi...On}, where Oi represents the image data feature of the category i product, where Oi = [Material(i), color(i), style(i)];

[0024] S202: Establish an intrinsic feature set of each category of the product in the product category set C, define Inside = {I1, I2, I3...Ii...In}, Ii represents the text description feature of the product of category i, where Ii includes but is not limited to the function, usage, and material features of the product, specifically Ii = [function(i), usage(i), material(i)](i), style2(i)...stylen(i)];

[0025] S203: extracting features from the product image data, classifying the image labels according to the material, color, and style in S201, and abstracting the digital image into a feature vector representation;

[0026] S204: Extracting the features of the product description text, extracting the key information of the product attributes in the text; specifically: first, performing word segmentation on the text data; second, extracting keywords from the text data according to the intrinsic features in step S202; and finally representing the feature word vector;

[0027] S205: Encoding at the text character level to identify unregistered words and phrases;

[0028] S206: Concatenate the image feature vector extracted in step S203 with the text description feature vector extracted in step S204 to construct a shared semantic subspace based on images and texts, so as to provide services for the generation of marketing copy.

[0029] As a preferred technical solution of the present invention, the specific steps of S205 are as follows:

[0030] S20501: The word segmenter encodes the text data of all the product descriptions in the database after word segmentation, where one code represents a word;

[0031] S20502: Identify the largest substring: In the encoded string, use the sliding window algorithm to find the substring with the largest number of occurrences;

[0032] Replace substring: replace the substring with a new single character;

[0033] Repeat the process: In the new string, continue to find the substring with the most occurrences and replace it until no substring appears more than 5 times;

[0034] Identify unknown words: In the final string, frequently occurring substrings are identified as unknown words or unknown phrases.

[0035] As a preferred technical solution of the present invention, the specific steps of step three are:

[0036] S301: The training data of the copywriting generation model is composed of the sum of pic and word embedding, segment embedding and position embedding;

[0037] S302: Text generation uses the seq2seq model architecture, where the input is the sum of the content vectors in S301, and the output is the sum of the paperwork vectors. It is mainly divided into two parts: encoder and decoder. The encoder converts the input sequence into a fixed-size vector n+N, and then the decoder uses the vector n+N to recursively generate the output sequence in one direction, that is:

[0038] P(Y|W)=P(Y1|W)P(Y2|W,Y1)P(Y3|W,Y1,Y2)......P(Yn|W,Y1,Y2..Yn-1)

[0039] Among them, Y is the word sequence in the paperwork, W is the feature vector, the encoder uses BiLSTM, and the decoder uses a unidirectional LSTM. When the output of the encoder is used as the initial hidden state of the decoder, the hidden state of the encoder is reduced in dimension, and then the loss value of the paperwork part is calculated according to the token type_id.

[0040] S303: Generate marketing copy Using the model trained in step S302, generate top 5 high-quality product marketing copy, then use the customer preferences in S1 to perform similarity calculation and matching with the generated top 5 marketing copy to select the best product marketing copy, and finally output the marketing copy.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention extracts the features of product images and description texts and constructs a shared semantic subspace between images and texts, which can accurately identify users' interests and preferences, thereby generating marketing copy that meets the preferences of different users and improving the display effect of the internal and external features of the product.

[0043] The present invention realizes feature extraction of commodity images and description texts by identifying the material, color and style in commodity images, as well as the intrinsic characteristics of the commodities, thereby enhancing the comprehensiveness and accuracy of commodity information.

[0044] The present invention utilizes the interest and preference keyword sets of commodity categories to perform binary classification processing on user data, which can effectively identify the user's interest and preference for commodities and their characteristics, and improve the accuracy of user portraits.

[0045] The present invention forms training data through pic and word embedding, segment embedding and position embedding, generates top 5 high-quality product marketing copywriting, calculates the similarity between customer preferences and these copywritings, screens out the copywriting that best meets customer preferences, and improves the effect of marketing copywriting.

[0046] The present invention performs word segmentation processing on text data, extracts keywords in combination with intrinsic characteristics of commodities, and vectorizes the extracted characteristic words to represent them, thereby realizing feature extraction of commodity description text and enhancing the analysis capability of text data.

[0047] The invention encodes text characters, extracts frequently occurring maximum common substrings, identifies unregistered words and phrases, and improves the accuracy and efficiency of text data processing.

[0048] The present invention uses a word segmenter to encode the text data of all commodity text descriptions in the database after word segmentation, identifies the largest common substring that appears more than 5 times as unregistered words and phrases, and realizes the effective extraction of the frequently occurring largest common substring. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 It is a flow chart of the method for generating marketing copy based on multi-modality of images and texts of the present invention;

[0051] Figure 2 It is a training data structure diagram of the marketing copy generation model of the present invention; DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0053] Example 1

[0054] like Figure 1-2 As shown, the present invention provides a method for generating marketing copy based on multi-modal images and texts, which specifically includes the following steps:

[0055] Step 1: Determine whether the historical data contains user preference information, and extract the user preference information and keywords in the preference information from the historical data; the specific steps are:

[0056] S101: Perform word segmentation on the customer's transaction product details and user comment text data, and use a binary classification algorithm to identify text information of interest and preference;

[0057] S102: Classify the user's products and define a product category set C, where C = {C1, C2, C3...Ci...Cn}, where Ci represents the i-th category of products and n represents the total number of product categories;

[0058] S103: Establish a set B of all users' interests and preference keywords for each category in the product category set C, and define B = {B1, B2, B3...Bj...Bn}, where Bj represents the interest and preference word list for product category Cj; for example, the preferred keywords for the mobile phone category include "cost-effectiveness" and "photography", and the preferred keywords for the clothing category include "V-neck" and "material", etc.;

[0059] S104: In the above implementation, extract the content features of the text of interest and preference identified in S101 to establish a relationship matrix A between the customer and the interest preference keyword set B of each category of goods in S103, A includes A1, A2, ...An, where An represents the preference vector of the nth product, An = [Wn1, Wn2, ...Wnk], where Wnk represents the kth interest and preference keyword of the nth category of goods;

[0060] S105: If the database does not have the user's preference information for a certain type of product, or has the user's preference information for this type of product but the preference words are not in the preference word set B, then based on the user's age, gender and other information, by calculating the similarity of the user's feature vectors, select the user with the highest similarity to the user in the database, extract the user's interest keywords for this type of product, and then output the customer's interest and preference relationship keyword matrix A for each type of product, so as to extract the user's preference information

[0061] Step 2: Different modal data such as pictures and texts have different levels of knowledge expression effects to different degrees. Through feature extraction and vectorization processing, information of different modalities such as language and visual and auditory senses are projected into a mutually related common subspace to achieve collaborative representation of multimodal data and provide a shared semantic subspace for marketing copy generation. The specific steps are as follows:

[0062] S201: Extract and establish the external feature set of the material, color, and style of each category of goods in the product category set C, and define Outside = {O1, O2, O3...Oi...On}, where Oi represents the image data feature of the category i product, where Oi = [Material(i), color(i), style(i)]; if Ci is a T-shirt type, then Oi = ["cotton and linen", "white", "V-neck"], and if Ci is a shoe type, then Oi = ["canvas", "maroon", "sports"];

[0063] S202: Establish an intrinsic feature set of each category of the product in the product category set C, define Inside = {I1, I2, I3...Ii...In}, Ii represents the text description feature of the product in category i, where Ii includes but is not limited to the function, use, and material features of the product, specifically Ii = [function(i), usage(i), material(i)](i), style2(i)...stylen(i)]; if Ci is a T-shirt type, then Ii = ["sunscreen", "youthful"... "slimming"], Ci is a shoe type, then Ii = ["breathable", "mature"... "exquisite"];

[0064] S203: In the above implementation, the features of the product image data are extracted, and the image tags are classified according to the material, color, and style in S201, so that the digital image is abstracted into a feature vector representation;

[0065] S204: In the above implementation, the text description features of the product are extracted to extract the key information of the product attributes in the text; specifically: first, the text data is segmented; second, the text data is extracted for keywords according to the intrinsic features in step S202; finally, the feature word vector is represented;

[0066] S205: In the above implementation, when the text segmentation technology processes the text data of steps S101 and S204, the commonly used word segmenter cannot recognize unregistered words well (unregistered words are words that are not included in the word segmentation vocabulary but must be segmented). Such as: "Ingenious, breathable, fashionable, starry sky, flying needle, coconut shoes, comfortable for sports and suitable for jogging." The traditional word segmentation effect of the word segmentation device is: "Ingenious / breathable / fashionable / sky / stars / flying needle / coconut / shoes / , / soft / comfortable / suitable / jogging / ." It is not possible to recognize "starry sky" and "coconut shoes" as a word. Therefore, encoding is performed at the text character level to identify unregistered words and phrases; the specific steps of S205 are as follows:

[0067] S20501: The word segmenter encodes the text data of all the product descriptions in the database after word segmentation, where one code represents a word; for example, "ingenuity / breathable / fashion / sky full of stars / flying needles / coconut / shoes" is encoded as abcdefgh;

[0068] S20502: Identify the largest substring: In the encoded string, use the sliding window algorithm to find the substring with the largest number of occurrences;

[0069] Replace substring: replace the substring with a new single character;

[0070] Repeat the process: In the new string, continue to find the substring with the most occurrences and replace it until no substring appears more than 5 times;

[0071] Identify unknown words: In the final string, frequently occurring substrings are identified as unknown words or unknown phrases;

[0072] For example, for the encoding of "ppbdppbacpp", "pp" appears the most times, so it is replaced with a character "P" to get "PbdPbacP", and then the process is repeated, with K=Pb, to get "KdKacp", until no repeated character pairs appear more than 5 times or more; at this time, the common substrings that appear in large numbers in the text data (such as P and K here) are registered words or unregistered phrases. For example, "Ingenious breathable and fashionable starry sky flying needle coconut shoes, comfortable and suitable for jogging." The word segmentation result under this method is "Ingenious / breathable / fashionable / starry sky / flying needle / coconut shoes / , / soft and comfortable / suitable / jogging / . / ", it can be seen that the word segmentation effect of the text is significantly improved;

[0073] S206: In the above implementation, the image feature vector extracted in step S203 is concatenated with the text description feature vector extracted in step S204 to construct a shared semantic subspace based on images and texts, thereby providing services for the generation of marketing copy.

[0074] Step 3: Generate marketing copy using the shared semantic subspace of the product image feature vector, the product text description feature vector, and customer preference information; the specific steps of step 3 are:

[0075] S301: The training data of the copywriting generation model is composed of the pic and word embedding, segment embedding and position embedding. The details are as follows Figure 2 As shown:

[0076] S302: In the above implementation, text generation adopts the seq2seq model architecture, in which the input is composed of the addition of the content part vectors of the image and text features in S301, and the output is composed of the addition of the paperwork part vectors. It is mainly divided into two parts: encoder and decoder. The encoder converts the input sequence into a fixed-size vector n+N, and then the decoder uses the vector n+N to recursively generate the output sequence in one direction, that is:

[0077] P(Y|W)=P(Y1|W)P(Y2|W,Y1)P(Y3|W,Y1,Y2)......P(Yn|W,Y1,Y2..Yn-1)

[0078] Among them, Y is the word sequence in the paperwork, W is the feature vector, the encoder uses BiLSTM, and the decoder uses a unidirectional LSTM. When the output of the encoder is used as the initial hidden state of the decoder, the hidden state of the encoder is reduced in dimension, and then the loss value of the paperwork part is calculated according to the token type_id;

[0079] S303: In the above implementation, the marketing copy is generated using the model trained in step S302 to generate top 5 high-quality product marketing copy, and then the customer preferences in S1 are used to perform similarity calculation and matching with the generated top 5 marketing copy to select the best product marketing copy, and finally the marketing copy is output.

[0080] The present invention solves the problem of lack of logic and poor accuracy of copy generated under a single mode in the prior art described in the background technology by adopting a multimodal copy generation method; secondly, customer preference information on product characteristics is added to the marketing copy generation, so that the generated marketing copy highlights the product selling points and product characteristics that more customers care about, thereby improving the practicality and reliability of the marketing copy; finally, in the word segmentation of the product text description data generated by the marketing copy, unregistered words and phrases are identified by extracting the maximum common substring of frequently occurring character strings, thereby compensating for the disadvantage of incorrect extraction of text semantic information due to word segmentation errors, making the feature extraction of the product text description data more accurate, thereby generating more accurate and reliable marketing copy.

[0081] Compared with the three inventions disclosed in the background technology

[0082] The copywriting generation method in the prior art 1 adopts a planned text generation method, which is suitable for copywriting generation in professional fields such as news types, but does not have good generalization ability, and the application range of copywriting generation is very narrow. The present invention adopts seq2seq generative copywriting generation, which has good generalization ability and can be well applied to text generation in various scenarios; secondly, in the generation logic, the method uses thresholds to limit entity indicators and modifiers, and the thresholds are given in advance, so that the copywriting content generated by the method is not accurate enough. The present invention uses BiLSTM in the encoder and a unidirectional LSTM training model in the decoder, so that the threshold of the model is learned through the neural network model, rather than manually set in advance, so that the generated copy is more accurate.

[0083] Prior art 2 is to search for historical copy information from historical data, and then extract a part of it as output. Because the data size of the search target library is limited, it is easy to cause the generated text to not express the semantic information well. The generated text is easily bound by the original text and has poor generalization ability, which easily leads to the same copy generated under different circumstances. The present invention uses seq2seq generative copy generation, which has good generalization ability and can be well applied to express text semantic information; secondly, the copy generation method uses fuzzy matching in the copy content in the search target library, which is not conducive to matching the best copy and is prone to ignoring some semantic information in the text. The present invention embeds the features of the text into the training data of the model. After being processed by the encoder encoder and the decoder decoder, the model can fully learn the semantic information before and after the text, which is conducive to generating copy rich in semantic information.

[0084] Prior art 3 only considers the generation of text under a single mode, and the text generated in this way is not conducive to the full and comprehensive expression of semantic information. The present invention is a text generation method based on multimodal data, which is conducive to obtaining semantic information from different levels, making the generated text semantics richer and more accurate; secondly, the scheme does not consider the processing of unregistered words in text processing, while the present invention considers the problem that unregistered words in word segmentation will lead to inaccurate semantic information, and adopts the method of identifying the largest common substring to identify unregistered words, so that the information learned by the model in machine learning is more accurate, thereby making the content of the generated text more accurate and the quality more reliable.

[0085] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating multimodal marketing copy based on images and texts, characterized in that: The specific steps include: Step 1: Determine whether the historical data contains user preference information, and extract the user preference information and keywords in the preference information from the historical data; Step 2: Different modal data such as pictures and texts have different levels of knowledge expression effects to different degrees. Through feature extraction and vectorization processing, information of different modalities such as language and visual and auditory senses is projected into a mutually related common subspace, realizing the collaborative representation of multimodal data and providing a shared semantic subspace for marketing copy generation; Step 3: Generate marketing copy using the shared semantic subspace of the product image feature vector, the product text description feature vector, and customer preference information.

2. The method for generating multimodal marketing copy based on images and texts according to claim 1, characterized in that: The specific steps of step one are: S101: Perform word segmentation on the customer's transaction product details and user comment text data, and use a binary classification algorithm to identify text information of interest and preference; S102: Classify the user's products and define a product category set C, where C = {C1, C2, C3...Ci...Cn}, where Ci represents the i-th category of products and n represents the total number of product categories; S103: Establish a keyword set B of interest and preference of all users for each category in the commodity category set C, and define B = {B1, B2, B3...Bj...Bn}, where Bj represents the interest and preference word list for commodity category Cj; S104: extract the content features of the texts of interest and preference identified in S101 to establish a relationship matrix A between the customer and the interest preference keyword set B of each category of goods in S103, A includes A1, A2, ...An, where An represents the preference vector of the nth product, An = [Wn1, Wn2, ...Wnk], where Wnk represents the kth interest and preference keyword of the nth category of goods; S105: If the database does not have the user's preference information for a certain type of product, or if the database has the user's preference information for this type of product but the preference vocabulary is not in the preference vocabulary set B, then based on the user's age, gender and other information, by calculating the similarity of the user's feature vectors, select the user with the highest similarity to the user in the database, extract the keywords of interest in this type of product, and then output the customer's interest and preference relationship keyword matrix A for each type of product, so as to extract the user's preference information.

3. The method for generating marketing copy based on multimodal images and texts according to claim 2, characterized in that: The specific steps of step 2 are: S201: Extract and establish the external feature set of material, color, and style of each category of goods in the product category set C, and define Outside = {O1, O2, O3...Oi...On}, where Oi represents the image data feature of the category i product, where Oi = [Material(i), color(i), style(i)]; S202: Establish an intrinsic feature set of each category of the product in the product category set C, define Inside = {I1, I2, I3...Ii...In}, Ii represents the text description feature of the product of category i, where Ii includes but is not limited to the function, usage, and material features of the product, specifically Ii = [function(i), usage(i), material(i)](i), style2(i)...stylen(i)]; S203: extracting features from the product image data, classifying the image labels according to the material, color, and style in S201, and abstracting the digital image into a feature vector representation; S204: Extracting the features of the product description text, extracting the key information of the product attributes in the text; specifically: first, performing word segmentation on the text data; second, extracting keywords from the text data according to the intrinsic features in step S202; and finally representing the feature word vector; S205: Encoding at the text character level to identify unregistered words and phrases; S206: Concatenate the image feature vector extracted in step S203 with the text description feature vector extracted in step S204 to construct a shared semantic subspace based on images and texts, so as to provide services for the generation of marketing copy.

4. The method for generating marketing copy based on multi-modal images and texts according to claim 3, characterized in that: The specific steps of S205 are as follows: S20501: The word segmenter encodes the text data of all the product descriptions in the database after word segmentation, where one code represents a word; S20502: Identify the largest substring: In the encoded string, use the sliding window algorithm to find the substring with the largest number of occurrences; Replace substring: replace the substring with a new single character; Repeat the process: In the new string, continue to find the substring with the most occurrences and replace it until no substring appears more than 5 times; Identify unknown words: In the final string, frequently occurring substrings are identified as unknown words or unknown phrases.

5. The method for generating marketing copy based on multi-modal images and texts according to claim 4, characterized in that: The specific steps of step three are: S301: The training data of the copywriting generation model is composed of the sum of pic and word embedding, segment embedding and position embedding; S302: Text generation uses the seq2seq model architecture, where the input is the sum of the content vectors in S301, and the output is the sum of the paperwork vectors. It is mainly divided into two parts: encoder and decoder. The encoder converts the input sequence into a fixed-size vector n+N, and then the decoder uses the vector n+N to recursively generate the output sequence in one direction, that is: P(Y|W)=P(Y1|W)P(Y2|W,Y1)P(Y3|W,Y1,Y2)...P(Yn|W,Y1,Y2..Yn-1); Among them, Y is the word sequence in the paperwork, W is the feature vector, the encoder uses BiLSTM, and the decoder uses a unidirectional LSTM. When the output of the encoder is used as the initial hidden state of the decoder, the hidden state of the encoder is reduced in dimension, and then the loss value of the paperwork part is calculated according to the token type_id; S303: Generate marketing copy Using the model trained in step S302, generate top 5 high-quality product marketing copy, then use the customer preferences in S1 to perform similarity calculation and matching with the generated top 5 marketing copy to select the best product marketing copy, and finally output the marketing copy.

Images