An advertising case recommendation system and method based on multimodal data

Abstract

The present invention discloses an advertising case recommendation system and method based on multimodal data, belonging to the technical field of advertising case recommendation. The system of the present invention includes: a data collection and knowledge graph construction module, a user classification and behavior data acquisition module, a key point screening and historical behavior data processing module, and a recommendation generation and effect evaluation module; the data collection and knowledge graph construction module uses the establishment of an advertising case knowledge graph to obtain a comprehensive advertising case feature vector; the user classification and behavior data acquisition module distinguishes the first target user and the second target user, and extracts the corresponding information data; the key point screening and historical behavior data processing module compares the historical data of the first target user and the second target user according to the user behavior data, thereby obtaining the screening key points; the recommendation generation and effect evaluation module generates an advertising case recommendation sequence based on the screening historical data of the second target user, evaluates the recommendation effect and outputs prompt information.

Description

Technical Field

[0001] The present invention relates to the technical field of advertisement case recommendation, and in particular to an advertisement case recommendation system and method based on multimodal data. Background Art

[0002] Driven by the wave of informatization and digitalization, multimodal data has become an indispensable part of the big data field. From a broad perspective, multimodal data refers to data types that integrate different senses, such as visual, auditory, tactile and other multi-dimensional information; while in a narrow sense, multimodal data is a mixture of diverse data types and data structures, including symbols, numbers, pictures, text, video, audio and time series data, which together constitute a rich and diverse data ecology. These multimodal data are widely present in various platforms and applications, such as social media, e-commerce platforms, search engines, etc., providing unprecedented opportunities for advertising case recommendation systems. As a core component of the advertising industry, the performance and accuracy of advertising case recommendation systems have a significant impact on advertising effectiveness and user experience.

[0003] However, existing advertising case recommendation systems often rely too much on historical user behavior data, such as click, browse and search records, to predict user interests and preferences. Although this method simplifies the processing of large amounts of complex data to a certain extent, it still has shortcomings. Specifically, not all user behavior data can accurately reflect their true intentions and preferences. Users' click, browse and search records may be affected by many factors, such as curiosity, chance or misoperation. Therefore, if the user's behavior data is not screened, it is not appropriate to directly use all the user's behavior data to analyze the user's true intentions and preferences, which will lead to inaccurate advertising case recommendation results. Summary of the invention

[0004] The purpose of the present invention is to provide an advertising case recommendation system and method based on multimodal data to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A method for recommending similar advertising cases based on multimodal data comprises the following steps:

[0007] Step S100. Collect multimodal data of advertising cases, construct an advertising case knowledge graph based on the multimodal data, and obtain low-dimensional vector representations of entities and relationships based on the advertising case knowledge graph; the multimodal data includes text data, image data, and video data; obtain a text vector representation of the advertising case based on the text data in the advertising case using a text vector embedding model; fuse the low-dimensional vector representation of the entity and relationship with the advertising case text vector representation to obtain a comprehensive advertising case feature representation;

[0008] Step S200. Obtain the historical behavior data of the user and the user's usage information of the recommended advertising case, and classify the user into a first target user and a second target user based on the user's usage information of the recommended advertising case;

[0009] Step S300. Combine the user's usage information of the recommended advertising case, compare the historical behavior data of the first target user with the historical behavior data of the second target user, so as to obtain the screening key points;

[0010] Step S400. Obtain historical behavior data of the second target user, and filter the historical behavior data of the second target user based on the filtering key points; generate a corresponding advertising case recommendation sequence according to the filtered historical behavior data;

[0011] Step S500. Analyze the generated advertising case recommendation sequence and the advertising case actually selected by the second target user, evaluate the advertising case recommendation effect, and thus obtain the advertising case recommendation index; and perform corresponding processing according to the size relationship between the advertising case recommendation index and the corresponding threshold.

[0012] Furthermore, the process of fusing the low-dimensional vector representation of entities and relationships with the vector representation of the advertisement case text in step S100 is as follows:

[0013] For each advertising case, obtain the low-dimensional vector representation of the relevant entities and attribute relationships from the advertising case knowledge graph. One advertising case involves entities (e1, e2, ...) and relationships (r1, r2, ...), and the corresponding vector representations are: V_ei and V_rj; i and j represent the indexes of the entity and relationship respectively; ei represents the i-th entity, and rj represents the j-th relationship; use the text vector embedding model to convert the text data of the advertising case into a text vector representation, and the text vector is T;

[0014] Define an attention mechanism so that during the fusion process, the text vector T and the vector representation (V_ei, V_rj) of each corresponding entity relationship pair (ei, rj) are dynamically weighted fused. The specific calculation process is as follows:

[0015] Calculate the attention weight between the text vector T and the corresponding vector representation (V_ei, V_rj) of each entity relationship pair (ei, rj). The specific calculation formula is:

[0016] α ij =exp(score(T,V_ei,V_rj)) / Σ (i’,j’) exp(score(T,V_ei',V_rj'));

[0017] Among them, α ij Represents the attention weight, which is used to dynamically weight the fusion of the text vector T and the vector representations V_ei and V_rj of the i-th entity ei and the j-th relationship rj; exp(score(T,V_ei,V_rj)) represents the correlation score between the text vector T and the entity ei and the relationship rj; Σ i’,j’ exp(score(T,V_ei',V_rj')) represents the sum of the scoring function values ​​of all entities ei' and relations rj';

[0018] Use attention weight α ij The entity vector V_ei and the relationship vector V_rj are weighted averaged and fused to obtain the comprehensive advertising case feature representation V_case, and V_case=Σ (i,j) α ij (V_ei⊕V_rj).

[0019] Further, step S200 includes:

[0020] S201. Obtain the user's historical behavior data and the user's usage information of the recommended advertising case. For each user, obtain the corresponding advertising case based on the historical behavior data, extract the corresponding entity in combination with the advertising case knowledge graph, and record the number of extracted entities, and each advertising case corresponds to a number of entities; summarize the number of advertising cases obtained based on the user's historical behavior data and the number of corresponding entities, so as to obtain data point A (a1, b1), where a1 represents the number of advertising cases obtained from the user's historical behavior data, and b1 represents the number of entities corresponding to the advertising case;

[0021] S202. Based on the user's usage information of the recommended advertising case, the browsing data of the user for the advertising case is obtained, and the browsing data includes the number of advertising cases browsed by the user and the browsing time of the corresponding advertising case; by comparing the browsing time with the preset browsing time, the advertising cases with a browsing time greater than or equal to the preset browsing time are screened, and based on the screening result, the number of advertising cases with a browsing time greater than or equal to the preset browsing time and the corresponding entities are summarized, thereby obtaining data point B (a2, b2), where a2 represents the number of advertising cases with a browsing time greater than or equal to the preset browsing time, and b2 represents the number of entities corresponding to the advertising cases with a browsing time greater than or equal to the preset browsing time;

[0022] S202. Obtain the corresponding data points A(a1, b1) and B(a2, b2) for each user, calculate the difference R between the two data points, and the specific calculation formula is: R = (a1-a2) 2 +(b1-b2) 2 ; Compare the difference R with the difference threshold R0. If R<R0, the corresponding user is classified as the first target user; if R≥R0, the corresponding user is classified as the second target user.

[0023] Furthermore, step S300 includes:

[0024] S301. For each first target user and second target user, the corresponding historical behavior data and the corresponding user's usage information on the recommended advertising case are obtained, and according to the corresponding advertising case, the historical behavior data and the user's usage information on the recommended advertising case are associated to form an associated data group, and each associated data group contains the advertising case number, historical behavior data and the user's usage information on the recommended advertising case; according to the associated data group and the advertising case knowledge graph, the initial entity path C and the actual entity path S corresponding to each user are obtained, and in the initial entity path C and the actual entity path S, the entity represents a node, and the line segment connecting the entity only represents the connection relationship, and the length of the line segment connecting all entities is equal; the initial entity path refers to the entity connection path containing the advertising cases corresponding to all historical behavior data; the actual entity path refers to the entity connection path corresponding to the user's usage information on the recommended advertising case; and the entities corresponding to the entity path can all be found on the initial entity path, and the number of entities corresponding to the initial entity path is greater than the number of entities corresponding to the actual path;

[0025] S302. For all first target users, obtain the corresponding initial entity path C1 and actual entity path S1, mark the nodes of the actual entity path S1 on the initial entity path C1, and use the entities not marked in the initial entity path C1 as indirect nodes, and use the line segment composed of nodes adjacent to and continuously connected to the indirect nodes as the indirect path J, and associate the historical behavior data corresponding to the indirect path J; for all second target users, obtain the corresponding initial entity path C2 and actual entity path S2, mark the nodes of the actual entity path S2 on the initial entity path C2, obtain the indirect path J corresponding to the first target user, and compare it with the entity line segments not marked in the initial entity path C2; use the nodes corresponding to the entity line segments that are not completely consistent with the indirect path J as the screening key points.

[0026] By comparing the historical behavior data of the first target user and the second target user, the key screening points are found and used to screen the historical behavior data of the second target user, thereby improving the pertinence of the recommendation.

[0027] Furthermore, step S400 includes:

[0028] Obtain the historical behavior data of the second target user, filter the historical behavior data corresponding to the filtering key points of the second target user according to the filtering key points and in combination with the associated data group, calculate the similarity between the filtered historical behavior data and the historical behavior data of the first target user, and arrange the corresponding recommendation strategies of the first target user in descending order according to the similarity calculation result; based on the recommendation strategy of the first target user, generate a corresponding advertising case recommendation sequence for the second target user.

[0029] Based on the filtered historical behavior data and combined with the recommendation strategy of the first target user, an advertising case recommendation sequence for the second target user is generated, thus realizing personalized advertising case recommendation.

[0030] Furthermore, step S500 includes:

[0031] S501. Obtain the generated advertising case recommendation sequence for the second target user and the advertising case sequence actually selected by the second target user, mark the generated advertising case recommendation sequence for the second target user as X1, and mark the advertising case sequence actually selected by the second target user as X2; based on the advertising case recommendation sequence, obtain the corresponding comprehensive advertising case feature representation, and calculate the similarity D between the advertising case recommendation sequences corresponding to X1 and X2 according to the comprehensive advertising case feature representation;

[0032] S502. According to the advertising case recommendation sequence and similarity D corresponding to X1 and X2, the advertising case recommendation effect for the second target user is evaluated, and the advertising case recommendation index Z is calculated. The specific calculation formula is:

[0033] Z=D×[N(X1∩X2) / N(X1∪X2)];

[0034] Among them, N(X1∩X2) represents the number of identical advertising cases in the advertising case recommendation sequence corresponding to X1 and X2, and N(X1∪X2) represents the total number of advertising cases in the advertising case recommendation sequence corresponding to X1 and X2; the advertising case recommendation index Z is compared with the corresponding advertising case recommendation index threshold Z0. If Z≥Z0, it means that the advertising case recommendation sequence for the current second target user meets the expected effect; if Z<Z0, it means that the advertising case recommendation sequence for the current second target user does not meet the expected effect, then the corresponding prompt information is output and the relevant personnel shall perform corresponding processing.

[0035] An advertising case recommendation system based on multimodal data, the system includes: a data collection and knowledge graph construction module, a user classification and behavior data acquisition module, a key point screening and historical behavior data processing module, and a recommendation generation and effect evaluation module;

[0036] The data collection and knowledge graph construction module constructs the advertising case knowledge graph based on multimodal data, extracts entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; uses the text vector embedding model to obtain the text vector representation of the advertising case, and fuses it with the entity and relationship vectors to obtain a comprehensive advertising case feature representation;

[0037] The user classification and behavior data acquisition module divides users into first target users and second target users according to their behavior data and usage information, and extracts users' browsing behavior data and corresponding advertising case entity information;

[0038] The key point screening and historical behavior data processing module compares the historical behavior data of the first target user and the second target user based on the user behavior data, thereby obtaining the screening key points; and screens the historical behavior data of the second target user according to the screening key points;

[0039] The recommendation generation and effect evaluation module generates a corresponding advertising case recommendation sequence based on the historical behavior data of the screened second target user; compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, calculates the advertising case recommendation index, and determines whether the recommendation effect meets expectations based on the set threshold, and outputs corresponding prompt information.

[0040] The data collection and knowledge graph construction module includes a data collection unit, a knowledge graph construction unit, and a vector representation generation unit;

[0041] The data collection unit is responsible for collecting multimodal data from advertising cases, including text, image and video data; the knowledge graph construction unit constructs an advertising case knowledge graph based on the collected multimodal data, which includes the modeling of entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; the vector representation generation unit uses technologies such as text vector embedding models to convert the text data of advertising cases into advertising case text vector representations, and fuses the low-dimensional vector representations of entities and relationships with the advertising case text vector representations to obtain a comprehensive advertising case feature representation.

[0042] Further, the user classification and behavior data acquisition module includes a behavior data acquisition unit and a user classification unit;

[0043] The behavior data acquisition unit is responsible for acquiring the user's historical behavior data and usage information of the recommended advertising case; the user classification unit divides the user into a first target user and a second target user based on the user's usage information of the recommended advertising case;

[0044] The key point screening and historical behavior data processing module includes a key point screening unit and a historical behavior data processing unit;

[0045] The key point screening unit combines the user's usage information of the recommended advertising case, compares the historical behavior data of the first target user and the second target user, and obtains the screening key points; the historical behavior data processing unit screens and processes the historical behavior data of the second target user based on the screening key points.

[0046] Further, the recommendation generation and effect evaluation module includes a recommendation generation unit, an effect evaluation unit, and a processing strategy unit;

[0047] The recommendation generation unit generates a corresponding advertising case recommendation sequence based on the processed user characteristics and behavior data, as well as the advertising case knowledge graph; the effect evaluation unit compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, evaluates the recommendation effect, and calculates the advertising case recommendation index; the processing strategy unit performs corresponding processing based on the size relationship between the advertising case recommendation index and the corresponding threshold.

[0048] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention adopts multimodal data, including text, image and video data, and constructs an advertising case knowledge graph; this method does not only rely on the user's click, browse and search records, but comprehensively analyzes the content and features of advertising cases from multiple angles, and can more accurately reflect the richness and diversity of advertising content; through the low-dimensional vector representation of entities and relationships based on the advertising case knowledge graph, combined with the text vector embedding model, a comprehensive feature representation of advertising cases is achieved; this comprehensive feature not only includes the static features of the advertising case, but also takes into account the semantic information of the text content, so as to more accurately Capture the real interests and preferences of users; the present invention divides users into first target users and second target users according to the users' usage information of recommended advertising cases, and uses the user behavior data to screen key points; this classification and screening process effectively improves the ability to understand and identify the real intentions of users, and avoids the problem of inaccurate recommendations that may be caused by relying solely on full behavior data; the present invention not only generates an advertising case recommendation sequence, but also evaluates the conformity of the recommendation effect by calculating the recommendation index; by comparing and analyzing with the advertising cases actually selected by users, the recommendation strategy is further optimized and adjusted to ensure that the advertising recommendation results are more in line with the expectations and needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0050] Figure 1 It is a schematic diagram of a module of an advertisement case recommendation system based on multimodal data according to the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1 , the present invention provides a technical solution:

[0053] An advertising case recommendation system based on multimodal data, the system includes: a data collection and knowledge graph construction module, a user classification and behavior data acquisition module, a key point screening and historical behavior data processing module, and a recommendation generation and effect evaluation module;

[0054] The data collection and knowledge graph construction module constructs the advertising case knowledge graph based on multimodal data, extracts entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; uses the text vector embedding model to obtain the text vector representation of the advertising case, and fuses it with the entity and relationship vectors to obtain a comprehensive advertising case feature representation;

[0055] The user classification and behavior data acquisition module divides users into first target users and second target users according to their behavior data and usage information, and extracts users' browsing behavior data and corresponding advertising case entity information;

[0056] The key point screening and historical behavior data processing module compares the historical behavior data of the first target user and the second target user based on the user behavior data, thereby obtaining the screening key points; and screens the historical behavior data of the second target user according to the screening key points;

[0057] The recommendation generation and effect evaluation module generates a corresponding advertising case recommendation sequence based on the historical behavior data of the screened second target user; compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, calculates the advertising case recommendation index, and determines whether the recommendation effect meets expectations based on the set threshold, and outputs corresponding prompt information.

[0058] The data collection and knowledge graph construction module includes a data collection unit, a knowledge graph construction unit, and a vector representation generation unit;

[0059] The data collection unit is responsible for collecting multimodal data from advertising cases, including text, image and video data; the knowledge graph construction unit constructs an advertising case knowledge graph based on the collected multimodal data, which includes the modeling of entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; the vector representation generation unit uses technologies such as text vector embedding models to convert the text data of advertising cases into advertising case text vector representations, and fuses the low-dimensional vector representations of entities and relationships with the advertising case text vector representations to obtain a comprehensive advertising case feature representation.

[0060] The user classification and behavior data acquisition module includes a behavior data acquisition unit and a user classification unit;

[0061] The behavior data acquisition unit is responsible for acquiring the user's historical behavior data and usage information of the recommended advertising case; the user classification unit divides the user into a first target user and a second target user based on the user's usage information of the recommended advertising case;

[0062] The key point screening and historical behavior data processing module includes a key point screening unit and a historical behavior data processing unit;

[0063] The key point screening unit combines the user's usage information of the recommended advertising case, compares the historical behavior data of the first target user and the second target user, and obtains the screening key points; the historical behavior data processing unit screens and processes the historical behavior data of the second target user based on the screening key points.

[0064] The recommendation generation and effect evaluation module includes a recommendation generation unit, an effect evaluation unit and a processing strategy unit;

[0065] The recommendation generation unit generates a corresponding advertising case recommendation sequence based on the processed user characteristics and behavior data, as well as the advertising case knowledge graph; the effect evaluation unit compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, evaluates the recommendation effect, and calculates the advertising case recommendation index; the processing strategy unit performs corresponding processing based on the size relationship between the advertising case recommendation index and the corresponding threshold.

[0066] A method for recommending similar advertising cases based on multimodal data comprises the following steps:

[0067] Step S100. Collect multimodal data of advertising cases, construct an advertising case knowledge graph based on the multimodal data, and obtain low-dimensional vector representations of entities and relationships based on the advertising case knowledge graph; the multimodal data includes text data, image data, and video data; obtain a text vector representation of the advertising case based on the text data in the advertising case using a text vector embedding model; fuse the low-dimensional vector representation of the entity and relationship with the advertising case text vector representation to obtain a comprehensive advertising case feature representation;

[0068] Step S200. Obtain the historical behavior data of the user and the user's usage information of the recommended advertising case, and classify the user into a first target user and a second target user based on the user's usage information of the recommended advertising case;

[0069] Step S300. Combine the user's usage information of the recommended advertising case, compare the historical behavior data of the first target user with the historical behavior data of the second target user, so as to obtain the screening key points;

[0070] Step S400. Obtain historical behavior data of the second target user, and filter the historical behavior data of the second target user based on the filtering key points; generate a corresponding advertising case recommendation sequence according to the filtered historical behavior data;

[0071] Step S500. Analyze the generated advertising case recommendation sequence and the advertising case actually selected by the second target user, evaluate the advertising case recommendation effect, and thus obtain the advertising case recommendation index; and perform corresponding processing according to the size relationship between the advertising case recommendation index and the corresponding threshold.

[0072] The process of fusing the low-dimensional vector representation of entities and relationships with the vector representation of the advertisement case text in step S100 is as follows:

[0073] For each advertising case, obtain the low-dimensional vector representation of the relevant entities and attribute relationships from the advertising case knowledge graph. One advertising case involves entities (e1, e2, ...) and relationships (r1, r2, ...), and the corresponding vector representations are: V_ei and V_rj; i and j represent the indexes of the entity and relationship respectively; ei represents the i-th entity, and rj represents the j-th relationship; use the text vector embedding model to convert the text data of the advertising case into a text vector representation, and the text vector is T;

[0074] Define an attention mechanism so that during the fusion process, the text vector T and the vector representation (V_ei, V_rj) of each corresponding entity relationship pair (ei, rj) are dynamically weighted fused. The specific calculation process is as follows:

[0075] Calculate the attention weight between the text vector T and the corresponding vector representation (V_ei, V_rj) of each entity relationship pair (ei, rj). The specific calculation formula is:

[0076] α ij =exp(score(T,V_ei,V_rj)) / Σ (i’,j’) exp(score(T,V_ei',V_rj'));

[0077] Among them, α ij Represents the attention weight, which is used to dynamically weight the fusion of the text vector T and the vector representations V_ei and V_rj of the i-th entity ei and the j-th relationship rj; exp(score(T,V_ei,V_rj)) represents the correlation score between the text vector T and the entity ei and the relationship rj; Σ i’,j’ exp(score(T,V_ei',V_rj')) represents the sum of the scoring function values ​​of all entities ei' and relations rj';

[0078] Use attention weight α ij The entity vector V_ei and the relationship vector V_rj are weighted averaged and fused to obtain the comprehensive advertising case feature representation V_case, and V_case=Σ (i,j) α ij (V_ei⊕V_rj).

[0079] Step S200 includes:

[0080] S201. Obtain the user's historical behavior data and the user's usage information of the recommended advertising case. For each user, obtain the corresponding advertising case based on the historical behavior data, extract the corresponding entity in combination with the advertising case knowledge graph, and record the number of extracted entities, and each advertising case corresponds to a number of entities; summarize the number of advertising cases obtained based on the user's historical behavior data and the number of corresponding entities, so as to obtain data point A (a1, b1), where a1 represents the number of advertising cases obtained from the user's historical behavior data, and b1 represents the number of entities corresponding to the advertising case;

[0081] S202. Based on the user's usage information of the recommended advertising case, the browsing data of the user for the advertising case is obtained, and the browsing data includes the number of advertising cases browsed by the user and the browsing time of the corresponding advertising case; by comparing the browsing time with the preset browsing time, the advertising cases with a browsing time greater than or equal to the preset browsing time are screened, and based on the screening result, the number of advertising cases with a browsing time greater than or equal to the preset browsing time and the corresponding entities are summarized, thereby obtaining data point B (a2, b2), where a2 represents the number of advertising cases with a browsing time greater than or equal to the preset browsing time, and b2 represents the number of entities corresponding to the advertising cases with a browsing time greater than or equal to the preset browsing time;

[0082] S202. Obtain the corresponding data points A(a1, b1) and B(a2, b2) for each user, calculate the difference R between the two data points, and the specific calculation formula is: R = (a1-a2) 2 +(b1-b2) 2 ; Compare the difference R with the difference threshold R0. If R<R0, the corresponding user is classified as the first target user; if R≥R0, the corresponding user is classified as the second target user.

[0083] In this embodiment, the data point A(a, b) is constructed as follows:

[0084] Assume that there are two corresponding advertising cases based on historical behavior data, namely: Advertising Case 1 and Advertising Case 2, and the corresponding entities are:

[0085] Advertising case 1: e1, e2, e3, e4; Advertising case 2: e2, e4, e5, e6;

[0086] The number of advertising cases and the corresponding number of entities obtained based on the user's historical behavior data are summarized to obtain the data point A(a,b)=(2,6); where b=N({e1,e2,e3,e4}∩{e2,e4,e5,e6}), and N() represents the counting function.

[0087] Step S300 includes:

[0088] S301. For each first target user and second target user, the corresponding historical behavior data and the corresponding user's usage information on the recommended advertising case are obtained, and according to the corresponding advertising case, the historical behavior data and the user's usage information on the recommended advertising case are associated to form an associated data group, and each associated data group contains the advertising case number, historical behavior data and the user's usage information on the recommended advertising case; according to the associated data group and the advertising case knowledge graph, the initial entity path C and the actual entity path S corresponding to each user are obtained, and in the initial entity path C and the actual entity path S, the entity represents a node, and the line segment connecting the entity only represents the connection relationship, and the length of the line segment connecting all entities is equal; the initial entity path refers to the entity connection path containing the advertising cases corresponding to all historical behavior data; the actual entity path refers to the entity connection path corresponding to the user's usage information on the recommended advertising case; and the entities corresponding to the entity path can all be found on the initial entity path, and the number of entities corresponding to the initial entity path is greater than the number of entities corresponding to the actual path;

[0089] S302. For all first target users, obtain the corresponding initial entity path C1 and actual entity path S1, mark the nodes of the actual entity path S1 on the initial entity path C1, and use the entities not marked in the initial entity path C1 as indirect nodes, and use the line segment composed of nodes adjacent to and continuously connected to the indirect nodes as the indirect path J, and associate the historical behavior data corresponding to the indirect path J; for all second target users, obtain the corresponding initial entity path C2 and actual entity path S2, mark the nodes of the actual entity path S2 on the initial entity path C2, obtain the indirect path J corresponding to the first target user, and compare it with the entity line segments not marked in the initial entity path C2; use the nodes corresponding to the entity line segments that are not completely consistent with the indirect path J as the screening key points.

[0090] By comparing the historical behavior data of the first target user and the second target user, the key screening points are found and used to screen the historical behavior data of the second target user, thereby improving the pertinence of the recommendation.

[0091] In this embodiment, for the first target user A and the second target user B:

[0092] User A:

[0093] According to the historical behavior data of user A and the usage information of the advertising case, an associated data group is established; the initial entity path C1 and the actual entity path S1 of user A are constructed; the node of S1 is marked on C1, and the indirect path J is identified and formed; the historical behavior data corresponding to the indirect path J is associated.

[0094] User B:

[0095] According to user B's historical behavior data and information on the use of advertising cases, an associated data group is established; the initial entity path C2 and the actual entity path S2 of user B are constructed; the position of the node of S2 on C2 is marked; the indirect path J of user A is obtained and compared with the unmarked entity segments in C2; the nodes corresponding to the entity segments that are not completely consistent with the indirect path J are determined as the key points for screening.

[0096] Specifically, assume that user A's initial entity path C1 is:

[0097] C1:e1→e2→e3→e4→e5→e6→e7;

[0098] The actual entity path S1 of user A is:

[0099] S1: e1→e5→e7;

[0100] It can be seen that the indirect path J is: e1→e2→e3→e4→e5 and e5→e6→e7;

[0101] Assume that user B's initial entity path C2 is:

[0102] C2:e1→e2→e5→e4→e3→e4→e5→e6→e7;

[0103] The actual entity path S2 of user B is:

[0104] S1: e1→e5→e7;

[0105] Obtain the indirect path J of user A and compare it with the unmarked entity line segments in C2; determine the nodes corresponding to the entity line segments that are not completely consistent with the indirect path J as the screening key points; thus, it can be seen that the first appearance of e5 is used as the screening key point.

[0106] Step S400 includes:

[0107] Obtain the historical behavior data of the second target user, filter the historical behavior data corresponding to the filtering key points of the second target user according to the filtering key points and in combination with the associated data group, calculate the similarity between the filtered historical behavior data and the historical behavior data of the first target user, and arrange the corresponding recommendation strategies of the first target user in descending order according to the similarity calculation result; based on the recommendation strategy of the first target user, generate a corresponding advertising case recommendation sequence for the second target user.

[0108] Based on the filtered historical behavior data and combined with the recommendation strategy of the first target user, an advertising case recommendation sequence for the second target user is generated, thus realizing personalized advertising case recommendation.

[0109] Step S500 includes:

[0110] S501. Obtain the generated advertising case recommendation sequence for the second target user and the advertising case sequence actually selected by the second target user, mark the generated advertising case recommendation sequence for the second target user as X1, and mark the advertising case sequence actually selected by the second target user as X2; based on the advertising case recommendation sequence, obtain the corresponding comprehensive advertising case feature representation, and calculate the similarity D between the advertising case recommendation sequences corresponding to X1 and X2 according to the comprehensive advertising case feature representation;

[0111] S502. According to the advertising case recommendation sequence and similarity D corresponding to X1 and X2, the advertising case recommendation effect for the second target user is evaluated, and the advertising case recommendation index Z is calculated. The specific calculation formula is:

[0112] Z=D×[N(X1∩X2) / N(X1∪X2)];

[0113] Among them, N(X1∩X2) represents the number of identical advertising cases in the advertising case recommendation sequence corresponding to X1 and X2, and N(X1∪X2) represents the total number of advertising cases in the advertising case recommendation sequence corresponding to X1 and X2; the advertising case recommendation index Z is compared with the corresponding advertising case recommendation index threshold Z0. If Z≥Z0, it means that the advertising case recommendation sequence for the current second target user meets the expected effect; if Z<Z0, it means that the advertising case recommendation sequence for the current second target user does not meet the expected effect, then the corresponding prompt information is output and the relevant personnel shall perform corresponding processing.

[0114] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0115] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for recommending similar advertising cases based on multimodal data, characterized in that: The method comprises the following steps: Step S100. Collect multimodal data of advertising cases, construct an advertising case knowledge graph based on the multimodal data, and obtain low-dimensional vector representations of entities and relationships based on the advertising case knowledge graph; the multimodal data includes text data, image data, and video data; obtain a text vector representation of the advertising case based on the text data in the advertising case using a text vector embedding model; fuse the low-dimensional vector representation of the entity and relationship with the advertising case text vector representation to obtain a comprehensive advertising case feature representation; Step S200. Obtain the historical behavior data of the user and the user's usage information of the recommended advertising case, and classify the user into a first target user and a second target user based on the user's usage information of the recommended advertising case; Step S300. Combine the user's usage information of the recommended advertising case, compare the historical behavior data of the first target user with the historical behavior data of the second target user, so as to obtain the screening key points; Step S400. Obtain historical behavior data of the second target user, and filter the historical behavior data of the second target user based on the filtering key points; generate a corresponding advertising case recommendation sequence according to the filtered historical behavior data; Step S500. Analyze the generated advertising case recommendation sequence and the advertising case actually selected by the second target user, evaluate the advertising case recommendation effect, and thus obtain the advertising case recommendation index; and perform corresponding processing according to the size relationship between the advertising case recommendation index and the corresponding threshold; The process of fusing the low-dimensional vector representation of entities and relationships with the vector representation of the advertisement case text in step S100 is as follows: For each advertising case, obtain the low-dimensional vector representation of the relevant entities and attribute relationships from the advertising case knowledge graph. One advertising case involves entities (e1, e2, ...) and relationships (r1, r2, ...), and the corresponding vector representations are: V_ei and V_rj; i and j represent the indexes of the entity and relationship respectively; ei represents the i-th entity, and rj represents the j-th relationship; use the text vector embedding model to convert the text data of the advertising case into a text vector representation, and the text vector is T; Define an attention mechanism so that during the fusion process, the text vector T and the vector representation (V_ei, V_rj) of each corresponding entity relationship pair (ei, rj) are dynamically weighted fused. The specific calculation process is as follows: Calculate the attention weight between the text vector T and the corresponding vector representation (V_ei, V_rj) of each entity relationship pair (ei, rj). The specific calculation formula is: α ij =exp(score(T,V_ei,V_rj)) / Σ (i’,j’) exp(score(T,V_ei’,V_rj’)); Among them, α ij Represents the attention weight, which is used to dynamically weight the fusion of the text vector T and the vector representations V_ei and V_rj of the i-th entity ei and the j-th relationship rj; exp(score(T,V_ei,V_rj)) represents the correlation score between the text vector T and the entity ei and the relationship rj; Σ i’,j’ exp(score(T,V_ei',V_rj')) represents the sum of the scoring function values ​​of all entities ei' and relations rj'; Use attention weight α ij The entity vector V_ei and the relationship vector V_rj are weighted averaged to obtain the comprehensive advertising case feature representation V_case, and V_case=Σ (i,j) α ij (V_ei⊕V_rj); Step S200 includes: S201. Obtain the user's historical behavior data and the user's usage information of the recommended advertising case. For each user, obtain the corresponding advertising case based on the historical behavior data, extract the corresponding entity in combination with the advertising case knowledge graph, and record the number of extracted entities, and each advertising case corresponds to a number of entities; summarize the number of advertising cases obtained based on the user's historical behavior data and the number of corresponding entities, so as to obtain data point A (a1, b1), where a1 represents the number of advertising cases obtained from the user's historical behavior data, and b1 represents the number of entities corresponding to the advertising case; S202. Based on the user's usage information of the recommended advertising case, the browsing data of the user for the advertising case is obtained, and the browsing data includes the number of advertising cases browsed by the user and the browsing time of the corresponding advertising case; by comparing the browsing time with the preset browsing time, the advertising cases with a browsing time greater than or equal to the preset browsing time are screened, and based on the screening result, the number of advertising cases with a browsing time greater than or equal to the preset browsing time and the corresponding entities are summarized, thereby obtaining data point B (a2, b2), where a2 represents the number of advertising cases with a browsing time greater than or equal to the preset browsing time, and b2 represents the number of entities corresponding to the advertising cases with a browsing time greater than or equal to the preset browsing time; S202. Obtain the corresponding data points A(a1, b1) and B(a2, b2) for each user, and calculate the difference R between the two data points. The specific calculation formula is: R=(a1-a2) 2 +(b1-b2) 2 ; Compare the difference R with the difference threshold R0. If R<R0, the corresponding user is classified as the first target user; if R≥R0, the corresponding user is classified as the second target user; The step S300 includes: S301. For each first target user and second target user, the corresponding historical behavior data and the corresponding user's usage information on the recommended advertising case are obtained, and according to the corresponding advertising case, the historical behavior data and the user's usage information on the recommended advertising case are associated to form an associated data group, and each associated data group contains the advertising case number, historical behavior data and the user's usage information on the recommended advertising case; according to the associated data group and the advertising case knowledge graph, the initial entity path C and the actual entity path S corresponding to each user are obtained, and in the initial entity path C and the actual entity path S, the entity represents a node, and the line segment connecting the entity only represents the connection relationship, and the length of the line segment connecting all entities is equal; the initial entity path refers to the entity connection path containing the advertising cases corresponding to all historical behavior data; the actual entity path refers to the entity connection path corresponding to the user's usage information on the recommended advertising case; and the entities corresponding to the entity path can all be found on the initial entity path, and the number of entities corresponding to the initial entity path is greater than the number of entities corresponding to the actual path; S302. For all first target users, obtain the corresponding initial entity path C1 and actual entity path S1, mark the nodes of the actual entity path S1 on the initial entity path C1, and use the entities not marked on the initial entity path C1 as indirect nodes, and use the line segment composed of the nodes adjacent to and continuously connected to the indirect nodes as the indirect path J, and associate the historical behavior data corresponding to the indirect path J; for all second target users, obtain the corresponding initial entity path C2 and actual entity path S2, mark the nodes of the actual entity path S2 on the initial entity path C2, obtain the indirect path J corresponding to the first target user, and compare it with the entity line segments not marked on the initial entity path C2; use the nodes corresponding to the entity line segments that are not completely consistent with the indirect path J as the screening key points; The step S400 includes: Obtain the historical behavior data of the second target user, filter the historical behavior data corresponding to the filtering key points of the second target user according to the filtering key points and in combination with the associated data group, calculate the similarity between the filtered historical behavior data and the historical behavior data of the first target user, and arrange the corresponding recommendation strategies of the first target user in descending order according to the similarity calculation result; based on the recommendation strategy of the first target user, generate a corresponding advertising case recommendation sequence for the second target user.

2. The method for recommending similar advertisements based on multimodal data according to claim 1, characterized in that: The step S500 includes: S501. Obtain the generated advertising case recommendation sequence for the second target user and the advertising case sequence actually selected by the second target user, mark the generated advertising case recommendation sequence for the second target user as X1, and mark the advertising case sequence actually selected by the second target user as X2; based on the advertising case recommendation sequence, obtain the corresponding comprehensive advertising case feature representation, and calculate the similarity D between the advertising case recommendation sequences corresponding to X1 and X2 according to the comprehensive advertising case feature representation; S502. According to the advertising case recommendation sequence and similarity D corresponding to X1 and X2, the advertising case recommendation effect for the second target user is evaluated, and the advertising case recommendation index Z is calculated. The specific calculation formula is: Z = D × [N (X1 ∩ X2) / N (X1 ∪ X2)]; Among them, N(X1∩X2) represents the number of identical advertising cases in the advertising case recommendation sequence corresponding to X1 and X2, and N(X1∪X2) represents the total number of advertising cases in the advertising case recommendation sequence corresponding to X1 and X2; the advertising case recommendation index Z is compared with the corresponding advertising case recommendation index threshold Z0. If Z≥Z0, it means that the advertising case recommendation sequence for the current second target user meets the expected effect; if Z<Z0, it means that the advertising case recommendation sequence for the current second target user does not meet the expected effect, then the corresponding prompt information is output and the relevant personnel shall perform corresponding processing.

3. An advertising case recommendation system based on multimodal data, applied to an advertising case recommendation method based on multimodal data as claimed in any one of claims 1 to 2, characterized in that: The system includes: a data collection and knowledge graph construction module, a user classification and behavior data acquisition module, a key point screening and historical behavior data processing module, and a recommendation generation and effect evaluation module; The data collection and knowledge graph construction module constructs an advertising case knowledge graph based on multimodal data, extracts entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; uses a text vector embedding model to obtain a text vector representation of the advertising case, and fuses it with entity and relationship vectors to obtain a comprehensive advertising case feature representation; The user classification and behavior data acquisition module classifies users into first target users and second target users according to the user's behavior data and usage information, and extracts the user's browsing behavior data and corresponding advertising case entity information; The key point screening and historical behavior data processing module compares the historical behavior data of the first target user and the second target user based on the user behavior data, thereby obtaining the screening key points; and screening the historical behavior data of the second target user according to the screening key points; The recommendation generation and effect evaluation module generates a corresponding advertising case recommendation sequence based on the historical behavior data of the screened second target user; compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, calculates the advertising case recommendation index, and determines whether the recommendation effect meets expectations based on a set threshold, and outputs corresponding prompt information.

4. The advertising case recommendation system based on multimodal data according to claim 3, characterized in that: The data collection and knowledge graph construction module includes a data collection unit, a knowledge graph construction unit and a vector representation generation unit; The data collection unit is responsible for collecting multimodal data from advertising cases, including text, image and video data; the knowledge graph construction unit constructs an advertising case knowledge graph based on the collected multimodal data, which includes modeling of entity and attribute relationships, and generates low-dimensional vector representations of entities and relationships; the vector representation generation unit uses text vector embedding model technology to convert the text data of the advertising case into an advertising case text vector representation, and fuses the low-dimensional vector representation of entities and relationships with the advertising case text vector representation to obtain a comprehensive advertising case feature representation.

5. The advertising case recommendation system based on multimodal data according to claim 3, characterized in that: The user classification and behavior data acquisition module includes a behavior data acquisition unit and a user classification unit; The behavior data acquisition unit is responsible for acquiring the user's historical behavior data and usage information of the recommended advertising case; the user classification unit classifies the user into a first target user and a second target user based on the user's usage information of the recommended advertising case; The key point screening and historical behavior data processing module includes a key point screening unit and a historical behavior data processing unit; The key point screening unit combines the user's usage information of the recommended advertising case, compares the historical behavior data of the first target user and the second target user, and obtains the screening key points; the historical behavior data processing unit screens and processes the historical behavior data of the second target user based on the screening key points.

6. The advertising case recommendation system based on multimodal data according to claim 3, characterized in that: The recommendation generation and effect evaluation module includes a recommendation generation unit, an effect evaluation unit and a processing strategy unit; The recommendation generation unit generates a corresponding advertising case recommendation sequence based on the processed user characteristics and behavior data and the advertising case knowledge graph; The effect evaluation unit compares the generated advertising case recommendation sequence with the advertising case actually selected by the user, evaluates the recommendation effect, and calculates the advertising case recommendation index; The processing strategy unit performs corresponding processing according to the size relationship between the advertisement case recommendation index and the corresponding threshold value.

Images