A big data information processing method

Abstract

The application discloses a big data information processing method, relates to data processing, acquires a plurality of similar products i of a product to be promoted p, and acquires product data of the similar products i; a product-user heterogeneous graph network is constructed according to the product data of the product to be promoted p and the similar products i; message passing calculation is performed on the heterogeneous graph network by using a graph neural network, and all product vectors fused with context information are acquired, the product vectors including a product to be promoted vector z p and each similar product vector z i ; according to the product vectors, direct similarity Sim1(p, i) and indirect similarity Sim2(p, i) between the product to be promoted p and each similar product i are calculated; and a comprehensive score of the product to be promoted p is obtained through weighted fusion according to the direct similarity Sim1(p, i) and the indirect similarity Sim2(p, i). The application effectively fuses the direct similarity and the indirect similarity through an attention mechanism and a target-oriented random walk strategy, and realizes mining of deep-level correlations between products, so that the problem that implicit correlation between products is neglected in promotion of a new product is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a big data information processing method. BACKGROUND

[0002] With the rapid development of the Internet and e-commerce, the global digital marketing market is expanding. The successful promotion of new products is crucial for enterprises to maintain market competitiveness. The main challenges faced by new product promotion include low user awareness, uncertain market acceptance, and limited promotion resources. The rise of big data and artificial intelligence technology provides new solutions for new product promotion. In particular, in the field of product similarity calculation and recommendation systems, multi-modal data fusion and graph structure data mining technology shows great potential.

[0003] The existing new product promotion technology mainly has the following problems and limitations: most of the existing methods ignore the implicit relationships between products reflected by user interaction behavior. Users' browsing, collecting, and purchasing behaviors contain rich product association information, but this information has not been fully utilized in existing technology.

[0004] For example, related patent CN118864008A discloses a personalized advertisement pushing method based on product data, which includes: obtaining product data of a product to be promoted; the product data contains basic attributes of the product to be promoted and sales volume statistics of the product to be promoted; determining at least two similar products of the product to be promoted, and determining product data of each of the at least two similar products; the product to be promoted and the at least two similar products belong to the same product category; calculating the weight coefficients of the at least two similar products, and using data sorting to extract the top k weight coefficients from all data weight coefficients, and determining the similar products corresponding to the top k weight coefficients as recommended products; k is an integer greater than or equal to 1; the product data corresponding to the recommended products is displayed on the user's display desktop or browser page. However, this scheme only relies on explicit data such as product basic attributes and sales volume, lacks deep mining ability for product association, and cannot discover non-obvious similar products. SUMMARY

[0005] In view of the fact that existing technology ignores the implicit relationship between products in new product promotion, the present application provides a big data information processing method, which simultaneously utilizes product text data, image data, and user interaction behavior to construct a multi-relation heterogeneous graph network, effectively fuses direct similarity and indirect similarity through attention mechanism and target-oriented random walk strategy, realizes the mining of deep-level association between products, and improves the promotion accuracy of new products.

[0006] The application provides a big data information processing method, comprising: obtaining product data of a product to be promoted p, the product data comprising text data and image data of the product to be promoted; obtaining a plurality of similar products i of the product to be promoted p, and obtaining product data of the similar products i; constructing a product-user heterogeneous graph network according to the product data of the product to be promoted p and the similar products i, wherein a plurality of types of edge relationships are defined between product nodes, and the plurality of types of edge relationships comprise functional similarity calculated based on the text data and visual similarity calculated based on the image data; performing message passing calculation on the heterogeneous graph network by using a graph neural network to obtain all product vectors fused with context information, wherein the product vectors comprise a product to be promoted vector z p and each similar product vector z i ; calculating direct similarity Sim1(p, i) and indirect similarity Sim2(p, i) between the product to be promoted p and each similar product i according to the product vectors; and obtaining a comprehensive score of the product to be promoted p through weighted fusion according to the direct similarity Sim1(p, i) and the indirect similarity Sim2(p, i).

[0007] Further, the construction of the product-user heterogeneous graph network comprises: performing natural language processing on the text data of the product to be promoted p and the similar products i to extract product functional features and construct text feature vectors; processing the image data of the product to be promoted p and the similar products i by using a convolutional neural network to extract product visual features and construct image feature vectors; calculating the cosine similarity between the text feature vectors as the functional similarity, and establishing a functional similarity edge between two product nodes when the functional similarity of the two product nodes is greater than a preset threshold; calculating the Euclidean distance between the image feature vectors as the visual similarity, and establishing a visual similarity edge between two product nodes when the visual similarity of the two product nodes is greater than a preset threshold; collecting user-product interaction data to construct user nodes, and establishing an interaction edge between a user node and a corresponding product node when the user and the product have an interaction behavior; and integrating the product nodes and the user nodes to construct the heterogeneous graph network.

[0008] In particular, new product promotion often has a serious data sparsity problem due to the lack of user interaction history and preference data, and traditional flat feature representation regards the product as an isolated vector, and the information transmission capacity is limited. The application constructs a multi-modal heterogeneous graph network and designs a meta-path-aware graph neural network, on the one hand, in graph theory, the topological position of a node itself contains rich structural information, and the information density is higher than that of a vector representation. By embedding the product position into the graph structure, even without direct interaction data, implicit semantics can be extracted from the topological relationship.

[0009] Further, the product vectors fused with context information comprise: constructing a plurality of meta-paths Φ rEach meta-path Φ r represents the connection mode of different types of nodes and edges in the graph; multiple meta-paths Φ r include: a meta-path Φ1 of product node-user node-product node, representing the product relationship that a user has interaction behavior with two different product nodes at the same time; a meta-path Φ2 of product node-function similarity edge-product node, representing the product relationship connected by the function similarity edge; a meta-path Φ3 of product node-visual similarity edge-product node, representing the product relationship connected by the visual similarity edge.

[0010] In particular, the key limitation of traditional graph neural networks is the insufficient processing of semantic differentiation. The present application constructs meta-paths, three core meta-paths (product-user-product, product-function-product, and product-visual-product), which are equivalent to creating three different relationship projection spaces. Through these projection spaces, multiple implicit association patterns that may exist between products can be captured. The information transmission process along a specific meta-path can be understood as a conditional random walk with semantic constraints, which ensures the semantic consistency of information and prevents irrelevant information from interfering.

[0011] Using a graph neural network, each product node v n in the heterogeneous graph network is processed. r The meta-path-based message aggregation calculation is performed according to different types of meta-paths Φ r The aggregated neighbor node information is fused with the node itself information to obtain the product vector with fused context information, including: for each meta-path Φ n , for the product node v n , the set of all neighbor nodes reachable along the meta-path Φ r is obtained According to the set , the product node v n and each neighbor node v r on the corresponding meta-path Φ m are obtained, the feature vectors of nodes v n and v m are input into the shared attention calculation neural network to obtain the attention coefficient a nm representing the importance between nodes; according to the attention coefficient a nm , the feature vectors of all neighbor nodes of the product node v n are weighted and summed to obtain the semantic representation of the node v n under the meta-path Φ r The semantic representation of the product node v n under each meta-path Φ r Each meta-path Φ​​r importance weight w of the current product to be promoted r ; according to the importance weight w r , the product node v n In the semantic representation of each meta path weighted sum, get the same product vector z i When the product node v n is the product node to be promoted, get the product vector z p to be promoted

[0012] Further, the direct similarity degree between the product to be promoted and each same product is calculated, including: obtaining the product vector z p to be promoted and each same product vector z i ; calculate the direct similarity Sim1(p, i) of the product vector z p to be promoted and each same product vector z i , the calculation formula is: Wherein, ||z p || and ||z i || respectively represent the Euclidean norm of the vector.

[0013] Further, the direct similarity degree between the product to be promoted and each same product is calculated, including: obtaining the product vector z p to be promoted and each same product vector z i ; calculate the direct similarity Sim1(p, i) of the product vector z p to be promoted and each same product vector z i , the calculation formula is: Wherein, ||z p || and ||z i || respectively represent the Euclidean norm of the vector.

[0014] Calculate the indirect similarity, including: set the maximum jump rope K of the heterogeneous graph network as the upper limit of the path length; according to the connection relationship between the nodes in the heterogeneous graph network, construct the transition probability matrix P reflecting the node to each neighbor node, including: extract the connection information between all node pairs in the heterogeneous graph network G, and according to the type of edge, it is divided into functional similarity edge set E func , visual similarity edge set E visual and user interaction edge set E user ; for functional similarity edge, based on the calculated cosine similarity value, the edge weight w func (v n ,v m ) is valued as the corresponding similarity value, wherein v n and v mconnected product node pairs; for visual similarity edges, convert them into similarity metrics based on the calculated Euclidean distance value, obtaining edge weights w visual (v n ,v m ), the conversion formula is: where d(v n ,v m ) represents the Euclidean distance, and sigma is a scaling parameter; for user interaction edges, calculate the edge weight w user (u,v) according to the interaction frequency f(u,v) between the user and the product, where u represents a user node and v represents a product node; for each node v n in the heterogeneous graph network G, collect the weights of all its outgoing edges {w(v n ,v m )} and perform normalization processing to obtain the transition probability P(v n ,v m ) from the node v n to each of its neighbor nodes v m , the calculation formula is: where v k iterates through all neighbor nodes of v n ; organize the transition probabilities P(v n ,v m ) between all node pairs into a matrix form to form a complete transition probability matrix P, which is used for subsequent random walk path sampling.

[0015] Take the direct similarity Sim1(p,i) as prior knowledge to construct the target-oriented function h(v n ,i) for evaluating the structural and semantic similarity between the node v n and the target node i, including: obtaining the vector representation z n of the intermediate node v n and the vector representation z i of the target node i, and calculating the preliminary cosine similarity Take the direct similarity Sim1(p,i) between the product to be promoted p and the target product i as an adjustment factor to construct an enhanced vector similarity calculation formula: sim vec (v n ,i) = cos sim (v n ,i) x [1 + mu x Sim1(p,i) x f(v n ,p)], where mu is a weight coefficient, and f(v n ,p) is a decay function reflecting the structural association of the node v n with the product to be promoted p; based on the topological structure of the heterogeneous graph network G, calculate the intermediate node v nsim(v struct (v n ,i) is obtained by analyzing the number of common neighbors of two nodes and their location relationship in the network; the intermediate node v n sim(v path (v n ,i) is obtained by calculating the reachability within a limited hop range; according to the size of the direct similarity Sim1(p, i), the weight parameters λ1, λ2, and λ3 are dynamically adjusted, and when Sim1(p, i) is higher, the value of λ1 is increased to strengthen the influence of vector similarity, and vice versa; the three similarity indexes are integrated to construct the target-oriented function h(v n ,i), and the calculation formula is: h(v n ,i) = λ1 × sim vec (v n ,i) + λ2 × sim struct (v n ,i) + λ3 × sim path (v n ,i), wherein λ1, λ2, and λ3 are weight parameters, and λ1 + λ2 + λ3 = 1; the value of h(v n ,i) obtained by calculation is normalized to ensure that its value range is between [0, 1], which is convenient for subsequent combination with the transition probability matrix P to form an effective search strategy.

[0016] In particular, the traditional random walk is completely random, and the fixed transition probability is used to walk in the graph without a clear target, but there are serious defects of low computational efficiency and insufficient result relevance in dealing with new product promotion. Specifically, the traditional random walk is based on a Markov chain model, and the transition probability of the current node only depends on the local connection structure without considering the global target, resulting in a large waste of computing resources in the exploration of irrelevant paths. In addition, the traditional random walk is an entropy increasing process, and with the increase of the number of steps, the system entropy increases continuously, and the walking track becomes more and more random, which reduces the probability of discovering meaningful paths. Finally, the computational efficiency decreases exponentially with the path length, which is particularly poor in the multi-hop path exploration scene, and it is difficult to apply to the new product promotion scene which needs to discover long path correlation.

[0017] In this application, on the one hand, the direct similarity is injected into the random walk process as prior knowledge, which is equivalent to introducing a prior probability distribution from the perspective of Bayesian inference, so that the random process changes from “uninformed exploration” to “informed guidance”, forming a “knowledge-guided exploration” mode. On the other hand, the biased transition probability provides a framework for dynamically adjusting the transition probability matrix P according to the target-oriented function h(v An accurate balance between the original structure (exploration) and the target-oriented (utilization) of the graph solves the classic "exploration-utilization" dilemma in reinforcement learning. On the other hand, the present application explores the implicit high-order connection between products through multi-hop path discovery. Even if the new product lacks direct user interaction, potential associations can be discovered through network structure, and the association link construction of "new product→similar product→active user" can be realized through multi-hop path discovery; it can migrate user group knowledge from products with established market position to new products.

[0018] For each similar product i, according to the transition probability matrix P and the target-oriented function h(v n ,i), through the preset exploration-balance parameter , the biased transition probability P' is constructed, and the target-oriented random walk strategy is executed under the constraint of the maximum number of hops K to obtain the connection path set Path set (p,i) from product p to product i; the weight of each path in the obtained path set Path set (p,i) is calculated, and the weight of all paths is weightedly averaged to obtain the indirect similarity Sim2(p,i) between the product to be promoted p and the similar product i.

[0019] For each similar product i, according to the transition probability matrix P and the target-oriented function h(v n ,i), through the preset exploration-balance parameter , the biased transition probability P' is constructed, and the target-oriented random walk strategy is executed under the constraint of the maximum number of hops K to obtain the connection path set Path set (p,i) from product p to product i, including: setting the product node p to be promoted as the starting node of the random walk, and setting the similar product node i as the target node; according to the transition probability matrix P and the target-oriented function h(v n ,i), the exploration-utilization balance parameter is used to construct the biased transition probability P'(v n ,v m ), and the calculation formula is: Wherein, control the degree of dependence on known transition probability and the degree of inclination to target-oriented in the random walk process; initialize the path counter count=0 and the maximum path number threshold to control the number of sampled paths; start from the starting node p, and at each step according to the current node v n , select the next node to be accessed v n , according to the biased transition probability P'(v m ,v m ,; record the node sequence in the random walk process to form a path starting from p; when the path length reaches the maximum hop count K or the current node is the target node i, end the generation of the current path; if the generated path successfully connects the starting node p and the target node i, add the path to the path set Path set (p,i) and add 1 to the counter count; repeat until count reaches threshold or other termination conditions are met, ultimately obtaining the connection path set Path from product p to product i set (p,i) and add 1 to the counter count; repeat until count reaches threshold or other termination conditions are met, ultimately obtaining the connection path set Path from product p to product i set (p,i) and add 1 to the counter count; repeat until count reaches threshold or other termination conditions are met, ultimately obtaining the connection path set Path from product p to product i

[0020] In particular, the target-oriented random walk in the present application reduces the brute-force search complexity of O(b d ) (b is the average branching factor and d is the path length) to approximately O(b x d), making long path discovery computationally feasible.

[0021] Compared with the prior art, the present application has the following advantages:

[0022] On the one hand, the prior art often uses traditional collaborative filtering or content matching algorithms, but when there is insufficient user interaction data for new products, traditional collaborative filtering methods cannot provide effective recommendations. The present application constructs a heterogeneous graph network by fusing text data, image data and user interaction data, and uses a meta-path-based attention mechanism to realize bidirectional information flow under different semantic relationships, achieving multi-dimensional feature representation of new products and effectively solving the cold start problem.

[0023] On the other hand, the prior art often only considers the direct similarity between products, resulting in the defects of single recommendation results and inability to discover potential related products. The present application calculates indirect similarity through a target-oriented random walk strategy, and mines implicit association paths and deep semantic relationships between products. In addition, traditional path mining algorithms generally use the method of enumerating all paths, but for product recommendation, directly enumerating all paths in a large-scale graph network will lead to computational explosion. The present application realizes efficient implicit association path search by fusing transition probability matrix and target-oriented function, significantly reducing computational complexity while ensuring recommendation quality. BRIEF DESCRIPTION OF DRAWINGS

[0024] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same reference numbers represent the same structures, wherein:

[0025] Figure 1 is an exemplary flowchart of a big data information processing method according to some embodiments of the present application;

[0026] Figure 2 is an exemplary flowchart of constructing a heterogeneous graph network according to some embodiments of the present application;

[0027] Figure 3 is an exemplary flowchart of generating indirect similarity according to some embodiments of the present application. DETAILED DESCRIPTION

[0028] The method and system provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0029] As shown in Figure 1 , product data of a product to be promoted p is obtained, and the product data includes text data and image data of the product to be promoted; specifically, product titles, descriptions, specification parameters and other text information are grabbed from an e-commerce platform, a product database or an enterprise resource planning (ERP) system through an API interface. The obtained original text data is preprocessed through steps such as word segmentation, stop word removal and stemming, and is converted into standardized text corpus. Product main images, detail images and other image resources are collected and preprocessed through uniform size scaling (such as adjusting to 224x224 pixels), color standardization, data enhancement and the like. The image preprocessing adopts a standard process in computer vision to ensure the consistency of the input of the subsequent convolutional neural network processing.

[0030] A plurality of similar products i of the product to be promoted p are obtained, and product data of the similar products i is obtained; specifically, a classification path (such as “electronic products→mobile phones→smartphones→5G smartphones”) of the product to be promoted p is first extracted from a product classification system, and then a preliminary candidate set is screened out within the most detailed classification level (such as “5G smartphones”). This fast filtering based on the classification tree can reduce the search space from the entire product library (which may contain millions of products) to a specific category (usually a few hundred to a few thousand products), significantly improving the efficiency of subsequent fine matching.

[0031] Semantic expansion is performed using a pre-constructed product ontology (Product Ontology). The ontology includes relationships such as “is-a”, “part-of”, “functional-equivalent” and the like between products. Through an ontology reasoning engine, the system can identify products that are similar in function although they are classified differently, for example, although “Bluetooth earphones” and “wireless earphones” may be located in different classifications, they have a “functional-equivalent” relationship in the ontology, and therefore will be included in the candidate set.

[0032] The feature vector (combined from text and image features) of the product to be promoted p is input into an efficient vector database (such as FAISS or Milvus), a K-nearest neighbor (KNN) search is performed, and the N closest products in the vector space are obtained. Technically, an approximate nearest neighbor algorithm (such as IVFPQ based on product quantization) is used to achieve efficient retrieval of large-scale high-dimensional vectors, and the search complexity is reduced from O(n d ) to O(n 0.5d ), where n is the number of products and d is the vector dimension.

[0033] Preferably, to avoid information redundancy caused by too similar recommended results, the system uses a deterministic greedy algorithm (Determinantal Point Process, DPP) to optimize the diversity of the initial ranking results. This algorithm ensures that the final recommended similar products present diversity in key attributes (such as price range, brand, special features, etc.) while maintaining relevance, improving the user's selection space.

[0034] For the final determined similar product set (usually 10-20 products), the system extracts its complete information from the product database through a parallel data acquisition mechanism, including: basic attribute data: product ID, name, brand, price, etc. Structured information; rich text description: product detailed description, specification parameters, selling point characteristics, etc.; image resources: product main picture, detail picture, application scenario picture, etc. Visual materials; user feedback data: evaluation data, purchase behavior data, etc.

[0035] As shown in Figure 2 , in this embodiment, a platform needs to promote a new "XYZ Pro" smartphone. The platform has a large amount of historical product data and user interaction records, and now needs to build a product-user heterogeneous graph network to provide accurate promotion support for the new product.

[0036] For the "XYZ Pro" smartphone and its similar products, the system collects the following text data: product title: "XYZ Pro 5G smartphone 8GB+256GB super light three-camera 6.7-inch full-screen"; product description: contains detailed function introduction, technical specifications, use scenarios, etc.; user evaluation: evaluation text from early testers; technical parameter table: processor, memory, battery, etc. Structured parameters; the system uses an industry-adapted BERT variant model to process these texts. In the preprocessing stage, the system performs word segmentation, stop word removal, and entity standardization (such as standardizing "8GB+256GB" to storage parameters). Then, the product feature extractor in the specific field identifies key function points (such as "super light three-camera", "full-screen", etc.) and assigns appropriate weights. Finally, the system generates a 768-dimensional text feature vector that encodes the product's functional characteristics, performance parameters, and application scenarios.

[0037] For each mobile phone product, the system collects the following image data: product main view: showing the front and back appearance of the phone; detailed close-up: close-up of camera module, screen display, interfaces, etc.; usage scenario view: showing the application effect of the product in different scenarios; the system uses a pre-trained ResNet101 model, which has been fine-tuned on a large number of electronic product images. The image is first preprocessed: adjusted to a uniform resolution (224x224 pixels), brightness and contrast standardized, data augmentation (rotation, flipping) to enhance model robustness. After processing by the convolutional neural network, the system extracts a 2048-dimensional feature vector, which contains visual features of the product, such as design style, color scheme, material feel, appearance structure, etc.

[0038] The system calculates the cosine similarity of the text feature vectors of "XYZ Pro" and candidate similar products (such as "ABC Ultra", "DEF Max" and other competing products). For example: sim_func("XYZ Pro" "ABC Ultra") = 0.82 (both products focus on high-end photography and large screens); sim_func("XYZ Pro" "DEF Max") = 0.76 (function positioning is close but emphasis is different); sim_func("XYZ Pro" "GHI Lite") = 0.58 (one is a high-end flagship, one is an entry-level model); the system sets the function similarity threshold θ func = 0.75, so a function similarity edge is established between "XYZ Pro" and "ABC Ultra" and "XYZ Pro" and "DEF Max", with edge weights of 0.82 and 0.76 respectively. These edges reflect the degree of functional approximation between products, providing channels for subsequent information propagation.

[0039] Similarly, the system calculates the visual similarity between products: sim_vis("XYZ Pro" "ABC Ultra") = 0.69 (both products have similar design styles); sim_vis("XYZ Pro" "JKL Neo") = 0.77 (although the function positioning is different, the appearance design is very similar); sim_vis("XYZ Pro" "DEF Max") = 0.62 (appearance design difference is large); the system sets the visual similarity threshold θ vis = 0.65, so a visual similarity edge is established between "XYZ Pro" and "ABC Ultra" and "XYZ Pro" and "JKL Neo", with edge weights of 0.69 and 0.77 respectively. These visual similarity edges capture the similarity relationship of products in appearance design, providing a basis for product recommendation for users with similar visual preferences.

[0040] The system collects the following types of user interaction data: purchase behavior: user U1 purchased "ABC Ultra", user U2 purchased "DEF Max"; browsing behavior: user U3 browsed the detail pages of "XYZ Pro" and "JKL Neo"; collection behavior: user U4 added "XYZ Pro" and "ABC Ultra" to the collection; evaluation behavior: user U5 evaluated "DEF Max"; the system assigns different weights to different interaction types (purchase: 1.0, collection: 0.7, evaluation: 0.6, browsing: 0.3). When the comprehensive interaction intensity of a user with a product exceeds the threshold θ inter = 0.5, an interaction edge is established between the user node and the product node. For example, an interaction edge is established between user U4 and "XYZ Pro" (collection behavior, weight 0.7). For the newly launched "XYZ Pro", due to the relatively sparse user interaction data, the system pays special attention to the user groups of those products that have established functional or visual similarity edges with "XYZ Pro". For example, through the functional similarity edge of "XYZ Pro"→"ABC Ultra", the system can identify users U1 and U4 who have strong interactions with "ABC Ultra", and these users become the high-quality target groups for potential promotion of "XYZ Pro".

[0041] As a newly launched product, "XYZ Pro" faces the typical cold start problem. The system alleviates this problem through the following steps: first, identify the product set P' = {"ABC Ultra", "DEF Max", "JKL Neo"} that has established functional similarity edges or visual similarity edges with "XYZ Pro"; for each product in P', find the user set that has strong interaction with it: U("ABC Ultra") = {U1, U4, U7, U9}; U("DEF Max") = {U2, U5, U8}; U("JKL Neo") = {U3, U6, U10}; the system constructs two-hop paths from "XYZ Pro" to these users: "XYZ Pro"→"ABC Ultra"→U1, U4, U7, U9; "XYZ Pro"→"DEF Max"→U2, U5, U8; "XYZ Pro"→"JKL Neo"→U3, U6, U10; through these paths, the system can calculate the indirect relevance of "XYZ Pro" to these users, and preferentially promote "XYZ Pro" to those users who have strong interactions with similar products, effectively alleviating the cold start problem of new products.

[0042] Finally, the system integrates the above constructed nodes and edges into a unified heterogeneous graph network G: the product node set P: contains "XYZ Pro" and its similar products, a total of about 50 nodes; the user node set U: contains users who interact with these products, about 10,000 nodes; the edge set E: contains three types of edges; the functional similarity edge E func : about 100, connecting functionally similar product pairs; the visual similarity edge E vis : about 80, connecting appearance similar product pairs; the user interaction edge E inter : about 25,000, connecting users and products they have interacted with; this heterogeneous graph network is stored in the form of an adjacency matrix and a feature matrix, and an efficient index structure is established. The system also constructs indexes for three types of meta-paths: product → user → product: reflecting product associations resulting from users' common interaction behaviors; product → functional similarity → product: reflecting product associations in the functional dimension; product → visual similarity → product: reflecting product associations in the visual dimension; this heterogeneous graph structure supports subsequent graph neural network message passing and multi-hop similarity calculation, enabling the system to consider multiple dimensions of information and provide accurate target user identification and similar product recommendation for the promotion of "XYZ Pro".

[0043] Through this multi-dimensional heterogeneous graph network construction method, the system successfully integrates the newly launched "XYZ Pro" smartphone into the existing product-user relationship network, even if it has not accumulated enough user interaction data. This graph-structured product representation and association mining method significantly improves the accuracy and efficiency of new product promotion.

[0044] The system has successfully constructed a product-user heterogeneous graph network centered on the new product "XYZ Pro" smartphone. Next, the graph neural network is used to perform message passing calculations on this heterogeneous graph to obtain product vectors that integrate context information.

[0045] Based on the constructed heterogeneous graph network G, the system defines three types of semantically rich meta-paths: meta-path Φ1 (product-user-product): in the smartphone market, this path captures the product associations implied by user preferences. For example, user U4 has collected both "XYZ Pro" and "ABC Ultra", indicating that these two products are associated in the user's mind. Through this meta-path, the system can discover products that have different specifications but high user group overlap.

[0046] Meta-path Φ2 (Product-Function Similarity-Product): This path reflects the association of products in the functional dimension. As mentioned earlier, "XYZ Pro" establishes a function similarity edge with both "ABC Ultra" and "DEF Max," indicating that these products have similarities in performance specifications, camera functions, screen displays, etc. This meta-path is particularly important for understanding the functional positioning and market segmentation of products.

[0047] Meta-path Φ3 (Product-Visual Similarity-Product): This path reflects the association of products in the design dimension. "XYZ Pro" establishes a visual similarity edge with both "ABC Ultra" and "JKL Neo," indicating that these products are similar in industrial design, material selection, appearance style, etc. In the smartphone market, visual design is one of the key factors affecting user selection.

[0048] For the new product "XYZ Pro" (node v p ), the system obtains its neighbor node set along three meta-paths: Φ1 path neighbors: The system first identifies the user set U direct = {U3, U4} (respectively, browsing and collecting behavior) that has direct interaction with "XYZ Pro," and then finds other products that these users have interacted with, forming the neighbor set

[0049] Φ2 path neighbors: Based on the function similarity edge, directly obtain These two products are highly similar to "XYZ Pro" in the functional dimension.

[0050] Φ3 path neighbors: Based on the visual similarity edge, directly obtain These two products are highly similar to "XYZ Pro" in the design dimension.

[0051] It is worth noting that "ABC Ultra" appears in the neighbor set of the three meta-paths, indicating that it has strong associations with "XYZ Pro" in multiple dimensions.

[0052] The system calculates the attention coefficient for "XYZ Pro" and its neighbor products under each meta-path: Attention calculation for meta-path Φ1 (Product-User-Product): For "XYZ Pro" and "ABC Ultra," the system considers the interaction intensity of user U4 (collection, weight 0.7); for "XYZ Pro" and "JKL Neo," the system considers the interaction intensity of user U3 (browsing, weight 0.3); after neural network calculation and softmax normalization, a(p, ABC) = 0.71, a(p, JKL) = 0.29.

[0053] Attention calculation for metapath Φ2 (product-functional similarity-product): Based on functional similarity and product feature vector, the attention coefficient is calculated; after processing, a(p, ABC) = 0.54 and a(p, DEF) = 0.46 are obtained.

[0054] Attention calculation for metapath Φ3 (product-visual similarity-product): Based on visual similarity and product feature vector, the attention coefficient is calculated; after processing, a(p, ABC) = 0.48 and a(p, JKL) = 0.52 are obtained.

[0055] During the attention calculation process, the system specifically considered the characteristics of "XYZ Pro" as a new product. For example, in the metapath Φ1, despite the sparse interaction data, the system enhanced the attention weight of products that had high-quality interactions with multiple users (such as purchasing rather than just browsing).

[0056] Based on the calculated attention coefficient, the system aggregates the neighbor information for each meta-path:

[0057] Φ1 Path Aggregation: Where h(ABC) and h(JKL) are the initial feature vectors of "ABC Ultra" and "JKL Neo", respectively. The aggregated... The vector reflects the product characteristics that share a common user group with "XYZ Pro".

[0058] Φ2 path aggregation: After aggregation The vector reflects product features similar to those of "XYZ Pro".

[0059] Φ3 path aggregation: After aggregation The vector reflects product characteristics similar to the "XYZ Pro" design.

[0060] To integrate information from different meta-paths, the system implements a semantic-level attention mechanism: First, for each semantically specific representation... A linear transformation is performed using a trainable weight matrix W to generate... Then, a nonlinear transformation is performed on the transformed vector using the hyperbolic tangent function (tanh), resulting in... Next, the transformed result is multiplied by the trainable semantic-level attention vector q to calculate the attention score for each meta-path. Finally, the attention scores {s1,s2,s3,.....} of all metapaths are normalized using the softmax function to obtain the Φ for each metapath. r Importance weight w of the current promotion task r =softmax(si Specifically, for "XYZ Pro", the system analyzes historical promotion data and determines that in the high-end smartphone market, functional features and visual design are the main factors influencing user decisions. Therefore, the importance weights calculated are: w1=0.25 (user behavior path), w2=0.42 (functional similarity path), w3=0.33 (visual similarity path).

[0061] Finally, the system weights and fuses the semantic representations under the three meta-paths according to the importance weights:

[0062] The generated product vector z p The fusion vector combines context information from three dimensions: user behavior association information (25%), functional similarity association information (42%), and visual design similarity association information (33%). This fusion vector not only contains the features of "XYZ Pro" itself, but also incorporates product information related to it in different dimensions, forming a more comprehensive and context-aware representation.

[0063] To enhance information transmission, the system performs bidirectional message passing on the heterogeneous graph network: Forward flow: information transmission from "XYZ Pro" to its neighbor products. For example, "XYZ Pro", as an innovative design of high-end smartphones, its design philosophy and technical features are transmitted to "ABC Ultra" and "JKL Neo", enabling the representations of these products to partially capture the innovative characteristics of "XYZ Pro".

[0064] Reverse flow: information transmission from neighbor products to "XYZ Pro". For example, "ABC Ultra", as a product with a certain user base in the market, its successful functional features and user feedback are transmitted to "XYZ Pro", helping the new product better position the market.

[0065] The system achieves bidirectional flow through multiple rounds of iteration updates, and in each round of iteration, all product nodes update their representations simultaneously. In this process, special attention is paid to paths connecting "XYZ Pro" with popular products to ensure that important market information can be effectively transmitted to the new product.

[0066] As shown in Figure 3 Based on the obtained fusion product vector, the system calculates the similarity between "XYZ Pro" and each similar product: Direct similarity reflects the closeness of product representation vectors in semantic space, but does not consider the implicit association of products in the network structure.

[0067] To explore the indirect associations between products, the system constructs a transition probability matrix P: Functional similarity edge: cosine similarity is used as weight, like w func (p,abc) = 0.82; Visual similarity edge: Euclidean distance is converted to similarity, like w visual (p,jkl) = 0.77; User interaction edge: based on user interaction intensity, like w user (U4,p) = 0.7 (collection behavior).

[0068] For each node v n , collect all its outgoing edge weights and normalize them to get the transition probability:

[0069] For the node p of “XYZ Pro”, its transition probability distribution is: P(p,abc) = 0.28 (based on the integrated similarity of function and vision); P(p,def) = 0.17 (based on functional similarity); P(p,jkl) = 0.23 (based on visual similarity); P(p,U3) = 0.12 (based on browsing interaction); P(p,U4) = 0.20 (based on collection interaction); these transition probabilities constitute the matrix P, which is used for subsequent random walk path sampling.

[0070] The system uses direct similarity as prior knowledge to construct a target-oriented function h(v n ,i): for the intermediate node v n (like “ABC Ultra”) and the target node i (like “DEF Max”), calculate the preliminary cosine similarity, and then adjust it in combination with the direct similarity between “XYZ Pro” and “DEF Max”:

[0071] sim vec (abc,def) = cos sim (abc,def) × [1 + 0.5 × Sim direct (p,def) × f(abc,p)], where f(abc,p) is a decay function reflecting the association strength between “ABC Ultra” and “XYZ Pro”.

[0072] Analyze the common neighbors and network positions of “ABC Ultra” and “DEF Max” to get the structural similarity Sim struct (abc,def). Detect whether there is a short path connection between “ABC Ultra” and “DEF Max” to calculate the path similarity sim path (abc,def). According to the value of Sim1(p,def) = 0.76, the system sets the weight parameters λ1 = 0.5, λ2 = 0.3, and λ3 = 0.2.

[0073] h(abc,def) = 0.5 x sim vec (abc,def) + 0.3 x sim struct (abc,def) + 0.2 x sim path (abc,def), which evaluates the attractiveness from any intermediate node to the target node, guiding the random walk process to find relevant paths more effectively.

[0074] For "XYZ Pro" (p) and each similar product i (such as "DEF Max"), the system performs goal-oriented random walk: combining the standard transition probability P and the goal-oriented function h, construct the biased transition probability P':

[0075] The system sets the exploration-exploitation balance parameter While maintaining the original network structure exploration, increase the tendency to approach the target product.

[0076] Set the maximum hop count K = 3, the maximum path number 1000; starting from "XYZ Pro", select the next node according to P'; for example, a possible path is: "XYZ Pro" → "ABC Ultra" → user U7 → "DEF Max"; record the node sequence during path generation; when the path reaches "DEF Max" or the length reaches K, end the current path generation.

[0077] Collect all paths that successfully connect "XYZ Pro" and "DEF Max", remove duplicates and filter, retain high-quality paths, and finally get the path set Path set (p,def). For "XYZ Pro" and "DEF Max", the system finds multiple meaningful connection paths, including: product function path: XYZ Pro → DEF Max (direct functional similarity); user preference path: XYZ Pro → ABC Ultra → user U1 → DEF Max; visual design path: XYZ Pro → JKL Neo → DEF Max (through design style association).

[0078] Assign weights to each path in Path set (p,def), the weight is inversely proportional to the path length and proportional to the importance of the nodes on the path, calculate the weighted average, get the indirect similarity:

[0079] For "XYZ Pro" and "DEFMax", the system calculates the indirect similarity Sim2(p,def) = 0.64; by similar methods, the system calculates the indirect similarity of "XYZ Pro" with other similar products: Sim2(p,abc) = 0.79; Sim2(p,jkl) = 0.68; Sim2(p,ghi) = 0.43.

[0080] Finally, the system fuses the direct similarity and the indirect similarity to obtain a comprehensive score: Score(p,i) = a x Siml(p,i) + (1-a) x Sim2(p,i). For the smartphone market, the system sets the weight a = 0.6 according to historical data analysis, giving higher weight to the direct similarity while retaining the network structure information reflected by the indirect similarity.

[0081] Finally, the system calculates the comprehensive scores of "XYZ Pro" with each similar product:

[0082] Score(p,abc) = 0.6 x 0.88 + 0.4 x 0.79 = 0.844;

[0083] Score(p,def) = 0.6 x 0.76 + 0.4 x 0.64 = 0.712;

[0084] Score(p,jkl) = 0.6 x 0.72 + 0.4 x 0.68 = 0.704;

[0085] Score(p,ghi) = 0.6 x 0.61 + 0.4 x 0.43 = 0.538.

[0086] Based on these comprehensive scores, the system generates the final promotion strategy for "XYZ Pro": focus on promoting to the user group with the highest similarity to "ABC Ultra"; highlight the common advantages and differentiated characteristics of "XYZ Pro" and "ABC Ultra" in marketing content; secondary targets include the user groups of "DEFMax" and "JKLNeo"; this promotion strategy that combines direct feature similarity and network structure association significantly improves the market penetration rate and user acceptance of "XYZ Pro".

[0087] The above description of the application and its embodiments is illustrative, and not restrictive, and the application can be practiced in other specific forms without departing from the spirit or essential character thereof. The drawings described herein are only schematic and are non-limiting, and the description herein with reference to the attached drawings only serve to explain embodiments of the application and is not intended to limit the scope of the application. Any reference signs in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The implementation of a software element, such as a program, may be through use of digital storage media, of a application specific integrated circuit, a programmable logic device or other hardware. The processing of hard coded device or reprogrammable device programs, software storage media programs, etc. can be done simultaneously with the processing of a computer application in a shared computer environment. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A big data information processing method, characterized by, The method comprises the following steps: obtaining product data of a product to be promoted p, wherein the product data comprises text data and image data of the product to be promoted; obtaining a plurality of similar products i of the product to be promoted p, and obtaining product data of the similar products i; constructing a product-user heterogeneous graph network according to the product data of the product to be promoted p and the similar products i, wherein a plurality of types of edge relationships are defined between product nodes, and the plurality of types of edge relationships comprise functional similarity calculated based on the text data and visual similarity calculated based on the image data; The heterogeneous graph network is subjected to message passing calculation by using a graph neural network to obtain all product vectors fused with context information, and the product vectors include a to-be-promoted product vector and each similar product vector ; According to the product vector, calculate the direct similarity between the product to be promoted p and each similar product i and indirect similarity ; According to the direct similarity and the indirect similarity , a comprehensive score of the product p to be promoted is obtained by weighted fusion. calculating indirect similarity, comprising: setting a maximum jump rope K of the heterogeneous graph network as an upper limit of path length; constructing a transition probability matrix P reflecting the transition of nodes to neighbor nodes according to the connection relationship between nodes in the heterogeneous graph network; direct similarity constructing a goal-oriented function for evaluating the structural and semantic similarity between a node and a goal node i; For each product p to be promoted and each product i of the same kind, according to the transition probability matrix P and the target-oriented function , the biased transition probability is constructed by a preset exploration-balance parameter , and the target-oriented random walk strategy is executed under the constraint of the maximum hop number K to obtain a connection path set from the product p to the product i ;​ The set of paths is calculated The weight of each path is calculated, and the weight of all paths is weightedly averaged to obtain the indirect similarity between the product p to be promoted and the similar product i ； wherein the construction of the product-user heterogeneous graph network comprises: performing natural language processing on the text data of the product to be promoted p and the similar products i, extracting product functional features, and constructing a text feature vector; processing the image data of the product to be promoted p and the similar products i through a convolutional neural network, extracting product visual features, and constructing an image feature vector; calculating the cosine similarity between the text feature vectors as the functional similarity, and establishing a functional similarity edge between two product nodes when the functional similarity of the two product nodes is greater than a preset threshold; calculating the Euclidean distance between the image feature vectors as the visual similarity, and establishing a visual similarity edge between two product nodes when the visual similarity of the two product nodes is greater than a preset threshold; collecting user-product interaction data, constructing a user node, and establishing an interaction edge between the user node and the corresponding product node when the user and the product have an interaction behavior; integrating the product nodes and the user nodes to construct a heterogeneous graph network.

2. The big data information processing method according to claim 1, wherein: obtaining a product vector fused with context information comprises: Multiple meta-paths of constructing heterogeneous graph networks Each meta-path represents a connection pattern of different types of nodes and edges in a graph; Using a graph neural network, each product node in the heterogeneous graph network is processed Performing meta-path-based message aggregation calculation according to different types of meta-paths Aggregating neighbor node information, and fusing the aggregated information with the node itself information to obtain a product vector of fused context information.

3. The big data information processing method according to claim 2, wherein: Multiple meta-paths comprising: Product node - user node - product node meta-path , indicating the existence of product relationships in which a user has interaction behaviors with two different product nodes simultaneously; Product node - function similarity edge - product node meta-path , indicating the product relationship connected by the function similarity edge; Product node - visual similarity edge - product node meta-path , representing product relationships connected by visual similarity edges.

4. The big data information processing method according to claim 3, wherein: performing meta-path-based message aggregation calculation on each product node by using a graph neural network comprises: for each meta-path , for product nodes , get nodes all neighbor nodes reachable along the meta-path ;​ According to the set , obtain each neighbor node on the product node and the corresponding meta path , , input the feature vectors corresponding to the nodes and to the shared attention calculation neural network to obtain the attention coefficients representing the importance between nodes ; According to the attention coefficient , the feature vector of all neighbor nodes of the product node is weighted and summed to obtain the semantic representation of the node under the meta-path ;​ Product nodes In each meta-path The semantic representation The importance weight of each meta-path To the current product to be promoted ; According to the importance weight , the product node , the semantic representation under each meta-path , and the weighted sum of the same kind of product vectors , when the product node , the product vector to be promoted is obtained 。 5. The big data information processing method according to any one of claims 1 to 4, wherein: calculating the direct similarity between the product to be promoted and each similar product comprises: acquiring a product to be promoted vector and each same kind product vector ; Computing the product vector to be promoted The direct similarity to each product vector of the same kind The direct similarity to each product vector of the same kind The direct similarity to each product vector of the same kind wherein and denote the Euclidean norm of a vector, respectively.

6. The big data information processing method according to claim 1, wherein: constructing a transition probability matrix P reflecting the transition of nodes to neighbor nodes comprises: extracting node connection information from the heterogeneous graph network, and classifying and processing the information according to the types of edges; calculating the weight of each edge based on the attribute features of different types of edges; calculating the transition probability from a node to each neighbor node based on the edge weight of each node in the heterogeneous graph network; constructing the transition probability matrix P according to the transition probability.

7. The big data information processing method according to claim 1, wherein: Target-oriented function using the following equation: wherein , , are weight parameters, denotes a vector representation of a node similarity to the vector representation of node i; denotes a network structure similarity of a node to node i; denotes a path connectivity measure of a node to node i.

8. The big data information processing method according to claim 1, wherein: biasing transition probabilities , which is expressed as follows: ,in, Controlling the original transition probability during random walk With target-oriented function The trade-off; For nodes arrive The standard transition probability; For nodes The attractiveness measure for target node i.

Images