Big data information processing method

Abstract

The invention discloses a big data information processing method, and relates to data processing, and the method comprises the steps: obtaining a plurality of similar products i of a to-be-popularized product p, and obtaining the product data of the similar products i; constructing a product-user heterogeneous graph network according to the product data of the to-be-popularized product p and the similar product i; a graph neural network is used for carrying out message passing calculation on the heterogeneous graph network, all product vectors fusing context information are obtained, and the product vectors comprise to-be-popularized product vectors zp and all similar product vectors zi; according to the product vector, calculating a direct similarity Sim1 (p, i) and an indirect similarity Sim2 (p, i) between the to-be-popularized product p and each similar product i; and according to the direct similarity Sim1 (p, i) and the indirect similarity Sim2 (p, i), obtaining a comprehensive score of the to-be-popularized product p through weighted fusion. For neglecting the implicit association relationship between the products in new product promotion, the direct similarity and the indirect similarity are effectively fused through an attention mechanism and a target-oriented random walk strategy, and mining of deep association between the products is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method for processing big data information. Background Art

[0002] With the rapid development of the internet and e-commerce, the global digital marketing market continues to expand. Successful new product promotion is crucial for companies to maintain their market competitiveness. Key challenges in promoting new products include low user awareness, uncertain market acceptance, and limited promotional resources. The rise of big data and artificial intelligence technologies offers new solutions for new product promotion. In particular, multimodal data fusion and graph data mining technologies demonstrate significant potential in areas such as product similarity calculation and recommendation systems.

[0003] Existing new product promotion technologies have the following main problems and limitations: Most existing methods ignore the implicit relationships between products reflected by user interactions. Users' browsing, saving, and purchasing behaviors contain rich product association information, but this information is not fully utilized in existing technologies.

[0004] For example, the related patent CN118864008A discloses a method for personalized advertising push based on product data, including: obtaining product data of a product to be promoted; the product data containing the basic attributes of the product to be promoted and sales statistics of the product to be promoted; determining at least two similar products of the product to be promoted and determining the product data of each of the at least two similar products; the product to be promoted and the at least two similar products belonging to the same product category; calculating weight coefficients of the at least two similar products, extracting the top k weight coefficients from all weight coefficients using a data sorting method, and determining the similar products corresponding to the top k weight coefficients as recommended products; k being an integer greater than or equal to 1; and displaying the product data corresponding to the recommended products on the user's desktop or browser page. However, this solution relies solely on explicit data such as the product's basic attributes and sales volume, lacks the ability to deeply explore product associations, and is unable to discover non-obvious similar products. Summary of the Invention

[0005] In response to the fact that the existing technology for promoting new products ignores the implicit correlation between products, this application provides a big data information processing method, which simultaneously utilizes the product's text data, image data and user interaction behavior to construct a multi-relationship heterogeneous graph network, and effectively integrates direct similarity and indirect similarity through the attention mechanism and goal-oriented random walk strategy to realize the mining of deep-level correlations between products and improve the promotion accuracy of new products.

[0006] The present application provides a big data information processing method, comprising: obtaining product data of a product p to be promoted, wherein the product data includes text data and image data of the product to be promoted; obtaining multiple similar products i of the product p to be promoted, and obtaining product data of the similar products i; constructing a product-user heterogeneous graph network based on the product data of the product p to be promoted and the similar products i, wherein multiple types of edge relationships are defined between product nodes, wherein the multiple types of edge relationships include functional similarity calculated based on text data and visual similarity calculated based on image data; using a graph neural network to perform message passing calculations on the heterogeneous graph network to obtain all product vectors that integrate context information, wherein the product vectors include a vector z of the product to be promoted. p and the vector z of similar products i ; According to the product vector, calculate the direct similarity Sim1(p,i) and indirect similarity Sim2(p,i) between the product to be promoted and each similar product i; According to the direct similarity Sim1(p,i) and the indirect similarity Sim2(p,i), through weighted fusion, obtain the comprehensive score of the product to be promoted p.

[0007] Furthermore, a product-user heterogeneous graph network is constructed, including: performing natural language processing on the text data of the promoted product p and similar products i, extracting product functional features, and constructing text feature vectors; processing the image data of the promoted product p and similar products i through a convolutional neural network, extracting product visual features, and constructing image feature vectors; calculating the cosine similarity between the text feature vectors as the functional similarity, and when the functional similarity of two product nodes is greater than a preset threshold, establishing a functional similarity edge between the two product nodes; calculating the Euclidean distance between the image feature vectors as the visual similarity, and when the visual similarity of two product nodes is greater than a preset threshold, establishing a visual similarity edge between the two product nodes; collecting user-product interaction data, constructing user nodes, and when there is interaction between the user and the product, establishing an interaction edge between the user node and the corresponding product node; integrating the product nodes and the user nodes to construct a heterogeneous graph network.

[0008] In particular, new product promotions often suffer from severe data sparsity due to the lack of user interaction history and preference data. Traditional planar feature representations treat products as isolated vectors with limited information transmission capabilities. This application constructs a multimodal heterogeneous graph network and designs a meta-path-aware graph neural network. On the one hand, in graph theory, the topological position of nodes itself contains rich structural information and has a higher information density than vector representations. By embedding product positions into the graph structure, implicit semantics can be extracted from topological relationships even without direct interaction data.

[0009] Furthermore, we obtain product vectors that integrate contextual information, including: constructing multiple meta-paths Φ of heterogeneous graph networks r, each meta-path Φ r Represents the connection patterns of different types of nodes and edges in the graph; multiple meta-paths Φ r It includes: the meta-path Φ1 of product node-user node-product node, which indicates that there is a product relationship in which a user has interactive behaviors with two different product nodes at the same time; the meta-path Φ2 of product node-functional similarity edge-product node, which indicates the product relationship connected by functional similarity edge; the meta-path Φ3 of product node-visual similarity edge-product node, which indicates the product relationship connected by visual similarity edge.

[0010] In particular, the key limitation of traditional graph neural networks is their insufficient handling of semantic differentiation. This application constructs meta-paths, three core meta-paths (product-user-product, product-function-product, product-visual-product), which are equivalent to creating three different relationship projection spaces. These projection spaces can capture various implicit association patterns that may exist between products. The information transmission process along a specific meta-path can be understood as a conditional random walk with semantic constraints. This constraint ensures the semantic consistency of the information and prevents interference from irrelevant information.

[0011] Using graph neural network, for each product node v in the heterogeneous graph network n Perform meta-path-based message aggregation calculations, according to different types of meta-paths Φ r Aggregate neighbor node information and fuse the aggregated information with the node's own information to obtain a product vector of fused context information, including: r , for product node v n , get node v n Along the element path Φ r The set of all reachable neighbor nodes According to the collection Get product node v n and the corresponding meta-path Φ r Each neighbor node v on m , change node v n and v m The corresponding feature vector is input into the shared attention calculation neural network to obtain the attention coefficient a representing the importance between nodes. nm ; According to the attention coefficient a nm , for product node v n The weighted sum of the feature vectors of all neighboring nodes of node v is obtained n In the meta-path Φ r The semantic representation of For product node v n In each element path Φ r The semantic representation of Calculate each meta-path Φ through the semantic-level attention neural networkr The importance weight of the current product to be promoted w r ; According to the importance weight w r , for product node v n Semantic representation under each meta-path Perform weighted summation to obtain the vector z of similar products i , when the product node v n When it is the node of the product to be promoted, the vector z of the product to be promoted is obtained. p ;

[0012] Furthermore, the direct similarity between the product to be promoted and each similar product is calculated, including: obtaining the vector z of the product to be promoted p and each similar product vector z i ; Calculate the product vector z to be promoted p With each similar product vector z i The direct similarity Sim1(p,i) is calculated as follows: Among them, ||z p || and ||z i || respectively represent the Euclidean norm of the vector.

[0013] Furthermore, the direct similarity between the product to be promoted and each similar product is calculated, including: obtaining the vector z of the product to be promoted p and each similar product vector z i ; Calculate the product vector z to be promoted p With each similar product vector z i The direct similarity Sim1(p,i) is calculated as follows: Among them, ||z p || and ||z i || respectively represent the Euclidean norm of the vector.

[0014] Calculate indirect similarity, including: setting the maximum skipping rope K of the heterogeneous graph network as the upper limit of the path length; constructing the transition probability matrix P reflecting the node to each neighbor node based on the connection relationship between the nodes in the heterogeneous graph network, including: extracting the connection information between all node pairs in the heterogeneous graph network G, and dividing them into functional similarity edge sets E according to the type of edge func , visual similarity edge set E visual and user interaction edge set E user ; For functional similarity edges, based on the calculated cosine similarity value, the edge weight w func (v n ,v m ) is assigned to the corresponding similarity value, where v n and v mRepresents the connected product node pair; for the visual similarity edge, based on the calculated Euclidean distance value, it is converted into a similarity measure to obtain the edge weight w visual (v n ,v m ), the conversion formula is: Among them, d(v n ,v m ) represents the Euclidean distance, σ is the scaling parameter; for user interaction edges, the edge weight w is calculated based on the interaction frequency f(u,v) between the user and the product user (u,v), where u represents the user node and v represents the product node; for each node v in the heterogeneous graph network G n , collect the weights of all its outgoing edges {w(v n ,v m )}, and normalize it to get the n To each of its neighbor nodes v m The transition probability P(v n ,v m ), the calculation formula is: Among them, v k Traverse node v n All neighbor nodes; the transition probability P(v n ,v m ) are organized into a matrix form to form a complete transition probability matrix P for subsequent random walk path sampling.

[0015] Take the direct similarity Sim1(p,i) as prior knowledge and construct the goal-oriented function h(v n ,i), used to evaluate node v n The structural and semantic similarity between the target node i, including: obtaining the intermediate node v n The vector representation z n and the vector representation z of the target node i i , calculate the preliminary cosine similarity between node vectors The direct similarity Sim1(p,i) between the product to be promoted p and the target product i is used as an adjustment factor to construct an enhanced vector similarity calculation formula: sim vec (v n ,i)=cos sim (v n ,i)×[1+μ×Sim1(p,i)×f(v n ,p)], where μ is the weight coefficient, f(v n ,p) reflects the node v n The decay function associated with the structure of the product to be promoted p; Based on the topological structure of the heterogeneous graph network G, calculate the intermediate node v nThe structural similarity sim between the target node i struct (v n ,i), This indicator is obtained by analyzing the number of common neighbors of two nodes and their position relationship in the network; detecting the intermediate node v n Whether there is a short path connection with the target node i, the path similarity sim is obtained by calculating the reachability within the limited hop range path (v n ,i); According to the size of the direct similarity Sim1(p,i), the weight parameters λ1, λ2, and λ3 are dynamically adjusted. When Sim1(p,i) is high, the λ1 value is increased to strengthen the influence of vector similarity, and vice versa, the λ2 and λ3 values are increased; Combining the above three similarity indicators, the goal-oriented function h(v n ,i), the calculation formula is: h(v n ,i)=λ1×sim vec (v n ,i)+λ2×sim struct (v n ,i)+λ3×sim path (v n ,i), where λ1, λ2, λ3 are weight parameters, and λ1+λ2+λ3=1; for the calculated h(v n ,i) The value is normalized to ensure that its value range is between [0, 1], so that it can be combined with the transition probability matrix P to form an effective search strategy.

[0016] In particular, traditional random walks are completely random, wandering through a graph with fixed transition probabilities and no clear goal. However, they suffer from severe computational inefficiency and insufficient correlation in their results when dealing with new product promotions. Specifically, traditional random walks are based on a Markov chain model, where the transition probability of the current node depends solely on the local connection structure and does not consider the global goal, resulting in a large amount of computational resources being wasted on exploring irrelevant paths. Furthermore, traditional random walks are an entropy-increasing process. As the number of steps increases, the system entropy continues to increase, and the wandering trajectory becomes increasingly random, reducing the probability of discovering meaningful paths. Finally, computational efficiency decreases exponentially with path length, performing particularly poorly in multi-hop path exploration scenarios, making it difficult to apply to new product promotion scenarios where long path associations need to be discovered.

[0017] This application, on the one hand, injects direct similarity as prior knowledge into the random walk process. From the perspective of Bayesian reasoning, it is equivalent to introducing a priori probability distribution, which transforms the random process from "uninformed exploration" to "informed guidance", forming a "knowledge-guided exploration" mode. On the other hand, it constructs the biased transfer probability Provides a framework to pass parameters A precise balance is achieved between the original structure of the graph (exploration) and the goal orientation (utilization), solving the classic "exploration-utilization" dilemma in reinforcement learning. On the other hand, this application discovers implicit high-order connections between products through multi-hop path discovery. Even if new products lack direct user interaction, potential associations can be discovered through the network structure. Through multi-hop path discovery, the construction of the association link "new product → similar product → active user" can be achieved; user group knowledge can be transferred from products with established market status to new products.

[0018] For the product to be promoted p and each similar product i, according to the transition probability matrix P and the goal-oriented function h(v n ,i), through the preset exploration-balance parameters Construct the biased transfer probability P' and execute the goal-oriented random walk strategy under the constraint of the maximum number of hops K to obtain the connection path set Path from product p to product i set (p,i); calculate the path set Path set The weight of each path in (p,i) is calculated, and the weighted average of the weights of all paths is performed to obtain the indirect similarity Sim2(p,i) between the product to be promoted p and similar products i.

[0019] For the product to be promoted p and each similar product i, according to the transition probability matrix P and the goal-oriented function h(v n ,i), through the preset exploration-exploitation balance parameters Construct the biased transfer probability P' and execute the goal-oriented random walk strategy under the constraint of the maximum number of hops K to obtain the connection path set Path from product p to product i set (p,i), including: setting the node p of the product to be promoted as the starting node of the random walk, and the node i of the same product as the target node; according to the transition probability matrix P and the goal-oriented function h(v n ,i), using the exploration-exploitation balance parameter Construct biased transition probability P'(v n ,v m ), the calculation formula is: in, Control the degree of dependence on known transition probabilities and the degree of tendency towards goal orientation during the random walk; initialize the path counter count = 0 and the maximum number of paths threshold to control the number of sampled paths; start from the starting node p, at each step according to the current node v n , according to the biased transfer probability P'(v n ,v m ) Select the next node v to visit m; Record the node sequence during the random walk to form a path starting from p; when the path length reaches the maximum number of hops K or the current node is the target node i, the generation of the current path ends; if the generated path successfully connects the starting node p and the target node i, the path is added to the path set Path set (p,i), and add 1 to the counter count; repeat until count reaches the threshold or meets other termination conditions, and finally obtain the connection path set Path from product p to product i set (p,i); for the collected path set Path set (p,i) is deduplicated and filtered to remove redundant or low-quality paths and retain representative high-quality paths to provide basic data for subsequent indirect similarity calculations.

[0020] In particular, the goal-directed random walk in this application will be O(b d )’s brute force search complexity (b is the average branching factor, d is the path length) is reduced to approximately O(b×d), making long path discovery computationally feasible.

[0021] Compared with the existing technology, the advantages of this application are:

[0022] On the one hand, existing technologies often use traditional collaborative filtering or content matching algorithms, but when there is a lack of sufficient user interaction data for new products, traditional collaborative filtering methods are difficult to provide effective recommendations. This application constructs a heterogeneous graph network by fusing text data, image data, and user interaction data. It uses a meta-path-based attention mechanism to achieve bidirectional information flow under different semantic relationships, realize multi-dimensional feature representation of new products, and effectively solve the cold start problem.

[0023] On the other hand, the existing technology often only considers the direct similarity between products, resulting in a single recommendation result and the inability to discover potentially related products. This application calculates indirect similarity through a goal-oriented random walk strategy to mine implicit association paths and deep semantic relationships between products. In addition, traditional path mining algorithms generally enumerate all paths, but for product recommendations, directly enumerating all paths in a large-scale graph network will lead to a computational explosion. This application achieves efficient implicit association path search by fusing the transition probability matrix and the goal-oriented function, significantly reducing the computational complexity while ensuring the quality of recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0025] Figure 1 is an exemplary flow chart of a big data information processing method according to some embodiments of the present application;

[0026] Figure 2 is an exemplary flow chart of constructing a heterogeneous graph network according to some embodiments of the present application;

[0027] Figure 3 This is an exemplary flow chart of generating indirect similarity according to some embodiments of the present application. DETAILED DESCRIPTION

[0028] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0029] like Figure 1 As shown, the product data of the product p to be promoted is obtained, and the product data includes text data and image data of the product to be promoted. Specifically, the product title, description, specification parameters and other text information are captured from the e-commerce platform, product database or enterprise resource planning (ERP) system through the API interface. The obtained raw text data is converted into a standardized text corpus through preprocessing steps such as word segmentation, stop word removal, and stemming. Image resources such as product main pictures and detail pictures are collected and preprocessed by uniform size scaling (such as adjusting to 224×224 pixels), color standardization, and data enhancement. Image preprocessing adopts the standard process in computer vision to ensure the input consistency of subsequent convolutional neural network processing.

[0030] Obtain multiple similar products i of the product to be promoted, p, and obtain product data for similar products i. Specifically, first extract the classification path of the product to be promoted p from the product classification system (e.g., "electronic products → mobile phones → smartphones → 5G phones"), then filter out a preliminary set of candidates within the most refined classification level (e.g., "5G phones"). This rapid filtering based on the classification tree narrows the search space from the entire product library (potentially containing millions of products) to specific categories (typically hundreds to thousands of products), significantly improving the efficiency of subsequent fine-grained matching.

[0031] Semantic extension is performed using a pre-built product ontology. This ontology contains relationships between products, such as "is-a," "part-of," and "functional-equivalent." Through the ontology reasoning engine, the system can identify products with similar functions despite being categorized differently. For example, while "Bluetooth headsets" and "wireless headsets" may be in different categories, they share a "functional equivalence" relationship in the ontology and are therefore included in the candidate set.

[0032] Input the feature vector of the product p to be promoted (composed of text and image features) into an efficient vector database (such as FAISS or Milvus), perform K-nearest neighbor (KNN) search, and obtain the N closest products in the vector space. Technically, an approximate nearest neighbor algorithm (such as IVFPQ based on product quantization) is used to achieve efficient retrieval of large-scale high-dimensional vectors, and the search complexity is reduced from O(n d ) is reduced to O(n 0.5d ), where n is the number of products and d is the vector dimension.

[0033] To avoid information redundancy caused by overly similar recommendation results, the system uses a deterministic greedy algorithm (DPP) to optimize the diversity of the initial ranking results. This algorithm ensures that the final recommended products of the same category exhibit diversity in key attributes (such as price range, brand, and features) while maintaining relevance, thereby increasing user choice.

[0034] For the final set of similar products (usually 10-20 products), the system extracts their complete information from the product database through a parallel data acquisition mechanism, including: basic attribute data: product ID, name, brand, price and other structured information; rich text description: product detailed description, specifications, selling points and features, etc.; image resources: product main pictures, detail pictures, application scenario pictures and other visual materials; user feedback data: evaluation data, purchase behavior data, etc.

[0035] like Figure 2 As shown in this example, a platform needs to promote the newly released "XYZPro" smartphone. The platform has a large amount of historical product data and user interaction records. Now it needs to build a product-user heterogeneous graph network to provide precise promotion support for this new product.

[0036] For the "XYZ Pro" smartphone and its similar products, the system collected the following text data: Product title: "XYZ Pro 5G smartphone 8GB+256GB ultra-sensitive triple camera 6.7-inch full screen"; Product description: contains detailed function introduction, technical specifications, usage scenarios, etc.; User evaluation: evaluation text from early experiencers; Technical parameter table: structured parameters such as processor, memory, battery, etc.; The system uses the industry-adapted BERT variant model to process these texts. In the preprocessing stage, the system performs word segmentation, stop word removal, and entity normalization (such as normalizing "8GB+256GB" to storage parameters). Then, the product feature extractor for specific fields identifies key functional points (such as "ultra-sensitive triple camera", "full screen", etc.) and assigns appropriate weights. Finally, the system generates a 768-dimensional text feature vector, which encodes semantic information such as the product's functional characteristics, performance parameters, and application scenarios.

[0037] For each mobile phone product, the system collected the following image data: Product main image: showing the front and back appearance of the phone; Detail close-up: close-up of the camera module, screen display, interface, etc.; Usage scenario image: showing the application effect of the product in different scenarios; The system uses a pre-trained ResNet101 model, which has been fine-tuned on a large number of electronic product images. The image is first standardized and pre-processed: adjusted to a uniform resolution (224×224 pixels), brightness and contrast normalized, and data augmented (rotated, flipped) to enhance the robustness of the model. After processing through the convolutional neural network, the system extracts a 2048-dimensional feature vector, which contains the visual characteristics of the product, such as design style, color scheme, material feel, appearance structure and other information.

[0038] The system calculates the cosine similarity between the text feature vectors of "XYZPro" and candidate similar products (such as "ABCUltra", "DEFMax" and other competing products). For example: sim_func("XYZPro" "ABC Ultra") = 0.82 (both products focus on high-end photography and large screens); sim_func("XYZPro" "DEFMax") = 0.76 (similar functional positioning but different emphasis); sim_func("XYZPro" "GHI Lite") = 0.58 (one is a high-end flagship, the other is an entry-level model); the system sets the functional similarity threshold θ func = 0.75, thus establishing functional similarity edges between "XYZPro" and "ABCUltra," and between "XYZ Pro" and "DEFMax," with edge weights of 0.82 and 0.76, respectively. These edges reflect the functional similarity between products and provide channels for subsequent information dissemination.

[0039] Similarly, the system calculates the visual similarity between products: sim_vis("XYZPro" "ABCUltra") = 0.69 (the two products have similar design styles); sim_vis("XYZPro" "JKLNeo") = 0.77 (although the functional positioning is different, the appearance design is very similar); sim_vis("XYZPro" "DEF Max") = 0.62 (the appearance design is quite different); the system sets the visual similarity threshold θ vis = 0.65, so visual similarity edges are established between "XYZPro" and "ABCUltra" and between "XYZPro" and "JKLNeo", with edge weights of 0.69 and 0.77, respectively. These visual similarity edges capture the similarity in product design and provide a basis for recommending products based on user-defined visual preferences.

[0040] The system collects the following types of user interaction data: Purchase behavior: User U1 purchased "ABCUltra", User U2 purchased "DEF Max"; Browsing behavior: User U3 browsed the detail pages of "XYZPro" and "JKLNeo"; Collection behavior: User U4 added "XYZPro" and "ABC Ultra" to favorites; Evaluation behavior: User U5 evaluated "DEF Max"; The system assigns different weights to different interaction types (purchase: 1.0, collection: 0.7, evaluation: 0.6, browsing: 0.3). When the comprehensive interaction strength between the user and the product exceeds the threshold θ inter =0.5, an interaction edge is established between the user node and the product node. For example, an interaction edge is established between user U4 and "XYZPro" (collection behavior, weight 0.7). For the newly launched "XYZPro", since user interaction data is relatively sparse, the system pays special attention to the user groups of products that have established functional or visual similarity edges with "XYZPro". For example, through the functional similarity edge of "XYZPro" → "ABCUltra", the system can identify users U1 and U4 who have strong interactions with "ABCUltra", and these users become high-quality target groups for potential promotion of "XYZPro".

[0041] As a newly launched product, "XYZPro" faces a typical cold start problem. The system alleviates this problem through the following steps: First, identify the product set P' = {"ABCUltra", "DEF Max", "JKLNeo"} that has established functional similarity or visual similarity with "XYZPro"; for each product in P', find the user set that has a strong interactive relationship with it: U("ABCUltra") = {U1, U4, U7, U9}; U("DEF Max") = {U2, U5, U8}; U("JKLNeo") = {U3, U6, U10}; The system constructs a two-hop path from "XYZPro" to these users: "XYZPro" → "ABC Ultra" → U1, U4, U7, U9; "XYZPro" → "DEF Max" → U2, U5, U8; "XYZ Pro” → “JKLNeo” → U3, U6, U10; through these paths, the system can calculate the indirect relevance of “XYZPro” to these users, and prioritize promoting “XYZPro” to users who have strong interactions with similar products, effectively alleviating the cold start problem of new products.

[0042] Finally, the system integrates the various nodes and edges constructed above into a unified heterogeneous graph network G: Product node set P: includes "XYZPro" and its similar products, a total of about 50 nodes; User node set U: includes users who interact with these products, about 10,000 nodes; Edge set E: includes three types of edges; Functionally similar edges E func : About 100, connecting functionally similar product pairs; visually similar edges E vis : about 80, connecting product pairs with similar appearance; user interaction edge E inter Approximately 25,000 entries connect users with the products they interacted with. This heterogeneous graph network is stored as an adjacency matrix and a feature matrix, with an efficient index structure. The system also constructs indexes for three meta-paths: product → user → product, reflecting product associations resulting from shared user interactions; product → functional similarity → product, reflecting product associations along the functional dimension; and product → visual similarity → product, reflecting product associations along the visual dimension. This heterogeneous graph structure supports subsequent graph neural network message passing and multi-hop similarity calculations, enabling the system to comprehensively consider multi-dimensional information and provide precise target user identification and similar product recommendations for the promotion of "XYZ Pro."

[0043] By building this multi-dimensional, heterogeneous graph network, the system successfully integrated the newly released "XYZPro" smartphone into the existing product-user relationship network, even though it had not yet accumulated sufficient user interaction data. This graph-structured product representation and association mining method significantly improved the accuracy and efficiency of new product promotion.

[0044] The system has successfully constructed a heterogeneous product-user graph network centered around the new "XYZPro" smartphone. Next, a graph neural network is used to perform message passing computations on this heterogeneous graph, obtaining product vectors that incorporate contextual information.

[0045] Based on the constructed heterogeneous graph network G, the system defines three semantically rich meta-paths: Meta-path Φ1 (product-user-product): In the smartphone market, this path captures the product associations implicit in user preferences. For example, user U4 has added both "XYZPro" and "ABC Ultra" to their favorites, indicating a connection between the two products in the user's mind. Using this meta-path, the system can identify products with different specifications but high user group overlap.

[0046] Meta-path Φ2 (product - functional similarity - product): This path reflects the functional relationship between products. As mentioned earlier, "XYZ Pro" establishes functional similarity edges with "ABC Ultra" and "DEF Max," indicating that these products share similarities in performance specifications, camera capabilities, and screen displays. This meta-path is particularly important for understanding product functional positioning and market segmentation.

[0047] Meta-path Φ3 (product - visual similarity - product): This path reflects the relationship between products in terms of appearance and design. "XYZ Pro" establishes a visual similarity edge with "ABCUltra" and "JKLNeo," indicating that these products are similar in industrial design, material selection, and appearance. In the smartphone market, visual design is a key factor influencing user choices.

[0048] For the new product "XYZPro" (node v p ), the system obtains its neighbor node set along three meta-paths: Φ1 path neighbors: The system first identifies the user set U that has direct interactions with “XYZPro” direct = {U3, U4} (browsing and collection behaviors, respectively), and then find other products that these users have interacted with to form a neighbor set

[0049] Φ2 path neighbors: directly obtain based on functional similarity edges These two products are highly similar to "XYZ Pro" in terms of functionality.

[0050] Φ3 path neighbors: directly obtain based on visual similarity edges These two products are highly similar to "XYZ Pro" in appearance design.

[0051] It is worth noting that “ABC Ultra” appears in the neighbor sets of the three meta-paths at the same time, indicating that it is strongly associated with “XYZPro” in multiple dimensions.

[0052] The system calculates the attention coefficient for "XYZ Pro" and its neighboring products under each meta-path: Attention calculation of meta-path Φ1 (product-user-product): For "XYZ Pro" and "ABC Ultra", the system considers the interaction intensity of user U4 (collection, weight 0.7); for "XYZ Pro" and "JKLNeo", the system considers the interaction intensity of user U3 (browsing, weight 0.3); after neural network calculation and softmax normalization, a(p, ABC) = 0.71, a(p, JKL) = 0.29 are obtained.

[0053] Attention calculation of meta-path Φ2 (product-functional similarity-product): Based on functional similarity and product feature vectors, the attention coefficient is calculated; after processing, a(p, ABC) = 0.54 and a(p, DEF) = 0.46 are obtained.

[0054] Attention calculation for meta-path Φ3 (product-visual similarity-product): Calculate the attention coefficient based on visual similarity and product feature vectors; after processing, we obtain a(p, ABC) = 0.48 and a(p, JKL) = 0.52.

[0055] During attention calculation, the system specifically considers the new product feature of "XYZ Pro." For example, in meta-path Φ1, despite sparse interaction data, the system increases the attention weight of products that have high-quality interactions with multiple users (such as purchases rather than just browsing).

[0056] Based on the calculated attention coefficient, the system aggregates the neighbor information of each meta-path:

[0057] Φ1 path aggregation: Among them, h(ABC) and h(JKL) are the initial feature vectors of “ABC Ultra” and “JKL Neo” respectively. The vector reflects the characteristics of products that share the same user group as "XYZ Pro".

[0058] Φ2 path aggregation: After aggregation Vectors reflect product features similar to the functionality of "XYZ Pro".

[0059] Φ3 path aggregation: After aggregation Vectors reflect product features similar to the "XYZ Pro" design.

[0060] In order to fuse the information of different meta-paths, the system implements a semantic-level attention mechanism: first, for each semantic-specific representation Apply a trainable weight matrix W to perform a linear transformation to generate Then, the hyperbolic tangent function (tanh) is applied to the transformed vector for nonlinear transformation, and we get Next, the transformed result is multiplied by the trainable semantic-level attention vector q to calculate the attention score of each meta-path Finally, the attention scores of all meta-paths {s1, s2, s3, .....} are normalized by applying the softmax function to obtain Φ for each meta-path r The importance weight w for the current promotion task r =softmax(si Specifically, for "XYZPro," the system analyzes historical promotion data and determines that in the high-end smartphone market, functional features and visual design are the primary factors influencing user decision-making. Therefore, the calculated importance weights are: w1 = 0.25 (user behavior path), w2 = 0.42 (functional similarity path), and w3 = 0.33 (visual similarity path).

[0061] Finally, the system performs weighted fusion of the semantic representations under the three meta-paths according to their importance weights:

[0062] The resulting product vector z p It integrates three dimensions of contextual information: user behavior association information (25%); functional similarity association information (42%); and visual design similarity association information (33%). This fusion vector not only contains the characteristics of "XYZ Pro" itself, but also incorporates product information related to it in different dimensions, forming a more comprehensive and context-aware representation.

[0063] To enhance information transfer, the system performs bidirectional message passing on the heterogeneous graph network: Forward flow: Information is transferred from "XYZPro" to its neighboring products. For example, as an innovative high-end smartphone, the design concept and technical features of "XYZ Pro" are transferred to "ABC Ultra" and "JKLNeo", allowing the representations of these products to partially capture the innovative characteristics of "XYZ Pro".

[0064] Reverse flow: Information transfer from neighboring products to "XYZ Pro." For example, "ABC Ultra," a product with a strong user base, has its successful features and user feedback passed on to "XYZ Pro," helping the new product better position itself in the market.

[0065] The system achieves bidirectional flow through multiple rounds of iterative updates, with all product nodes updating their representations simultaneously during each iteration. During this process, special attention is paid to paths connecting "XYZ Pro" with popular products, ensuring that important market information is effectively transmitted to new products.

[0066] like Figure 3 As shown, based on the obtained fusion product vector, the system calculates the similarity between "XYZ Pro" and each similar product. The system first calculates the direct similarity between "XYZ Pro" (p) and each similar product (i): Direct similarity reflects the proximity of product representation vectors in the semantic space, but does not consider the implicit associations between products in the network structure.

[0067] In order to discover the indirect relationship between products, the system constructs the transition probability matrix P: Functional similarity edge: directly use cosine similarity as the weight, such as w func (p,abc)=0.82; Visual similarity edge: Convert Euclidean distance to similarity, such as w visual (p,jkl)=0.77; User interaction edge: based on user interaction intensity, such as w user (U4, p) = 0.7 (collection behavior).

[0068] For each node v n , collect all its outgoing edge weights and normalize them to get the transition probability:

[0069] For the "XYZPro" node p, its transition probability distribution is: P(p,abc)=0.28 (based on the comprehensive similarity of function and vision); P(p,def)=0.17 (based on functional similarity); P(p,jkl)=0.23 (based on visual similarity); P(p,U3)=0.12 (based on browsing interaction); P(p,U4)=0.20 (based on collection interaction); These transition probabilities constitute the matrix P, which is used for subsequent random walk path sampling.

[0070] The system uses direct similarity as prior knowledge to construct a goal-oriented function h(v n ,i): For the intermediate node v n (such as "ABC Ultra") and the target node i (such as "DEFMax"), calculate the preliminary cosine similarity, and then adjust it based on the direct similarity between "XYZPro" and "DEFMax":

[0071] sim vec (abc,def)=cos sim (abc,def)×[1+0.5×Sim direct (p,def)×f(abc,p)], where f(abc,p) is the attenuation function that reflects the strength of the association between “ABCUltra” and “XYZPro”.

[0072] Analyze the common neighbors and network positions of "ABCUltra" and "DEFMax" to obtain the structural similarity Sim struct (abc,def). Check whether there is a short path connection between "ABCUltra" and "DEFMax" and calculate the path similarity sim path (abc,def). According to the value of Sim1(p,def)=0.76, the system sets the weight parameters λ1=0.5, λ2=0.3, and λ3=0.2.

[0073] h(abc,def)=0.5×sim vec (abc,def)+0.3×sim struct (abc,def)+0.2×sim path (abc,def), this function evaluates the attraction from any intermediate node to the target node, guiding the random walk process to find relevant paths more effectively.

[0074] For "XYZPro" (p) and each similar product i (such as "DEFMax"), the system performs a goal-directed random walk: combining the standard transition probability P and the goal-directed function h to construct a biased transition probability P':

[0075] System Setup Exploration - Utilizing Balance Parameters While maintaining the original network structure exploration, increase the tendency to move closer to the target product.

[0076] Set the maximum number of hops, K, to 3 and the maximum number of paths, 1000. Starting from "XYZ Pro," select the next node based on P'. For example, a possible path might be: "XYZ Pro" → "ABC Ultra" → user U7 → "DEF Max." Record the node sequence during path generation. End path generation when the path reaches "DEF Max" or its length reaches K.

[0077] Collect all paths that successfully connect "XYZ Pro" and "DEF Max", remove duplicates and filter the paths, retain high-quality paths, and finally obtain the path set Path set For "XYZ Pro" and "DEF Max," the system discovered multiple meaningful connection paths, including: product function path: XYZ Pro → DEF Max (direct functional similarity); user preference path: XYZ Pro → ABC Ultra → user U1 → DEF Max; and visual design path: XYZ Pro → JKLNeo → DEF Max (association through design style).

[0078] Path set Each path in (p, def) is assigned a weight, which is inversely proportional to the path length and proportional to the importance of the nodes on the path. The weighted average is calculated to obtain the indirect similarity:

[0079] For "XYZ Pro" and "DEFMax", the calculated indirect similarity Sim2(p,def)=0.64; using a similar method, the system calculates the indirect similarity between "XYZPro" and other similar products: Sim2(p,abc)=0.79; Sim2(p,jkl)=0.68; Sim2(p,ghi)=0.43.

[0080] Finally, the system obtains a comprehensive score based on the weighted fusion of direct and indirect similarities: Score(p,i)=α×Sim1(p,i)+(1-α)×Sim2(p,i). For the smartphone market, the system sets the weight α=0.6 based on historical data analysis, giving a higher weight to direct similarity while retaining the network structure information reflected by indirect similarity.

[0081] The final calculation yields the comprehensive scores of "XYZPro" and similar products:

[0082] Score(p,abc)=0.6×0.88+0.4×0.79=0.844;

[0083] Score(p,def)=0.6×0.76+0.4×0.64=0.712;

[0084] Score(p,jkl)=0.6×0.72+0.4×0.68=0.704;

[0085] Score(p,ghi)=0.6×0.61+0.4×0.43=0.538.

[0086] Based on these comprehensive scores, the system generated a final promotion strategy for "XYZPro": focusing on the user group with the highest similarity to "ABCUltra"; highlighting the common advantages and differentiated features of "XYZPro" and "ABCUltra" in marketing content; and secondary targets including the user groups of "DEFMax" and "JKLNeo". This promotion strategy, which combines direct feature similarity with network structure association, significantly improved the market penetration and user acceptance of "XYZPro".

[0087] The above schematically describes the invention of the present application and its implementation methods. This description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and embodiment similar to the technical solution are designed without creativity, which should fall within the scope of protection of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A method for processing big data information, characterized in that: include: Acquire product data of the product to be promoted, wherein the product data includes text data and image data of the product to be promoted; Obtain multiple similar products i of the product to be promoted p, and obtain product data of similar products i; Based on the product data of the product to be promoted p and similar products i, a product-user heterogeneous graph network is constructed, in which multiple types of edge relationships are defined between product nodes. The multiple types of edge relationships include functional similarity calculated based on text data and visual similarity calculated based on image data. Use graph neural network to perform message passing calculation on heterogeneous graph network to obtain all product vectors of fused context information. The product vector contains the product vector z to be promoted. p and the vector z of similar products i ; Based on the product vector, calculate the direct similarity Sim1(p,i) and indirect similarity Sim2(p,i) between the product to be promoted p and each similar product i; According to the direct similarity Sim1(p,i) and the indirect similarity Sim2(p,i), the comprehensive score of the product p to be promoted is obtained through weighted fusion.

2. The big data information processing method according to claim 1, characterized in that: Build a product-user heterogeneous graph network, including: Perform natural language processing on the text data of the promoted product p and similar products i, extract product functional features, and construct text feature vectors; The image data of the promoted product p and similar products i are processed through convolutional neural networks to extract product visual features and construct image feature vectors; Calculate the cosine similarity between text feature vectors as functional similarity. When the functional similarity of two product nodes is greater than a preset threshold, establish a functional similarity edge between the two product nodes. Calculate the Euclidean distance between image feature vectors as visual similarity. When the visual similarity between two product nodes is greater than a preset threshold, establish a visual similarity edge between the two product nodes. Collect interaction data between users and products, build user nodes, and establish interaction edges between user nodes and corresponding product nodes when users interact with products. Integrate product nodes and user nodes into a heterogeneous graph network.

3. The big data information processing method according to claim 2, characterized in that: Obtain product vectors that incorporate contextual information, including: Constructing multiple meta-paths Φ for heterogeneous graph networks r , each meta-path Φ r Represents the connection patterns of different types of nodes and edges in the graph; Using graph neural network, for each product node v in the heterogeneous graph network n Perform meta-path-based message aggregation calculations, according to different types of meta-paths Φ r Aggregate neighbor node information and fuse the aggregated information with the node's own information to obtain a product vector that integrates contextual information.

4. The big data information processing method according to claim 3, characterized in that: Multiple metapaths Φ r include: The meta-path Φ1 of product node-user node-product node indicates that there is a product relationship in which the user interacts with two different product nodes at the same time; The meta-path Φ2 of product node-functional similarity edge-product node represents the product relationship connected by functional similarity edges; The meta-path Φ3 of product node-visual similarity edge-product node represents the product relationship connected by visual similarity edges.

5. The big data information processing method according to claim 4, characterized in that: Using graph neural networks, we perform meta-path-based message aggregation calculations on each product node, including: For each meta-path Φ r , for product node v n , get node v n Along the element path Φ r The set of all reachable neighbor nodes According to the collection Get product node v n and the corresponding meta-path Φ r Each neighbor node v on m , change node v n and v m The corresponding feature vector is input into the shared attention calculation neural network to obtain the attention coefficient a representing the importance between nodes. nm ; According to the attention coefficient a nm , for product node v n The weighted sum of the feature vectors of all neighboring nodes of node v is obtained n In the meta-path Φ r The semantic representation of For product node v n In each element path Φ r The semantic representation of Calculate each meta-path Φ through the semantic-level attention neural network r The importance weight of the current product to be promoted w r ; According to the importance weight w r , for product node v n Semantic representation under each meta-path Perform weighted summation to obtain the vector z of similar products i , when the product node v n When it is the node of the product to be promoted, the vector z of the product to be promoted is obtained. p .

6. The big data information processing method according to any one of claims 2 to 5, characterized in that: Calculate the direct similarity between the product to be promoted and each similar product, including: Get the product vector z to be promoted p and each similar product vector z i ; Calculate the product vector z to be promoted p With each similar product vector z i The direct similarity Sim1(p,i) is calculated as follows: Among them, ||z p || and ||z i || respectively represent the Euclidean norm of the vector.

7. The big data information processing method according to claim 6, characterized in that: Calculate indirect similarity, including: Set the maximum skipping rope K of the heterogeneous graph network as the upper limit of the path length; According to the connection relationship between nodes in the heterogeneous graph network, a transition probability matrix P reflecting the node to each neighboring node is constructed; Take the direct similarity Sim1(p,i) as prior knowledge and construct the goal-oriented function h(v n ,i), used to evaluate node v n Structural and semantic similarity with the target node i; For the product to be promoted p and each similar product i, according to the transition probability matrix P and the goal-oriented function h(v n ,i), through the preset exploration-balance parameters Construct biased transition probability P'(v n ,v m ), and execute the goal-oriented random walk strategy under the constraint of the maximum number of hops K to obtain the connection path set Path from product p to product i set (p,i); Calculate the obtained path set Path set The weight of each path in (p,i) is calculated, and the weighted average of the weights of all paths is performed to obtain the indirect similarity Sim2(p,i) between the product to be promoted p and similar products i.

8. The big data information processing method according to claim 7, characterized in that: Construct the transition probability matrix P reflecting the node to each neighboring node, including: Extract node connection information from heterogeneous graph networks and classify them according to edge types; Based on the attribute characteristics of different types of edges, the weight of each edge is calculated separately; For each node's edge weight in a heterogeneous graph network, calculate the transition probability from the node to each neighboring node; According to the transition probability, construct the transition probability matrix P.

9. The big data information processing method according to claim 7, characterized in that: Goal-oriented function h(v n ,i), using the following formula: h(v n ,i)=λ1×sim vec (v n ,i)+λ2×sim struct (v n ,i)+λ3×sim path (v n ,i), where λ1, λ2, λ3 are Weight parameter, sim vec (v n ,i) represents node v n The vector representation similarity with node i; sim struct (v n ,i) represents node v n The network structure similarity with node i; sim path (v n ,i) represents node v n Path connectivity measure to node i.

10. The big data information processing method according to claim 7, characterized in that: Biased transition probability P'(v n ,v m ), the expression is as follows: in, Control the original transition probability P(v n ,v m ) and the goal-oriented function h(v m ,i) trade-off; P(v n ,v m ) is the node v n to v m The standard transition probability of h(v m ,i) is node v m The attractiveness measure for target node i.

Images