A dynamic quantile filtering method for commodity co-occurrence network

Abstract

This invention discloses a dynamic quantile filtering method for product co-occurrence networks, comprising: firstly, constructing a product co-occurrence network based on the co-occurrence relationships of products in orders; then, employing a dynamic quantile filtering strategy to denoise the network, specifically: obtaining the co-occurrence frequency weights of all product pairs and forming a co-occurrence matrix; extracting all non-zero co-occurrence frequency weights for each node; defining a local adaptive threshold based on the quantiles of these weights; retaining edges greater than or equal to the threshold; resetting the weights of edges less than the threshold to zero; and retaining isolated nodes; symmetricizing the adjacency matrix using an intersection strategy to obtain a weighted adjacency matrix; updating the network to obtain the denoised product co-occurrence network; and finally, using the Louvain algorithm to perform community partitioning on the denoised network to obtain a product community network. This invention can adaptively filter out weak connections and noisy edges in the product co-occurrence network, effectively improving the accuracy and robustness of community detection.

Description

Technical Field

[0001] This invention relates to the fields of data processing and network science analysis, and in particular to a dynamic quantile filtering method for commodity co-occurrence networks. Background Technology

[0002] With the rapid development of e-commerce and retail, one of the most fundamental elements in the supply chain center warehouse of online shopping is the goods themselves. Understanding the relationships between goods in an order directly impacts core issues such as location allocation and picking strategies. Different goods in the same order are not merely random customer behavior, but reflect complex interactions of demand preferences, functional complementarity, and promotional strategies. However, traditional warehousing research, focusing on minimizing picking distances or optimizing storage locations, often treats orders as simple collections of goods, neglecting the potential structural relationships between them. Addressing this deficiency can better reveal the complex and deep relationships within order goods, thus providing more accurate and reliable guidance for business needs.

[0003] Early basic methods, such as ABC classification, divide goods into different priorities based on individual item turnover rate. Although this method is simple to operate, it is essentially a static division at the individual item level and fails to fully capture the interaction relationships between goods. Current mainstream methods, such as building mathematical models to minimize picking distance for storage optimization, or mining association rules for goods, still remain at the level of co-occurrence frequency or similarity, without deeply exploring the complex structural relationships in the product network. Summary of the Invention

[0004] This invention provides a dynamic quantile filtering method for product co-occurrence networks to overcome the problems of failing to fully capture the interaction relationships between products and failing to deeply explore the complex structural relationships in the product network.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A dynamic quantile filtering method for product co-occurrence networks includes: S1. Construct a product co-occurrence network based on the co-occurrence relationships between products in orders; S2. Based on the product co-occurrence network, denoise it using a dynamic quantile filtering strategy to obtain a denoised product co-occurrence network; The execution process of the dynamic quantile filtering strategy includes: S21. Based on the product co-occurrence network, obtain the co-occurrence frequency weights between all two products and combine them into a product co-occurrence matrix; The co-occurrence frequency weight is the total number of times the two products appear together in all orders; S22. Based on the product co-occurrence matrix, obtain the weights of all non-zero co-occurrence frequencies for each node; S23. Define a local adaptive threshold for obtaining the weights of all non-zero co-occurrence frequencies of a node, based on the weights of all non-zero co-occurrence frequencies of each node. The local adaptive threshold is the q-quantile of the weights of all non-zero co-occurrence frequencies of a node. The steps to obtain the local adaptive threshold are as follows: S2311. Arrange all non-zero co-occurrence frequency weights of the nodes in ascending order to obtain the non-zero co-occurrence frequency weight sequence; S2312. Using linear interpolation, calculate the position index of the globally optimal q-quantile in the non-zero co-occurrence frequency weight sequence; S2313. Calculate the local adaptive threshold based on the position index and the non-zero co-occurrence frequency weight sequence; S24. Filter nodes using a local adaptive threshold: If the non-zero co-occurrence frequency weight is greater than or equal to the local adaptive threshold, retain the node's non-zero co-occurrence frequency weight; otherwise, reset the node's non-zero co-occurrence frequency weight to zero; if all of the node's non-zero co-occurrence frequency weights are reset to zero, retain the node as an isolated node in the product co-occurrence network. S25. Construct an adjacency matrix based on the retained non-zero co-occurrence frequency weights; use an intersection strategy to symmetricize the adjacency matrix to obtain a weighted adjacency matrix; S26. Update the product co-occurrence network according to the weighted adjacency matrix to obtain the denoised product co-occurrence network; S3. Based on the denoised product co-occurrence network, the Louvain algorithm is used to divide it, and finally the product community network is obtained.

[0006] Furthermore, the globally optimal q-quantile is obtained through a parameter selection method, the specific process of which is as follows: S2321. Select two adjacent global q-quantiles and construct denoised product co-occurrence networks corresponding to the two global q-quantiles respectively; based on the denoised product co-occurrence networks corresponding to the two global q-quantiles, obtain the modularity of the two adjacent global q-quantiles using the Louvain algorithm. S2322. Based on the modularity of two adjacent global q-quantiles, calculate and obtain the change in modularity when the q-quantile changes. S2323. Based on the co-occurrence frequency weights of the product co-occurrence network and the co-occurrence frequency weights of the denoised product co-occurrence network, calculate and obtain the retention change of the co-occurrence frequency weights. S2324. Based on the change in modularity and the change in retention of co-occurrence frequency weights when the q quantile changes, calculate the modularity-weight efficiency ratio, expressed as follows:

[0007] In the formula, Modularity-weight efficiency ratio; This represents the change in modularity when the q-quantile changes; The retained change in co-occurrence frequency weight; S2325. By selecting multiple pairs of adjacent global q quantiles, multiple modularity-weight efficiency ratios are obtained; the peak value of the modularity-weight efficiency ratio is determined by traversing multiple modularity-weight efficiency ratios. S2326. Based on the peak value of the modularity-weight efficiency ratio, determine the selected adjacent global q quantiles, and take the median of the adjacent global q quantiles as the optimal global q quantile.

[0008] Furthermore, the expression for calculating the modularity is:

[0009] In the formula, Modularity; The total weight of the product co-occurrence network for denoising; The number of nodes in the noise reduction product co-occurrence network; and For the index of the node; For nodes and Connection weights between them; The Kronecker function; For nodes The associated tag; For nodes The associated tag; To maintain node degree and In an invariant random network, the expected probability of two nodes being connected is given by: .

[0010] Furthermore, the specific steps of the Louvain algorithm include: S31. Initialize the denoised product co-occurrence network and treat the nodes in the denoised product co-occurrence network as communities; S32. Traverse all nodes and calculate the change in global modularity for each node when it moves to multiple adjacent communities; S33. If the largest change in global modularity among all changes in global modularity is less than or equal to 0, then the node is retained in its original community; otherwise, the node is added to the community corresponding to the largest change in global modularity, thus obtaining a new community. S34. Repeat the traversal of all nodes until no node movement occurs in a round of traversal, and a new community is obtained. S35. Using the new community as a supernode, reconstruct a new product co-occurrence network, and iteratively execute S31 to S34 until the convergence condition is met, finally obtaining the product community network.

[0011] Furthermore, the expression for the change in global modularity is:

[0012] In the formula, For nodes From the community Move to the neighboring community The change in the degree of the denoised product co-occurrence network module; For the community The sum of the weights of all internal edges; For nodes With the community The sum of the connection weights of the internal nodes; For the community Total degree of all nodes; For nodes In the community before the move The sum of the weights of all internal edges; For nodes Before moving, with the community The sum of the connection weights of all other nodes within the node; For nodes Within the community before the relocation Total degree of all nodes.

[0013] Beneficial Effects: This invention provides a dynamic quantile filtering method for product co-occurrence networks. By introducing a dynamic quantile filtering strategy, the product co-occurrence network is preprocessed and denoised. The dynamic quantile can adaptively handle the heterogeneity of degree distribution among different nodes in the product co-occurrence network, eliminating weak signal noise in the network, and making real and stable product associations (such as complementary goods, substitutes, or goods in the same consumption scenario) more prominent in the topology. This dynamic adaptive mechanism fundamentally avoids the "peak shaving and valley filling" information loss caused by a globally uniform threshold, thereby significantly improving the quality of the input graph structure for the subsequent Louvain algorithm.

[0014] By employing the Louvain algorithm to perform community discovery based on the optimized network, tightly coupled product communities can be identified more accurately. These communities correspond to real consumption scenarios (such as "barbecue ingredients community", "baby products community" or "office stationery community") and have higher business interpretability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the dynamic quantile filtering method of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This embodiment provides a dynamic quantile filtering method for product co-occurrence networks, such as... Figure 1 As shown, it includes: S1. Construct a product co-occurrence network based on the co-occurrence relationships between products in orders; Specifically, the product co-occurrence network includes nodes, edges between nodes, and edge weights; wherein, a node is a product entity, the edges between nodes represent the co-occurrence relationship of products in an order, and the edge weight is the total number of times two products co-occur in all orders.

[0019] S2. Based on the product co-occurrence network, denoise it using a dynamic quantile filtering strategy to obtain a denoised product co-occurrence network; The execution process of the dynamic quantile filtering strategy includes: S21. Based on the product co-occurrence network, obtain the co-occurrence frequency weights between all two products and combine them into a product co-occurrence matrix; The co-occurrence frequency weight is the total number of times the two products appear together in all orders; S22. Based on the product co-occurrence matrix, obtain the weights of all non-zero co-occurrence frequencies for each node; S23. Define a local adaptive threshold for obtaining the weights of all non-zero co-occurrence frequencies of a node, based on the weights of all non-zero co-occurrence frequencies of each node. The local adaptive threshold is the q-quantile of the weights of all non-zero co-occurrence frequencies of a node. The steps to obtain the local adaptive threshold are as follows: S2311. Arrange all non-zero co-occurrence frequency weights of the nodes in ascending order to obtain the non-zero co-occurrence frequency weight sequence; S2312. Using linear interpolation, calculate the position index of the globally optimal q-quantile in the non-zero co-occurrence frequency weight sequence. The expression is:

[0020] In the formula, This is the index of the globally optimal q-quantile in the non-zero co-occurrence frequency weight sequence; The globally optimal q-quantile; The degree of the node; S2313. Based on the position index and the non-zero co-occurrence frequency weight sequence, the local adaptive threshold is calculated, and its expression is:

[0021] In the formula, This is a locally adaptive threshold; For the integer part of the position index The corresponding non-zero co-occurrence frequency weights; The decimal part of the position index; For the integer part of the position index The corresponding non-zero co-occurrence frequency weights; S24. Filter nodes using a local adaptive threshold: If the non-zero co-occurrence frequency weight is greater than or equal to the local adaptive threshold, retain the node's non-zero co-occurrence frequency weight; otherwise, reset the node's non-zero co-occurrence frequency weight to zero; if all of the node's non-zero co-occurrence frequency weights are reset to zero, retain the node as an isolated node in the product co-occurrence network. S25. Construct an adjacency matrix based on the retained non-zero co-occurrence frequency weights; use an intersection strategy to symmetricize the adjacency matrix to obtain a weighted adjacency matrix; S26. Update the product co-occurrence network according to the weighted adjacency matrix to obtain the denoised product co-occurrence network.

[0022] Specifically, the intersection strategy is defined as follows: Since the edges are undirected, to ensure the symmetry of the final network, the following intersection condition is adopted when deciding whether to retain the weights of non-zero co-occurrence frequencies of nodes: Non-zero co-occurrence frequency weights are retained if and only if the non-zero co-occurrence frequency weights of the first node and the second node are greater than or equal to the local adaptive threshold, and the non-zero co-occurrence frequency weights of the second node and the first node are greater than or equal to the local adaptive threshold.

[0023] Preferably, the globally optimal q-quantile is obtained through a parameter selection method, the specific process of which is as follows: S2321. Select two adjacent global q-quantiles and construct denoised product co-occurrence networks corresponding to the two global q-quantiles respectively; based on the denoised product co-occurrence networks corresponding to the two global q-quantiles, obtain the modularity of the two adjacent global q-quantiles using the Louvain algorithm. S2322. Based on the modularity of two adjacent global q-quantiles, calculate and obtain the change in modularity when the q-quantile changes. S2323. Based on the co-occurrence frequency weights of the product co-occurrence network and the co-occurrence frequency weights of the denoised product co-occurrence network, calculate and obtain the retention change of the co-occurrence frequency weights. S2324. Based on the change in modularity and the change in retention of co-occurrence frequency weights when the q quantile changes, calculate the modularity-weight efficiency ratio, expressed as follows:

[0024] In the formula, Modularity-weight efficiency ratio; This represents the change in modularity when the q-quantile changes; The retained change in co-occurrence frequency weight; S2325. By selecting multiple pairs of adjacent global q quantiles, multiple modularity-weight efficiency ratios are obtained; the peak value of the modularity-weight efficiency ratio is determined by traversing multiple modularity-weight efficiency ratios. S2326. Based on the peak value of the modularity-weight efficiency ratio, determine the selected adjacent global q quantiles, and take the median of the adjacent global q quantiles as the optimal global q quantile.

[0025] In this embodiment, the choice of q quantile affects the size of the modularity. By calculating the modularity-weight efficiency ratio, the optimal q quantile can be accurately determined. The modularity-weight efficiency ratio quantifies the modularity gain that can be obtained by sacrificing 1% of the total network weight, reflecting the "cost-effectiveness" of the dynamic quantile filtering strategy. By iterating through the modularity-weight efficiency ratio, the modularity-weight efficiency ratio is... The range reached its peak, and then... The range showed a precipitous drop, therefore ,Right now This is the optimal q-quantile.

[0026] Preferably, the expression for calculating the modularity is:

[0027] In the formula, Modularity; The total weight of the product co-occurrence network for denoising; The number of nodes in the noise reduction product co-occurrence network; and For the index of the node; For nodes and Connection weights between them; The Kronecker function; For nodes The associated tag; For nodes The associated tag; To maintain node degree and In an invariant random network, the expected probability of two nodes being connected is given by: .

[0028] S3. Based on the denoised product co-occurrence network, the Louvain algorithm is used to divide it, and finally the product community network is obtained; Preferably, the specific steps of the Louvain algorithm include: S31. Initialize the denoised product co-occurrence network and treat the nodes in the denoised product co-occurrence network as communities; S32. Traverse all nodes and calculate the change in global modularity for each node when it moves to multiple adjacent communities; S33. If the largest change in global modularity among all changes in global modularity is less than or equal to 0, then the node is retained in its original community; otherwise, the node is added to the community corresponding to the largest change in global modularity, thus obtaining a new community. S34. Repeat the traversal of all nodes until no node movement occurs in a round of traversal, and a new community is obtained. S35. Using the new community as a supernode, reconstruct a new product co-occurrence network, and iteratively execute S31 to S34 until the convergence condition is met, finally obtaining the product community network.

[0029] Specifically, the reconstructed product co-occurrence network must satisfy the following mathematical properties: First, the modularity space of the new product co-occurrence network is the same as the structure of the original product co-occurrence network; Second, add the edges between communities so that the weights between communities satisfy the following additivity:

[0030] In the formula, The weights between communities C and D; This is the index of the node in community C; This is the index of the node in community D; For nodes and Connection weights between them; Third, add the edges within the community to transform them into self-loops, as expressed in the following expression:

[0031] In the formula, The total edge weight within community C; and This is the index of the node in community C; For nodes and The connection weights between them.

[0032] Preferably, the expression for the change in global modularity is:

[0033] In the formula, For nodes From the community Move to the neighboring community The change in the degree of the denoised product co-occurrence network module; For the community The sum of the weights of all internal edges; For nodes With the community The sum of the connection weights of the internal nodes; For the community Total degree of all nodes; For nodes In the community before the move The sum of the weights of all internal edges; For nodes Before moving, with the community The sum of the connection weights of all other nodes within the node; For nodes Within the community before the relocation Total degree of all nodes.

[0034] Specifically, when a node joins a neighboring community, the change in global modularity can be simplified as follows:

[0035] In the formula, For nodes Join the community At that time, the increment of global modularity; Specifically, the process of a node moving to an adjacent community implies a dual mathematical requirement for a high-quality community structure, including maximizing community cohesion and maximizing inter-community separation. The community cohesion is maximized when the actual number of connections within the community is greater than the random expectation, that is, the nodes within the community have an interaction strength greater than the random expectation. The principle of maximizing the separability between communities is to ensure that the actual number of connections between different communities is less than the random expectation, thus guaranteeing that there are clear structural boundaries between communities.

[0036] In this embodiment, multiple mechanisms are used to evaluate and optimize the results of the Louvain algorithm. These multiple mechanisms include: multiple stability enhancement mechanisms, convergence criteria, and community structure statistical indicators. The multiple stability enhancement mechanism involves running the Louvain algorithm independently multiple times, recording the partitioned product community network and its corresponding modularity after each run, and selecting the result with the highest modularity from all the results as the final result; the highest modularity can be expressed as:

[0037] In the formula, Maximum modularity; The convergence criterion, by comparing the relative rate of change of modularity between two adjacent iterations with a preset threshold, ensures that the modularity reaches a stable state. The expression is:

[0038] In the formula, Let be the modularity of the t-th iteration; Let be the modularity of the (t-1)th iteration; The community structure statistical indicators are used to quantify the cohesion and separation of a community, including intra-community density and inter-community separation; the intra-community density measures the closeness within a community, and is expressed as:

[0039] In the formula, Density within the community; This is the sum of the weights of all edges within the community; The number of nodes within the community; The inter-community separation degree measures the degree of separation between different communities, and its expression is:

[0040] In the formula, Density of connections between communities; This is the sum of the weights of all edges between communities; This represents the number of nodes in other communities.

[0041] In this embodiment, the denoising effect of the dynamic quantile filtering strategy is evaluated by constructing topology verification metrics; the topology verification metrics include network density, average path length and network diameter, and average clustering coefficient. Network density is one of the most fundamental global metrics in graph theory, measuring the overall completeness of connections between nodes in a network. In the context of a community network, network density reflects the prevalence of connections between nodes. The expression for network density is:

[0042] In the formula, Network density; The number of edges; This refers to the number of nodes in the noise reduction product co-occurrence network; in, It represents The maximum number of possible edges in an undirected graph with n nodes, therefore, the grid density The range of values ​​is ; The average path length and network diameter are key indicators for measuring network information transmission efficiency and connectivity. In this embodiment, the path length is determined by the co-occurrence strength (weight) between items, representing the reachability distance based on association strength. The average path length represents the "small-world" nature of the network, i.e., the average weighted topological distance between nodes in the network. The network diameter specifies the maximum weighted distance of the network. The steps for calculating the average path length and network diameter are as follows: In a product co-occurrence network, the co-occurrence strength (weight) between products is inversely proportional to the topological distance. This topological distance is defined using the inverse weighting method and is expressed as:

[0043] In the formula, Topological distance, i.e., node With nodes The direct distance; For nodes With nodes The number of times they co-occur, i.e., their weight; Define connection nodes With nodes The set of all paths is , where path The path The path length is the sum of the distances of all edges on the path, expressed as:

[0044] In the formula, This represents the path length. and For path The nodes on; For nodes and nodes The distance; Based on path length, node To the node The shortest distance is:

[0045] In the formula, For nodes To the node The shortest distance; To accurately calculate the shortest path between all node pairs, the following method is used: Floyd-Warshall The algorithm systematically iteratively updates the distance matrix between nodes to gradually approximate the global shortest path solution. The rules for the iterative update are as follows:

[0046] In the formula, These are temporary distance estimates during the iteration process; This is an intermediate node in the path; For nodes To the intermediate node The distance; intermediate node To the node The distance; Based on the path distance of the shortest path solution, the average path length is calculated, and the expression is:

[0047] In the formula, This is the average path length, which is the arithmetic mean of the distances between all reachable node pairs; This refers to the number of network nodes in the product co-occurrence network. The network diameter is calculated based on the path distance of the shortest path solution, expressed as:

[0048] In the formula, The network diameter is the maximum value of the shortest paths between all pairs of nodes. The average clustering coefficient is a mathematical description of the phenomenon of "friends of friends are also friends" in commodity community networks, and can effectively reveal the local clustering or community tendency in the network; the average clustering coefficient measures: at nodes i The maximum possible number of triangles constrained by the local topology (by simplicity) Within the framework of (determined by), the average clustering coefficient represents the level of actual connection strength (measured by the geometric mean of edge weights) between its neighboring nodes; the process of calculating the average clustering coefficient is as follows: The local clustering coefficient is calculated using the geometric mean method, and the expression is as follows:

[0049] In the formula, Local clustering coefficients; For nodes The degree, that is, the number of nodes. The number of triangles; For nodes With nodes The normalized weights of the edges between them; node With nodes The normalized weights of the edges between them; For nodes With nodes The normalized weights of the edges between them; For nodes A triangle with one vertex - - The strength is measured by the geometric mean of the weights of the three sides; Based on the local clustering coefficients, the average weighted clustering coefficient of the entire commodity community network is calculated, and the expression is as follows:

[0050] In the formula, The average weighted clustering coefficient is the arithmetic mean of the locally weighted clustering coefficients of all nodes, and .

[0051] In this embodiment, a dynamic quantile filtering strategy is used to denoise the product co-occurrence network, and the topology verification indexes of the two product co-occurrence networks before and after denoising are compared and analyzed. The results are shown in Table 1. Table 1 Comparison Results

[0052] Comparative analysis showed that the dynamic quantile filtering strategy significantly reduced the overall connectivity of the product co-occurrence network, decreasing the network density by 65.7%, from 31.01% to 10.64%, while the average weighted degree also decreased by 58.1%. This indicates that non-critical redundant connections in the product co-occurrence network were effectively simplified. Meanwhile, the weighted average path length, which characterizes global connectivity efficiency, remained essentially unchanged (increasing only slightly by 0.05%), while the weighted network diameter remained completely constant. The high stability of both indicates that despite a significant reduction in the number of connections, the communication efficiency between core nodes and the overall breadth of the network were fully preserved. In addition, the average weighted clustering coefficient decreased slightly by 7.5%, reflecting the simplification of local triangular connections, which is consistent with the overall optimization goal of "removing redundancy and retaining the backbone," meaning that the local structure is becoming purer. When the modularity is Q96, it can successfully avoid the risk of accidentally deleting "bridge edges" that have low local weight but are important in the global topology in practice. The most powerful proof comes from the analysis of path length and network diameter, as well as direct observation of visualization. Among them, the near-perfect stability of the weighted average path length and network diameter is the decisive evidence that the core connectivity framework of the network has not been destroyed; the network diameter is defined as the maximum value among all shortest paths, which is the embodiment of its "ultimate accessibility"; it means that even the two nodes that are farthest apart in the network have not had their access efficiency reduced; if a large number of critical global bridge edges are mistakenly deleted, the path length and diameter will inevitably increase significantly. Through force-directed layout visualization, the macroscopic topology of the product co-occurrence network before and after denoising remains basically unchanged; this is consistent with the stability of the path index, which together indicate that most of the removed edges are "unstructured redundancies" attached to the core skeleton, and their removal did not cause the skeleton to collapse or reconstruct. The 65.7% reduction in network density confirms that the dynamic quantile filtering strategy effectively removes a massive number of low-weight connections; the slight decrease of only 7.5% in the average weighted clustering coefficient indicates that the filtering process accurately removes a large number of "shallow" triangles that contribute little to the local clustering structure, while retaining the "deep" triangles that define the core of the community, thus achieving purification of the local structure. Based on the above analysis, the dynamic quantile filtering strategy achieves precise control over the network architecture by imposing constraints based on the local weight distribution of nodes. The core of this intervention mechanism lies in achieving an optimal balance between maximizing the preservation and enhancement of local connection density and systematically removing topological noise to optimize global routing efficiency. Ultimately, this strategy successfully reconstructs an initially inefficiently connected network into a network with both high modularity and efficient global integration capabilities.

[0053] The present invention has the following beneficial effects: This invention presents a dynamic quantile filtering method for product co-occurrence networks. By introducing a dynamic quantile filtering strategy, the product co-occurrence network is preprocessed and denoised. The dynamic quantile can adaptively handle the heterogeneity of degree distribution among different nodes in the product co-occurrence network, eliminating weak signal noise in the network, and making real and stable product associations (such as complementary goods, substitutes, or goods in the same consumption scenario) more prominent in the topology. This dynamic adaptive mechanism fundamentally avoids the "peak shaving and valley filling" information loss caused by a globally uniform threshold, thereby significantly improving the quality of the input graph structure for the subsequent Louvain algorithm.

[0054] By employing the Louvain algorithm to perform community discovery based on the optimized network, tightly coupled product communities can be identified more accurately. These communities correspond to real consumption scenarios (such as "barbecue ingredients community", "baby products community" or "office stationery community") and have higher business interpretability.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic quantile filtering method for product co-occurrence networks, characterized in that, include: S1. Construct a product co-occurrence network based on the co-occurrence relationships between products in orders; S2. Based on the product co-occurrence network, denoise it using a dynamic quantile filtering strategy to obtain a denoised product co-occurrence network; The execution process of the dynamic quantile filtering strategy includes: S21. Based on the product co-occurrence network, obtain the co-occurrence frequency weights between all two products and combine them into a product co-occurrence matrix; The co-occurrence frequency weight is the total number of times the two products appear together in all orders; S22. Based on the product co-occurrence matrix, obtain the weights of all non-zero co-occurrence frequencies for each node; S23. Define a local adaptive threshold for obtaining the weights of all non-zero co-occurrence frequencies of a node, based on the weights of all non-zero co-occurrence frequencies of each node. The local adaptive threshold is the q-quantile of the weights of all non-zero co-occurrence frequencies of a node. The steps to obtain the local adaptive threshold are as follows: S2311. Arrange all non-zero co-occurrence frequency weights of the nodes in ascending order to obtain the non-zero co-occurrence frequency weight sequence; S2312. Using linear interpolation, calculate the position index of the globally optimal q-quantile in the non-zero co-occurrence frequency weight sequence; The globally optimal q-quantile is obtained through a parameter selection method, specifically as follows: S23121. Select two adjacent global q-quantiles and construct denoised product co-occurrence networks corresponding to the two global q-quantiles respectively; based on the denoised product co-occurrence networks corresponding to the two global q-quantiles, obtain the modularity of the two adjacent global q-quantiles using the Louvain algorithm. S23122. Based on the modularity of two adjacent global q-quantiles, calculate and obtain the change in modularity when the global q-quantile changes. S23123. Based on the co-occurrence frequency weights of the product co-occurrence network and the co-occurrence frequency weights of the denoised product co-occurrence network, calculate and obtain the retention change of the co-occurrence frequency weights. S23124. Based on the change in modularity and the change in retention of co-occurrence frequency weights when the global q-quantile changes, calculate the modularity-weight efficiency ratio, expressed as follows: In the formula, Modularity-weight efficiency ratio; This represents the change in modularity when the global q-quantile changes; The retained change in co-occurrence frequency weight; S23125. By selecting multiple pairs of adjacent global q-quantiles, multiple modularity-weight efficiency ratios are obtained; the peak value of the modularity-weight efficiency ratio is determined by traversing multiple modularity-weight efficiency ratios. S23126. Based on the peak value of the modularity-weight efficiency ratio, determine the selected adjacent global q quantiles, and take the median of the adjacent global q quantiles as the optimal global q quantile. S2313. Calculate the local adaptive threshold based on the position index and the non-zero co-occurrence frequency weight sequence; S24. Filter nodes using a local adaptive threshold: If the non-zero co-occurrence frequency weight is greater than or equal to the local adaptive threshold, retain the node's non-zero co-occurrence frequency weight; otherwise, reset the node's non-zero co-occurrence frequency weight to zero; if all of the node's non-zero co-occurrence frequency weights are reset to zero, retain the node as an isolated node in the product co-occurrence network. S25. Construct an adjacency matrix based on the retained non-zero co-occurrence frequency weights; use an intersection strategy to symmetricize the adjacency matrix to obtain a weighted adjacency matrix; S26. Update the product co-occurrence network according to the weighted adjacency matrix to obtain the denoised product co-occurrence network; S3. Based on the denoised product co-occurrence network, the Louvain algorithm is used to divide it, and finally the product community network is obtained.

2. The dynamic quantile filtering method for product co-occurrence networks according to claim 1, characterized in that, The expression for calculating the modularity is: In the formula, Modularity; The total weight of the noise-reducing product co-occurrence network; The number of nodes in the noise reduction product co-occurrence network; and For the index of the node; For nodes and Connection weights between them; The Kronecker function; For nodes The associated tag; For nodes The associated tag; To maintain node degree and In an invariant random network, the expected probability of two nodes being connected is given by: .

3. The dynamic quantile filtering method for product co-occurrence networks according to claim 2, characterized in that, The specific steps of the Louvain algorithm include: S31. Initialize the denoised product co-occurrence network and treat the nodes in the denoised product co-occurrence network as communities; S32. Traverse all nodes and calculate the change in global modularity for each node when it moves to multiple adjacent communities; S33. If the largest change in global modularity among all changes in global modularity is less than or equal to 0, then the node is retained in its original community; otherwise, the node is added to the community corresponding to the largest change in global modularity, thus obtaining a new community. S34. Repeat the traversal of all nodes until no node movement occurs in a round of traversal, and a new community is obtained. S35. Using the new community as a supernode, reconstruct a new product co-occurrence network, and iteratively execute S31 to S34 until the convergence condition is met, finally obtaining the product community network.

4. The dynamic quantile filtering method for product co-occurrence networks according to claim 3, characterized in that, The expression for the change in global modularity is: In the formula, For nodes From the community Move to the neighboring community The change in the degree of the denoised product co-occurrence network module; For the community The sum of the weights of all internal edges; For nodes With the community The sum of the connection weights of the internal nodes; For the community The total degree of all nodes within the node; For nodes In the community before the move The sum of the weights of all internal edges; For nodes Before moving, with the community The sum of the connection weights of all other nodes within the node; For nodes In the community before the move The total degree of all nodes within the range.

Images