Joint method, device and equipment for network reconstruction and overlapping community detection, and medium

Abstract

The invention discloses a network reconstruction and overlapping community detection combination method and device, equipment and a medium. The method comprises the steps that binary time sequence data and the community number of a community network are acquired; initializing a community membership matrix and a community interaction matrix; iterating the community membership matrix and the community interaction matrix by using an expectation maximization algorithm to obtain a target community membership matrix; and determining a community structure of the community network and overlapping communities in the community network based on the target community membership matrix. Community membership variables are explicitly introduced into a parameter space, so that network inference and community detection which are regarded as independent problems form a unified optimization architecture, specific contribution of each edge to observation dynamics is tracked through responsibility variables, deep coupling and mutual promotion of network inference and community detection are ensured, and the network inference and community detection efficiency is improved. According to the method, the bottom layer overlapping community structure of the complex network is detected, the hidden network topology structure is accurately reconstructed, and the reconstruction precision of the complex network is ensured.

Description

Technical Field

[0001] This application relates to the field of complex network reconstruction technology, and in particular to a combined method, apparatus, device and medium for network reconstruction and overlapping community detection. Background Technology

[0002] The study of complex networks occupies an important position in contemporary science. From protein-protein interaction networks in molecular biology to disease transmission contact networks in epidemiology, and interpersonal relationship networks in social sciences, these networks generally exhibit rich organizational structures. Therefore, studying the topology and properties of complex networks is crucial for promoting technological progress in related fields. However, existing schemes for constructing complex network models for community detection mostly only observe the evolution of node states (infected / susceptible, adopted / not adopted, etc.) over time, without studying the community structure information of binary state time series. Furthermore, the generative models do not explicitly model the relationship between community information and observation dynamics, thus failing to fully utilize the characteristic traces left by community structure in the binary state evolution, which in turn affects the accuracy of complex network reconstruction.

[0003] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a combined method, apparatus, device and medium for network reconstruction and overlapping community detection, addressing the shortcomings of the prior art.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a joint method for network reconstruction and overlapping community detection, wherein the joint method for network reconstruction and overlapping community detection specifically includes: Obtain binary time series data and the number of communities in the community network. The binary time series data includes the state data of each node in the community network at each time step, and the state data is the infection state or the susceptible state. Initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix is ​​used to reflect the probability that each node in the community network belongs to each community, and the community interaction matrix is ​​used to reflect the interaction relationship between communities. The target community membership matrix is ​​obtained by iterating through the community membership matrix and community interaction matrix using the expectation-maximization algorithm. The community structure of the community network and the overlapping communities in the community network are determined based on the target community membership matrix; In the process of iterating the community membership matrix and community interaction matrix using the expectation-maximization algorithm, each iteration includes an expectation step and a maximization step, wherein: The expected step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining the responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition. The maximization step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iterative community membership matrix and community interaction matrix.

[0006] The joint method for network reconstruction and overlapping community detection, wherein obtaining the observation transition probability of each node in the community network specifically includes: For each node in the community network, the edge weights between the node and other nodes in the community network are determined based on the community interaction matrix and the community membership matrix. Obtain the neighbor set of the node, and calculate the observation transition probability of the node using the edge weights between the node and the neighbor nodes in the neighbor set whose state data is infected.

[0007] The joint method for network reconstruction and overlapping community detection, wherein obtaining the observation transition probability of each node in the community network specifically includes: The conditional probability of each node being infected is calculated based on the binary time series data, and this conditional probability is used as the observation transition probability of each node.

[0008] The joint method for network reconstruction and overlapping community detection includes a community responsibility variable and a noise responsibility variable. The community responsibility variable is used to determine the contribution of other nodes in the community network to the cause of node infection, and the noise responsibility variable is used to determine the contribution of the noise vector to the cause of node infection.

[0009] The joint method for network reconstruction and overlapping community detection, wherein the community membership matrix and the community interaction matrix are updated based on the fact that the interaction strength of the community and the number of infections observed within the community are proportional to the interaction strength of the community.

[0010] The joint method for network reconstruction and overlapping community detection, wherein the process of determining the community structure based on the community network specifically includes: Calculate the edge probability between any two nodes in the community network based on the target community membership matrix; For each node in the community network, the main community to which the node belongs is determined based on the target community membership matrix. The threshold within the community is calculated based on the edge probability between the node and the first node, and the threshold outside the community is calculated based on the edge probability between the node and the second node. The main community to which the first node belongs is the same as the main community to which the node belongs, and the main community to which the second node belongs is the same as the main community to which the node belongs. Based on the in-community threshold and the out-of-community threshold, edges are constructed between the node and other nodes in the community network.

[0011] The joint method for network reconstruction and overlapping community detection further includes: If there is a time step in the binary time series data where the state data of each node is the same, then the state data of some nodes in the time step are randomly adjusted so that there are at least two nodes with different state data in the time step.

[0012] A second aspect of this application provides a combined apparatus for network reconstruction and overlapping community detection, wherein the combined apparatus for network reconstruction and overlapping community detection specifically includes: The acquisition module is used to acquire binary time series data of the community network and the number of communities. The binary time series data includes the state data of each node in the community network at each time step, and the state data is the infection state or the susceptible state. An initialization module is used to initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix reflects the probability that each node in the community network belongs to each community, and the community interaction matrix reflects the interaction relationship between communities. An iterative module is used to iterate the community membership matrix and community interaction matrix using the expectation-maximization algorithm to obtain the target community membership matrix; The determination module is used to determine the community structure of the community network and the overlapping communities in the community network based on the target community membership matrix; In the process of iterating the community membership matrix and community interaction matrix using the expectation-maximization algorithm, each iteration includes an expectation step and a maximization step, wherein: The expected step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining a responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition. The maximization step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iteratively updated community membership matrix and community interaction matrix.

[0013] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the joint method of network reconstruction and overlapping community detection as described above.

[0014] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the joint method of network reconstruction and overlapping community detection as described above.

[0015] Beneficial effects: 1. This application explicitly introduces community membership variables into the parameter space, enabling network inference and community detection, which are treated as independent problems, to form a unified optimization architecture. Then, by using a responsibility variable that reflects the contribution of each infected node to the observed state transition, the specific contribution of each edge to the observation dynamics is tracked, ensuring deep coupling and mutual promotion between network inference and community detection. This not only enables the detection of the underlying overlapping community structure of complex networks but also accurately reconstructs the hidden network topology, while ensuring the reconstruction accuracy of complex networks.

[0016] 2. This application explicitly models the community structure in the generative model of network dynamics, making it an intrinsic element of the inference process. Then, by encoding edge weights through community membership, the unique signature of community information in observation dynamics can be fully utilized. This allows community information to be accurately tracked and estimated, thereby improving the inference accuracy of complex networks.

[0017] 3. This application, through community structure constraints, significantly reduces the dimensionality of model parameters compared to traditional statistical inference methods, directly improving data efficiency. Compared to compressed sensing, which requires the selection of sparsity parameters λ, and Lasso, which requires cross-validation of regularization parameters, the parameters of this invention are entirely determined by EM optimization, requiring no additional hyperparameter tuning. Compared to deep learning methods, which require large amounts of data and computational resources, this invention achieves its goal with a simpler probabilistic model.

[0018] 4. This application makes the noise term more tolerant to errors in the observation data by explicitly modeling it and estimating it in the EM framework, so that low-quality observation data can also be used for complex network reconstruction, thus reducing the requirements for observation data. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of the joint method for network reconstruction and overlapping community detection provided in the embodiments of this application.

[0021] Figure 2 A flowchart illustrating the principle of the joint method for network reconstruction and overlapping community detection provided in this application embodiment.

[0022] Figure 3 A schematic diagram of the combined device for network reconstruction and overlapping community detection provided in the embodiments of this application.

[0023] Figure 4 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0024] This application provides a combined method, apparatus, device, and medium for network reconstruction and overlapping community detection. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0025] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0026] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0027] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0028] Research has shown that complex networks occupy an important position in contemporary science. From protein-protein interaction networks in molecular biology to disease transmission and contact networks in epidemiology, and interpersonal relationship networks in social sciences, these networks generally exhibit rich organizational structures. Therefore, studying the topology and properties of complex networks plays a crucial role in promoting technological progress in related fields. However, existing schemes for constructing complex network models for community detection mostly only observe the evolution of node states (infected / susceptible, adopted / not adopted, etc.) over time, without studying the community structure information of binary state time series. Furthermore, the generative models do not explicitly model the relationship between community information and observation dynamics, thus failing to fully utilize the characteristic traces left by community structure in the binary state evolution, which in turn affects the accuracy of complex network reconstruction.

[0029] Furthermore, existing network inference methods, even when specifically designed for network reconstruction tasks, still have their limitations. For example, they typically require tens of thousands of time steps to achieve acceptable reconstruction accuracy, or they rely on explicit sparsity patterns and the randomness of the measurement matrix, making them insufficient for specific organizational patterns such as community structures. They also struggle to handle nonlinear dynamics and require cross-validation to select regularization parameters. Moreover, existing network inference methods lack sufficient robustness to noise and errors in the observation data, making reconstruction accuracy heavily dependent on the quality of the observation data; when the quality of the observation data deteriorates, the reconstruction accuracy rapidly declines.

[0030] Therefore, the core technical problem that this application needs to solve is how to detect the underlying overlapping community structure of complex networks and accurately reconstruct the hidden network topology from extremely limited binary state time series observation data that may contain noise.

[0031] To address the core technical challenges, this application acquires binary time-series data and the number of communities in a community network; initializes a community membership matrix and a community interaction matrix, whereby the community membership matrix reflects the probability of each node belonging to each community, and the community interaction matrix reflects the interactions between communities; iterates the community membership matrix and community interaction matrix using an expectation-maximization algorithm to obtain a target community membership matrix; and determines the community structure and overlapping communities in the community network based on the target community membership matrix. This application explicitly introduces community membership variables into the parameter space, enabling network inference and community detection, treated as independent problems, to form a unified optimization architecture. Then, by using a responsibility variable reflecting the contribution of nodes in each infected state to the observed state transition, the specific contribution of each edge to the observation dynamics is tracked, ensuring deep coupling and mutual promotion between network inference and community detection. This not only enables the detection of the underlying overlapping community structure of complex networks but also accurately reconstructs the hidden network topology, while ensuring the reconstruction accuracy of complex networks.

[0032] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0033] This embodiment provides a joint method for network reconstruction and overlapping community detection, such as... Figure 1 and Figure 2 As shown, the method specifically includes: S10. Obtain the binary time series data of the community network and the number of communities.

[0034] Specifically, community networks can be social relationship networks, computer networks, power system networks, transportation networks, and biomolecular networks, etc. The number of communities refers to the number of communities within the community network. A community is defined as a combination of nodes in the network model corresponding to the community network. Nodes within the same community are tightly connected, while connections between nodes in different communities are sparse. Different communities can include the same nodes; that is, a node can belong to multiple communities.

[0035] A community network model can be described as a community network graph. , Represents a set of nodes. Let represent the set of edges. At time ... Nodes in a node set The binary state is 0 represents a susceptible state, and 1 represents an infected state. The binary time series data is used to reflect the state data of each node in the network model of the community network at each time step, and the state data is either an infected state or a susceptible state.

[0036] For example, binary time series data includes Node at If the state data at each time step is given, then the binary time series data is: 3D matrix Indicates the number of time steps. Indicating the number of nodes, binary time series data can reflect the community network's... Each node in the node is The state data for each time step in the matrix. Where, if the elements in the matrix... A value of 0 indicates a node. At time step The state data represents susceptible states; if the elements in the matrix... A value of 1 indicates a node. At time step The status data indicates an infection status.

[0037] Furthermore, since many real-world nodes in practical applications do indeed have multiple identities or belong to multiple groups, this application addresses each stage in each community network. Categorize one Community membership vector of dimension ,in, Represents a node Belonging to the community The probability, Represents community data, The value range is 0-1, and This application achieves this by providing each node segment. Categorize one Community membership vector of dimension This community membership vector allows nodes in a community network to belong to multiple communities with varying probabilities, providing a more accurate reflection of real-world network structures. For example, a community network might include community A and community B, with nodes... The probability of belonging to community A is 0.7, and the probability of belonging to community B is 0.3; Node The probability of belonging to community A is 0.1, and the probability of belonging to community B is 0.9. Then the node... The community membership vector is [0.7, 0.3], and the node... The community membership vector is [0.1, 0.9].

[0038] S20. Initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix is ​​used to reflect the probability that each node in the community network belongs to each community, and the community interaction matrix is ​​used to reflect the interaction relationship between communities.

[0039] Specifically, the community membership matrix includes the community membership vector of each node in the community network. In other words, the community membership vector is a matrix formed by the community membership vectors of all nodes in the community network. This community membership matrix can be used to reflect the probability that each node in the community network belongs to each community.

[0040] Since nodes in a community network interact with each other, this application constructs a community interaction matrix to reflect the interaction relationships between communities within the network. The community interaction matrix is ​​as follows: The community interaction matrix represents the probability of nodes forming edges within the same community. However, in practical applications, nodes within the same community tend to connect to each other, while connections between nodes in different communities are less frequent; that is, the probability of edges forming between nodes within the same community is high, while the probability of edges forming between nodes in different communities is low. Therefore, the community interaction matrix can be simplified to a single matrix. Dimensional Community Interaction Vector , Indicates community Average edge weight between pairs of nodes.

[0041] It should be noted that when the joint processing of network reconstruction and overlapping community detection begins, both the community membership matrix and the community interaction matrix are unknown and need to be solved iteratively. Therefore, during the joint processing of network reconstruction and overlapping community detection, initial community membership and community interaction matrices are constructed through initialization. For example, the community membership and community interaction matrices can be constructed using random initialization.

[0042] S30. The community membership matrix and community interaction matrix are iterated using the expectation-maximization algorithm to obtain the target community membership matrix.

[0043] Specifically, the Expectation-Maximization (EM) algorithm is an iterative parameter estimation algorithm that solves for the maximum likelihood (MLE) or maximum a posteriori (MAP) estimate of a probability model containing latent variables or missing data by alternately executing the expectation step (E-step) and the maximization step (M-step). In other words, this application uses the Expectation-Maximization algorithm to solve for the community membership matrix and the community interaction matrix to obtain the target community membership matrix.

[0044] In this embodiment of the application, when using the EM algorithm to solve the target community membership matrix, the expected infection intensity is first determined through the E-step, and the responsibility variable is determined using the expected infection intensity to reflect the contribution of each infected node to the observed state transition; then, the community membership matrix, the community interaction matrix, and the noise vector are updated based on the responsibility variable through the M-step to achieve the iteration of the community membership matrix and the community interaction matrix.

[0045] Based on this, in the process of iterating the community membership matrix and community interaction matrix using the EM algorithm, each iteration process includes an E-step and an M-step, where: The E-step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining the responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition.

[0046] The M-step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iterative community membership matrix and community interaction matrix.

[0047] Specifically, in order to model network dynamics as a probabilistic model, nodes are defined. At any moment The expected value of transitioning from susceptible to infected is: , in, Represents a node At any moment The expected value of transitioning from susceptible to infected. It is a state exponential function, when the node In time When infected, ,node In time When not infected, , Represents a node Belonging to the community The probability, Represents a node Belonging to the community The probability, Indicates community Average edge weight between pairs of nodes Indicates the probability of observation transition. This represents a noise vector used to explain randomness and measurement errors that the model cannot capture.

[0048] It can be understood that the observed transition probability can be the conditional probability directly calculated from the binary time series data, i.e., at the node... Current susceptible status, nodes At all points in the current infection state, the node The frequency of infection at the next moment. This conditional probability requires no prior knowledge of the network structure and relies solely on binary time series data. In other words, obtaining the observed transition probability of each node in the community network can be calculated based on the binary time series data, showing the node's position at time step [missing information]. The conditional probability of infection is calculated, and this conditional probability is used as the observation transition probability for each node.

[0049] However, in practical applications, the probability of edges forming between nodes in the same community is high, while the probability of edges forming between nodes in different communities is low. To capture the key characteristics of this community structure (i.e., strong intra-community connections and weak cross-community connections), this embodiment utilizes community membership vectors to encode the observation transition probabilities of nodes, so that the observation transition probabilities can reflect the key characteristics of the community structure. Therefore, obtaining the observation transition probability of each node in the community network specifically includes: For each node in the community network, the edge weights between the node and other nodes in the community network are determined based on the community interaction matrix and the community membership matrix. Obtain the neighbor set of the node, and calculate the observation transition probability of the node using the edge weights between the node and the neighbor nodes in the neighbor set whose state data is infected.

[0050] Specifically, the node observation transition probability is used to associate edge weights with the community structure. The edge weights are encoded using the community interaction matrix and the community membership matrix. The edge weight of each node in the community network can be expressed as: , in, Represents a node and nodes Edge weights between them Represents a node Belonging to the community The probability, Represents a node Belonging to the community The probability, Indicates community Average edge weight between pairs of nodes.

[0051] This application utilizes community interaction matrices and community membership matrices to determine edge weights, capturing the key characteristic of strong intra-community connectivity and weak cross-community connectivity in community structures. This is because when nodes... and nodes Belongs to the community At that time, on the side The edge weights will be through Term increases, conversely, node decreases. Belongs to the community ,node Belongs to the community At that time, on the side The edge weights will be through The term decreases.

[0052] After obtaining the edge weights between any two nodes, the observation transition probability of the nodes is determined using these edge weights. Nodes with observation transition probabilities indicating a susceptible state are identified. At any moment The probability of infection can be expressed as: , in, Nodes representing susceptible states At any moment The probability of being infected It is a node The neighborhood group, It is a standardized infection rate. It is a constant that controls the overall infection tendency. It is the average degree.

[0053] Furthermore, the Poisson distribution is used to model the state transition, which can be expressed as: , in, Represents a node The state indicator function indicates all time steps when the function value is 0. Indicates the number of time steps. This indicates the number of nodes in the community network. The parameters to be inferred include the community membership matrix, community interaction matrix, and noise vector. Represents a node At any moment The expected value of transitioning from susceptible to infected.

[0054] To further infer, the log-likelihood function is calculated for all observed state transition events. The log-likelihood function can be expressed as: , Summation only iterates through those nodes. The model is in a vulnerable state, because state transitions are only possible under such circumstances. This log-likelihood function expresses how well the model parameters fit the observed data: the higher the likelihood value, the greater the probability that the model will generate the observed data.

[0055] However, directly maximizing this likelihood function is difficult because the logarithm inherently involves summation. Therefore, embodiments of this application apply Jensen's inequality to obtain an optimizable lower bound: , in, Represents the community responsibility variable. This represents the noise liability variable.

[0056] Therefore, the responsibility variables determined through the E-step include community responsibility variables and noise responsibility variables. The community responsibility variable represents the contribution of other nodes in the community network to the cause of node infection, while the noise responsibility variable represents the contribution of the noise vector to the cause of node infection. Specifically, for each time step... Each node pair and every community The community responsibility variable and the noise responsibility variable can be expressed as: , .

[0057] The above completes the explanation of step E. The following section explains step M.

[0058] In the M-step, the parameters to be inferred are updated using the responsibility variables. The community membership matrix, community interaction matrix, and noise vector are updated using the community responsibility variable matrix and noise responsibility variable matrix formed by the community responsibility variables. Specifically, the community membership matrix and the community interaction matrix are updated based on the principle that they are proportional to the interaction strength within a community and the number of infections observed within that community. For the community membership matrix, the update equation is: , For the community interaction matrix, the update equation is: , The community interaction matrix and its update equation reflect the community The intensity of interaction is directly proportional to the number of infection events observed within the community, but according to the two nodes in the community The degree of membership in the data is weighted.

[0059] For the noise vector, the update equation is: , The update equation for this noise vector is the node The average number of "unexplained infection" events at all times, i.e. infection events that cannot be explained from known infected neighbors, is used to quantify model uncertainty and observation noise.

[0060] In this embodiment, the E-step and M-step are executed alternately using the EM algorithm. In each iteration, the log-likelihood function value is monotonically increased (or at least not decreased) until the parameter change is less than the convergence threshold, so as to obtain the target community membership matrix.

[0061] S40. Determine the community structure of the community network and the overlapping communities in the community network based on the target community membership matrix.

[0062] Specifically, the community structure reflects the topological structure of nodes in a community network, while overlapping communities reflect the overlapping structure of the node community network. Since the community membership matrix includes the community membership vector of each node, the community to which a node belongs can be determined through this vector, and overlapping communities in the community network can be identified based on the target community membership matrix.

[0063] Furthermore, after obtaining the target community membership matrix, the probability of edges forming between node pairs in the community network can be determined. The formula for calculating the probability of edges forming between node pairs is as follows: This probability integrates edge weights predicted based on community affiliation and node values ​​determined from binary time series data. and The degree of mutual influence between nodes improves the accuracy of this probability. Furthermore, after determining the probability of forming an edge between node pairs, this probability can be directly used to form edges between node pairs to obtain the community node.

[0064] In one embodiment, the process of determining the community structure based on the community network specifically includes: Calculate the edge probability between any two nodes in the community network based on the target community membership matrix; For each node in the community network, the main community to which the node belongs is determined based on the target community membership matrix. The threshold within the community is calculated based on the edge probability between the node and the first node, and the threshold outside the community is calculated based on the edge probability between the node and the second node. Based on the in-community threshold and the out-of-community threshold, edges are constructed between the node and other nodes in the community network.

[0065] Specifically, the main community is determined based on the node's community membership vector. For example, the community with the highest membership degree in the community membership vector can be used as the main community, or the community with a membership degree greater than a preset threshold (e.g., 0.5) can be used as the main community. After determining the main community of a node, an in-community threshold and an out-of-community threshold are calculated based on this main community. The in-community threshold is calculated based on the edge probability between the node and a first node, and the out-of-community threshold is calculated based on the edge probability between the node and a second node. The main community to which the first node belongs is the same as the main community to which the node belongs, and the main community to which the second node belongs is the same as the main community to which the node belongs. For example, the in-community threshold is the average edge probability between the node and all first nodes, and the out-of-community threshold is the average edge probability between the node and all second nodes. Of course, in practical applications, the in-community threshold and the out-of-community threshold can also be calculated in other ways, such as selecting the median of the edge probability between the node and all first nodes as the in-community threshold, and selecting the median of the edge probability between the node and all second nodes as the out-of-community threshold.

[0066] After obtaining the intra-community threshold and the extra-community threshold, edges are constructed for intra-community node pairs with edge probabilities greater than the intra-community threshold, and for intra-community node pairs with edge probabilities greater than the extra-community threshold, so as to construct edges between the node and other nodes in the community network, thereby obtaining the network structure of the community network.

[0067] This application does not use hard thresholding of community membership to obtain hard community partitions. Instead, it preserves the complete community membership vector, then uses this vector to determine the main community, and constructs intra-community and extra-community thresholds for each node using the main community. Finally, it uses these thresholds to construct edges between nodes. Preserving the complete community membership vector allows for a precise description of the community structure, thus improving edge accuracy. Furthermore, this community membership vector provides a precise description of overlapping community structures, enabling fine-grained analysis of overlapping community structures. This method directly detects overlapping communities from binary time-series data, and simultaneously detects the underlying overlapping community structure of complex networks while accurately reconstructing the hidden network topology.

[0068] Furthermore, in practical applications, to avoid the system falling into an absorption state (where all nodes are susceptible), the method further includes randomly adjusting the state data of some nodes in a time step in which the state data of each node is the same, so that there are at least two nodes with different state data in the time step.

[0069] In summary, this embodiment provides a joint method for network reconstruction and overlapping community detection. The method includes acquiring binary time-series data and the number of communities in a community network; initializing a community membership matrix and a community interaction matrix for the community network, whereby the community membership matrix reflects the probability of each node belonging to each community and the community interaction matrix reflects the interaction relationships between communities; iterating the community membership matrix and the community interaction matrix using an expectation-maximization algorithm to obtain a target community membership matrix; and determining the community structure and overlapping communities in the community network based on the target community membership matrix. This application explicitly introduces community membership variables into the parameter space, enabling network inference and community detection, which are treated as independent problems, to form a unified optimization architecture. Then, by using a responsibility variable reflecting the contribution of nodes in each infected state to the observed state transition, the specific contribution of each edge to the observation dynamics is tracked, ensuring deep coupling and mutual promotion between network inference and community detection. This not only enables the detection of the underlying overlapping community structure of complex networks but also accurately reconstructs the hidden network topology, while ensuring the reconstruction accuracy of complex networks.

[0070] In addition, to illustrate the feasibility of the embodiments of this application, this application uses simulation experiments and real data verification, including multiple aspects such as comparative studies, parameter sensitivity analysis, and robustness evaluation.

[0071] The simulation experiment used a hybrid membership random block model to generate a network with a controllable community structure. The network size was set to 100 nodes, and the number of communities was set to 2, 3, and 4 to cover different complexities. Community strength was adjusted by modifying the community affinity matrix. The off-diagonal elements (range 0.0-0.2) are used for control, with larger off-diagonal elements indicating weaker community structure. Parameters of the susceptibility-infection-susceptibility dynamics include: recovery rate. Initial infection rate Standardized infection rate Each experiment was independently repeated 5 times, and the results were averaged to eliminate random fluctuations.

[0072] Experimental results fully demonstrate the superiority of this application. (Regarding time series length...) As the number of networks increased from 500 to 2000, the network reconstruction F1 score of this application remained stable at a high level, outperforming the traditional EM algorithm and the Lasso method. Regarding overlapping community detection, the cosine similarity (CS) score of this application improved with increasing frequency and density. It increases and approaches 1.0. The RMSE metric reflects the accuracy of the community affiliation probability. This application... At that time, the RMSE metric, which reflects the accuracy of community affiliation probability, is close to 0.

[0073] Parameter sensitivity analysis reveals the robustness of the algorithm. When the number of communities... When the score increased from 2 to 10, both the F1 and CS scores showed a reasonable decrease, which was expected—increasing the question dimensionality does indeed increase the difficulty of inference. However, even... In high-dimensional cases, the performance of this application is still significantly better than the comparative methods. When the community structure weakens, the F1 score remains stable, while the CS score drops more significantly, reflecting that community detection is naturally more difficult when communities are hard to distinguish, but network reconstruction is unaffected.

[0074] Noise robustness testing was conducted by randomly flipping binary observations at different ratios. Results show that the invention maintains usable performance even at extremely high noise levels (30%).

[0075] Real-world validation was performed on two standard benchmark datasets. The Zachary Club Network (34 nodes, 78 edges, 2 known communities) and the Dolphins Social Network (62 nodes, 159 edges, 2 known communities) are classic test sets in the field of community detection, with clear annotations of real communities. Experimental results fully demonstrate the generalization ability of this invention on real-world data.

[0076] Based on the aforementioned joint method for network reconstruction and overlapping community detection, this embodiment provides a joint apparatus for network reconstruction and overlapping community detection, such as... Figure 3 As shown, the joint device for network reconstruction and overlapping community detection specifically includes: The acquisition module 100 is used to acquire binary time series data of the community network and the number of communities. The binary time series data includes the state data of each node in the community network at each time step, and the state data is the infection state or the susceptible state. The initialization module 200 is used to initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix is ​​used to reflect the probability that each node in the community network belongs to each community, and the community interaction matrix is ​​used to reflect the interaction relationship between communities. The iteration module 300 is used to iterate the community membership matrix and community interaction matrix using the expectation-maximization algorithm to obtain the target community membership matrix; The determination module 400 is used to determine the community structure of the community network and the overlapping communities in the community network based on the target community membership matrix; In the process of iterating the community membership matrix and community interaction matrix using the expectation-maximization algorithm, each iteration includes an expectation step and a maximization step, wherein: The expected step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining a responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition. The maximization step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iteratively updated community membership matrix and community interaction matrix.

[0077] Based on the above-described joint method for network reconstruction and overlapping community detection, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the joint method for network reconstruction and overlapping community detection as described in the above embodiment.

[0078] Based on the joint method of network reconstruction and overlapping community detection described above, this application also provides a terminal device, such as... Figure 4 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0079] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0080] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0081] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0082] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A joint method for network reconstruction and overlapping community detection, characterized in that, The joint method of network reconstruction and overlapping community detection specifically includes: Obtain binary time series data and the number of communities in the community network. The binary time series data includes the state data of each node in the community network at each time step, and the state data is the infection state or the susceptible state. Initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix is ​​used to reflect the probability that each node in the community network belongs to each community, and the community interaction matrix is ​​used to reflect the interaction relationship between communities. The target community membership matrix is ​​obtained by iterating through the community membership matrix and community interaction matrix using the expectation-maximization algorithm. The community structure of the community network and the overlapping communities in the community network are determined based on the target community membership matrix; In the process of iterating the community membership matrix and community interaction matrix using the expectation-maximization algorithm, each iteration includes an expectation step and a maximization step, wherein: The expected step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining the responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition. The maximization step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iterative community membership matrix and community interaction matrix.

2. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The specific steps of obtaining the observed transition probability of each node in the community network include: For each node in the community network, the edge weights between the node and other nodes in the community network are determined based on the community interaction matrix and the community membership matrix. Obtain the neighbor set of the node, and calculate the observation transition probability of the node using the edge weights between the node and the neighbor nodes in the neighbor set whose state data is infected.

3. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The specific steps of obtaining the observed transition probability of each node in the community network include: The conditional probability of each node being infected is calculated based on the binary time series data, and this conditional probability is used as the observation transition probability of each node.

4. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The responsibility variables include community responsibility variables and noise responsibility variables. The community responsibility variables are used to determine the contribution of other nodes in the community network to the cause of node infection, and the noise responsibility variables are used to determine the contribution of the noise vector to the cause of node infection.

5. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The community membership matrix and the community interaction matrix are updated based on the fact that they are proportional to the interaction strength of the community and the number of infections observed within the community.

6. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The process of determining the community structure based on the community network specifically includes: Calculate the edge probability between any two nodes in the community network based on the target community membership matrix; For each node in the community network, the main community to which the node belongs is determined based on the target community membership matrix. The threshold within the community is calculated based on the edge probability between the node and the first node, and the threshold outside the community is calculated based on the edge probability between the node and the second node. The main community to which the first node belongs is the same as the main community to which the node belongs, and the main community to which the second node belongs is the same as the main community to which the node belongs. Based on the in-community threshold and the out-of-community threshold, edges are constructed between the node and other nodes in the community network.

7. The joint method for network reconstruction and overlapping community detection according to claim 1, characterized in that, The method further includes: If there is a time step in the binary time series data where the state data of each node is the same, then the state data of some nodes in the time step are randomly adjusted so that there are at least two nodes with different state data in the time step.

8. A combined device for network reconstruction and overlapping community detection, characterized in that, The combined device for network reconstruction and overlapping community detection specifically includes: The acquisition module is used to acquire binary time series data of the community network and the number of communities. The binary time series data includes the state data of each node in the community network at each time step, and the state data is the infection state or the susceptible state. An initialization module is used to initialize the community membership matrix and community interaction matrix of the community network. The community membership matrix reflects the probability that each node in the community network belongs to each community, and the community interaction matrix reflects the interaction relationship between communities. An iterative module is used to iterate the community membership matrix and community interaction matrix using the expectation-maximization algorithm to obtain the target community membership matrix; The determination module is used to determine the community structure of the community network and the overlapping communities in the community network based on the target community membership matrix; In the process of iterating the community membership matrix and community interaction matrix using the expectation-maximization algorithm, each iteration includes an expectation step and a maximization step, wherein: The expected step specifically includes: obtaining the observed transition probability of each node in the community network; determining the expected infection intensity of each node based on the community membership matrix, the community interaction matrix, the observed transition probability, and the noise vector; and determining a responsibility variable based on the expected infection intensity of each node, wherein the responsibility variable is used to reflect the degree of contribution of each infected node to the observed state transition. The maximization step specifically includes: updating the community membership matrix, the community interaction matrix, and the noise vector based on the responsibility variable to obtain the iterative community membership matrix and community interaction matrix.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the steps in the joint method for network reconstruction and overlapping community detection as described in any one of claims 1-7.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps of the joint method for network reconstruction and overlapping community detection as described in any one of claims 1-7.

Images