Method and system for identifying potential information in combination with heterogeneous network environment

Abstract

The invention provides a potential information identification method and system in combination with a heterogeneous network environment, and the method comprises the steps: firstly obtaining a heterogeneous network environment data set of a target network environment, which comprises original information resources of various data formats, and carrying out the feature extraction of the heterogeneous network environment data set to obtain an associated feature set; performing hierarchical pattern matching processing on the associated feature set based on a pre-trained information pattern analysis model to generate a potential information identification result set containing significant value information resource attribute identifiers and distribution paths, and generating a dynamic resource integration strategy according to the potential information identification result set; and the resource management system is used for adjusting a resource scheduling rule and a storage path mapping relation, finally feeding back a dynamic resource integration strategy to the data management system, triggering a resource directory updating operation, and improving heterogeneous network resource utilization and management efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of information resource optimization, and more particularly, to a method and system for identifying potential information in combination with a heterogeneous network environment. Background Art

[0002] In today's network environment, with the rapid development of information technology, the network architecture has become increasingly complex, and heterogeneous network environments have gradually become the mainstream. A heterogeneous network environment includes a variety of different types of network devices, operating systems, and applications, which results in a large number of original information resources with different data formats in the network.

[0003] Existing methods for identifying potential information often can only process single data formats or homogeneous network environments, lacking effective means for integrating and analyzing original information resources with multiple data formats in a heterogeneous network environment. These methods are difficult to extract the semantic association features and dynamic interaction features between the original information resources, and cannot accurately identify the potentially valuable information among them.

[0004] In addition, existing methods cannot generate reasonable resource integration strategies based on the recognition results, and are difficult to effectively adjust the resource scheduling rules and storage path mapping relationships in a heterogeneous network environment, resulting in low utilization efficiency of network resources, and also facing many difficulties in the update and maintenance of data management systems. Therefore, a method for identifying potential information in combination with a heterogeneous network environment is needed to solve the above problems. Summary of the Invention

[0005] In view of the problems mentioned above, in combination with the first aspect of the present invention, embodiments of the present invention provide a method for identifying potential information in combination with a heterogeneous network environment, the method comprising: Obtaining a heterogeneous network environment data set of a target network environment, the heterogeneous network environment data set including original information resources with multiple data formats; Performing feature extraction on the heterogeneous network environment data set to obtain an association feature set, the association feature set including semantic association features and dynamic interaction features between the original information resources; Based on a pre-trained information pattern analysis model, performing hierarchical pattern matching processing on the association feature set to generate a potential information recognition result set, the potential information recognition result set including attribute identifiers and distribution paths of multiple significantly valuable information resources; Generating a dynamic resource integration strategy according to the potential information recognition result set, the dynamic resource integration strategy being used to adjust the resource scheduling rules and storage path mapping relationships in a heterogeneous network environment; Feeding back the dynamic resource integration strategy to a data management system of the target network environment to trigger the data management system to perform a resource directory update operation.

[0006] In another aspect, an embodiment of the present invention further provides a potential information recognition system combined with a heterogeneous network environment, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0007] Based on the above aspects, in the embodiment of the present invention, by obtaining a heterogeneous network environment data set containing original information resources in multiple data formats, extracting features from the heterogeneous network environment data set to obtain an association feature set of semantic association features and dynamic interaction features, mining the internal relationship between information resources at a deeper level, performing hierarchical pattern matching processing based on a pre-trained information pattern analysis model, accurately identifying the attribute identifiers and distribution paths of significant value information resources, generating a dynamic resource integration strategy according to the recognition results, effectively adjusting the resource scheduling rules and storage path mapping relationships, improving the utilization efficiency of resources in the heterogeneous network environment, and feeding back the dynamic resource integration strategy to the data management system to trigger a resource directory update operation, making the data management of the entire heterogeneous network environment more scientific and efficient, and comprehensively improving the information processing and management capabilities of the heterogeneous network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a schematic flowchart of the execution of the potential information recognition method combined with a heterogeneous network environment provided by an embodiment of the present invention.

[0009] Figure 2 is a schematic diagram of exemplary hardware and software components of the potential information recognition system combined with a heterogeneous network environment provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] The present invention will be specifically described below with reference to the accompanying drawings of the specification. Figure 1 is a schematic flowchart of the potential information recognition method combined with a heterogeneous network environment provided by an embodiment of the present invention. The potential information recognition method combined with a heterogeneous network environment will be introduced in detail below.

[0011] Step S110: Obtain a heterogeneous network environment data set of the target network environment, where the heterogeneous network environment data set contains original information resources in multiple data formats.

[0012] In this embodiment, there are multiple different data sources in the target network environment. Data source A generates information in data format X, data source B generates information in data format Y, and data source C generates information in data format Z. Here, data formats X, Y, and Z represent different data organization and representation methods. For example, data format X may be a structured record format, data format Y may be a semi-structured markup language format, and data format Z may be an unstructured data stream format. Through network connections, the data acquisition module is used to obtain data from these data sources, and the information in data format X from data source A, the information in data format Y from data source B, and the information in data format Z from data source C are integrated together to form a heterogeneous network environment data set. For example, the data acquisition module establishes a connection with data source A through a network protocol, collects the output data according to the data interface specification of data source A, and performs the same operation on data sources B and C. Finally, a heterogeneous network environment data set is obtained, which contains the original information resources of multiple data formats.

[0013] Step S120: Extract features from the heterogeneous network environment data set to obtain an associated feature set, where the associated feature set includes semantic association features and dynamic interaction features between each original information resource.

[0014] In this embodiment, feature extraction operations are performed separately for different data types in the heterogeneous network environment data set.

[0015] Step S121: Perform semantic vectorization processing on the text data units in the heterogeneous network environment data set to generate a text semantic feature set, where each text semantic feature in the text semantic feature set corresponds to a semantic vector representation of a text data unit.

[0016] Suppose there are several text data units in the heterogeneous network environment data set, denoted as T1, T2,..., Tn. For each text data unit Ti, first perform word segmentation on it to split Ti into individual words, denoted as w1, w2,..., wm. Then, adopt a semantic vectorization method, such as using a language model trained conventionally based on related technologies, to assign a vector value Vj to each word wj. Assume the dimension of this vector value is d. Next, through a certain aggregation method, such as weighted summation, calculate the semantic vector representation of the text data unit Ti. Assume the weight of each word wj is aj, then the semantic vector representation Si of the text data unit Ti is: Si = a1V1 + a2V2 +... + amVm. Perform such processing on all text data units T1, T2,..., Tn, and a text semantic feature set is obtained, where each text semantic feature is the semantic vector representation Si of the corresponding text data unit.

[0017] Step S122: Perform spatial feature encoding processing on the image data units in the heterogeneous network environment data set to generate an image spatial feature set, where each image spatial feature in the image spatial feature set contains object distribution information and context relationship information in the image data unit.

[0018] Suppose there are image data units I1, I2, …, In in the heterogeneous network environment data set. For each image data unit Ij, a spatial feature encoding algorithm is adopted. First, the image is divided into multiple regions. For example, according to a fixed grid division method, the image Ij is divided into A1, A2, …, Ak regions. Then, for each region Al, through a feature extraction operator, such as an edge detection operator, a corner detection operator, etc., extract the features in the region Al, such as edge features, corner features, etc., and denote these features as Fl. At the same time, use the conventional context analysis method in related technologies to analyze the relationship between the region Al and the surrounding regions to obtain the context relationship information Cl. Combine the feature Fl of the region Al and the context relationship information Cl to form a feature description corresponding to the region Al. Perform such processing on all regions A1, A2, …, Ak in the image Ij, and then integrate the feature descriptions of these regions in a certain way, such as by splicing, to obtain the image spatial feature of the image data unit Ij, denoted as Ij-Feature. Perform such processing on all image data units I1, I2, …, In, and an image spatial feature set is generated, where each image spatial feature Ij-Feature contains the object distribution information (reflected by the region features) and context relationship information in the image data unit Ij.

[0019] Step S123: Perform dynamic pattern analysis processing on the time series data units in the heterogeneous network environment data set to generate a time series dynamic feature set, where each time series dynamic feature in the time series dynamic feature set contains data change trend and periodic fluctuation information.

[0020] Suppose there are time-series data units S1, S2, …, Sn in the heterogeneous network environment data set. For each time-series data unit Sk, it is a data sequence arranged in chronological order, denoted as sk1, sk2, …, skm. First, an analysis method, such as the sliding window method, is adopted to slide a window of size w on the data sequence sk1, sk2, …, skm. Within each window, the trend of the data is analyzed by calculating the slope of the data. Suppose within the window [ski, ski+1, …, ski+w-1], the slope between adjacent data points is calculated. For example, the slope from ski to ski+1 is (ski+1 - ski) / (t(ski+1) - t(ski)), where t(ski) represents the time corresponding to the data point ski. By analyzing multiple slopes within the window, the trend feature Tk of the data within this window is obtained. At the same time, methods such as Fourier transform are used to analyze the periodic fluctuation information of the data sequence, and the periodic fluctuation feature Pk is obtained. The trend feature Tk and the periodic fluctuation feature Pk are combined to form the time-series dynamic feature of the time-series data unit Sk, denoted as Sk-Feature. Such processing is performed on all time-series data units S1, S2, …, Sn, and a time-series dynamic feature set is generated, where each time-series dynamic feature Sk-Feature contains information on data trend and periodic fluctuation.

[0021] Step S124: Perform cross-modal alignment processing on the text semantic feature set, the image spatial feature set, and the time-series dynamic feature set to determine the semantic association features between different modal features, and generate dynamic interaction features based on the feature interaction frequency.

[0022] In this embodiment, cross-modal alignment operations are performed on the text semantic feature set, the image spatial feature set, and the time-series dynamic feature set.

[0023] Step S1241: For the text semantic feature corresponding to each text data unit, match the image data unit and the time-series data unit with synchronous timestamps within a preset time window according to its timestamp.

[0024] Suppose there is a text semantic feature T-Si in the text semantic feature set, and its corresponding text data unit is Ti, which carries a timestamp t(Ti). The preset time window is [-Δt, +Δt]. Within this time window, search for the image data unit Ij in the image space feature set and the time series data unit Sk in the time series dynamic feature set, such that |t(Ti)-t(Ij)|≤Δt and |t(Ti)-t(Sk)|≤Δt, where t(Ij) and t(Sk) are the timestamps of the image data unit Ij and the time series data unit Sk respectively. After finding the matching image data unit Ij and time series data unit Sk, associate their corresponding image space feature Ij-Feature and time series dynamic feature Sk-Feature with the text semantic feature T-Si to form a feature triple (T-Si, Ij-Feature, Sk-Feature). Perform such matching operations on all text semantic features in the text semantic feature set to obtain a series of feature triples.

[0025] Step S1242: After mapping the text semantic feature, image space feature, and time series dynamic feature to a unified dimensional space through a projection layer, call the cross-modal alignment model to perform joint encoding processing on the mapped features to generate cross-modal joint encoding features.

[0026] Suppose the dimension of the text semantic feature T-Si is d1, the dimension of the image space feature Ij-Feature is d2, and the dimension of the time series dynamic feature Sk-Feature is d3. Through the projection layer, project the text semantic feature T-Si into a space of a unified dimension d to obtain the projected text semantic feature T-Si'. Project the image space feature Ij-Feature into the space of the same dimension d to obtain the projected image space feature Ij-Feature'. Project the time series dynamic feature Sk-Feature into the space of dimension d to obtain the projected time series dynamic feature Sk-Feature'. Then, input the projected feature triple (T-Si', Ij-Feature', Sk-Feature') into the cross-modal alignment model. The cross-modal alignment model contains multiple neural network layers. For example, the input layer receives the projected feature triple, and through a series of non-linear transformations in the hidden layer, fuses and encodes the features of different modalities, and finally outputs the cross-modal joint encoding feature, denoted as C(T-Si, Ij, Sk), in the output layer. Perform such processing on all feature triples to obtain multiple cross-modal joint encoding features.

[0027] Step S1243: Perform a similarity calculation process on the cross-modal joint encoding features to determine the first correlation score between the text semantic features and the image spatial features, and the second correlation score between the text semantic features and the temporal dynamic features.

[0028] For each cross-modal joint encoding feature C(T-Si, Ij, Sk), extract the parts related to the text semantic features and the image spatial features from this joint encoding feature. Through a similarity calculation method, such as the cosine similarity calculation method, calculate the similarity between the text semantic feature T-Si' and the image spatial feature Ij-Feature' to obtain the first correlation score R1(T-Si, Ij). Similarly, extract the parts related to the text semantic features and the temporal dynamic features from the cross-modal joint encoding feature C(T-Si, Ij, Sk), and calculate the similarity between the text semantic feature T-Si' and the temporal dynamic feature Sk-Feature' to obtain the second correlation score R2(T-Si, Sk). Perform such calculations on all cross-modal joint encoding features to obtain a series of first correlation scores R1(T-Si, Ij) and second correlation scores R2(T-Si, Sk).

[0029] Step S1244: Construct a semantic association matrix based on the first correlation score and the second correlation score. The rows of the semantic association matrix represent text data unit identifiers, the columns represent image data unit or temporal data unit identifiers, and the matrix elements are the corresponding correlation scores.

[0030] Let the set of text data unit identifiers be {T1, T2,..., Tn}, the set of image data unit identifiers be {I1, I2,..., In}, and the set of temporal data unit identifiers be {S1, S2,..., Sn}. Construct a semantic association matrix M, the number of rows of which is equal to the number n of text data units, and the number of columns is equal to the sum 2n of the number of image data units and temporal data units. For the i-th row and the j-th column (1 ≤ j ≤ n) of the matrix M, if j corresponds to the image data unit Ij, then M(i, j) = R1(Ti, Ij); for the i-th row and the k-th column (n + 1 ≤ k ≤ 2n), if k - n corresponds to the temporal data unit Sk - n, then M(i, k) = R2(Ti, Sk - n). In this way, fill all the first correlation scores and the second correlation scores into the semantic association matrix M.

[0031] Step S1245: Extract the feature combinations corresponding to the correlation scores exceeding the preset threshold in the semantic association matrix to generate the semantic association features.

[0032] Set a preset threshold θ and traverse all elements in the semantic association matrix M. For the element M(i, j), if M(i, j) > θ, when 1 ≤ j ≤ n, extract the text semantic feature T-Si of the corresponding text data unit Ti and the image spatial feature Ij-Feature of the image data unit Ij to form a feature combination (T-Si, Ij-Feature); when n + 1 ≤ j ≤ 2n, extract the text semantic feature T-Si of the corresponding text data unit Ti and the temporal dynamic feature Sj-n-Feature of the temporal data unit Sj-n to form a feature combination (T-Si, Sj-n-Feature). Collect all feature combinations that meet the conditions to form a semantic association feature set, that is, obtain the semantic association feature.

[0033] Step S1246: Divide each data unit in the heterogeneous network environment data set into blocks according to the data category, count the number of interaction events between data units within each block, and construct a block interaction frequency matrix. The rows and columns of the interaction frequency matrix represent different data unit identifiers, and the matrix elements represent the number of interactions between the corresponding data units.

[0034] Divide the heterogeneous network environment data set into different blocks according to the data category, such as text data blocks, image data blocks, and temporal data blocks. For the text data block, assume there are text data units T1, T2,..., Tn. Define an interaction event, for example, a text data unit Ti sending a message to a text data unit Tj is an interaction event. Count the number of interaction events between each text data unit Ti and other text data units Tj within the text data block over a period of time, denoted as N(Ti, Tj). Similarly, perform similar operations on the image data block and the temporal data block to count the number of interaction events between image data units and between temporal data units. Then construct a block interaction frequency matrix F, whose number of rows and columns is equal to the total number of all data units. For the i-th row and j-th column of the matrix F, if the i-th data unit is the text data unit Ti and the j-th data unit is the text data unit Tj, then F(i, j) = N(Ti, Tj); if the i-th data unit is the image data unit Ii and the j-th data unit is the image data unit Ij, then F(i, j) = N(Ii, Ij); if the i-th data unit is the temporal data unit Si and the j-th data unit is the temporal data unit Sj, then F(i, j) = N(Si, Sj). Fill the block interaction frequency matrix F in this way.

[0035] Step S1247: Perform time dimension decomposition processing on the interaction frequency matrix to obtain short-term interaction frequency features and long-term interaction trend features.

[0036] A time dimension decomposition method is adopted. For example, time is divided into short-term time periods and long-term time periods. Assume the short-term time period is [t1, t2], the long-term time period is [t3, t4], and [t1, t2] is included in [t3, t4]. For the block interaction frequency matrix F, within the short-term time period [t1, t2], the interaction frequency between each pair of data units is counted to obtain the short-term interaction frequency matrix Fs, and its element Fs(i, j) represents the interaction frequency between the ith data unit and the jth data unit within the short-term time period. Some features are extracted from the short-term interaction frequency matrix Fs, such as the average interaction frequency of each data unit, the standard deviation of the interaction frequency, etc., to form the short-term interaction frequency features. Within the long-term time period [t3, t4], the change trend of the interaction frequency between each pair of data units is counted. For example, the change curve of the interaction frequency over time is fitted by methods such as linear regression to obtain the long-term interaction trend features. The long-term interaction trend features can be represented by some parameters, such as the slope and intercept of the change curve.

[0037] Step S1248: Calculate the difference between the short-term interaction frequency features and the long-term interaction trend features to generate dynamic fluctuation features.

[0038] Calculate the difference between the short-term interaction frequency features and the long-term interaction trend features. Assume the short-term interaction frequency features are represented by a set of parameters A1, A2,..., Am, and the long-term interaction trend features are represented by a set of parameters B1, B2,..., Bm, where m is the number of feature parameters. For each pair of parameters Ai and Bi, calculate their difference Di = Ai - Bi. Combine all the differences D1, D2,..., Dm to form the dynamic fluctuation features. The dynamic fluctuation features reflect the change situation of the interaction frequency between data units in the short term and the long term.

[0039] Step S1249: Determine the dynamic interaction intensity between each pair of data units according to the change direction and amplitude of the dynamic fluctuation features.

[0040] Analyze each difference Di in the dynamic fluctuation features. If Di > 0, it means that the short-term interaction frequency is higher than the long-term interaction trend, and the greater the absolute value of Di, the higher the dynamic interaction intensity; if Di < 0, it means that the short-term interaction frequency is lower than the long-term interaction trend, and the greater |Di| is, the lower the dynamic interaction intensity. According to such rules, a dynamic interaction intensity value is determined for each pair of data units. For example, for the pair of data units (i, j), the dynamic interaction intensity value I(i, j) is calculated according to the corresponding difference Di. In this way, the dynamic interaction intensity between all data units is determined.

[0041] Step S12410: Based on the dynamic interaction intensity, perform weighted correction processing on the semantic association features to generate the dynamic interaction features.

[0042] For each feature combination in the semantic association feature, such as the feature combination (T-Si, Ij-Feature), a weighted correction is performed on the feature combination based on the dynamic interaction strength I(Ti, Ij) between the corresponding text data unit Ti and image data unit Ij. Assuming that the feature combination (T-Si, Ij-Feature) can be expressed as a vector form V(T-Si, Ij-Feature), the weighted correction feature combination is V'(T-Si, Ij-Feature) = I(Ti, Ij) * V(T-Si, Ij-Feature). This weighted correction process is performed on all feature combinations in the semantic association feature, and then the weighted correction feature combinations are spliced (not added) to form the dynamic interaction feature.

[0043] Step S125: Aggregate the semantic association features and the dynamic interaction features to construct the association feature set.

[0044] Aggregate all feature combinations and dynamic interaction features in the semantic association feature set. For example, assuming the semantic association feature set is {Semantic1, Semantic2, …, Semantick} and the dynamic interaction feature is Dynamic, concatenate them together in a set order to form an associated feature set, denoted as AssociatedFeatures = [Semantic1, Semantic2, …, Semantick, Dynamic]. In this way, an associated feature set is constructed.

[0045] Step S130: Based on the pre-trained information pattern analysis model, hierarchical pattern matching processing is performed on the associated feature set to generate a potential information recognition result set, which includes attribute identifiers and distribution paths of multiple significant value information resources.

[0046] In this embodiment, the associated feature set is processed using a pre-trained information pattern analysis model.

[0047] Step S131: input the associated feature set into the feature coding layer of the information pattern analysis model, perform layer-by-layer nonlinear transformation processing, and generate a multi-layer abstract feature representation sequence, wherein the dimension of the n-th layer abstract feature representation and the dimension of the n-1-th layer abstract feature representation are in a preset dimensionality reduction ratio.

[0048] The feature encoding layer of the information pattern analysis model includes multiple neural network layers. The associated feature set AssociatedFeatures is input into the first layer of the feature encoding layer. The first layer performs a non-linear transformation on the input associated features, for example, through an activation function (such as the ReLU function), mapping the associated features from the original dimension to a new dimension. Suppose the original dimension of the associated feature set is d0, the feature dimension output by the first layer is d1, and d1 and d0 satisfy the preset dimensionality reduction ratio r, that is, d1 = r * d0. Then, the features output by the first layer are input into the second layer, and the second layer also performs a non-linear transformation on the input features, outputting features with a dimension of d2 = r * d1. And so on, after several layers of non-linear transformations, a multi-layer abstract feature representation sequence is generated, denoted as [L1, L2, …, Ln], where Li represents the abstract feature representation of the i-th layer. The non-linear transformation of each layer will cause the features to be transformed in different feature spaces. Through this layer-by-layer processing, the original associated feature set is gradually abstracted, enabling the high-level abstract features to capture more complex and representative pattern information. For example, the first layer may initially integrate and transform some local features in the associated feature set, and the second layer, based on the output of the first layer, further mines higher-level feature combination information. As the number of layers increases, the feature representation becomes more and more abstract, gradually refining the key features related to the potential information pattern.

[0049] Step S132: In the pattern matching layer of the information pattern analysis model, calculate the cross-layer correlation degree of the multi-layer abstract feature representation sequence to determine the pattern similarity index between adjacent layer abstract feature representations.

[0050] From the multi-layer abstract feature representation sequence [L1, L2, …, Ln], two adjacent layers are sequentially selected, that is, the k-th layer abstract feature representation Lk and the (k + 1)-th layer abstract feature representation Lk+1 are processed. First, calculate the similarity between them at the same feature unit position. Here, taking a common similarity calculation method as an example, the cosine similarity between them is calculated. Suppose Lk consists of a series of feature vectors, denoted as Lk = [v1_k, v2_k, …, vm_k], and Lk+1 consists of corresponding feature vectors, denoted as Lk+1 = [v1_k+1, v2_k+1, …, vm_k+1] (here it is assumed that the number of feature vectors in the two layers is the same, and in actual situations, some alignment methods may be needed to ensure dimension matching). For each pair of feature vectors vi_k and vi_k+1 at the same position, calculate their cosine similarity. The calculation logic of the cosine similarity is roughly as follows: first calculate the dot product of the two vectors, that is, the sum of the products of the corresponding dimension elements of vi_k and vi_k+1; then calculate the norms of the two vectors respectively, that is, the square root of the sum of the squares of the dimension elements; finally, divide the dot product result by the product of the norms of the two vectors to obtain the cosine similarity value of this pair of feature vectors. Combine the cosine similarity values of the feature vectors at all positions to form a local similarity matrix, denoted as LocalSimMatrix_k.

[0051] Next, perform sliding window mean pooling on the local similarity matrix LocalSimMatrix_k. Set a sliding window, the size of which is related to the dimension scaling factor of the k-th layer abstract feature representation. For example, if the dimension scaling factor from the k-th layer to the (k + 1)-th layer is s, then the sliding window size may be set to a value related to s, such as the side length of the window being an integer multiple of s. Slide this window on the local similarity matrix, and for the elements within each window, calculate their average value. For example, within the area covered by the window, add up all the elements and divide by the number of elements within the window to obtain the average value at this window position. Combine the average values at each window position to form a new distribution map, that is, the windowed similarity distribution map WindowedSimDistribution_k.

[0052] Then, extract the regions in the windowed similarity distribution map WindowedSimDistribution_k that exceed the preset activation threshold. The preset activation threshold is a value set based on experience or previous experiments, and is used to screen out the key regions with higher similarity. Traverse each value in the windowed similarity distribution map, and mark the regions corresponding to the values greater than the preset activation threshold. These marked regions form the target association region set TargetAssociationRegions_k.

[0053] After that, calculate the coverage ratio of the target association region set TargetAssociationRegions_k in the windowed similarity distribution graph WindowedSimDistribution_k. Calculate the ratio of the number of elements in the target association region set to the total number of elements in the windowed similarity distribution graph to obtain a ratio value, denoted as CoverageRatio_k. This ratio value reflects the proportion of regions with high similarity between adjacent two layers in the overall distribution graph.

[0054] Then, multiply the coverage ratio value CoverageRatio_k by the preset hierarchical attenuation coefficient to generate the pattern similarity index. The preset hierarchical attenuation coefficient is a parameter preset for adjusting the similarity index. Assuming the preset hierarchical attenuation coefficient is α, then the pattern similarity index SimilarityIndex_k = CoverageRatio_k × α.

[0055] Finally, determine whether to add the abstract feature representation of the corresponding layer to the hierarchical feature subset according to the comparison result between the pattern similarity index SimilarityIndex_k and the preset hierarchical transition threshold. The preset hierarchical transition threshold is another value set according to experience or previous experiments, used to determine whether the pattern similarity between adjacent two layers is high enough for hierarchical transition. If the pattern similarity index SimilarityIndex_k is greater than the preset hierarchical transition threshold, add the abstract feature representations of the k-th layer and the (k + 1)-th layer to the hierarchical feature subset at the same time; if the pattern similarity index SimilarityIndex_k is less than or equal to the preset hierarchical transition threshold and the (k + 1)-th layer is the bottom layer, only add the abstract feature representation of the (k + 1)-th layer to the hierarchical feature subset. In this way, by calculating the cross-layer correlation degree of the abstract feature representations of adjacent layers, the layers with high pattern similarity can be screened out, preparing for the subsequent generation of the hierarchical feature subset.

[0056] Step S133: Screen the abstract feature representations that meet the hierarchical transition conditions from the multi-layer abstract feature representation sequence according to the comparison result between the pattern similarity index and the preset dynamic threshold, and generate a hierarchical feature subset.

[0057] After calculating the pattern similarity index SimilarityIndex_k between adjacent layers as described in the above steps, these indices are compared with a preset dynamic threshold. The preset dynamic threshold may be a value that dynamically adjusts with model training or data characteristics. Here, it is assumed to be DynamicThreshold. For each pattern similarity index SimilarityIndex_k, if SimilarityIndex_k is greater than DynamicThreshold, it indicates that the pattern similarity between the k-th layer and the (k + 1)-th layer meets the hierarchical transition condition. At this time, the abstract feature representation Lk of the k-th layer and the abstract feature representation Lk+1 of the (k + 1)-th layer are selected. In this way, all adjacent layers in the multi-layer abstract feature representation sequence [L1, L2, …, Ln] are judged, and the abstract feature representations of the adjacent layers that meet the hierarchical transition condition are selected to form the hierarchical feature subset HierarchicalFeatureSubset. This hierarchical feature subset contains the abstract feature representations screened from the multi-layer abstract feature representation sequence, which have a relatively high pattern similarity and meet the hierarchical transition condition. These feature representations are more likely to contain key information related to the potential information pattern, providing more targeted data for subsequent pattern clustering processing.

[0058] Step S134: Perform pattern clustering processing on the hierarchical feature subset to generate an initial candidate pattern set, where each initial candidate pattern includes a pattern center vector and a membership degree range.

[0059] Taking the hierarchical feature subset HierarchicalFeatureSubset as the input, a clustering algorithm is adopted, such as the K-Means clustering algorithm (here is just an example to illustrate the type of clustering algorithm, and in practice, a more suitable clustering algorithm for this scenario may be used). At the beginning of the clustering process, the number of clusters is first determined. Assume that the number of clusters is set to K. Then, K feature vectors are randomly selected from the hierarchical feature subset as the initial pattern center vectors, denoted as CenterVectors = [c1, c2, …, cK].

[0060] For each feature vector f in the hierarchical feature subset, calculate the distance between it and each pattern center vector ci. The distance can be calculated using common distance metrics such as the Euclidean distance. Taking the Euclidean distance as an example, assuming that the feature vector f and the pattern center vector ci have the same dimension, both are d-dimensional, f = [f1, f2, …, fd], and ci = [ci1, ci2, …, cid], then the Euclidean distance between f and ci is the square root of the sum of the squares of the differences in each dimension, that is, first calculate (f1 - ci1)² + (f2 - ci2)² + … + (fd - cid)², and then take the square root of the result. After calculating the distances between f and all pattern center vectors, assign f to the category corresponding to the pattern center vector with the closest distance.

[0061] After all the feature vectors in the hierarchical feature subset have been assigned, recalculate the mean of all the feature vectors in each category and use this mean as the new pattern center vector. For example, for category j, which contains feature vectors f1_j, f2_j, …, fm_j, the values of each dimension of the new pattern center vector cj_new are the average of the corresponding dimension values of all the feature vectors in this category, that is, the k-th dimension value of cj_new is (f1_j[k] + f2_j[k] + … + fm_j[k]) / m (assuming there are m feature vectors in this category).

[0062] Repeat the above process of assigning feature vectors and updating pattern center vectors until the change in the pattern center vector is less than a preset convergence threshold or the preset maximum number of iterations is reached. After these steps, K pattern center vectors are obtained, and each pattern center vector corresponds to a clustering category.

[0063] For each clustering category, calculate the distance distribution of the feature vectors in this category from the pattern center vector to determine the membership degree range. For example, the maximum and minimum distances between all the feature vectors in this category and the pattern center vector can be calculated, and this distance range is used as the membership degree range of the feature vectors in this category. In this way, each initial candidate pattern consists of the pattern center vector and the corresponding membership degree range, and all the initial candidate patterns form the initial candidate pattern set InitialCandidatePatternSet.

[0064] Step S135: Based on the distance distribution between the pattern center vector and each feature vector in the associated feature set, calculate the feature vector distribution density within a preset neighborhood range for each initial candidate pattern as the information density value, and determine the coverage radius according to the maximum distance threshold.

[0065] For each initial candidate pattern in the InitialCandidatePatternSet, its pattern center vector is denoted as cp. In the AssociatedFeatures set, calculate the distance between each feature vector af and the pattern center vector cp. Here, the Euclidean distance is still used for calculation, as described in the above method for calculating the Euclidean distance.

[0066] Set a preset neighborhood range. For example, a spherical region centered at the pattern center vector cp with a radius of r (where r is the preset neighborhood radius value). Count the number of feature vectors in the AssociatedFeatures set within this preset neighborhood range, denoted as N. At the same time, calculate the volume of this neighborhood range (for a spherical neighborhood, the volume formula is 4 / 3πr³), denoted as V. Then the distribution density of feature vectors of this initial candidate pattern within the preset neighborhood range, that is, the information density value InfoDensity = N / V.

[0067] In addition, determine the coverage radius according to the maximum distance threshold. Set a maximum distance threshold MaxDistance, and find the feature vector in the AssociatedFeatures set that is farthest from the pattern center vector cp and less than or equal to MaxDistance. The distance between this farthest feature vector and the pattern center vector is the coverage radius CoverageRadius. By calculating the information density value and determining the coverage radius, it provides a basis for subsequent redundant merging processing of the InitialCandidatePatternSet.

[0068] Step S136: Perform redundant merging processing on the InitialCandidatePatternSet according to the information density value and the coverage radius, and merge the initial candidate patterns with information density value differences less than the preset merging threshold within the overlapping coverage radius to generate an optimized pattern set.

[0069] Traverse every two initial candidate patterns in the InitialCandidatePatternSet, assume they are pattern i and pattern j, their pattern center vectors are cpi and cpj respectively, the information density values are InfoDensity_i and InfoDensity_j respectively, and the coverage radii are CoverageRadius_i and CoverageRadius_j respectively.

[0070] First, determine whether the coverage radii of the two patterns overlap, that is, determine whether there is an intersection between the area centered at cpi with a radius of CoverageRadius_i and the area centered at cpj with a radius of CoverageRadius_j. If there is an intersection, further compare the difference in their information density values. Calculate the absolute value of the difference between InfoDensity_i and InfoDensity_j, denoted as |InfoDensity_i - InfoDensity_j|.

[0071] Compare the absolute value of this difference with the preset merging threshold MergeThreshold. If |InfoDensity_i - InfoDensity_j| is less than MergeThreshold, it indicates that the two initial candidate patterns have a small difference in information density and their coverage radii overlap. At this time, merge these two patterns. The merging method can be to recalculate the pattern center vector of the merged pattern. For example, calculate the mean value of all feature vectors included in the two patterns (the feature vectors from the corresponding regions in the associated feature set) to obtain the new pattern center vector; at the same time, recalculate the coverage radius and information density value and other related parameters of the merged pattern.

[0072] In this way, judge and process all pairs of patterns in the initial candidate pattern set, merge the patterns that meet the merging conditions, and finally generate the optimized pattern set OptimizedPatternSet. This optimized pattern set reduces redundant patterns, makes the patterns more refined, and can more accurately represent potential information patterns.

[0073] Step S137: Extract the feature unit identifiers corresponding to each pattern center vector from the optimized pattern set, associate the resource attribute tags and path topology relationships in the semantic association features, and generate the attribute identifiers and distribution paths of the significant value information resources.

[0074] In the optimized pattern set OptimizedPatternSet, for each pattern, its pattern center vector corresponds to some feature vectors in the associated feature set. By recording these corresponding relationships, the feature unit identifiers corresponding to the pattern center vector can be found. Suppose the pattern center vector of pattern p is cp. By searching for the feature vectors in the associated feature set that are closest to cp or meet the set matching conditions, determine the feature units to which these feature vectors belong, so as to obtain the feature unit identifier FeatureUnitIDs_p.

[0075] Then, in the semantic association feature set, find the resource attribute labels and path topology relationships associated with these feature unit identifiers. The semantic association feature set records the semantic associations between different types of data units, and the corresponding resource attribute labels ResourceAttributeLabels and path topology relationships PathTopologyRelations can be indexed through the feature unit identifiers.

[0076] Associate and combine the feature unit identifiers FeatureUnitIDs_p, resource attribute labels ResourceAttributeLabels, and path topology relationships PathTopologyRelations to generate the attribute identifiers and distribution paths of the significant value information resources. For example, using the feature unit identifiers as indexes, combine the corresponding resource attribute labels and path topology relationships together to form an information pair (AttributeID, DistributionPath) containing the attribute identifier and the distribution path, where AttributeID is composed of the resource attribute labels and DistributionPath is determined by the path topology relationship. Perform such processing on all the patterns in the optimization pattern set to obtain a series of attribute identifiers and distribution paths of the significant value information resources.

[0077] Step S138: Aggregate the attribute identifiers and distribution paths of all the significant value information resources to construct the potential information recognition result set.

[0078] Aggregate the attribute identifiers and distribution paths of all the significant value information resources obtained through the above steps. For example, collect the (AttributeID, DistributionPath) information pairs generated by each pattern to form a set, denoted as PotentialInfoRecognitionResultSet. This potential information recognition result set contains the attribute identifiers and distribution paths of multiple significant value information resources, which is obtained based on the hierarchical pattern matching process of the association feature set and provides key information for generating the dynamic resource integration strategy in the follow-up.

[0079] Step S140: Generate a dynamic resource integration strategy according to the potential information recognition result set, and the dynamic resource integration strategy is used to adjust the resource scheduling rules and storage path mapping relationships in the heterogeneous network environment.

[0080] In this embodiment, a dynamic resource integration strategy is generated based on the potential information recognition result set to optimize the resource scheduling and storage paths in the heterogeneous network environment.

[0081] Step S141: Analyze the attribute identifiers of each significant value information resource in the set of potential information recognition results to determine its data category and access permission level to which it belongs.

[0082] In the set of potential information recognition results PotentialInfoRecognitionResultSet, for the attribute identifier AttributeID of each significant value information resource, determine its data category through a preset parsing rule. Assume that different data category identifiers are predefined, such as DataCategory1, DataCategory2,.... The attribute identifier may contain specific fields or encodings to represent the data category. By extracting and analyzing these fields or encodings, determine which category of data this significant value information resource belongs to, such as determining it belongs to DataCategoryX.

[0083] At the same time, parse the information related to the access permission level from the attribute identifier. The access permission level may be represented by a certain encoding or numerical range, such as AccessLevel1, AccessLevel2,.... By parsing the attribute identifier, determine that the access permission level of this significant value information resource is AccessLevelY. Perform such parsing on the attribute identifiers of all significant value information resources in the set of potential information recognition results to determine the data category and access permission level to which each resource belongs.

[0084] Step S142: Extract the storage location identifier and transmission path topology relationship of each significant value information resource according to the distribution path.

[0085] For the distribution path DistributionPath of each significant value information resource in the set of potential information recognition results, parse it according to a predefined format and rule. The distribution path may exist in a form containing a storage location and a description of the transmission path. For example, it may be a representation of a series of nodes and connection relationships. By parsing this distribution path, extract the storage location identifier StorageLocationIDs. The storage location identifier may be the number, address of the storage device, or other information that can uniquely identify the storage location.

[0086] At the same time, extract the transmission path topology relationship TransmissionPathTopology from the distribution path. The transmission path topology relationship describes the path structure of data transmission in the network and may include information such as the connection order and direction between nodes. For example, the transmission path topology relationship may be represented as a directed connection relationship from node A through node B to node C. Perform such extraction operations on the distribution paths of all significant value information resources to obtain the storage location identifier and transmission path topology relationship of each resource.

[0087] Step S143: Generate a resource scheduling priority rule based on the data category and the access privilege level, where the resource scheduling priority rule defines the response order and the concurrent processing quantity of different categories of resources.

[0088] Generate a resource scheduling priority rule according to the previously determined data category and access privilege level. First, classify according to the data category. Different data categories may have different characteristics such as importance or usage frequency. For example, for the data category DataCategory1, if it is a data category that is critical to the business and is frequently accessed, then assign a higher priority to it; for the data category DataCategory2, if its importance is relatively low, then assign a lower priority. Here, the priority can be represented by numbers or a certain priority code. Assume they are PriorityCode1, PriorityCode2, …, and assign corresponding priority codes to different data categories respectively.

[0089] Meanwhile, consider the access privilege level. Resources with a higher access privilege level, such as the resources corresponding to AccessLevel1, may have a higher priority during scheduling. Integrate the data category and the access privilege level to determine the final priority of each significant value information resource.

[0090] Based on these priorities, define the response order of resource scheduling. For example, resources with a higher priority are processed first during scheduling. That is, when there are multiple resources waiting for scheduling, first process the resources with the priority of PriorityCode1, then process the resources with the priority of PriorityCode2, and so on.

[0091] In addition, it is also necessary to determine the concurrent processing quantity. For resources with different priorities, set different concurrent processing quantities according to the processing capacity of the system and the resource characteristics. For example, for the resources with the highest priority, the allowed concurrent processing quantity may be ConcurrentNum1; for the resources with a lower priority, the allowed concurrent processing quantity is ConcurrentNum2 (here, ConcurrentNum1 and ConcurrentNum2 are only for example, and the actual quantity is determined according to the system situation). In this way, generate a resource scheduling priority rule, which clarifies the response order and the concurrent processing quantity of different categories of resources.

[0092] Step S144: Generate a storage path optimization mapping table based on the storage location identifier and the transmission path topology relationship, where the storage path optimization mapping table contains the correspondence between the resource identifier and the optimal storage node.

[0093] First, for each significant value information resource, the optimal storage node is determined by combining its storage location identifier and transmission path topology. The storage location identifier may contain multiple pieces of relevant information, such as the storage area number and storage device type. Assuming the storage location identifier consists of multiple fields, such as LocationField1 and LocationField2, potential storage nodes are initially screened by analyzing these fields and applying specific rules.

[0094] The transmission path topology provides structural information about data transmission within the network. For example, it indicates characteristics such as the transmission path length and transmission delay from the source node to different potential storage nodes. Assume that the transmission path topology can be represented by the connection weights between nodes. The weights may represent factors such as transmission delay or bandwidth. For each potential storage node, the comprehensive transmission cost from the data source (the data source's related nodes can be determined based on the transmission path topology) to that storage node is calculated. This comprehensive transmission cost calculation may involve some combination of the weights of the connections along the transmission path, such as summing the weights of all connections along the path (this is just an example logic).

[0095] By comparing the comprehensive transmission costs of all potential storage nodes, the storage node with the lowest comprehensive transmission cost is selected as the optimal storage node for the significant value information resource. Assume that for the significant value information resource identified by ResourceID, the optimal storage node determined by the above calculation is OptimalNode. The ResourceID and OptimalNode form a corresponding relationship pair (ResourceID, OptimalNode).

[0096] This process is repeated for all significant information resources in the potential information identification result set, and all corresponding relationship pairs are summarized to form a storage path optimization mapping table, StoragePathOptimizationMapTable. This storage path optimization mapping table clearly records the relationship between each resource identifier and the corresponding optimal storage node.

[0097] Step S145: performing policy fusion processing on the resource scheduling priority rule and the storage path optimization mapping table to generate the dynamic resource integration policy.

[0098] This step aims to integrate resource scheduling priority rules and storage path optimization mapping tables to generate a comprehensive dynamic resource integration strategy.

[0099] Step S1451: Invoke the node monitoring interface of the target network environment to obtain the real-time load status data of the storage nodes. Combine the response order in the resource scheduling priority rules to perform load balancing analysis on the storage nodes in the storage path optimization mapping table, and determine the real-time load status of each storage node.

[0100] By invoking the node monitoring interface provided by the target network environment, the real-time load status data of each storage node can be obtained. Assume that this data includes information on multiple dimensions such as the current storage utilization rate, data transfer rate, and number of processing tasks of the storage node, which are respectively denoted as StorageUsageRate, DataTransferRate, TaskCount, etc.

[0101] Combine the response order in the resource scheduling priority rules to perform load balancing analysis on the storage nodes in the storage path optimization mapping table. For resources with higher priorities, the corresponding storage nodes have higher weights in the load balancing analysis. For example, for resources with a priority of PriorityHigh, the load situation of the corresponding storage node has a weight of WeightHigh in the overall analysis; for resources with a priority of PriorityLow, the weight of the load situation of the corresponding storage node is WeightLow, and WeightHigh is greater than WeightLow.

[0102] Through a load balancing analysis algorithm, such as comprehensively considering factors such as storage utilization rate, data transfer rate, and number of processing tasks to calculate the comprehensive load index of each storage node. Assume that the calculation logic of the comprehensive load index is: multiply the storage utilization rate by a weight coefficient WeightUsage, add the data transfer rate multiplied by the weight coefficient WeightTransfer, and then add the number of processing tasks multiplied by the weight coefficient WeightTask, that is, CompositeLoadIndex = StorageUsageRate × WeightUsage + DataTransferRate × WeightTransfer + TaskCount × WeightTask. Through such calculations, the real-time load status of each storage node is determined.

[0103] Step S1452: If the real-time load status of the target storage node exceeds the capacity threshold, search for alternative storage nodes in the storage path optimization mapping table according to the business priority weights of the resources, and preferentially migrate resources with business priority weights lower than the preset migration threshold and update the corresponding relationships.

[0104] Set a capacity threshold, CapacityThreshold, to determine whether a storage node is overloaded. For each storage node, compare the calculated composite load index, CompositeLoadIndex, with the capacity threshold, CapacityThreshold. If the CompositeLoadIndex of a target storage node exceeds the CapacityThreshold, it indicates that the storage node is overloaded and resource migration is required.

[0105] In the resource scheduling priority rules, a business priority weight, BusinessPriorityWeight, is assigned to each resource. In the storage path optimization mapping table, search for other backup storage nodes except the current overloaded storage node. Check whether the business priority weight of a resource is lower than a preset migration threshold, MigrationThreshold, in ascending order of the business priority weight. For resources with a business priority weight lower than the MigrationThreshold, migrate them from the overloaded target storage node to the backup storage node.

[0106] Suppose a resource, ResourceA, was originally stored in the overloaded target storage node, TargetNode. When searching for a backup storage node, it is found that the backup storage node, BackupNode, has sufficient space and performance to receive ResourceA. After migrating ResourceA to BackupNode, update the corresponding relationship in the storage path optimization mapping table, form a new corresponding relationship pair (ResourceA, BackupNode) between ResourceA and BackupNode, and at the same time delete the original (ResourceA, TargetNode) corresponding relationship.

[0107] Step S1453: Adjust the number of concurrent processes in the resource scheduling priority rules according to the updated storage path optimization mapping table, so that the number of concurrent processes of the preset first-priority resources matches the capacity limit of the storage nodes with sufficient remaining available capacity.

[0108] After completing the resource migration and updating the storage path optimization mapping table, adjust the number of concurrent processes in the resource scheduling priority rules. First, determine the preset first-priority resources, assumed to be PriorityFirstResources. For these resources, it is necessary to adjust the number of concurrent processes according to the capacity limit of the storage nodes with sufficient remaining available capacity.

[0109] Traverse the updated storage path optimization mapping table and count the remaining available capacity of each storage node. Assume that the remaining available capacity of storage node Node1 is AvailableCapacity1, the remaining available capacity of Node2 is AvailableCapacity2, etc. Find the storage nodes with sufficient remaining available capacity and set a criterion for sufficient capacity. For example, a storage node with a remaining available capacity greater than a certain threshold AdequateCapacityThreshold is a node with sufficient available capacity.

[0110] For these storage nodes with sufficient available capacity, determine their capacity upper limits CapacityUpperLimit. For example, for a group of storage nodes with sufficient available capacity, their capacity upper limits are CapacityUpperLimit1, CapacityUpperLimit2, etc. According to these capacity upper limits, calculate the concurrent processing quantity of the preset first-priority resources that can be supported. Assume the calculation logic is: add up the capacity upper limits of all storage nodes with sufficient available capacity to get the total capacity upper limit TotalCapacityUpperLimit, and then determine the concurrent processing quantity AdjustedConcurrentNum of the preset first-priority resources according to the preset resource allocation strategy (such as average allocation or allocation according to resource characteristics). For example, if the average allocation strategy is adopted and the number of preset first-priority resources is NumPriorityFirstResources, then AdjustedConcurrentNum = TotalCapacityUpperLimit / NumPriorityFirstResources.

[0111] Then, in the resource scheduling priority rule, adjust the concurrent processing quantity of the preset first-priority resources to AdjustedConcurrentNum.

[0112] Step S1454: Combine the adjusted resource scheduling priority rule with the updated storage path optimization mapping table to generate a fused dynamic resource integration strategy; Combine the resource scheduling priority rule adjusted above with the updated storage path optimization mapping table. The resource scheduling priority rule clarifies the response order and concurrent processing quantity of different types of resources, while the storage path optimization mapping table determines the correspondence between resources and the optimal storage nodes.

[0113] Integrate these two parts together to form a unified policy document or data structure as the integrated Dynamic Resource Integration Strategy. For example, the resource scheduling priority rules can be represented in a certain data format (such as JSON format), including information such as resource category, priority, response order, and number of concurrent processes; the storage path optimization mapping table can also be represented in a corresponding data format (such as key-value pairs, with the resource identifier as the key and the optimal storage node as the value). Then combine these two data structures into a larger data structure containing policy information on both resource scheduling and storage paths, thereby generating a complete dynamic resource integration strategy.

[0114] Step S1455: Perform conflict detection and handling on the dynamic resource integration strategy to ensure that there is no logical contradiction in the execution order between different rules.

[0115] Perform conflict detection on the generated dynamic resource integration strategy DynamicResourceIntegrationStrategy. Check whether there is a logical conflict between the resource scheduling priority rules and the storage path optimization mapping table. For example, in the resource scheduling priority rules, it may be stipulated that certain high-priority resources need to be processed first, but in the storage path optimization mapping table, the storage nodes corresponding to these high-priority resources may not be able to provide services in a timely manner due to other reasons (such as maintenance status or overload and incomplete processing), which may lead to conflicts.

[0116] For this situation, a series of detection rules can be set. First, check whether the response order in the resource scheduling priority rules matches the availability of the storage nodes. For resources at each priority level, check whether the corresponding storage nodes are in an available state (obtain storage node information through the storage path optimization mapping table and combine it with the real-time status data obtained from the node monitoring interface for judgment). If the storage node corresponding to a certain high-priority resource is unavailable, and the resource scheduling rule requires that this resource be processed first, a conflict occurs.

[0117] Secondly, check whether the number of concurrent processes matches the capacity of the storage nodes. Based on the capacity information of the storage nodes in the storage path optimization mapping table and the number of concurrent processes set in the resource scheduling priority rules, judge whether the number of resources processed simultaneously will exceed the processing capacity or storage capacity of the storage nodes. For example, if the remaining available capacity of a certain storage node can only support a certain number of resources to be stored and processed simultaneously, and the number of concurrent processes set in the resource scheduling rule exceeds this limit, a conflict will occur.

[0118] If a conflict is detected, it is processed according to a preset conflict resolution strategy. For example, if the conflict is caused by the unavailability of a storage node, a backup storage node can be searched again in the storage path optimization mapping table, and the relevant corresponding relationship can be updated; if the conflict is that the number of concurrent processes does not match the capacity of the storage node, the number of concurrent processes in the resource scheduling priority rule can be further adjusted to adapt to the actual capacity of the storage node. Through such conflict detection and processing, it is ensured that there is no logical contradiction in the execution order between different rules in the dynamic resource integration strategy, and the effectiveness and feasibility of the strategy are guaranteed.

[0119] Step S150: Feed back the dynamic resource integration strategy to the data management system of the target network environment, and trigger the data management system to perform a resource directory update operation.

[0120] The dynamic resource integration strategy DynamicResourceIntegrationStrategy after conflict detection and processing is fed back to the data management system of the target network environment. The data management system is responsible for managing various resource information in the network environment, including the storage location, attributes, etc. of the resources.

[0121] After receiving the dynamic resource integration strategy, the data management system first parses it. According to the storage path optimization mapping table part in the strategy, the resource storage location information to be updated is determined. For example, if the mapping table indicates that resource ResourceX needs to be migrated from storage node NodeA to storage node NodeB, the data management system will record this change.

[0122] At the same time, according to the resource scheduling priority rule, the data management system updates the scheduling priority information of the resources. For example, the priority of some resources is raised from a lower level to a higher level, or the number of concurrent processes is adjusted, etc.

[0123] After parsing the dynamic resource integration strategy, the data management system starts to perform the resource directory update operation. It traverses the resource directory in the system, and for each resource, corresponding updates are made according to the information in the dynamic resource integration strategy. If the storage location of the resource has changed, the data management system will modify the storage location field of the resource in the resource directory to point to the new storage node. If the scheduling priority of the resource has changed, the data management system will update the priority field of the resource in the resource directory.

[0124] In this way, the data management system completes the resource directory update operation, making the resource management in the system consistent with the dynamic resource integration strategy, thereby realizing the optimal scheduling of resources and the reasonable adjustment of storage paths in the heterogeneous network environment, and improving the resource utilization efficiency and operation performance of the entire network environment.

[0125] In the above embodiments, an information pattern analysis model involving pre-training is concerned. The construction, training of this artificial intelligence model and its combination with the overall technical solution will be elaborated in detail below.

[0126] The information pattern analysis model mainly consists of a feature encoding layer and a pattern matching layer. The role of the feature encoding layer is to perform layer-by-layer non-linear transformation on the input associated feature set to generate a multi-layer abstract feature representation sequence. This layer contains multiple neural network layers, and there is a tight connection relationship between each layer. Assume starting from the input layer, which are the 1st layer, the 2nd layer... the nth layer in sequence. The input layer receives the associated feature set as input, and the number of its neurons corresponds to the dimension of the associated feature set to ensure that the input information can be completely received.

[0127] The 1st layer is fully connected to the input layer, and it performs the first non-linear transformation on the input associated features. Common activation functions, such as the ReLU function, can be used here to introduce non-linearity. In this layer, the input features are weighted and summed through a weight matrix, and then processed by the activation function to map the associated features from the original dimension to a new dimension. Assume the original dimension of the associated feature set is d0, and the feature dimension output by the 1st layer is d1, and d1 and d0 satisfy the preset dimensionality reduction ratio r, that is, d1 = r * d0. This dimensionality reduction process helps to extract more critical feature information and reduce data redundancy.

[0128] The 2nd layer is also fully connected to the 1st layer, and it further processes the features output by the 1st layer. Similar to the 1st layer, the 2nd layer weights and sums the input features through a weight matrix, and then processes them through the activation function (which can also be the ReLU function), and outputs features with a dimension of d2 = r * d1. And so on, each layer takes the output of the previous layer as input, performs non-linear transformation through the weight matrix and the activation function, and performs dimensionality transformation according to the preset dimensionality reduction ratio, and finally generates a multi-layer abstract feature representation sequence [L1, L2,..., Ln].

[0129] The pattern matching layer is responsible for calculating the cross-layer correlation degree of the multi-layer abstract feature representation sequence to determine the pattern similarity index between adjacent layer abstract feature representations. This layer mainly operates based on the multi-layer abstract feature representation sequence output by the feature encoding layer.

[0130] In terms of model training, a large amount of training data needs to be prepared first. These training data should have similar characteristics to the heterogeneous network environment data in the actual application scenario, include the original information resources in various data formats, and have been labeled with the potential information recognition results, that is, the attribute identifiers and distribution paths of the significant value information resources.

[0131] The training steps are as follows: 1. Initialize model parameters: Randomly initialize the weight matrices and bias terms of each layer in the feature encoding layer. These initial values will be continuously adjusted during the training process through the backpropagation algorithm to optimize the model performance.

[0132] 2. Forward propagation: Input the set of associated features in the training data into the information pattern analysis model. Through the layer-by-layer non-linear transformation of the feature encoding layer, a sequence of multi-layer abstract feature representations is generated. Then, this sequence enters the pattern matching layer to calculate the cross-layer correlation degree, obtain the pattern similarity index, and generate potential information recognition results based on this.

[0133] 3. Calculate the loss: Compare the potential information recognition results generated by the model with the labeled true results, and calculate the difference between the two through a suitable loss function (such as the cross-entropy loss function) to obtain the loss value. The loss value reflects the deviation degree of the current prediction result of the model from the true result.

[0134] 4. Backward propagation: According to the loss value, calculate the gradients of the loss value with respect to the parameters (weight matrices and bias terms) of each layer of the model through the backpropagation algorithm. The gradient represents the direction and degree of the impact of a small change in the parameter on the loss value.

[0135] 5. Parameter update: Use an optimizer (such as Stochastic Gradient Descent SGD, Adagrad, Adadelta, etc.) to update the parameters of the model according to the calculated gradients. The optimizer adjusts the parameters to make the loss value change in the decreasing direction. For example, in the Stochastic Gradient Descent algorithm, update the parameters in the opposite direction of the gradient at a set learning rate. The learning rate is a pre-set parameter that controls the step size of each parameter update.

[0136] 6. Repeat training: Repeat the above steps of forward propagation, calculating the loss, backward propagation, and parameter update, and perform multiple iterative trainings on a large amount of training data. In each iteration, the parameters of the model are continuously adjusted, the loss value gradually decreases, and the performance of the model is gradually improved. The training process ends until the loss value converges to a small value or reaches the preset maximum number of iterations.

[0137] In the actual application scenario, the information pattern analysis model is closely combined with the entire potential information recognition method. The input of the model is the set of associated features obtained after feature extraction, which contains the semantic association features and dynamic interaction features between each original information resource in the heterogeneous network environment data set. Through the processing of the feature encoding layer and the pattern matching layer, the model outputs a set of potential information recognition results, including the attribute identifiers and distribution paths of multiple significant value information resources. These output results are directly used to generate dynamic resource integration strategies in the subsequent stage to adjust the resource scheduling rules and storage path mapping relationships in the heterogeneous network environment.

[0138] In the processes of data collection, label management, rule setting, etc. above, strictly abide by laws, regulations and ethical guidelines to ensure that there are no situations of violating laws, going against fairness and justice, or having discrimination and prejudice. For example, in the data collection stage, clearly inform the data providers of the purpose, scope and protection measures of data use to ensure the legality and transparency of data collection. In terms of label management, strictly review the identified results of potential information in the annotations to avoid unfair decisions caused by incorrect labels or biases. In rule setting, ensure that the formulation of rules such as resource scheduling priority rules and storage path optimization mapping tables is based on objective resource characteristics and requirements, rather than discriminatory factors.

[0139] Figure 2 FIG. shows a schematic diagram of exemplary hardware and software components of a potential information recognition system 100 that combines heterogeneous network environments and can implement the idea of the present application. For example, the processor 120 can be used in the potential information recognition system 100 that combines heterogeneous network environments and is used to execute the functions in the present application.

[0140] The potential information recognition system 100 that combines heterogeneous network environments can be a general-purpose server or a special-purpose server, both of which can be used to implement the potential information recognition method that combines heterogeneous network environments of the present application. Although only one server is shown in the present application, for convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0141] For example, the potential information recognition system 100 that combines heterogeneous network environments can include a network port 110 connected to the network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as disks, ROM, or RAM, or any combination thereof. Exemplarily, the potential information recognition system 100 that combines heterogeneous network environments can also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The potential information recognition system 100 that combines heterogeneous network environments also includes an I / O interface 150 between the computer and other input / output devices.

[0142] For ease of description, only one processor is described in the potential information recognition system 100 for integrating heterogeneous network environments. However, it should be noted that the potential information recognition system 100 for integrating heterogeneous network environments in this application may also include multiple processors. Therefore, the steps performed by one processor described in this application can also be jointly performed or separately performed by multiple processors. For example, if the processor of the potential information recognition system 100 for integrating heterogeneous network environments performs steps A and B, it should be understood that steps A and B can also be jointly performed by two different processors or separately performed in one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0143] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the potential information recognition method for integrating heterogeneous network environments as described above is implemented.

[0144] It should be noted that, in order to simplify the description of the present invention disclosure and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, sometimes multiple features are merged into one embodiment, drawing, or description thereof.

Claims

1. A potential information recognition method combining heterogeneous network environments, characterized in that, The method includes: Obtaining a heterogeneous network environment data set of a target network environment, where the heterogeneous network environment data set contains original information resources in multiple data formats; Performing feature extraction on the heterogeneous network environment data set to obtain an associated feature set, where the associated feature set includes semantic association features and dynamic interaction features between the original information resources; Based on a pre-trained information pattern analysis model, performing hierarchical pattern matching processing on the associated feature set to generate a set of potential information recognition results, where the set of potential information recognition results contains attribute identifiers and distribution paths of multiple significant value information resources; Generating a dynamic resource integration strategy according to the set of potential information recognition results, where the dynamic resource integration strategy is used to adjust resource scheduling rules and storage path mapping relationships in the heterogeneous network environment; Feeding back the dynamic resource integration strategy to the data management system of the target network environment to trigger the data management system to perform a resource directory update operation.

2. The potential information recognition method in combination with a heterogeneous network environment according to claim 1, characterized in that The performing feature extraction on the heterogeneous network environment data set to obtain an associated feature set includes: Performing semantic vectorization processing on text data units in the heterogeneous network environment data set to generate a text semantic feature set, where each text semantic feature in the text semantic feature set corresponds to a semantic vector representation of a text data unit; Performing spatial feature encoding processing on image data units in the heterogeneous network environment data set to generate an image spatial feature set, where each image spatial feature in the image spatial feature set contains object distribution information and context relationship information in the image data unit; Performing dynamic pattern analysis processing on time series data units in the heterogeneous network environment data set to generate a time series dynamic feature set, where each time series dynamic feature in the time series dynamic feature set contains data change trends and periodic fluctuation information; Performing cross-modal alignment processing on the text semantic feature set, the image spatial feature set, and the time series dynamic feature set to determine semantic association features between different modal features, and generating dynamic interaction features based on feature interaction frequencies; Aggregating the semantic association features and the dynamic interaction features to construct the associated feature set.

3. The potential information recognition method for combining heterogeneous network environments according to claim 2, wherein The performing cross-modal alignment processing on the text semantic feature set, the image spatial feature set, and the time series dynamic feature set to determine semantic association features between different modal features includes: For each text semantic feature corresponding to a text data unit, matching image data units and time series data units with synchronous timestamps within a preset time window according to its timestamp; After mapping text semantic features, image spatial features, and time series dynamic features to a unified dimensional space through a projection layer, calling a cross-modal alignment model to perform joint encoding processing on the mapped features to generate cross-modal joint encoding features; Performing similarity calculation processing on the cross-modal joint encoding features to determine a first association degree score between the text semantic feature and the image spatial feature, and a second association degree score between the text semantic feature and the time series dynamic feature; Construct a semantic association matrix based on the first association degree score and the second association degree score. The rows of the semantic association matrix represent text data unit identifiers, the columns represent image data unit or time series data unit identifiers, and the matrix elements are the corresponding association degree scores; Extract the feature combinations corresponding to the association degree scores exceeding a preset threshold in the semantic association matrix to generate the semantic association features.

4. The potential information recognition method for combining heterogeneous network environments according to claim 3, characterized in that The generation of dynamic interaction features based on the feature interaction frequency includes: Partition each data unit in the heterogeneous network environment data set by data category, count the number of interaction events between data units within each partition, and construct a partition interaction frequency matrix. The rows and columns of the interaction frequency matrix represent different data unit identifiers respectively, and the matrix elements represent the number of interactions between the corresponding data units; Perform time dimension decomposition processing on the interaction frequency matrix to obtain short-term interaction frequency features and long-term interaction trend features; Calculate the difference between the short-term interaction frequency features and the long-term interaction trend features to generate dynamic fluctuation features; Determine the dynamic interaction intensity between each data unit according to the change direction and amplitude of the dynamic fluctuation features; Perform weighted correction processing on the semantic association features based on the dynamic interaction intensity to generate the dynamic interaction features.

5. The potential information recognition method in combination with a heterogeneous network environment according to claim 1, wherein The hierarchical pattern matching process of the association feature set based on the pre-trained information pattern analysis model to generate a set of potential information recognition results includes: Input the association feature set into the feature encoding layer of the information pattern analysis model, perform layer-by-layer non-linear transformation processing to generate a multi-layer abstract feature representation sequence. Among them, the dimension of the nth layer abstract feature representation and the dimension of the (n - 1)th layer abstract feature representation have a preset dimensionality reduction ratio; In the pattern matching layer of the information pattern analysis model, calculate the cross-layer association degree of the multi-layer abstract feature representation sequence to determine the pattern similarity index between adjacent layer abstract feature representations; According to the comparison result between the pattern similarity index and the preset dynamic threshold, screen out the abstract feature representations that meet the hierarchical transition conditions from the multi-layer abstract feature representation sequence to generate a hierarchical feature subset; Perform pattern clustering processing on the hierarchical feature subset to generate an initial candidate pattern set, where each initial candidate pattern contains a pattern center vector and a membership degree range; Based on the distance distribution between the pattern center vector and each feature vector in the association feature set, calculate the feature vector distribution density within a preset neighborhood range of each initial candidate pattern as the information density value, and determine the coverage radius according to the maximum distance threshold; Perform redundant merging processing on the initial candidate pattern set according to the information density value and the coverage radius, and merge the initial candidate patterns with the difference in information density values within the overlapping coverage radius less than the preset merging threshold to generate an optimized pattern set; Extract the feature unit identifiers corresponding to each pattern center vector from the optimized pattern set, and associate the resource attribute labels and path topology relationships in the semantic association features to generate the attribute identifiers and distribution paths of the significant value information resources; Aggregate the attribute identifiers and distribution paths of all significant value information resources to construct the potential information identification result set.

6. The potential information recognition method combining heterogeneous network environments according to claim 5, characterized in that The performing cross-layer correlation calculation on the multi-layer abstract feature representation sequence to determine the pattern similarity index between the abstract feature representations of adjacent layers includes: For the k-th layer abstract feature representation and the k+1-th layer abstract feature representation, calculate the cosine similarity of the two at the same feature unit position to generate a local similarity matrix; Performing a sliding window mean pooling process on the local similarity matrix to generate a windowed similarity distribution map, wherein the sliding window size matches the dimensionality scaling factor of the k-th layer abstract feature representation; Extracting regions exceeding a preset activation threshold in the windowed similarity distribution map to generate a target associated region set; Counting the coverage ratio of the target associated region set in the windowed similarity distribution map, and calculating the global similarity weight between the k-th layer and the k+1-th layer; Generate the pattern similarity index according to the product of the global similarity weight and the preset level attenuation coefficient; If the pattern similarity index is greater than a preset level transition threshold, the k-th level and the k+1-th level abstract feature representations are simultaneously added to the hierarchical feature subset; If the pattern similarity index is less than or equal to the hierarchical transition threshold and the k+1th layer is the lowest layer, only the k+1th layer abstract feature representation is added to the hierarchical feature subset.

7. The potential information recognition method in combination with a heterogeneous network environment according to claim 1, wherein Generating a dynamic resource integration strategy based on the potential information identification result set includes: Analyze the attribute identifiers of each significant value information resource in the potential information identification result set to determine the data category and access permission level to which it belongs; Extracting the storage location identifier and transmission path topology relationship of each significant value information resource according to the distribution path; Generate resource scheduling priority rules based on the data category and access permission level, wherein the resource scheduling priority rules define the response order and concurrent processing quantity of different categories of resources; Generating a storage path optimization mapping table based on the storage location identifier and the transmission path topology relationship, wherein the storage path optimization mapping table includes a correspondence between the resource identifier and the optimal storage node; The resource scheduling priority rule is subjected to policy fusion processing with the storage path optimization mapping table to generate the dynamic resource integration strategy.

8. The potential information recognition method for integrating heterogeneous network environments according to claim 7, wherein The step of performing policy fusion processing on the resource scheduling priority rule and the storage path optimization mapping table to generate the dynamic resource integration strategy includes: Calling the node monitoring interface of the target network environment to obtain real-time load status data of the storage nodes, combining the response order in the resource scheduling priority rule, performing load balancing analysis on the storage nodes in the storage path optimization mapping table, and determining the real-time load status of each storage node; If the real-time load status of the target storage node exceeds the capacity threshold, the backup storage node is searched in the storage path optimization mapping table according to the service priority weight of the resource, and the resources with service priority weight lower than the preset migration threshold are preferentially migrated and the corresponding relationship is updated; Optimize the concurrent processing quantity in the resource scheduling priority rules according to the updated storage path optimization mapping table, so that the concurrent processing quantity of the preset first-priority resources matches the capacity upper limit of the storage nodes with sufficient remaining available capacity; Perform a policy combination of the adjusted resource scheduling priority rules and the updated storage path optimization mapping table to generate a fused dynamic resource integration strategy; Perform conflict detection processing on the dynamic resource integration strategy to ensure that there is no logical contradiction in the execution order between different rules.

9. The potential information recognition method for combining heterogeneous network environments according to claim 1, wherein Before feeding back the dynamic resource integration strategy to the data management system of the target network environment, it further includes: Perform executability verification processing on the dynamic resource integration strategy, including: Deploy the dynamic resource integration strategy in a simulation environment, and monitor the response time and error rate during the resource scheduling process by injecting multi-scenario stress test data; If the response time exceeds the preset time threshold or the error rate exceeds the preset fault tolerance threshold, perform iterative optimization processing on the resource scheduling rules in the dynamic resource integration strategy; Re-execute the executability verification processing until the response time and error rate meet the preset conditions; Mark the dynamically resource integration strategy that passes the verification as an executable strategy, and generate a strategy version identifier and an execution log template; Among them, the iterative optimization processing of the resource scheduling rules in the dynamic resource integration strategy includes: Analyze the scheduling tasks with exceeded response time, and determine the associated resource categories and path nodes; Reduce the priority of the resource category in the resource scheduling priority rules and allocate additional concurrent processing channels for it; For the scheduling tasks with exceeded error rate, check the node connection status in the storage path mapping relationship, and replace the unstable nodes with standby nodes; Update the node health status identifier in the storage path optimization mapping table and recalculate the optimal storage nodes; Recombine the optimized resource scheduling rules and the storage path mapping relationship to generate a new version of the dynamic resource integration strategy.

10. A potential information recognition system integrating heterogeneous network environments, characterized in that, It includes a processor and a memory. The memory is connected to the processor. The memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the potential information recognition method for combining heterogeneous network environments according to any one of claims 1-9 above.

Images