Intelligent bee colony type quality optimization management method and system for multi-source tag data

Abstract

The invention provides an intelligent bee colony type quality optimization management method and system for multi-source tag data, and relates to the technical field of data tags, and the method comprises the steps: extracting a comprehensive feature vector of a tag through a feature extraction module, carrying out the processing through a deep feature enhancement network, generating a similarity and dependency feature matrix, and constructing a tag quality evaluation model; mapping the label data into bee colony network nodes, performing three-level division, and constructing a node optimization association network; and performing optimization based on the collaborative optimization rule set until the quality improvement value is smaller than a preset threshold value. The label data quality can be effectively improved, the data processing cost is reduced, and the optimization efficiency is improved.

Description

Technical Field

[0001] The present invention relates to data labeling technology, and in particular to an intelligent bee colony quality optimization management method and system for multi-source label data. Background Art

[0002] With the advent of the big data era, labeled data has been widely used in various fields as an important way of data organization and management. Labeled data can not only effectively describe and classify information, but also help users quickly retrieve and understand relevant content. In practical applications, labeled data often comes from multiple different data sources, which may include user annotations, automatic generation, expert annotations and other channels. The quality of multi-source labeled data directly affects the availability and application effect of the data, so it is of great significance to optimize the quality management of multi-source labeled data.

[0003] At present, the quality management of multi-source labeled data mainly adopts methods such as manual review and simple rule filtering. These traditional methods have some obvious shortcomings when processing large-scale multi-source labeled data: Existing label quality assessment methods often only focus on features of a single dimension, ignoring the semantic relevance and timeliness characteristics between labels, resulting in incomplete and inaccurate quality assessment results.

[0004] Traditional label optimization methods lack systematicity and coordination. The optimization process of each label is relatively independent, and they fail to fully utilize the similarities and dependencies between labels to improve the optimization effect.

[0005] Existing optimization methods usually adopt static optimization strategies and cannot adaptively adjust the optimization scheme according to the dynamic changes of label quality, resulting in unsatisfactory optimization results and easy to fall into local optimality.

[0006] Therefore, there is an urgent need for a label data quality optimization management method that can comprehensively consider multi-dimensional features, support collaborative optimization, and have adaptive capabilities to improve the overall quality level of multi-source label data. Summary of the invention

[0007] The embodiments of the present invention provide an intelligent swarm-type quality optimization management method and system for multi-source label data, which can solve the problems in the prior art.

[0008] According to a first aspect of the embodiments of the present invention, Provides an intelligent swarm-style quality optimization management method for multi-source label data, including: Obtain multi-source label data input, extract the comprehensive feature vector of each label respectively through the feature extraction module, the comprehensive feature vector includes a semantic feature vector, an association feature vector and a time feature vector, input the comprehensive feature vector into the feature fusion layer to generate a label comprehensive feature matrix; process the label comprehensive feature matrix through a deep feature enhancement network, calculate the similarity feature matrix and the dependency feature matrix between labels; construct a label quality assessment model based on each feature matrix, calculate the initial quality score of each label, and generate a label quality grading sequence; According to the label quality classification sequence, the label data is mapped into bee colony network nodes, and the nodes are divided into three levels based on the initial quality score: the nodes with quality scores higher than the preset threshold are set as instruction bees, which are responsible for formulating optimization strategies; the nodes with quality scores equal to the preset threshold are set as follower bees, which are responsible for transmitting optimization instructions; the nodes with quality scores lower than the preset threshold are set as execution bees, which are responsible for implementing optimization operations; the node optimization association network is constructed using the similarity feature matrix and the dependency feature matrix, and a collaborative optimization rule set is designed based on the node optimization association network; The optimization process is started based on the collaborative optimization rule set, the instruction bee analyzes the feature pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan; after the optimization is completed, the optimization result is re-input into the feature extraction module for quality evaluation and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is ended and the final label data set is output.

[0009] The comprehensive feature matrix of the labels is processed by a deep feature enhancement network to calculate the similarity feature matrix and the dependency feature matrix between the labels; a label quality evaluation model is constructed based on each feature matrix to calculate the initial quality score of each label, and a label quality grading sequence is generated, including: Constructing a deep feature enhancement network, the deep feature enhancement network includes a feature mapping layer, a feature enhancement layer and a feature reconstruction layer, the feature mapping layer uses a nonlinear activation function to map the label comprehensive feature matrix to a high-dimensional latent space, the feature enhancement layer calculates the attention weights between feature vectors and performs weighted fusion based on the attention weights to achieve feature enhancement, and the feature reconstruction layer uses a deconvolution operation to remap the enhanced features to the original feature space; The cosine similarity method is used to calculate the semantic similarity between tags to obtain the similarity feature matrix. At the same time, the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain the dependency feature matrix. The similarity feature matrix and the dependency feature matrix are subjected to feature fusion with the original label comprehensive feature matrix to construct a label quality assessment model, wherein the label quality assessment model calculates the label feature quality, the similarity relationship between labels and the label dependency relationship respectively, and obtains an initial quality score by weighted summation; The initial quality score is normalized, a density clustering method is used to automatically determine a quality grade division threshold, and a label quality grading sequence containing quality grade information is generated.

[0010] The cosine similarity method is used to calculate the semantic similarity between tags to obtain the similarity feature matrix. At the same time, the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain the dependency feature matrix, including: Obtain a set of label texts, input the set of label texts into a pre-trained word vector model, and obtain an initial word vector corresponding to each label text; for a label containing multiple words in the label text set, calculate the word frequency-inverse document frequency value of each word in the label as a weight coefficient, and perform weighted averaging on the initial word vectors of each word according to the weight coefficient to obtain an overall semantic feature vector of the label; Based on the overall semantic feature vector, the cosine similarity between any two tags in the tag text set is calculated, and the cosine similarity is filled into the row and column positions corresponding to the similarity feature matrix to obtain the semantic similarity feature matrix of the tag text set, wherein the dimension of the semantic similarity feature matrix is ​​the product of the total number of tags and the total number of tags; Acquire the time series data of the tag text set, set a time window, analyze the co-occurrence pattern of the tag text set within the time window, count the co-occurrence frequency of tag pairs, record the relative appearance order of tags, and calculate the time interval distribution of tag pairs; based on the co-occurrence frequency, the relative appearance order and the time interval distribution, use a preset weight coefficient for weighted combination to obtain the time series dependency strength of the tag pairs; A directed weighted graph is constructed according to the temporal dependency strength, the labels in the label text set are mapped to nodes of the directed weighted graph, the connection direction between the nodes is determined based on the relative appearance order of the label pairs, and the temporal dependency strength is used as the weight value of the corresponding connection to obtain a directed temporal weighted graph that describes the temporal association of the labels; a dependency feature matrix is ​​constructed based on the connection relationship and weight value of the directed temporal weighted graph.

[0011] The similarity feature matrix and the dependency feature matrix are fused with the original label comprehensive feature matrix to construct a label quality assessment model. The label quality assessment model calculates the label feature quality, the similarity relationship between labels and the label dependency relationship respectively, and obtains the initial quality score by weighted summation, including: The semantic similarity features and the structural similarity features in the similarity feature matrix are weighted averaged to obtain the comprehensive similarity features between tags, and the hierarchical dependency features and the associated dependency features in the dependency feature matrix are weighted averaged to obtain the comprehensive dependency features between tags; The language standardization features, professional terminology standardization features and semantic integrity features in the original label comprehensive feature matrix are normalized to obtain the normalized features, and the word frequency distribution features and query coverage features are logarithmically transformed to obtain the statistical features; The comprehensive similarity feature between labels, the comprehensive dependency feature between labels, the normalized feature and the statistical feature are serially combined to construct a feature fusion matrix, and a label quality assessment model is constructed based on the feature fusion matrix; Calculating a quality score of the label's own features based on the label quality assessment model, wherein the quality score of the label's own features is obtained by weighted summing the normalized features and the statistical features; Calculating a similarity score between tags based on the tag quality assessment model, wherein the similarity score between tags is obtained by threshold screening of the comprehensive similarity features between tags; Calculating a tag dependency score based on the tag quality assessment model, wherein the tag dependency score is obtained by weighted calculation of the comprehensive dependency features between the tags; The tag quality initial score is obtained by weighted summing the tag feature quality score, the tag similarity score and the tag dependency score.

[0012] The optimization process is started based on the collaborative optimization rule set, the instruction bee analyzes the characteristic pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan, including: Constructing a node optimization association network and establishing a collaborative optimization rule set, wherein the collaborative optimization rule set includes feature extraction rules, optimization instruction generation rules, and quality optimization rules, and starting a label optimization process based on the collaborative optimization rule set; The instruction bee analyzes the first quality label according to the feature extraction rule, obtains semantic features through word vector representation, obtains structural features through hierarchical relationships, obtains usage features through application frequency, and weightedly combines the semantic features, the structural features and the usage features to generate label feature data; The instruction bee processes the label feature data based on the optimization instruction generation rule, inputs the label feature data into the feature vector space for cluster analysis, extracts feature patterns to establish a feature pattern library, and generates an optimization instruction set according to the feature pattern library; The follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability; The execution bee analyzes the semantic content and word vector features of the second quality label based on the quality optimization rules and the task allocation plan, selects words with similar semantic content from a preset synonym library to supplement them, and expands the dimensional information of the word vector features to generate an optimized label.

[0013] The follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability, including: receiving the optimization instruction set, extracting a priority parameter in the optimization instruction set, and determining a semantic distance weight, a structural distance weight, and a functional distance weight based on the priority parameter; Calculate the cosine similarity of the node label word vector to obtain the semantic distance, calculate the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculate the co-occurrence frequency and complementarity between nodes to obtain the functional distance; The semantic distance, the structural distance and the functional distance are respectively multiplied by corresponding weights and added to generate a comprehensive optimized distance matrix of the node optimized association network; Obtaining the task queue length and resource occupancy of the node to obtain the node load value, analyzing the time resources of the node to obtain the time constraint value, and calculating the optimization cost based on the comprehensive optimization distance matrix, the node load value and the time constraint value; Setting an initial pheromone distribution according to the priority information of the optimization instruction set, generating a state transition probability in combination with the optimization cost and the node optimization capability, and performing a path search based on the state transition probability; During the path search process, local pheromones are updated in real time, and the historical optimal path information is recorded. When the number of search iterations reaches the preset threshold, the optimal optimization path is selected; Decomposing the optimization task based on the optimal optimization path and the node processing capacity, generating a subtask set and determining the dependency between the subtasks, and calculating the resource requirement of each subtask; A task scheduling strategy is formulated according to the subtask set, the dependency relationship and the resource demand, and a task allocation plan including execution order and parallelism is generated.

[0014] Calculate the cosine similarity of the node label word vector to obtain the semantic distance, calculate the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculate the co-occurrence frequency and complementarity between nodes to obtain the functional distance, including: Obtain node label text, input the node label text into a pre-trained word vector model, and generate a word vector corresponding to the node label text; calculate the cosine similarity of the word vectors between the node labels to obtain the semantic distance between the node labels; The semantic distance calculation formula is as follows: ; in, is the semantic similarity, t i , t j For two label nodes, is the corresponding word vector; Obtain a pre-constructed label hierarchy tree, locate the node position in the label hierarchy tree based on the semantic distance; calculate the hierarchy span between nodes, the hierarchy span is determined by the path length between nodes; identify the common ancestor node between nodes and calculate the depth of the common ancestor node; perform a weighted combination of the hierarchy span and the depth to obtain the structural distance between node labels; The formula for calculating the structural distance is as follows: ; Among them, α is the weight coefficient of the structural distance, path(t i ,t j ) is the hierarchical span between nodes, maxPath is the maximum path length in the tree, depth(LCA) is the depth of the nearest common ancestor node, and maxDepth is the maximum depth of the tree; Determine the associated node pairs based on the semantic distance and the structural distance, and count the co-occurrence frequencies of the associated node pairs; extract usage scenario information of the associated node pairs, and calculate the degree of complementarity of the node labels in the usage scenarios; perform weighted combination of the co-occurrence frequency and the degree of complementarity to obtain the functional distance between the node labels; The functional distance calculation formula is as follows: ; Among them, t i is the i-th label node to be calculated, t j is the jth label node to be calculated, β is the weight coefficient of the functional distance, coot(t i ,t j ) is the number of co-occurrences of the tag pair, is the total number of occurrences of all label pairs, t k is the kth tag combination in the tag set, For label t i The distribution vector in all scenes, For label t j The distribution vector in all scenes, is the scene vector of the label, is the Jensen-Shannon divergence of the scene vector.

[0015] According to a second aspect of the embodiments of the present invention, Provides intelligent swarm-style quality optimization management system with multi-source label data, including: The first unit is used to obtain multi-source label data input, extract the comprehensive feature vector of each label respectively through the feature extraction module, the comprehensive feature vector includes a semantic feature vector, an associated feature vector and a time feature vector, and input the comprehensive feature vector into the feature fusion layer to generate a label comprehensive feature matrix; process the label comprehensive feature matrix through a deep feature enhancement network, calculate the similarity feature matrix and the dependency feature matrix between the labels; construct a label quality evaluation model based on each feature matrix, calculate the initial quality score of each label, and generate a label quality grading sequence; The second unit is used to map the label data into bee colony network nodes according to the label quality classification sequence, and divide the nodes into three levels based on the initial quality score: set the nodes with quality scores higher than the preset threshold as instruction bees, responsible for formulating optimization strategies; set the nodes with quality scores equal to the preset threshold as follower bees, responsible for transmitting optimization instructions; set the nodes with quality scores lower than the preset threshold as execution bees, responsible for implementing optimization operations; construct a node optimization association network using the similarity feature matrix and the dependency feature matrix, and design a collaborative optimization rule set based on the node optimization association network; The third unit is used to start the optimization process based on the collaborative optimization rule set, the instruction bee analyzes the feature pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan; after the optimization is completed, the optimization result is re-input into the feature extraction module for quality evaluation and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is terminated and the final label data set is output.

[0016] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0017] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0018] The beneficial effects of this application are as follows: 1. By constructing a deep feature enhancement network to process the comprehensive feature matrix of labels, it can effectively capture the similarity and dependency relationships between labels, realize accurate assessment of label data quality, and improve the accuracy and reliability of label quality assessment.

[0019] 2. The hierarchical structure design of the bee swarm network is adopted, and the label nodes are divided into three levels: command bees, follower bees and execution bees. A clear optimization division of labor mechanism is established. Through the formulation of collaborative optimization rule sets, efficient optimization of multi-source label data is achieved, which significantly improves the execution efficiency of label optimization.

[0020] 3. Task allocation and path planning are carried out based on the node optimization association network, combined with the dynamic feedback mechanism of quality assessment, to achieve adaptive adjustment of the label optimization process, ensure the continuous improvement of the optimization results, and finally output a high-quality label data set, providing a reliable data foundation for subsequent data applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of an intelligent bee colony quality optimization management method for multi-source tag data according to an embodiment of the present invention; Figure 2 The following is a general architecture diagram of the data tag platform according to an embodiment of the present invention; Figure 3 It is a structural schematic diagram of an intelligent bee colony quality optimization management system for multi-source label data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0024] Figure 1 FIG. 1 is a flow chart of an intelligent swarm-type quality optimization management method for multi-source tag data according to an embodiment of the present invention. Figure 1 As shown, the method includes: S11. Obtain multi-source label data input, extract the comprehensive feature vector of each label respectively through the feature extraction module, the comprehensive feature vector includes a semantic feature vector, an association feature vector and a time feature vector, input the comprehensive feature vector into the feature fusion layer to generate a label comprehensive feature matrix; process the label comprehensive feature matrix through a deep feature enhancement network, calculate the similarity feature matrix and the dependency feature matrix between labels; construct a label quality assessment model based on each feature matrix, calculate the initial quality score of each label, and generate a label quality grading sequence; S12. According to the label quality classification sequence, the label data is mapped into bee colony network nodes, and the nodes are divided into three levels based on the initial quality score: the nodes with quality scores higher than the preset threshold are set as instruction bees, responsible for formulating optimization strategies; the nodes with quality scores equal to the preset threshold are set as follower bees, responsible for transmitting optimization instructions; the nodes with quality scores lower than the preset threshold are set as execution bees, responsible for implementing optimization operations; the node optimization association network is constructed using the similarity feature matrix and the dependency feature matrix, and a collaborative optimization rule set is designed based on the node optimization association network; S13. The optimization process is started based on the collaborative optimization rule set, the instruction bee analyzes the feature pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan; after the optimization is completed, the optimization result is re-input into the feature extraction module for quality evaluation and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is terminated and the final label data set is output.

[0025] In an optional implementation, the comprehensive feature matrix of the labels is processed by a deep feature enhancement network to calculate the similarity feature matrix and the dependency feature matrix between the labels; a label quality assessment model is constructed based on each feature matrix, an initial quality score of each label is calculated, and a label quality grading sequence is generated, including: Constructing a deep feature enhancement network, the deep feature enhancement network includes a feature mapping layer, a feature enhancement layer and a feature reconstruction layer, the feature mapping layer uses a nonlinear activation function to map the label comprehensive feature matrix to a high-dimensional latent space, the feature enhancement layer calculates the attention weights between feature vectors and performs weighted fusion based on the attention weights to achieve feature enhancement, and the feature reconstruction layer uses a deconvolution operation to remap the enhanced features to the original feature space; The cosine similarity method is used to calculate the semantic similarity between tags to obtain the similarity feature matrix. At the same time, the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain the dependency feature matrix. The similarity feature matrix and the dependency feature matrix are subjected to feature fusion with the original label comprehensive feature matrix to construct a label quality assessment model, wherein the label quality assessment model calculates the label feature quality, the similarity relationship between labels and the label dependency relationship respectively, and obtains an initial quality score by weighted summation; The initial quality score is normalized, a density clustering method is used to automatically determine a quality grade division threshold, and a label quality grading sequence containing quality grade information is generated.

[0026] The present invention proposes a label quality assessment method based on a deep feature enhancement network. The method processes the label comprehensive feature matrix, calculates the similarity feature matrix and the dependency feature matrix between labels, and constructs a label quality assessment model based on these feature matrices, and finally generates a label quality grading sequence.

[0027] First, a deep feature enhancement network is constructed. The network includes a feature mapping layer, a feature enhancement layer, and a feature reconstruction layer. The feature mapping layer uses nonlinear activation functions such as ReLU to map the label comprehensive feature matrix to a high-dimensional latent space. For example, assuming that the original feature matrix dimension is 100x50, it may be mapped to a high-dimensional space of 100x200 through the feature mapping layer. The feature enhancement layer achieves feature enhancement by calculating the attention weights between feature vectors and performing weighted fusion based on these weights. Specifically, the dot product attention mechanism can be used to calculate the attention score between each pair of feature vectors, and then the softmax function is used to normalize these scores to weights. Then, these weights are used to weighted sum the feature vectors to obtain the enhanced features. The feature reconstruction layer uses a deconvolution operation to remap the enhanced features to the original feature space. In this step, a transposed convolution operation can be used to remap high-dimensional features (such as 100x200) back to the original dimension (100x50).

[0028] Next, calculate the similarity feature matrix and dependency feature matrix between tags. For the similarity feature matrix, the cosine similarity method is used to calculate the semantic similarity between tags. Specifically, for the feature vectors of any two tags, calculate their dot product divided by their respective L2 norms to obtain a similarity value between -1 and 1. In this way, for n tags, an nxn similarity matrix is ​​finally obtained. For the dependency feature matrix, the time series association analysis method is introduced. By analyzing the appearance pattern and co-occurrence frequency of tags in the time series, a directed weighted graph is constructed. For example, the frequency of tag B appearing immediately after tag A in a certain time window can be counted, and this frequency can be used as the weight of the edge from A to B. Finally, an nxn dependency matrix is ​​obtained, in which each element represents the degree of dependence of one tag on another tag.

[0029] Then, the similarity feature matrix and the dependency feature matrix are fused with the original label comprehensive feature matrix to construct a label quality assessment model. This model calculates the quality of the label's own features, the similarity relationship between labels, and the label dependency relationship. For the quality of the label's own features, factors such as the frequency of use and coverage of the label can be considered. The similarity relationship between labels can directly use the values ​​in the similarity feature matrix. The label dependency relationship can be calculated based on the dependency feature matrix. Finally, the initial quality score of each label is obtained by weighted summation. For example, the weights can be set to 0.4, 0.3, and 0.3 respectively, and the scores of the three aspects can be weighted summed.

[0030] Finally, the initial quality scores are normalized, and the density clustering method is used to automatically determine the quality grade division threshold, generating a label quality grading sequence containing quality grade information. Normalization can use the minimum-maximum scaling method to map all scores between 0 and 1. Then, using density clustering algorithms such as DBSCAN, several cluster centers are automatically determined based on the distribution characteristics of the scores, and the midpoints between these centers can be used as the quality grade division threshold. For example, if three cluster centers are determined, the label quality can be divided into four levels: excellent, good, general, and to be improved. Finally, according to these thresholds, each label is assigned to the corresponding quality level to form a label quality grading sequence.

[0031] The solution of this application can: This method improves the expressiveness of label features through a deep feature enhancement network, and can better capture the complex relationships between labels, thereby improving the accuracy and reliability of label quality assessment. By introducing temporal association analysis and constructing a directed weighted graph, this method not only considers the static similarity between labels, but also the dynamic dependency, making the quality assessment results more comprehensive and objective. The density clustering method is used to automatically determine the quality grade division threshold, avoiding the subjectivity and inaccuracy that may be caused by manually setting the threshold, making the label quality grading more scientific and reasonable, and facilitating subsequent label optimization and management.

[0032] In an optional implementation, the cosine similarity method is used to calculate the semantic similarity between tags to obtain a similarity feature matrix, and the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain a dependency feature matrix, including: Obtain a set of label texts, input the set of label texts into a pre-trained word vector model, and obtain an initial word vector corresponding to each label text; for a label containing multiple words in the label text set, calculate the word frequency-inverse document frequency value of each word in the label as a weight coefficient, and perform weighted averaging on the initial word vectors of each word according to the weight coefficient to obtain an overall semantic feature vector of the label; Based on the overall semantic feature vector, the cosine similarity between any two tags in the tag text set is calculated, and the cosine similarity is filled into the row and column positions corresponding to the similarity feature matrix to obtain the semantic similarity feature matrix of the tag text set, wherein the dimension of the semantic similarity feature matrix is ​​the product of the total number of tags and the total number of tags; Acquire the time series data of the tag text set, set a time window, analyze the co-occurrence pattern of the tag text set within the time window, count the co-occurrence frequency of tag pairs, record the relative appearance order of tags, and calculate the time interval distribution of tag pairs; based on the co-occurrence frequency, the relative appearance order and the time interval distribution, use a preset weight coefficient for weighted combination to obtain the time series dependency strength of the tag pairs; A directed weighted graph is constructed according to the temporal dependency strength, the labels in the label text set are mapped to nodes of the directed weighted graph, the connection direction between the nodes is determined based on the relative appearance order of the label pairs, and the temporal dependency strength is used as the weight value of the corresponding connection to obtain a directed temporal weighted graph that describes the temporal association of the labels; a dependency feature matrix is ​​constructed based on the connection relationship and weight value of the directed temporal weighted graph.

[0033] The present invention proposes a label feature matrix construction method based on semantic similarity and temporal association, the method comprising the following steps: First, get the tag text collection. This collection contains all the tags that need to be analyzed. For example, suppose there is a collection containing 5 tags: {"machine learning", "deep learning", "artificial intelligence", "data mining", "neural network"}.

[0034] Next, the obtained label text set is input into a pre-trained word vector model. This model can be Word2Vec, GloVe, or FastText. Assuming that a 300-dimensional word vector model is used, each label will get a 300-dimensional initial word vector.

[0035] For tags containing multiple words, the term frequency-inverse document frequency (TF-IDF) value of each word in the tag needs to be calculated as the weight coefficient. For example, for the tag "deep learning", the TF-IDF values ​​of "depth" and "learning" are calculated respectively. Assume that the TF-IDF value of "depth" is 0.6 and the TF-IDF value of "learning" is 0.4.

[0036] Then, the initial word vectors of each word are weighted averaged according to the calculated weight coefficient to obtain the overall semantic feature vector of the label. Continuing with "deep learning" as an example, the word vector of "depth" is multiplied by 0.6, the word vector of "learning" is multiplied by 0.4, and then the two results are added together to obtain the overall semantic feature vector of the label "deep learning".

[0037] Based on the obtained overall semantic feature vector, calculate the cosine similarity between any two tags in the tag text set. For example, calculate the cosine similarity between "machine learning" and "deep learning", assuming that the result is 0.85. Fill the calculated cosine similarity into the corresponding row and column positions of the similarity feature matrix. For the above example of 5 tags, the final semantic similarity feature matrix is ​​a 5x5 matrix.

[0038] Next, we get the time series data of the tag text collection, which includes the timestamp information of the tag appearance.

[0039] Set a suitable time window, such as 1 hour. Analyze the co-occurrence pattern of the tag text set within this time window. Count the co-occurrence frequency of tag pairs, for example, "machine learning" and "deep learning" appear together 10 times within 1 hour. At the same time, record the relative order of appearance of tags, such as "machine learning" appears before "deep learning" in 80% of cases.

[0040] Calculate the time interval distribution of tag pairs. For example, if we count the time intervals between the tags "machine learning" and "deep learning" when they co-occur, we may get the following results: 50% of the time intervals are within 5 minutes, 30% of the time intervals are between 5-15 minutes, and 20% of the time intervals are between 15-30 minutes.

[0041] Based on the co-occurrence frequency, relative order of occurrence, and time interval distribution, a weighted combination is performed using a preset weight coefficient to obtain the temporal dependency strength of the tag pair. Assume that the weight of the co-occurrence frequency is set to 0.5, the weight of the relative order of occurrence is set to 0.3, and the weight of the time interval distribution is set to 0.2. Then, the temporal dependency strength of "machine learning" on "deep learning" can be calculated based on these weights.

[0042] Based on the calculated temporal dependency strength, a directed weighted graph is constructed. The labels in the label text set are mapped to nodes of the directed weighted graph. The connection direction between nodes is determined based on the relative order of appearance of the label pairs, for example, a directed edge is established from the "machine learning" node to the "deep learning" node. The temporal dependency strength is used as the weight value of the corresponding connection.

[0043] Finally, based on the connection relationship and weight value of the constructed directed time-series weighted graph, a dependency feature matrix is ​​constructed. The dimension of this matrix is ​​also the product of the total number of labels and the total number of labels, that is, 5x5. Each element in the matrix represents the degree of dependence of the row label on the column label.

[0044] Through the above steps, two feature matrices are obtained: semantic similarity feature matrix and dependency feature matrix. These two matrices together describe the semantic relationship and temporal association between tags, providing rich feature information for subsequent tag analysis and recommendation tasks.

[0045] The solution of this application can: This method comprehensively describes the relationship between tags by combining semantic similarity and temporal association analysis, improving the expressiveness and accuracy of tag features. The pre-trained word vector model and TF-IDF weighted average are used to effectively capture the semantic information of tags, overcoming the limitation of traditional methods that rely only on literal matching. The introduction of temporal association analysis takes into account the time pattern and co-occurrence characteristics of tag appearance, reveals the dynamic dependency between tags, and provides more valuable information support for time-related application scenarios.

[0046] In an optional implementation, the similarity feature matrix and the dependency feature matrix are feature-fused with the original label comprehensive feature matrix to construct a label quality assessment model, which calculates the label's own feature quality, the similarity relationship between labels, and the label dependency relationship, and obtains an initial quality score by weighted summation, including: The semantic similarity features and the structural similarity features in the similarity feature matrix are weighted averaged to obtain the comprehensive similarity features between tags, and the hierarchical dependency features and the associated dependency features in the dependency feature matrix are weighted averaged to obtain the comprehensive dependency features between tags; The language standardization features, professional terminology standardization features and semantic integrity features in the original label comprehensive feature matrix are normalized to obtain the normalized features, and the word frequency distribution features and query coverage features are logarithmically transformed to obtain the statistical features; The comprehensive similarity feature between labels, the comprehensive dependency feature between labels, the normalized feature and the statistical feature are serially combined to construct a feature fusion matrix, and a label quality assessment model is constructed based on the feature fusion matrix; Calculating a quality score of the label's own features based on the label quality assessment model, wherein the quality score of the label's own features is obtained by weighted summing the normalized features and the statistical features; Calculating a similarity score between tags based on the tag quality assessment model, wherein the similarity score between tags is obtained by threshold screening of the comprehensive similarity features between tags; Calculating a tag dependency score based on the tag quality assessment model, wherein the tag dependency score is obtained by weighted calculation of the comprehensive dependency features between the tags; The tag quality initial score is obtained by weighted summing the tag feature quality score, the tag similarity score and the tag dependency score.

[0047] This embodiment provides a method for constructing a label quality assessment model, which achieves a comprehensive assessment of label quality through feature fusion and multi-dimensional assessment.

[0048] First, the semantic similarity features and structural similarity features in the similarity feature matrix are weighted averaged to obtain the comprehensive similarity features between tags. Specifically, different weights can be assigned to the semantic similarity features and the structural similarity features, for example, the semantic similarity weight is 0.6 and the structural similarity weight is 0.4. In this way, the semantic and structural similarities between tags can be comprehensively considered.

[0049] At the same time, the hierarchical dependency features and the associated dependency features in the dependency feature matrix are weighted averaged to obtain the comprehensive dependency features between tags. Different weights can be assigned to the hierarchical dependency features and the associated dependency features, for example, the hierarchical dependency weight is 0.7 and the associated dependency weight is 0.3. In this way, the dependency relationship between tags can be fully considered.

[0050] Next, the language normative features, professional terminology normative features, and semantic integrity features in the original label comprehensive feature matrix are normalized to obtain normalized features. Normalization can use the minimum-maximum normalization method to map the feature values ​​to the range of 0 to 1. For the word frequency distribution features and query coverage features, logarithmic transformation is performed to obtain statistical features. Logarithmic transformation can effectively compress the data range and make the feature distribution more uniform.

[0051] The above-obtained inter-label comprehensive similarity features, inter-label comprehensive dependency features, normalized features, and statistical features are combined in series to construct a feature fusion matrix. This feature fusion matrix contains all the feature information required for label quality assessment. Based on this feature fusion matrix, a label quality assessment model is constructed.

[0052] In the tag quality assessment model, the quality score of the tag's own features is calculated first. This score is obtained by weighted summation of normalized features and statistical features. Different weights can be assigned to different features, for example, the weight of language normative features is 0.3, the weight of professional terminology normative features is 0.2, the weight of semantic integrity features is 0.2, the weight of word frequency distribution features is 0.15, and the weight of query coverage features is 0.15.

[0053] Then, the score of the similarity relationship between tags is calculated. This score is obtained by threshold screening the comprehensive similarity features between tags. A similarity threshold can be set, for example, 0.8. When the comprehensive similarity between two tags is greater than or equal to this threshold, they are considered to have a significant similarity relationship and are given a higher score; otherwise, a lower score is given.

[0054] Next, the tag dependency score is calculated. This score is obtained by weighting the comprehensive dependency features between tags. Different weights can be set according to the strength of the dependency, for example, a strong dependency weight is 0.8, a medium dependency weight is 0.5, and a weak dependency weight is 0.2.

[0055] Finally, the weighted sum of the label's own feature quality score, the similarity score between labels, and the label dependency score is used to obtain the label quality initial score. Different weights can be assigned to these three scores, for example, the weight of the own feature quality score is 0.5, the weight of the similarity score is 0.3, and the weight of the dependency score is 0.2.

[0056] To better understand this process, we can use a specific data case to illustrate. Suppose there is a label "artificial intelligence" and its feature values ​​are as follows: Language standardization feature: 0.95; professional terminology standardization feature: 0.88; semantic completeness feature: 0.92; word frequency distribution feature: 1000 (6.91 after logarithmic transformation); query coverage feature: 0.85; comprehensive similarity feature with other tags: 0.82 (with "machine learning"), 0.75 (with "deep learning"); comprehensive dependency feature with other tags: 0.9 (with "algorithm"), 0.85 (with "big data").

[0057] First, the normalized features and statistical features are weighted and summed to obtain the quality score of the label itself: 0.95*0.3+0.88*0.2+0.92*0.2+6.91*0.15+0.85*0.15=1.8965; Then, a threshold screening is performed on the similarity relationships between labels. Assuming the threshold is 0.8, the similarity score between "artificial intelligence" and "machine learning" is 1, and the similarity score between "artificial intelligence" and "deep learning" is 0.

[0058] Next, calculate the label dependency score, assuming that "artificial intelligence" has a strong dependency with "algorithm" and a medium dependency with "big data": 0.9*0.8+0.85*0.5=1.145; Finally, the three scores are weighted and summed to get the initial score of label quality: 1.8965*0.5+1*0.3+1.145*0.2=1.4233; In this way, a label quality assessment result that comprehensively considers multiple factors can be obtained.

[0059] The solution of this application can: This technical solution achieves a comprehensive evaluation of label quality through feature fusion and multi-dimensional evaluation, and improves the accuracy and reliability of the evaluation. By weighted averaging similarity features and dependency features, and normalizing and logarithmically transforming the original features, different types of features can be effectively fused together, making full use of various feature information. The final label quality score is calculated by weighted summation, making the evaluation result more flexible and adjustable. The weights of different features can be adjusted according to actual needs to adapt to different application scenarios.

[0060] In an optional implementation, the optimization process is started based on the collaborative optimization rule set, the instruction bee analyzes the characteristic pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan, including: Constructing a node optimization association network and establishing a collaborative optimization rule set, wherein the collaborative optimization rule set includes feature extraction rules, optimization instruction generation rules, and quality optimization rules, and starting a label optimization process based on the collaborative optimization rule set; The instruction bee analyzes the first quality label according to the feature extraction rule, obtains semantic features through word vector representation, obtains structural features through hierarchical relationships, obtains usage features through application frequency, and weightedly combines the semantic features, the structural features and the usage features to generate label feature data; The instruction bee processes the label feature data based on the optimization instruction generation rule, inputs the label feature data into the feature vector space for cluster analysis, extracts feature patterns to establish a feature pattern library, and generates an optimization instruction set according to the feature pattern library; The follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability; The execution bee analyzes the semantic content and word vector features of the second quality label based on the quality optimization rules and the task allocation plan, selects words with similar semantic content from a preset synonym library to supplement them, and expands the dimensional information of the word vector features to generate an optimized label.

[0061] The label quality optimization system based on the node optimization association network first builds the optimization association network. By establishing the connection relationship between nodes, a network topology is formed. Each node has independent optimization capability indicators, including computing resources, storage capacity and other parameters. At the same time, a collaborative optimization rule set is formulated, which includes three parts: feature extraction rules, optimization instruction generation rules and quality optimization rules.

[0062] In the feature extraction stage, CommandBee performs a multi-dimensional analysis on the first quality label. The semantic features of the label are obtained through word vector models such as Word2Vec, and each word is mapped to a 300-dimensional vector space. The hierarchical position of the label in the classification system is analyzed, and the upper and lower relationships are extracted as structural features. The usage features such as the frequency and coverage of the label in historical data are counted. The three types of features are combined in a weighted manner to generate a label feature vector.

[0063] In the optimization instruction generation phase, the instruction bee inputs the label feature vector into the feature space. The K-means clustering algorithm is used to perform cluster analysis on the feature vector and extract feature patterns. Based on the clustering results, a feature pattern library is established, which contains typical feature patterns of various labels. A targeted optimization instruction set is generated based on the pattern library.

[0064] After receiving the optimization instruction set, the follower bee performs path planning through the node optimization association network. It calculates the optimized distance between nodes, taking into account factors such as network delay and bandwidth. It uses the ant colony algorithm to select the optimal path, weighing the path length and optimization effect. It generates a task allocation plan based on the optimization capabilities of the nodes.

[0065] The execution bee optimizes the second quality label according to the allocation plan. It analyzes the semantic content of the label and selects similar words from the synonym library to supplement it. At the same time, it expands the dimension of the word vector feature and improves the semantic expression ability. Finally, it generates an optimized high-quality label.

[0066] After analysis, the original label "mobile phone" obtains a semantic feature vector [0.82, 0.56, ...], with a structural feature of {"superordinate": "electronic products", "subordinate": "smart phone"}, and a usage frequency of 2850 times. After optimization, it is expanded to "mobile phone, smart phone, communication equipment, mobile terminal", and the word vector dimension is expanded from 300 to 500.

[0067] The solution of this application can: Through multi-dimensional feature analysis and collaborative optimization, the semantic expression ability of tags is improved, so that tags can more accurately reflect content characteristics. The distributed node collaborative optimization method is adopted to improve the processing efficiency of tag optimization, which is suitable for large-scale data processing scenarios. The optimization scheme based on feature patterns and heuristic algorithms has strong adaptability and can perform targeted optimization according to the characteristics of different types of tags.

[0068] In an optional implementation, the follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability, including: receiving the optimization instruction set, extracting a priority parameter in the optimization instruction set, and determining a semantic distance weight, a structural distance weight, and a functional distance weight based on the priority parameter; Calculate the cosine similarity of the node label word vector to obtain the semantic distance, calculate the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculate the co-occurrence frequency and complementarity between nodes to obtain the functional distance; The semantic distance, the structural distance and the functional distance are respectively multiplied by corresponding weights and added to generate a comprehensive optimized distance matrix of the node optimized association network; Obtaining the task queue length and resource occupancy of the node to obtain the node load value, analyzing the time resources of the node to obtain the time constraint value, and calculating the optimization cost based on the comprehensive optimization distance matrix, the node load value and the time constraint value; Setting an initial pheromone distribution according to the priority information of the optimization instruction set, generating a state transition probability in combination with the optimization cost and the node optimization capability, and performing a path search based on the state transition probability; During the path search process, local pheromones are updated in real time, and the historical optimal path information is recorded. When the number of search iterations reaches the preset threshold, the optimal optimization path is selected; Decomposing the optimization task based on the optimal optimization path and the node processing capacity, generating a subtask set and determining the dependency between the subtasks, and calculating the resource requirement of each subtask; A task scheduling strategy is formulated according to the subtask set, the dependency relationship and the resource demand, and a task allocation plan including execution order and parallelism is generated.

[0069] After receiving the optimization instruction set, the Follow Bee system first parses the priority parameter information contained in the instruction set. The priority parameter usually contains weight values ​​of three dimensions: the semantic distance weight ranges from 0.2 to 0.4, the structural distance weight ranges from 0.3 to 0.5, and the functional distance weight ranges from 0.2 to 0.4. For example, when the system emphasizes semantic similarity, the semantic distance weight can be set to 0.4, the structural distance weight to 0.3, and the functional distance weight to 0.3.

[0070] After determining the weights, the system starts to calculate the multi-dimensional distances between nodes. For semantic distance, the system uses a pre-trained word vector model to convert node labels into vector representations, and calculates the cosine similarity between vectors to obtain the semantic distance value. Taking nodes A and B as an example, if the similarity of their label word vectors is 0.85, the semantic distance is 0.15.

[0071] The calculation of structural distance is based on the label hierarchical tree. Assuming that node A is on the third level, node B is on the fifth level, and their nearest common ancestor node is on the second level, the level span is 3 and the common ancestor depth is 2. By combining these two factors, the structural distance value of 0.4 can be obtained.

[0072] Functional distance is obtained by analyzing historical task data. The system counts the task collaboration frequency and functional complementarity between nodes. For example, the historical collaboration times of nodes A and B are 50 times, and the functional complementarity is 0.7, so the functional distance value is calculated to be 0.3.

[0073] The distance values ​​of the three dimensions are multiplied by the corresponding weights and summed to obtain the comprehensive optimization distance. Taking nodes A and B as an example, their comprehensive optimization distance is 0.28. The system performs this calculation for all node pairs to generate a complete optimization distance matrix.

[0074] Next, the system collects node status information to calculate the optimization cost. For each node, obtain its current task queue length and resource utilization. For example, the task queue length of node A is 10, the CPU utilization is 75%, and the memory utilization is 60%, so the node load value is 0.8. At the same time, the time resource constraints of the node are considered. For example, the available time of node A in the next 2 hours is 1.5 hours, and the time constraint value is 0.75.

[0075] Based on the optimized distance matrix, node load value and time constraint value, the system calculates the complete optimized cost network. On this basis, the improved ant colony algorithm is used for path search. Initially, the pheromone distribution is set according to the priority information, and the higher priority path obtains more initial pheromones.

[0076] During the iterative search process, the system updates the local pheromone concentration in real time. Each time a path search is completed, the pheromone value is adjusted according to the optimization cost of the path. At the same time, a global optimal path record is maintained. When the number of iterations reaches 1000, the optimal path found during the period is selected as the final optimized path.

[0077] After obtaining the optimal path, the system decomposes the optimization task according to the processing capacity of the node. For example, a data processing task can be decomposed into subtasks such as data cleaning, feature extraction, and model training. The system analyzes the dependencies between subtasks, builds a task dependency graph, and estimates the resource requirements of each subtask.

[0078] Finally, the system generates a detailed task allocation plan based on the subtask set, dependencies, and resource requirements. The plan contains specific information such as the execution order of tasks, parallelism settings, and resource allocation.

[0079] The solution of this application can: Through multi-dimensional distance calculation and comprehensive weight allocation mechanism, the degree of association between nodes can be comprehensively evaluated, which improves the accuracy and rationality of task allocation. The improved ant colony algorithm is used for path optimization, combined with real-time pheromone updates and historical optimal path records, which significantly improves the optimization efficiency and reduces system resource consumption. Based on fine-grained task decomposition and dependency analysis, the optimal scheduling and resource allocation of tasks are achieved, which improves the overall execution efficiency and resource utilization of the system.

[0080] In an optional implementation, calculating the cosine similarity of the node label word vector to obtain the semantic distance, calculating the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculating the co-occurrence frequency and complementarity between nodes to obtain the functional distance includes: Obtain node label text, input the node label text into a pre-trained word vector model, and generate a word vector corresponding to the node label text; calculate the cosine similarity of the word vectors between the node labels to obtain the semantic distance between the node labels; The semantic distance calculation formula is as follows: ; in, is the semantic similarity, t i , t j For two label nodes, is the corresponding word vector; Obtain a pre-constructed label hierarchy tree, locate the node position in the label hierarchy tree based on the semantic distance; calculate the hierarchy span between nodes, the hierarchy span is determined by the path length between nodes; identify the common ancestor node between nodes and calculate the depth of the common ancestor node; perform a weighted combination of the hierarchy span and the depth to obtain the structural distance between node labels; The formula for calculating the structural distance is as follows: ; Among them, α is the weight coefficient of the structural distance, path(t i ,t j ) is the hierarchical span between nodes, maxPath is the maximum path length in the tree, depth(LCA) is the depth of the nearest common ancestor node, and maxDepth is the maximum depth of the tree; Determine the associated node pairs based on the semantic distance and the structural distance, and count the co-occurrence frequencies of the associated node pairs; extract usage scenario information of the associated node pairs, and calculate the degree of complementarity of the node labels in the usage scenarios; perform weighted combination of the co-occurrence frequency and the degree of complementarity to obtain the functional distance between the node labels; The functional distance calculation formula is as follows: ; Among them, t i is the i-th label node to be calculated, t j is the jth label node to be calculated, β is the weight coefficient of the functional distance, coot(t i ,t j ) is the number of co-occurrences of the tag pair, is the total number of occurrences of all label pairs, t k is the kth tag combination in the tag set, For label t i The distribution vector in all scenes, For label t j The distribution vector in all scenes, is the scene vector of the label, is the Jensen-Shannon divergence of the scene vector.

[0081] In the process of calculating the similarity of label nodes, the text content of the node label is first obtained. The label text is vectorized through a pre-trained word vector model. A mainstream word vector model such as Word2Vec or BERT can be used. Taking the two labels "data mining" and "machine learning" as examples, after inputting the two label texts into the pre-trained model, the corresponding high-dimensional vector representation can be obtained. By calculating the cosine similarity between the two vectors, the semantic distance between the labels can be obtained. The smaller the semantic distance, the closer the two labels are semantically.

[0082] After obtaining the semantic distance, it is necessary to locate the node position in the pre-built label hierarchy tree. Taking the technology field label tree as an example, "artificial intelligence" is the parent node, with child nodes such as "machine learning" and "deep learning". Calculate the hierarchical span between any two nodes, that is, the length of the shortest path between the nodes. At the same time, identify the nearest common ancestor node of the two nodes and calculate the depth value of the ancestor node. Combining the hierarchical span and depth information, we can get the structural distance that reflects the structural relationship between nodes.

[0083] After determining the semantic distance and structural distance, we can identify pairs of nodes with high correlation. We count the co-occurrence frequencies of these pairs of nodes in actual application scenarios. Taking technical document annotation as an example, we analyze the co-occurrence times of the tags "deep learning" and "neural network" in the document. At the same time, we extract the usage of these tags in different application scenarios and construct scenario distribution vectors. By calculating the difference between scenario distribution vectors, we can evaluate the complementarity between tags. Finally, we weight the co-occurrence frequency information and the complementarity degree information to obtain the functional distance that reflects the strength of the association between nodes.

[0084] In practical applications, the weights of semantic distance, structural distance, and functional distance can be adjusted according to specific needs. For example, in the scenario of technical document classification, the weight of semantic distance can be appropriately increased; in the scenario of knowledge graph construction, the weight of structural distance can be increased; in the scenario of recommendation system, the influence of functional distance can be strengthened.

[0085] The solution of this application can: The semantic knowledge contained in the pre-trained model is fully utilized through the calculation of semantic distance, which can accurately capture the semantic correlation between labels and effectively improve the accuracy of label association analysis. Combined with the structural information of the label hierarchy tree, it not only considers the hierarchical span between nodes, but also pays attention to the depth of the common ancestor node, fully reflects the hierarchical organizational relationship between labels, and improves the rationality of similarity calculation. The integration of co-occurrence frequency and scene complementarity analysis not only considers the statistical laws of label use, but also pays attention to the complementary role of labels in different scenarios, characterizes the association characteristics between labels from multiple dimensions, and enhances the comprehensiveness and reliability of label relationship analysis.

[0086] Figure 2 The overall architecture diagram of the data tag platform of the embodiment of the present invention is as follows: Figure 2 As shown, the method also includes: The platform supports multi-source heterogeneous data access, covering structured data (such as SQL databases, data warehouses), semi-structured data (such as JSON, XML files) and unstructured data (such as logs, text files). Through the data source management module, users can easily configure data source connection parameters to ensure real-time access and efficient processing of multi-source data. The addition, deletion and maintenance of data sources are all completed through the platform's built-in visual interface, and the system will strictly verify the connectivity and access rights of the data source.

[0087] The feature extraction module uses advanced feature engineering technology to automatically extract high-value features from access data. The system supports rule-based feature selection (such as correlation analysis based on statistical indicators, information gain, etc.), and provides automated clustering, classification and other model applications to generate preliminary user tags. This process not only reduces reliance on manual operations, but also improves the accuracy and timeliness of feature extraction. Users can further adjust feature selection rules through custom SQL scripts or drag-and-drop operation interfaces.

[0088] The platform's integrated automated configuration engine is based on predefined business rules and can automatically complete feature data matching, label generation and configuration. The configuration process includes entering basic label information, defining label values, associating data source tables, setting filtering conditions and update methods. The system will dynamically adjust label configurations based on preset rules to ensure that corresponding update strategies are automatically generated when data changes. Users can review and fine-tune configuration results on the platform interface.

[0089] The feedback optimization module adopts a closed-loop management mechanism, which automatically adjusts key parameters in the label system through user behavior analysis, historical data monitoring, and real-time data stream input. The module integrates incremental learning models and adaptive algorithms, and can quickly adjust the weights, screening conditions, and data update frequency of labels in response to business changes, ensuring that the label system can always maintain the best state. Through this mechanism, the platform achieves continuous optimization and adaptive adjustment of labels, greatly improving the accuracy and dynamic response capabilities of labels.

[0090] The platform adopts a containerized deployment solution based on a microservice architecture. The core technology stack includes the SpringBoot framework, Docker containers, and Kubernetes orchestration management tools. The platform supports horizontal expansion and elastic scaling, and can automatically expand processing capabilities according to business needs. The service modules in the system use Zookeeper for service discovery and registration, and the data storage uses a combination of MySQL and Elasticsearch to ensure fast reading and writing capabilities in a high-concurrency environment. The system also supports distributed log management, monitoring alarms, and automated operation and maintenance to ensure high availability and stability.

[0091] The present invention proposes an intelligent tag management and optimization platform for big data environments, aiming to solve the problems of poor flexibility and insufficient automation capabilities of traditional data tag management systems. The platform integrates functions such as multi-source data access, feature extraction, automatic tag generation and configuration, and dynamic feedback optimization. Through a microservice architecture and distributed computing framework, it provides efficient data processing and intelligent tag management. Its innovations include an automated configuration engine, a closed-loop feedback optimization mechanism, multi-level authority management, and scalability, and is suitable for enterprise-level users in scenarios such as user profiling and precision marketing. Big data management needs.

[0092] Figure 3 FIG. 1 is a schematic diagram of the structure of an intelligent bee colony quality optimization management system for multi-source tag data according to an embodiment of the present invention. Figure 3 As shown, the system comprises: The first unit is used to obtain multi-source label data input, extract the comprehensive feature vector of each label respectively through the feature extraction module, the comprehensive feature vector includes a semantic feature vector, an associated feature vector and a time feature vector, and input the comprehensive feature vector into the feature fusion layer to generate a label comprehensive feature matrix; process the label comprehensive feature matrix through a deep feature enhancement network, calculate the similarity feature matrix and the dependency feature matrix between the labels; construct a label quality evaluation model based on each feature matrix, calculate the initial quality score of each label, and generate a label quality grading sequence; The second unit is used to map the label data into bee colony network nodes according to the label quality classification sequence, and divide the nodes into three levels based on the initial quality score: set the nodes with quality scores higher than the preset threshold as instruction bees, responsible for formulating optimization strategies; set the nodes with quality scores equal to the preset threshold as follower bees, responsible for transmitting optimization instructions; set the nodes with quality scores lower than the preset threshold as execution bees, responsible for implementing optimization operations; construct a node optimization association network using the similarity feature matrix and the dependency feature matrix, and design a collaborative optimization rule set based on the node optimization association network; The third unit is used to start the optimization process based on the collaborative optimization rule set, the instruction bee analyzes the feature pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan; after the optimization is completed, the optimization result is re-input into the feature extraction module for quality evaluation and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is terminated and the final label data set is output.

[0093] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0094] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0095] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent swarm-type quality optimization management method for multi-source label data, characterized in that: include: Obtain multi-source label data input, extract comprehensive feature vectors of each label respectively through feature extraction module, the comprehensive feature vectors include semantic feature vectors, association feature vectors and time feature vectors, input the comprehensive feature vectors into feature fusion layer to generate label comprehensive feature matrix; process the label comprehensive feature matrix through deep feature enhancement network, calculate similarity feature matrix and dependency feature matrix between labels; A label quality assessment model is constructed based on each feature matrix, the initial quality score of each label is calculated, and a label quality grading sequence is generated; According to the label quality classification sequence, the label data is mapped into bee colony network nodes, and the nodes are divided into three levels based on the initial quality score: the nodes with quality scores higher than the preset threshold are set as instruction bees, which are responsible for formulating optimization strategies; the nodes with quality scores equal to the preset threshold are set as follower bees, which are responsible for transmitting optimization instructions; the nodes with quality scores lower than the preset threshold are set as execution bees, which are responsible for implementing optimization operations; the node optimization association network is constructed using the similarity feature matrix and the dependency feature matrix, and a collaborative optimization rule set is designed based on the node optimization association network; The optimization process is started based on the collaborative optimization rule set, the instruction bees analyze the characteristic pattern of the first quality label to generate an optimization instruction set, the follower bees perform optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bees perform feature optimization on the second quality label according to the task allocation plan; After the optimization is completed, the optimization results are re-input into the feature extraction module for quality assessment and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is terminated and the final label data set is output.

2. The method according to claim 1, characterized in that Processing the comprehensive feature matrix of labels through a deep feature enhancement network to calculate a similarity feature matrix and a dependency feature matrix between labels; A label quality assessment model is constructed based on each feature matrix, the initial quality score of each label is calculated, and a label quality grading sequence is generated, including: Constructing a deep feature enhancement network, the deep feature enhancement network includes a feature mapping layer, a feature enhancement layer and a feature reconstruction layer, the feature mapping layer uses a nonlinear activation function to map the label comprehensive feature matrix to a high-dimensional latent space, the feature enhancement layer calculates the attention weights between feature vectors and performs weighted fusion based on the attention weights to achieve feature enhancement, and the feature reconstruction layer uses a deconvolution operation to remap the enhanced features to the original feature space; The cosine similarity method is used to calculate the semantic similarity between tags to obtain the similarity feature matrix. At the same time, the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain the dependency feature matrix. The similarity feature matrix and the dependency feature matrix are feature-fused with the original label comprehensive feature matrix to construct a label quality assessment model, wherein the label quality assessment model calculates the label feature quality, the similarity relationship between labels and the label dependency relationship respectively, and obtains an initial quality score by weighted summation; The initial quality score is normalized, a density clustering method is used to automatically determine a quality grade division threshold, and a label quality grading sequence containing quality grade information is generated.

3. The method according to claim 2, characterized in that The cosine similarity method is used to calculate the semantic similarity between tags to obtain the similarity feature matrix. At the same time, the temporal association analysis method is introduced to construct a directed weighted graph through the temporal pattern and co-occurrence frequency of the tags to obtain the dependency feature matrix, including: Obtain a set of label texts, input the set of label texts into a pre-trained word vector model, and obtain an initial word vector corresponding to each label text; for a label containing multiple words in the label text set, calculate the word frequency-inverse document frequency value of each word in the label as a weight coefficient, and perform weighted averaging on the initial word vectors of each word according to the weight coefficient to obtain an overall semantic feature vector of the label; Based on the overall semantic feature vector, the cosine similarity between any two tags in the tag text set is calculated, and the cosine similarity is filled into the row and column positions corresponding to the similarity feature matrix to obtain the semantic similarity feature matrix of the tag text set, wherein the dimension of the semantic similarity feature matrix is ​​the product of the total number of tags and the total number of tags; Acquire the time series data of the tag text set, set a time window, analyze the co-occurrence pattern of the tag text set within the time window, count the co-occurrence frequency of tag pairs, record the relative appearance order of tags, and calculate the time interval distribution of tag pairs; based on the co-occurrence frequency, the relative appearance order and the time interval distribution, use a preset weight coefficient for weighted combination to obtain the time series dependency strength of the tag pairs; A directed weighted graph is constructed according to the temporal dependency strength, the labels in the label text set are mapped to nodes of the directed weighted graph, the connection direction between the nodes is determined based on the relative appearance order of the label pairs, and the temporal dependency strength is used as the weight value of the corresponding connection to obtain a directed temporal weighted graph that describes the temporal association of the labels; a dependency feature matrix is ​​constructed based on the connection relationship and weight value of the directed temporal weighted graph.

4. The method according to claim 2, characterized in that: The similarity feature matrix and the dependency feature matrix are fused with the original label comprehensive feature matrix to construct a label quality assessment model. The label quality assessment model calculates the label feature quality, the similarity relationship between labels and the label dependency relationship respectively, and obtains the initial quality score by weighted summation, including: The semantic similarity features and the structural similarity features in the similarity feature matrix are weighted averaged to obtain the comprehensive similarity features between tags, and the hierarchical dependency features and the associated dependency features in the dependency feature matrix are weighted averaged to obtain the comprehensive dependency features between tags; The language standardization features, professional terminology standardization features and semantic integrity features in the original label comprehensive feature matrix are normalized to obtain the normalized features, and the word frequency distribution features and query coverage features are logarithmically transformed to obtain the statistical features; The comprehensive similarity feature between labels, the comprehensive dependency feature between labels, the normalized feature and the statistical feature are serially combined to construct a feature fusion matrix, and a label quality assessment model is constructed based on the feature fusion matrix; Calculating a quality score of the label's own features based on the label quality assessment model, wherein the quality score of the label's own features is obtained by weighted summing the normalized features and the statistical features; Calculating the similarity score between tags based on the tag quality assessment model, wherein the similarity score between tags is obtained by threshold screening of the comprehensive similarity features between tags; Calculating a tag dependency score based on the tag quality assessment model, wherein the tag dependency score is obtained by weighted calculation of the comprehensive dependency features between the tags; The tag quality initial score is obtained by weighted summing the tag feature quality score, the tag similarity score and the tag dependency score.

5. The method according to claim 1, characterized in that The optimization process is started based on the collaborative optimization rule set, the instruction bee analyzes the characteristic pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan, including: Constructing a node optimization association network and establishing a collaborative optimization rule set, wherein the collaborative optimization rule set includes feature extraction rules, optimization instruction generation rules, and quality optimization rules, and starting a label optimization process based on the collaborative optimization rule set; The instruction bee analyzes the first quality label according to the feature extraction rule, obtains semantic features through word vector representation, obtains structural features through hierarchical relationships, obtains usage features through application frequency, and weightedly combines the semantic features, the structural features and the usage features to generate label feature data; The instruction bee processes the label feature data based on the optimization instruction generation rule, inputs the label feature data into the feature vector space for cluster analysis, extracts feature patterns to establish a feature pattern library, and generates an optimization instruction set according to the feature pattern library; The follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability; The execution bee analyzes the semantic content and word vector features of the second quality label based on the quality optimization rules and the task allocation plan, selects words with similar semantic content from a preset synonym library to supplement them, and expands the dimensional information of the word vector features to generate an optimized label.

6. The method according to claim 5, characterized in that The follower bee receives the optimization instruction set, calculates the optimization distance and optimization cost between nodes through the node optimization association network, selects the optimal optimization path using a heuristic algorithm, and generates a task allocation plan based on the optimal optimization path and node optimization capability, including: receiving the optimization instruction set, extracting a priority parameter in the optimization instruction set, and determining a semantic distance weight, a structural distance weight, and a functional distance weight based on the priority parameter; Calculate the cosine similarity of the node label word vector to obtain the semantic distance, calculate the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculate the co-occurrence frequency and complementarity between nodes to obtain the functional distance; The semantic distance, the structural distance and the functional distance are respectively multiplied by corresponding weights and added to generate a comprehensive optimized distance matrix of the node optimized association network; Obtaining the task queue length and resource occupancy of the node to obtain the node load value, analyzing the time resources of the node to obtain the time constraint value, and calculating the optimization cost based on the comprehensive optimization distance matrix, the node load value and the time constraint value; Setting an initial pheromone distribution according to the priority information of the optimization instruction set, generating a state transition probability in combination with the optimization cost and the node optimization capability, and performing a path search based on the state transition probability; During the path search process, local pheromones are updated in real time, and the historical optimal path information is recorded. When the number of search iterations reaches the preset threshold, the optimal optimization path is selected; Decomposing the optimization task based on the optimal optimization path and the node processing capacity, generating a subtask set and determining the dependency between the subtasks, and calculating the resource requirement of each subtask; A task scheduling strategy is formulated according to the subtask set, the dependency relationship and the resource demand, and a task allocation plan including execution order and parallelism is generated.

7. The method according to claim 6, characterized in that Calculate the cosine similarity of the node label word vector to obtain the semantic distance, calculate the level span and common ancestor node depth between nodes in the label hierarchy tree to obtain the structural distance, and calculate the co-occurrence frequency and complementarity between nodes to obtain the functional distance, including: Obtain node label text, input the node label text into a pre-trained word vector model, and generate a word vector corresponding to the node label text; calculate the cosine similarity of the word vectors between the node labels to obtain the semantic distance between the node labels; The semantic distance calculation formula is as follows: ； in, is the semantic similarity, t i , t j For two label nodes, is the corresponding word vector; Obtain a pre-constructed label hierarchy tree, locate the node position in the label hierarchy tree based on the semantic distance; calculate the hierarchy span between nodes, the hierarchy span is determined by the path length between nodes; identify the common ancestor node between nodes and calculate the depth of the common ancestor node; perform a weighted combination of the hierarchy span and the depth to obtain the structural distance between node labels; The formula for calculating the structural distance is as follows: ； Among them, α is the weight coefficient of the structural distance, path(t i ,t j ) is the hierarchical span between nodes, maxPath is the maximum path length in the tree, depth(LCA) is the depth of the nearest common ancestor node, and maxDepth is the maximum depth of the tree; Determine the associated node pairs based on the semantic distance and the structural distance, and count the co-occurrence frequencies of the associated node pairs; extract usage scenario information of the associated node pairs, and calculate the degree of complementarity of the node labels in the usage scenarios; perform weighted combination of the co-occurrence frequency and the degree of complementarity to obtain the functional distance between the node labels; The functional distance calculation formula is as follows: ； Among them, t i is the i-th label node to be calculated, t j is the jth label node to be calculated, β is the weight coefficient of the functional distance, coot(t i ,t j ) is the number of co-occurrences of the tag pair, is the total number of occurrences of all label pairs, t k is the kth tag combination in the tag set, For label t i The distribution vector in all scenes, For label t j The distribution vector in all scenes, is the scene vector of the label, is the Jensen-Shannon divergence of the scene vector.

8. An intelligent bee colony quality optimization management system for multi-source label data, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain multi-source label data input, extract the comprehensive feature vector of each label respectively through the feature extraction module, the comprehensive feature vector includes a semantic feature vector, an associated feature vector and a time feature vector, input the comprehensive feature vector into the feature fusion layer to generate a label comprehensive feature matrix; process the label comprehensive feature matrix through a deep feature enhancement network, and calculate the similarity feature matrix and the dependency feature matrix between the labels; A label quality assessment model is constructed based on each feature matrix, the initial quality score of each label is calculated, and a label quality grading sequence is generated; The second unit is used to map the label data into bee colony network nodes according to the label quality classification sequence, and divide the nodes into three levels based on the initial quality score: set the nodes with quality scores higher than the preset threshold as instruction bees, responsible for formulating optimization strategies; set the nodes with quality scores equal to the preset threshold as follower bees, responsible for transmitting optimization instructions; set the nodes with quality scores lower than the preset threshold as execution bees, responsible for implementing optimization operations; construct a node optimization association network using the similarity feature matrix and the dependency feature matrix, and design a collaborative optimization rule set based on the node optimization association network; The third unit is used to start the optimization process based on the collaborative optimization rule set, the instruction bee analyzes the characteristic pattern of the first quality label to generate an optimization instruction set, the follower bee performs optimization path planning through the node optimization association network to generate a task allocation plan, and the execution bee performs feature optimization on the second quality label according to the task allocation plan; After the optimization is completed, the optimization results are re-input into the feature extraction module for quality assessment and the label quality grading sequence is updated; when the quality score improvement value after multiple consecutive rounds of optimization is less than the preset improvement threshold, the optimization process is terminated and the final label data set is output.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images