Image data annotation method and system

Abstract

The present invention discloses an image data annotation method and system, which relates to the field of computer vision technology. The method comprises collecting image and text data, updating through kernel density estimation and dynamic equations, optimizing using Euler discretization combined with GLP propagation and EA, converting using amplitude coding, constructing through quantum state circuits, calculating using partial traces, generating topological weighted entanglement entropy through weighted fusion, updating based on an attention mechanism, generating potential conflict pairs, calculating priorities, generating final conflict pairs through a CNN model, outputting regional features through a NeRF model, extracting them using a CLIP model, and updating through a loss function. The present invention improves the accuracy and consistency of annotation by optimizing through VAE combined with GLP propagation and EA, and improves the efficiency and accuracy of annotation through joint verification through quantum entanglement entropy calculation and NeBF model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image data annotation method and system. Background Art

[0002] With the rapid development of deep learning, multimodal modeling and big data processing technologies, image data annotation, as one of the core technologies in the field of computer vision, has made great progress. Early image data annotation mainly relied on manual annotation or rule-based automation methods, which were costly and prone to subjective bias. In order to improve efficiency, technologies such as VAE, GAN and NeRF were gradually used. Clustering algorithms and dimensionality reduction techniques were also widely used in data preprocessing and feature extraction.

[0003] Existing image data annotation methods still have shortcomings. They rely on static graph structures or fixed weight matrices, which make it difficult to adapt to dynamic changes in data distribution, resulting in accumulated annotation propagation errors. Existing singular value decomposition methods usually ignore local topological structures, resulting in loss of semantic information. Clustering algorithms based on classical machine learning have limited processing capabilities for high-dimensional sparse data and lack a quantitative evaluation mechanism for potential conflicting labels. Quantum computing and NeRF models have failed to effectively solve the collaborative optimization of label propagation, resulting in deviations in annotation results. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an image data annotation method and system to solve the problems of relying on static graph structure or fixed weight matrix, which are difficult to adapt to dynamic changes in data distribution, resulting in accumulated annotation propagation errors. Existing singular value decomposition methods usually ignore local topological structures, resulting in loss of semantic information. Clustering algorithms based on classical machine learning have limited processing capabilities for high-dimensional sparse data and lack a quantitative evaluation mechanism for potential conflicting labels. Quantum computing and NeRF models have failed to effectively solve the collaborative optimization of label propagation, resulting in the problem of deviation in annotation results.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides an image data annotation method, which includes collecting image and text data, preprocessing the collected two types of data, generating hash codes through VAE and constructing a dynamic edge weight matrix, combining singular value decomposition to generate a low-rank approximation matrix, updating through kernel density estimation and dynamic equations, using Euler discretization combined with GLP propagation and EA for optimization, based on the optimization results, using amplitude coding for conversion, and constructing through quantum state circuits, using partial traces for calculation, generating topological weighted entanglement entropy through weighted fusion, updating based on the attention mechanism, generating potential conflict pairs, and calculating priorities, generating final conflict pairs through the CNN model, outputting regional features through the NeRF model, extracting using the CLIP model, and updating through the loss function, generating a verification fusion label distribution matrix through the MLP model, generating a heat map through ECharts, and storing the two collected data.

[0008] As a preferred embodiment of the image data annotation method of the present invention, the image and text data are collected and optimized using Euler discretization combined with GLP propagation and EA, including:

[0009] Based on the multimodal feature vector and attention region features, the global similarity and local attention similarity are calculated respectively using the cosine formula;

[0010] Use VAE to map the multimodal feature vector into a hash code and calculate the hash similarity using the Gaussian kernel function;

[0011] Global similarity, local attention similarity, and hash similarity are weightedly fused to obtain comprehensive similarity. The comprehensive similarity is defined as an edge, and the image ID is a node to construct a dynamic edge weight matrix.

[0012] The number of categories K is set based on the elbow theorem, and K-Means clustering is used to cluster the multimodal feature vectors into K clusters, which are then mapped to string labels to generate initial category labels. The initial probability distribution is then generated through maximum likelihood estimation.

[0013] Map the hash code to the initial probability distribution through kernel density estimation, and update the initial probability distribution;

[0014] Based on each node, the dynamic equations are defined, discretized using the Euler method, the initial probability distribution is updated iteratively, and GLP propagation is performed using a low-rank approximation matrix;

[0015] Perform weighted fusion on the propagation initial probability distribution and the update initial probability distribution to generate a fusion label distribution matrix;

[0016] Based on the dynamic edge weight matrix, the fusion label distribution matrix is ​​updated through EA;

[0017] Based on the updated fusion distribution label matrix, the EA objective function is defined to optimize the two weight parameters in the dynamic equation;

[0018] Based on each node, the maximum value in the fusion distribution label matrix is ​​filtered and optimized to generate the final label of each node.

[0019] As a preferred solution of the image data annotation method of the present invention, wherein: generating the final conflict pair based on the optimization result through the CNN model includes:

[0020] The optimized fusion label distribution matrix is ​​converted into a quantum state vector through amplitude coding. The edge set is defined based on the dynamic edge weight matrix. The quantum state vector is entangled through the quantum state circuit. The topological feature vector of each node is generated through TDA.

[0021] Based on the entangled state, the reduced density matrix is ​​calculated through partial traces;

[0022] The topological feature vector is normalized using the L2 norm and mapped using the Sigmoid function to generate the weight of the topological feature vector;

[0023] The entanglement entropy of the reduced density matrix is ​​calculated by Qiskit, and the weight of the topological eigenvector is combined for weighted fusion to generate the topological weighted entanglement entropy, which is then adjusted to generate the adjusted topological weighted entanglement entropy.

[0024] Based on the multimodal feature vector, the modal weight is obtained through the attention mechanism, and the adjusted topological weighted entanglement entropy is multiplied to generate the latest topological weighted entanglement entropy;

[0025] If the final label of each node is not equal to the neighbor label and the edge weight between the two is greater than the edge weight threshold, then this node and the neighbor node are marked as potential conflict pairs and the priority is calculated;

[0026] Sort the priorities in descending order, filter the first E potential conflict pairs, where E is the number of potential conflict pairs, and generate a potential conflict pair set;

[0027] Construct a GNN model, input the topological feature vector and edge weight matrix of the potential conflict pair set into the GNN model, output the visual weight and text weight, and calculate the cross-modal similarity of the potential conflict pair set through the CLIP model;

[0028] Calculate the difference between cross-modal similarity and comprehensive similarity, filter out potential conflicting pairs that are greater than the difference threshold, and generate the final conflicting pairs.

[0029] As a preferred solution of the image data annotation method of the present invention, the output of regional features by the NeRF model, the extraction using the CLIP model, and the updating using the loss function include:

[0030] Based on the final conflict pair, the NeRF model is initialized to generate a high-dimensional feature map, and CBAM is used to generate spatial and channel attention weights to output the initial region features;

[0031] Use the CLIP model to extract the regional semantic embedding of the final conflict pair, obtain the semantic feature vector, and use a linear layer to map it to the CLIP dimension to define the loss function of the model parameters;

[0032] Update regional features through Adam optimizer output;

[0033] Perform weighted fusion on the regional semantic embedding and mapping results to generate fusion features;

[0034] Extract the feature vector of each class of initial category labels, obtain the category average feature by summing and averaging, and calculate the weighted Euclidean distance between the fusion feature and the category average feature;

[0035] The weighted Euclidean distances are sorted in descending order, the fusion feature with the smallest weighted Euclidean distance is selected, the attention region feature is updated, and the updated attention region feature is generated.

[0036] As a preferred solution of the image data annotation method of the present invention, wherein: the generation and verification of the fusion label distribution matrix by the MLP model and the generation of the heat map by ECharts include:

[0037] Build an MLP model, input the updated attention region features and category mean features into the MLP model, map them to category probabilities, calculate the updated label distribution through the Softmax function, update the fusion label distribution matrix of the final conflict pair, and generate the verification fusion label distribution matrix;

[0038] The verification fusion label distribution matrix is ​​converted into a metadata table and a heat map is generated using ECharts.

[0039] As a preferred solution of the image data annotation method of the present invention, the preprocessing of the two collected data includes:

[0040] Preprocess the text data, including word segmentation using the BPE algorithm, extracting semantic features from the segmentation results using the BERT model, reducing the dimensionality of the semantic features using PCA, and normalizing them using the L2 norm to generate text features;

[0041] Use the VGG-16 model to extract the convolution features of the image data, extract the fully connected layer features through the FC7 layer, generate the FC7 vector, and encode it through the BOW model to generate the BOW vector;

[0042] Use the Conv5-3 layer of VGG-16 to extract the features of image data, generate feature maps, and use the L2 norm for normalization to generate multi-scale spatial features;

[0043] The text features, normalized FC7 vectors, normalized BOW vectors, and multi-scale spatial features are concatenated, dimensionally reduced through a fully connected layer, and normalized using the L2 norm to generate a multimodal feature vector.

[0044] Use the Selective Search algorithm to segment the feature map, generate candidate regions, and extract the features of each candidate region through the ResNet-50 model;

[0045] The discriminative scores of all candidate regions are sorted in descending order, the region with the highest discriminative score is selected, the features of the corresponding candidate region are extracted, and the multimodal feature vectors are combined for splicing to generate the attention region features.

[0046] As a preferred solution of the image data annotation method of the present invention, the storing of the two types of collected data includes:

[0047] The two types of data collected and the heat map generated by the analysis are transferred to the central database via the SFTP protocol for storage and regular maintenance and update.

[0048] In a second aspect, the present invention provides an image data annotation system, comprising:

[0049] The processing and optimization module is used to collect image and text data, preprocess the collected data, generate hash codes and construct a dynamic edge weight matrix through VAE, combine singular value decomposition to generate a low-rank approximation matrix, update it through kernel density estimation and dynamic equations, and use Euler discretization combined with GLP propagation and EA for optimization;

[0050] The calculation update module is used to convert the optimization results using amplitude coding, construct them through quantum state circuits, calculate using partial traces, generate topological weighted entanglement entropy through weighted fusion, update based on the attention mechanism, generate potential conflict pairs, calculate priorities, generate final conflict pairs through the CNN model, and output regional features through the NeRF model, extract them using the CLIP model, and update them through the loss function;

[0051] Verification upload module, used to generate verification fusion label distribution matrix through MLP model and generate heat map through ECharts;

[0052] The encrypted storage module is used to store the two types of collected data.

[0053] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the image data annotation method described in the first aspect of the present invention is implemented.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the image data annotation method described in the first aspect of the present invention is implemented.

[0055] The beneficial effects of the present invention are as follows: the present invention collects image and text data, pre-processes the two collected data, generates hash codes through VAE and constructs a dynamic edge weight matrix, generates a low-rank approximation matrix in combination with singular value decomposition, updates through kernel density estimation and dynamic equations, uses Euler discretization combined with GLP propagation and EA for optimization, converts based on the optimization results, constructs through quantum state circuits, uses partial traces for calculation, generates topological weighted entanglement entropy through weighted fusion, updates based on the attention mechanism, generates potential conflict pairs, and calculates priorities, generates final conflict pairs through the CNN model, outputs regional features through the NeRF model, uses the CLIP model for extraction, and updates through the loss function; improves the accuracy and consistency of annotation, and improves the efficiency and accuracy of annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a flow chart of the image data annotation method in Example 1;

[0058] Figure 2 Schematic diagram of the image data annotation system in Example 1;

[0059] Figure 3 Schematic diagram of conflict pair identification in Example 1;

[0060] Figure 4Schematic diagram of the generation of topological weighted entanglement entropy in Example 1. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0064] Example 1, reference Figures 1 to 4 , which is the first embodiment of the present invention, provides an image data annotation method, comprising the following steps:

[0065] S1. Collect image and text data, preprocess the collected data, generate hash codes and construct a dynamic edge weight matrix through VAE, combine singular value decomposition to generate a low-rank approximation matrix, update it through kernel density estimation and dynamic equations, and use Euler discretization combined with GLP propagation and EA for optimization;

[0066] Specifically, the two types of collected data are preprocessed, including:

[0067] Use cameras and social software to collect image data and text data;

[0068] Preprocess the text data, including word segmentation using the BPE algorithm, extracting semantic features from the segmentation results using the BERT model, reducing the dimensionality of the semantic features using PCA, and normalizing them using the L2 norm to generate text features;

[0069] Use the VGG-16 model to extract the convolution features of image data and perform global average pooling;

[0070] The pooled results are used to extract the fully connected layer features using the FC7 layer of the VGG-16 model, and the dimensionality is reduced by PCA to generate FC7 vectors, which are then encoded by the BOW model to generate BOW vectors.

[0071] Use L2 norm to normalize BOW vector and FC7 vector respectively;

[0072] Use the Conv5-3 layer of VGG-16 to extract the features of the image data, generate a feature map, perform pooling through SPP, perform dimensionality reduction through a fully connected layer, filter through the ReLU activation function, and use the L2 norm for normalization to generate multi-scale spatial features;

[0073] The text features, normalized FC7 vectors, normalized BOW vectors, and multi-scale spatial features are concatenated, dimensionally reduced through a fully connected layer, and normalized using the L2 norm to generate a multimodal feature vector.

[0074] Use the Selective Search algorithm to segment the feature map, generate candidate regions, and extract the features of each candidate region through the ResNet-50 model;

[0075] The variance of each candidate region feature and the category average feature (obtained based on the initial category label) is calculated through intra-class variance, and normalized through Min-Max normalization to generate a discriminant score;

[0076] The discriminative scores of all candidate regions are sorted in descending order, the region with the highest discriminative score is selected, the features of the corresponding candidate region are extracted, and the multimodal feature vectors are combined for splicing to generate the attention region features.

[0077] By collecting multi-source data, the diversity and representativeness of data are improved, the bias problems that may be caused by a single data source are avoided, the system can adapt to the needs of dynamic scenes, and provide data support for real-time labeling tasks, thereby enhancing the versatility and robustness of the system. The combination of BPE and BERT achieves deep mining of text semantics and retains complex semantic associations. CA dimensionality reduction and L2 normalization effectively reduce redundant information and noise interference, improve the computational efficiency and stability of features, and lay a high-quality text feature foundation for subsequent multimodal feature fusion. Through VGG-16 convolutional feature extraction, the complex texture and semantic structure of the image are accurately represented. The combination of FC7 vectors and BOW vectors achieves complementary expression of global semantics and local pattern features, enhancing the semantic richness of the features. Through SPP pooling, adaptive feature extraction is achieved for images of different sizes and resolutions, solving the problem of traditional fixed-size pooling being sensitive to image scale changes. The Selective Search algorithm combined with the ResNet-50 model achieves precise positioning and deep feature extraction of key image areas, improving the semantic relevance and feature quality of the regions.

[0078] Furthermore, we collected image and text data and used Euler discretization combined with GLP propagation and EA for optimization, including:

[0079] Based on the multimodal feature vector and attention region features, the global similarity and local attention similarity are calculated respectively using the cosine formula;

[0080] The formula for constructing the dynamic edge weight matrix is:

[0081] ,

[0082] in, For nodes and nodes In time The edge weights of is an exponential function, is the scalar parameter of the time decay factor (obtained based on experimental optimization), For nodes and nodes The dynamic time constant (based on dynamic adjustment of simulated annealing), is the comprehensive similarity;

[0083] The calculation obtains the number of neighbors, and the formula is:

[0084] ,

[0085] in, is the proportional coefficient (set based on the statistical method of data distribution), For nodes The number of neighbors, For nodes The local density of

[0086] Based on the sparse adjacency matrix, the degree matrix and Laplace matrix of each node are calculated as follows:

[0087] ,

[0088]

[0089] in, is the element of the degree matrix, representing the node In time The degree, For time The Laplace matrix of For time The sparse adjacency matrix of ;

[0090] Apply singular value decomposition to the Laplacian matrix to generate a low-rank approximation matrix;

[0091] Based on the elbow theorem, the number of categories K is set, and K-Means clustering is used to cluster the multimodal feature vectors into K clusters. The clusters are then mapped to string labels (e.g., K categories such as person, cat, etc.) to generate initial category labels.

[0092] Based on the initial category labels, the probability distribution is obtained by maximum likelihood estimation and normalized by softmax normalization to generate the initial probability distribution;

[0093] Based on the multimodal feature vector, attention region features (mapped by the fully connected layer) and timestamps (sine encoded), the LTC time constant is calculated by Laplace eigenvalue decomposition;

[0094] The hash code is mapped to the initial probability distribution through kernel density estimation, and the initial probability distribution is updated. The formula is:

[0095] ,

[0096] in, is the hash prior weight (based on cross-validation settings), For nodes The updated initial probability distribution of For nodes The initial probability distribution of the hash code, is the number of category labels;

[0097] Based on each node, the dynamic equation is defined as follows:

[0098] ,

[0099] in, To influence the neighbor weights (based on the Laplace regularization setting), is the attention weight (based on adversarial training settings), For nodes The attention guidance vector (sort the Laplacian matrix in descending order and filter the eigenvector corresponding to the minimum value as the attention guidance vector), For nodes The dynamics of updating the initial probability distribution, For nodes The dynamic time constant of

[0100] Use the Euler method for discretization, iterate the updated initial probability distribution, and use the low-rank approximation matrix for GLP propagation. Stop after reaching the maximum number of iterations (determined based on mathematical derivation) and output the iterative initial probability distribution. The formula is:

[0101] ,

[0102] in, is the retention weight factor of the initial label (obtained based on the label propagation algorithm), For nodes The initial probability distribution of propagation, For nodes Time The initial probability distribution of For nodes and nodes In time Update the initial probability distribution of ;

[0103] To update the initial probability distribution, VAE is used to generate a hash code for the updated initial probability distribution, and the hash loss is calculated as a constraint on the multimodal feature vector. The formula is:

[0104] ,

[0105] in, is the hash loss, For nodes The hash code of

[0106] Perform weighted fusion on the propagation initial probability distribution and the update initial probability distribution to generate a fusion label distribution matrix;

[0107] Based on the updated initial probability distribution and the propagated initial probability distribution, the square error loss function is used to calculate the consistency loss of the updated initial probability distribution and the initial label deviation of the propagated initial probability, and the total loss is generated by weighted fusion;

[0108] Based on the dynamic edge weight matrix, EA is used to calculate the voting weight of each node's neighbors and the node's voting distribution, and the fusion label distribution matrix is ​​updated;

[0109] Based on the updated fusion distribution label matrix, the EA objective function is defined to optimize the two weight parameters in the dynamic equation. The formula is:

[0110] ,

[0111] in, is the objective function value, is the total loss, is the variance weight (static setting based on empirical proportion), is the label distribution variance (calculated by global variance based on the updated fusion label distribution matrix), For nodes Update fusion distribution label matrix;

[0112] Based on each node, the maximum value in the fusion distribution label matrix is ​​filtered and optimized to generate the final label of each node.

[0113] By generating low-dimensional hash codes through VAE, the computational complexity of multimodal features is reduced. The dynamic edge weight matrix introduces a time decay factor, which enhances the adaptability of the model to dynamic scenes. The weighted fusion of comprehensive similarity improves semantic consistency and significantly improves the annotation accuracy and robustness. The low-rank approximation matrix effectively filters out noise in high-dimensional data and retains key semantic information. The introduction of sparse adjacency matrix and Laplace matrix enhances the modeling ability of data topology structure, which can better capture the potential correlation between data in complex scenes and improve the annotation consistency and accuracy. K-Means clustering combined with kernel density estimation significantly improves the semantic discrimination of the initial label. Density estimation introduces non-parametric probability modeling, which enhances the model's adaptability to uneven data distribution and can effectively reduce labeling deviations in dynamic scenarios. The efficient discretization of the Euler method reduces the complexity of iterative calculations. GLP propagation utilizes the topological information of the low-rank approximate matrix, significantly improving the stability and convergence speed of label propagation. It can better balance local and global semantics in complex network structures and reduce labeling errors. EA optimizes weight parameters through global search, avoiding the risk of traditional gradient optimization methods falling into local optimality. The voting mechanism combined with the dynamic edge weight matrix enhances the model's ability to handle potential data conflicts and significantly improves the robustness and adaptability of the labeling results.

[0114] S2. Based on the optimization results, amplitude coding is used for conversion, and quantum state circuits are used for construction. Partial traces are used for calculation, and topological weighted entanglement entropy is generated through weighted fusion. Updates are made based on the attention mechanism to generate potential conflict pairs and calculate priorities. The final conflict pairs are generated through the CNN model, and regional features are output through the NeRF model. The CLIP model is used for extraction and the update is made through the loss function.

[0115] Specifically, based on the optimization results, the CNN model is used to generate the final conflict pair, including:

[0116] The optimized fusion label distribution matrix is ​​converted into a quantum state vector through amplitude coding. The edge set is defined based on the dynamic edge weight matrix. The quantum state vector is entangled through the quantum state circuit. The topological feature vector of each node (including Betti number and persistence graph feature) is generated through TDA.

[0117] Based on the entangled state, the reduced density matrix is ​​calculated through partial traces;

[0118] The topological feature vector is normalized using the L2 norm and mapped using the Sigmoid function to generate the weight of the topological feature vector;

[0119] The entanglement entropy of the reduced density matrix is ​​calculated by Qiskit, and the weight of the topological eigenvector is combined for weighted fusion to generate the topological weighted entanglement entropy.

[0120] The final label is adjusted using temperature scaling, the adjustment result is normalized by the Sigmoid function, the adjustment factor is calculated, and the topologically weighted entanglement entropy is multiplied to generate the adjusted topologically weighted entanglement entropy;

[0121] Based on the multimodal feature vector, the modal weight is obtained through the attention mechanism, and the adjusted topological weighted entanglement entropy is multiplied to generate the latest topological weighted entanglement entropy;

[0122] Based on the latest topologically weighted entanglement entropy, the average value and standard deviation are calculated by arithmetic mean and sample standard deviation respectively, and the dynamic adjustment threshold is generated by adaptive statistical method;

[0123] Nodes whose latest topological entanglement entropy is greater than the dynamically adjusted threshold are screened and defined as agents (including the final label, the latest topological weighted entanglement entropy, the topological eigenvector, and the fusion distribution matrix);

[0124] Define the agent's actions based on the final labels (adjust the labels to the neighbor node label set) and set the reward function;

[0125] Initialize the Q table, update the Q value using the Bellman equation, stop when the maximum number of updates is reached (based on the early stopping method), output the final Q value of each node, and generate the agent Q table;

[0126] If the final label of each node is not equal to the neighbor label and the edge weight between the two is greater than the edge weight threshold (set based on the fixed threshold method), then this node and the neighbor node are marked as potential conflict pairs and the priority is calculated using the formula:

[0127] ,

[0128] in, For nodes The latest topologically weighted entanglement entropy of For nodes Q value, For nodes The final label, For potential conflict pairs (nodes and nodes ) priority;

[0129] Sort the priorities in descending order, filter the top E potential conflict pairs (Top-K filtering based on priority), where E is the number of potential conflict pairs, and generate a potential conflict pair set;

[0130] Build a GNN model, including input layer, convolution layer, fully connected layer and output layer;

[0131] Use the Cora dataset to train the GNN model;

[0132] The topological feature vector and edge weight matrix of the potential conflict pair set are input into the GNN model, the visual weight and text weight are output, and the cross-modal similarity of the potential conflict pair set is calculated through the CLIP model;

[0133] Calculate the difference between cross-modal similarity and comprehensive similarity, filter out potential conflicting pairs that are greater than the difference threshold, and generate the final conflicting pairs.

[0134] Through amplitude coding and quantum state circuits, efficient conversion of classical data to quantum states is achieved, which significantly reduces the data dimension while retaining key topological features. The parallelism of quantum computing improves computing efficiency. At the same time, TDA captures the intrinsic geometric structure of the data, which enhances the ability to recognize complex data patterns during the annotation process. The density matrix is ​​reduced by partial trace calculation to accurately extract the quantum state information of the subsystem. Through the principles of quantum mechanics, the modeling accuracy of complex relationships between multimodal data is significantly improved, providing reliable data support for subsequent entanglement entropy generation. Through the combination of L2 norm and Sigmoid function, the standardization and nonlinear weighting of feature vectors are achieved, which enhances the robustness of the model to sparse or high-dimensional data. The fusion with topological features enables accurate modeling of the intrinsic correlation of high-dimensional data and significantly improves the representation ability of feature correlation. The introduction of temperature scaling and adjustment factors further optimizes the dynamic range of entropy values ​​and enhances the adaptability of the model to dynamic data sets. Through the attention mechanism, adaptive weighting of modal features is achieved, which significantly improves the model's ability to capture semantic associations between image and text modalities. Through Q learning and priority sorting, intelligent screening of conflict pairs is achieved, which improves the accuracy and efficiency of conflict identification. The introduction of priority formulas further optimizes resource allocation, ensuring that high-impact conflict pairs are processed first. Through the synergistic effect of GNN and CLIP models, refined screening of conflict pairs is achieved, which significantly improves the semantic consistency of annotation results.

[0135] Furthermore, the regional features are output by the NeRF model, extracted using the CLIP model, and updated using the loss function, including:

[0136] Based on the final conflict pair, the NeRF model is initialized, and the regional features of the image data are rendered to generate a high-dimensional feature map. CBAM is used to generate spatial and channel attention weights and output the initial regional features.

[0137] The CLIP model is used to extract the regional semantic embedding of the final conflict pair, obtain the semantic feature vector, and use the linear layer to map it to the CLIP dimension. The loss function of the model parameters is defined as follows:

[0138] ,

[0139] in, is the loss function value of the model parameters, For the parameters (NeRF model) distribution The expected operation, Regional characteristics Generate an image of the final conflict node The logarithm of the probability, For parameters the posterior distribution (approximated via variational inference), For parameters The prior distribution of (obtained through the standard normal distribution), is the divergence, is the hyperparameter of semantic embedding (set based on heuristic scaling), For the region The semantic feature vector of is the hyperparameter of label loss (dynamically adjusted based on the number of training rounds), For nodes The final label The fusion label distribution matrix;

[0140] The Adam optimizer is used to minimize the loss function value of the model parameters. When the preset number of iterations is reached, it stops (based on the learning rate scheduling method). The parameters of the NeBF model are updated and the updated regional features are output.

[0141] Perform weighted fusion on the regional semantic embedding and mapping results to generate fusion features;

[0142] Extract the feature vector of each initial category label, obtain the category average feature by summing and averaging, and calculate the weighted Euclidean distance between the fusion feature and the category average feature. The formula is:

[0143] ,

[0144] in, For the region The weighted Euclidean distance of For the region The fusion feature vector of (regional feature and semantic feature vector are obtained by weighted fusion), is the Sigmoid activation function, For nodes The topological eigenvector of is the label category;

[0145] The weighted Euclidean distances are sorted in descending order, the fusion feature with the smallest weighted Euclidean distance is selected, the attention region feature is updated, and the updated attention region feature is generated.

[0146] By combining the NeRF model with CBAM's regional feature rendering and attention mechanism, the adaptive enhancement of high-dimensional feature maps is achieved, significantly improving the pertinence and robustness of feature extraction. It can retain more geometric information and reduce feature loss, providing a high-quality feature foundation for subsequent semantic embedding and label generation, and improving the accuracy and consistency of image data annotation. Through the semantic embedding of the CLIP model and the optimization of the comprehensive loss function, deep alignment of regional features and semantic information is achieved, improving the semantic consistency of the annotation task, effectively capturing the deep associations between images and text, and reducing annotation bias. The Adam optimizer combined with the learning rate scheduling method further ensures the stability of model convergence, thus providing reliable support for real-time annotation in dynamic scenes. By weighted fusion of regional semantic embedding and mapping results, the effective integration of visual and semantic features is achieved, significantly enhancing the information integrity and expressiveness of the feature vector. Compared with single-modal features, the fused features can better capture semantic associations in complex scenes, reduce information redundancy, and provide a more reliable feature foundation for subsequent distance calculation and feature update. By calculating and filtering the weighted Euclidean distance, the attention region features are accurately updated, significantly improving the model's ability to focus on key areas.

[0147] S3. Generate a verification fusion label distribution matrix through the MLP model and generate a heat map through ECharts;

[0148] Specifically, the MLP model is used to generate a verification fusion label distribution matrix, and ECharts is used to generate a heat map, including:

[0149] Build an MLP model, including input layer, hidden layer and output layer;

[0150] Use the COCO multimodal dataset to train the MLP model;

[0151] The updated attention region features and category mean features are input into the MLP model, mapped to category probabilities, and the updated label distribution is calculated through the Softmax function. The fusion label distribution matrix of the final conflict pair is updated to generate the verification fusion label distribution matrix;

[0152] The verification fusion label distribution matrix is ​​converted into a metadata table and a heat map is generated using ECharts.

[0153] By constructing an MLP model, the processing capability of high-dimensional multimodal features is significantly improved, thereby improving the accuracy and robustness of label distribution. Using the MNIST dataset to train the MLP model not only improves the model's adaptability to image data, but also reduces the training cost on specific task data. By combining the attention region features with the category mean features and mapping them to the probability space, the conflict problem in multimodal data fusion is alleviated, the semantic consistency and accuracy of the label distribution are significantly improved, and it can better adapt to dynamic data changes in complex scenarios. The generated fused label distribution matrix has higher credibility and application value. Generating heat maps through ECharts significantly improves the interpretability and user-friendliness of the annotation results.

[0154] S4. storing the two types of collected data;

[0155] Specifically, the two types of data collected and stored with the heat map include:

[0156] The two types of data collected and the heat map generated by the analysis are transferred to the central database via the SFTP protocol for storage and regular maintenance and update.

[0157] By transferring the two types of collected data and the heat map generated by analysis to the central database for storage via the SFTP protocol, data security risks are significantly reduced, and the credibility and compliance of the system are enhanced. By establishing a secure communication channel, the reliability and integrity of data transmission are improved, ensuring the safe migration of data from local to central databases, and reducing the risk of data loss or damage due to network vulnerabilities. Through regular maintenance and updates, the stability and scalability of the data storage system are enhanced. At the same time, through regular security updates, the security risks caused by system aging or vulnerabilities are further reduced.

[0158] This embodiment also provides an image data annotation system, including:

[0159] The processing and optimization module is used to collect image and text data, preprocess the collected data, generate hash codes and construct a dynamic edge weight matrix through VAE, combine singular value decomposition to generate a low-rank approximation matrix, update it through kernel density estimation and dynamic equations, and use Euler discretization combined with GLP propagation and EA for optimization;

[0160] The calculation update module is used to convert the optimization results using amplitude coding, construct them through quantum state circuits, calculate using partial traces, generate topological weighted entanglement entropy through weighted fusion, update based on the attention mechanism, generate potential conflict pairs, calculate priorities, generate final conflict pairs through the CNN model, and output regional features through the NeRF model, extract them using the CLIP model, and update them through the loss function;

[0161] Verification upload module, used to generate verification fusion label distribution matrix through MLP model and generate heat map through ECharts;

[0162] The encrypted storage module is used to store the two types of collected data.

[0163] This embodiment also provides a computer device suitable for the image data labeling method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the image data labeling method proposed in the above embodiment.

[0164] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0165] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the image data annotation method proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0166] In summary, the present invention collects image and text data, preprocesses the two collected data, generates hash codes and constructs a dynamic edge weight matrix through VAE, generates a low-rank approximation matrix in combination with singular value decomposition, updates through kernel density estimation and dynamic equations, uses Euler discretization combined with GLP propagation and EA for optimization, and based on the optimization results, uses amplitude coding for conversion, constructs through quantum state circuits, uses partial traces for calculation, generates topological weighted entanglement entropy through weighted fusion, updates based on the attention mechanism, generates potential conflict pairs, and calculates priorities, generates final conflict pairs through the CNN model, outputs regional features through the NeRF model, uses the CLIP model for extraction, and updates through the loss function; improves the accuracy and consistency of annotation, and improves the efficiency and accuracy of annotation.

[0167] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for labeling image data, characterized by: include, Collect image and text data, preprocess the collected data, generate hash codes through VAE and construct a dynamic edge weight matrix, use VAE to map multimodal feature vectors to hash codes, calculate hash similarity through Gaussian kernel function, and obtain comprehensive similarity by weighted fusion of global similarity, local attention similarity and hash similarity. Define comprehensive similarity as edges, image ID as nodes, obtain edge weights based on hash similarity, construct a dynamic edge weight matrix, extract the similarity of all nodes and sort them in descending order, filter the top N maximum similarities, where N is the number of nodes, and sum and average them to obtain local density. Based on the local density, calculate the number of neighbors. Based on the number of neighbors, build a sparse adjacency matrix for each node, update it through kernel density estimation and dynamic equations, and use Euler discretization combined with GLP propagation and EA for optimization; Based on the optimization results, amplitude coding is used for conversion, and quantum state circuits are used for construction. Partial traces are used for calculation, and topological weighted entanglement entropy is generated through weighted fusion. Updates are performed based on the attention mechanism to generate potential conflict pairs and calculate priorities. The GNN model is constructed, and the topological feature vectors and edge weight matrices of the potential conflict pair set are input into the GNN model. The visual weights and text weights are output, and the cross-modal similarity of the potential conflict pair set is calculated through the CLIP model. The difference between the cross-modal similarity and the comprehensive similarity is calculated, and potential conflict pairs greater than the difference threshold are screened to generate the final conflict pair. Based on the final conflict pair, the NeRF model is initialized to generate a high-dimensional feature map, and CBAM is used to generate spatial and channel attention weights. The initial regional features are output, and the regional semantic embedding of the final conflict pair is extracted using the CLIP model to obtain the semantic feature vector, which is mapped to the CLIP dimension using a linear layer. The loss function of the model parameters is defined, and the updated regional features are output through the Adam optimizer. Generate a verification fusion label distribution matrix through the MLP model, and generate a heat map through ECharts; The two types of collected data are stored.

2. The image data annotation method according to claim 1, wherein: The image and text data are collected and optimized using Euler discretization combined with GLP propagation and EA, including: Based on the multimodal feature vector and attention region features, the global similarity and local attention similarity are calculated respectively using the cosine formula; Use VAE to map the multimodal feature vector into a hash code and calculate the hash similarity using the Gaussian kernel function; The global similarity, local attention similarity and hash similarity are weightedly fused to obtain the comprehensive similarity. The comprehensive similarity is defined as the edge, the image ID is the node, and a dynamic edge weight matrix is ​​constructed. The number of categories K is set based on the elbow theorem, and K-Means clustering is used to cluster the multimodal feature vectors into K clusters, which are then mapped to string labels to generate initial category labels. The initial probability distribution is then generated through maximum likelihood estimation. Map the hash code to the initial probability distribution through kernel density estimation, update the initial probability distribution, define the dynamic equation based on each node, use the Euler method for discretization, iterate the updated initial probability distribution, and use the low-rank approximation matrix for GLP propagation, perform weighted fusion on the propagated initial probability distribution and the updated initial probability distribution to generate a fused label distribution matrix, update the fused label distribution matrix through EA based on the dynamic edge weight matrix, define the EA objective function based on the updated fused distribution label matrix, and optimize the two weight parameters in the dynamic equation; Based on each node, the maximum value in the fusion distribution label matrix is ​​filtered and optimized to generate the final label of each node.

3. The image data annotation method according to claim 2, wherein: Based on the optimization results, the final conflict pair is generated through the CNN model, including: The optimized fusion label distribution matrix is ​​converted into a quantum state vector through amplitude coding. The edge set is defined based on the dynamic edge weight matrix. The quantum state vector is entangled through the quantum state circuit. The topological feature vector of each node is generated through TDA. Based on the entangled state, the reduced density matrix is ​​calculated through partial traces; The topological feature vector is normalized using the L2 norm and mapped using the Sigmoid function to generate the weight of the topological feature vector; The entanglement entropy of the reduced density matrix is ​​calculated by Qiskit, and the weight of the topological eigenvector is combined for weighted fusion to generate the topological weighted entanglement entropy, which is then adjusted to generate the adjusted topological weighted entanglement entropy. Based on the multimodal feature vector, the modal weight is obtained through the attention mechanism, and the adjusted topological weighted entanglement entropy is multiplied to generate the latest topological weighted entanglement entropy; If the final label of each node is not equal to the neighbor label and the edge weight between the two is greater than the edge weight threshold, then this node and the neighbor node are marked as potential conflict pairs and the priority is calculated; Sort the priorities in descending order, filter the first E potential conflict pairs, where E is the number of potential conflict pairs, and generate a potential conflict pair set.

4. The image data annotation method according to claim 3, wherein: The NeRF model is used to output regional features, which are extracted using the CLIP model and updated using the loss function, including: Perform weighted fusion on the regional semantic embedding and mapping results to generate fusion features; Extract the feature vector of each class of initial category labels, obtain the category average feature by summing and averaging, and calculate the weighted Euclidean distance between the fusion feature and the category average feature; The weighted Euclidean distances are sorted in descending order, the fusion feature with the smallest weighted Euclidean distance is selected, the attention region feature is updated, and the updated attention region feature is generated.

5. The image data annotation method according to claim 4, wherein: The verification fusion label distribution matrix is ​​generated by the MLP model, and the heat map is generated by ECharts, including: Build an MLP model, input the updated attention region features and category mean features into the MLP model, map them to category probabilities, calculate the updated label distribution through the Softmax function, update the fusion label distribution matrix of the final conflict pair, and generate the verification fusion label distribution matrix; The verification fusion label distribution matrix is ​​converted into a metadata table and a heat map is generated using ECharts.

6. The image data annotation method according to claim 1, wherein: The preprocessing of the two collected data includes: Preprocess the text data, including word segmentation using the BPE algorithm, extracting semantic features from the segmentation results using the BERT model, reducing the dimensionality of the semantic features using PCA, and normalizing them using the L2 norm to generate text features; Use the VGG-16 model to extract the convolution features of the image data, extract the fully connected layer features through the FC7 layer, generate the FC7 vector, and encode it through the BOW model to generate the BOW vector; Use the Conv5-3 layer of VGG-16 to extract the features of image data, generate feature maps, and use the L2 norm for normalization to generate multi-scale spatial features; The text features, normalized FC7 vectors, normalized BOW vectors, and multi-scale spatial features are concatenated, dimensionally reduced through a fully connected layer, and normalized using the L2 norm to generate a multimodal feature vector. Use the Selective Search algorithm to segment the feature map, generate candidate regions, and extract the features of each candidate region through the ResNet-50 model; The discriminative scores of all candidate regions are sorted in descending order, the region with the highest discriminative score is selected, the features of the corresponding candidate region are extracted, and the multimodal feature vectors are combined for splicing to generate the attention region features.

7. The image data annotation method according to claim 5, wherein: The two types of data collected are stored, including: The two types of data collected and the heat map generated by the analysis are transferred to the central database via the SFTP protocol for storage and regular maintenance and update.

8. An image data annotation system, based on the image data annotation method according to any one of claims 1 to 7, characterized in that: include, The processing and optimization module is used to collect image and text data, preprocess the collected data, generate hash codes and construct a dynamic edge weight matrix through VAE, combine singular value decomposition to generate a low-rank approximation matrix, update it through kernel density estimation and dynamic equations, and use Euler discretization combined with GLP propagation and EA for optimization; The calculation update module is used to convert the optimization results using amplitude coding, construct them through quantum state circuits, calculate using partial traces, generate topological weighted entanglement entropy through weighted fusion, update based on the attention mechanism, generate potential conflict pairs, calculate priorities, generate final conflict pairs through the CNN model, and output regional features through the NeRF model, extract them using the CLIP model, and update them through the loss function; Verification upload module, used to generate verification fusion label distribution matrix through MLP model and generate heat map through ECharts; The encrypted storage module is used to store the two types of collected data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image data annotation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image data annotation method according to any one of claims 1 to 7 are implemented.

Images