Crowdsourcing data dynamic redundancy detection method based on contrastive learning

Abstract

The present application relates to crowd-sensing data dynamic redundancy detection technical field, especially in kind based on contrast learning crowd-sensing data dynamic redundancy detection method;The method comprises: obtaining the sensing data set of task target, redundancy detection is carried out using pre-trained redundancy detection model, and redundancy detection result is obtained, wherein the training steps of redundancy detection model include: constructing sensing data set, carrying out data enhancement to several sensing data, feature extraction and multi-modal feature fusion, obtaining fusion feature pair, constructing fusion feature matrix, constructing positive and negative samples, carrying out instance level and clustering level contrast learning, iterative training is carried out through loss function until convergence, and redundancy detection model is obtained;The present application realizes single-stage end-to-end redundancy clustering by integrating clustering task in contrast learning framework, accurately flexible redundancy data identification is implemented for dynamic crowd-sensing data, and the accuracy of redundancy detection and data quality are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of dynamic redundancy detection technology for crowd-sensing data, and particularly to a method for dynamic redundancy detection of crowd-sensing data based on contrastive learning. Background Technology

[0002] With the built-in cameras of smart mobile devices becoming common visual recording tools in daily life, mobile crowdsourcing photogrammetry (MCP) has gradually developed into an emerging research field. In the MCP process, user-contributed visual data can be further analyzed and integrated, significantly improving the quality of crowdsourcing services and decision-making efficiency. Thanks to its low cost, high scalability, and high energy efficiency, MCP is widely used in virtual tourism, smart cities, environmental monitoring, emergency management, and disaster relief, among other fields. However, to meet the task requirements of MCP applications, crowdsourcing platforms often adopt a distributed data collection method to collect image data from different data sources. This often results in the final collected dataset containing a large amount of low-quality redundant data, severely impacting data quality. Furthermore, the large size and multidimensionality of visual data exacerbate the impact on data quality.

[0003] Data optimization is considered an effective solution, and data redundancy detection is a key step in it. The crowd-sensing platform evaluates redundant data by utilizing the multimodal information (contextual metadata, visual content) in the sensing data, retaining the most valuable samples to reduce transmission costs and improve storage and processing efficiency. Current research discusses the important role of contextual and visual information in the redundancy detection process. However, there are still the following shortcomings: (1) Current work is not robust enough to low-quality redundant data, and noise interference from environmental factors has a certain impact on the accuracy of redundancy grouping of crowd-sensing data and the availability of subsequent data; (2) There is a lack of comprehensive consideration of the multimodal information of crowd-sensing data. Redundancy grouping using metadata and visual data in stages ignores the content differences of the images themselves, resulting in large internal differences between images in the same group; (3) The repeated uploading of feature data in the current redundancy detection work reduces the user's enthusiasm for uploading, thereby affecting the subsequent selection decision of crowd-sensing data and the quality of crowd-sensing application services. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, the present invention aims to propose a dynamic redundancy detection method for swarm intelligence sensing data based on contrastive learning. By integrating clustering tasks within the contrastive learning framework, single-stage end-to-end redundancy clustering is achieved. For dynamic swarm intelligence sensing data, accurate and flexible redundancy data identification is implemented, effectively improving the accuracy of redundancy detection and data quality.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for dynamic redundancy detection of crowd sensing data based on contrastive learning, the method comprising:

[0007] Obtain the perception dataset of the task objective;

[0008] A pre-trained redundancy detection model is used to perform redundancy detection on the perceptual dataset of the task target, and the redundancy detection results are obtained.

[0009] The training steps of the pre-trained redundancy detection model include:

[0010] Step 1: Construct a perception dataset for mobile swarm photography perception scenarios. The perception dataset includes several perception data points from the same task target or different task targets.

[0011] Step 2: Perform data augmentation, feature extraction, and multimodal feature fusion on several pieces of perceived data in sequence to obtain fused feature pairs and construct a fused feature matrix;

[0012] Step 3: Construct positive and negative samples based on the fused feature matrix, perform instance-level and cluster-level comparative learning, and use the loss function for iterative training until convergence, thus obtaining the trained redundant detection model.

[0013] The sensing data in step 1 includes image data and metadata; the metadata includes timestamps, camera position coordinates when the image was captured, and shooting angle.

[0014] In step 2, data augmentation, feature extraction, and multimodal feature fusion are sequentially performed on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix, including:

[0015] Step 2.1: Apply different data augmentation strategies to the image data in each piece of perceptual data, extract image features, obtain image feature pairs, and construct the image feature matrix pair of the perceptual dataset, represented as:

[0016]

[0017] Among them, v i For the image data of the i-th perception data, H 1 and H 2 Different data augmentation strategies, For H 1 Image features obtained through data augmentation strategies For H 2 Image features obtained through data augmentation strategies and For a set of image feature pairs, Image features from all perceived data The constructed image feature matrix, Image features from all perceived data The constructed image feature matrix, V 1 and V 2 For the image feature matrix pairs of the perception dataset, d1 is the dimension of the image representation, N is the batch size, and f(·) is the visual encoder with shared parameters;

[0018] Step 2.2: Utilize the Transformer architecture to augment the metadata in each piece of sensing data and extract metadata features to obtain metadata feature pairs. Construct the metadata feature matrix of the sensing dataset, represented as follows:

[0019]

[0020] Among them, t i This is the metadata for the i-th sensing data. The metadata features obtained after one forward propagation. Metadata features obtained after another forward propagation and For a set of metadata feature pairs, Metadata features from all perceived data The constructed metadata feature matrix, Metadata features from all perceived data The constructed metadata feature matrix, T 1 and T 2 For the metadata feature matrix pair of the perception dataset, d2 is the dimension of the metadata representation, N is the batch size, and g(·) is the text encoder;

[0021] Step 2.3: Perform multimodal feature fusion on the image feature pairs and metadata feature pairs in the same perceptual data to obtain fused feature pairs, represented as:

[0022]

[0023] in, Image features and metadata features The fusion characteristics after fusion Image features and metadata features The fusion characteristics after fusion and MHA(·) is a multimodal feature fusion layer containing 8 independent attention heads, which is a set of fused feature pairs.

[0024] Step 2.4: Perform multimodal feature fusion on the image feature matrix pairs of the perceptual dataset and the metadata feature matrix pairs of the perceptual dataset to obtain the fused feature matrix, represented as:

[0025]

[0026] Among them, F 1 For the image feature matrix V 1 and metadata feature matrix T 1 The constructed fusion feature matrix, F 2 For the image feature matrix V 2 and metadata feature matrix T 2 The constructed fusion feature matrix, d represents the dimension of the fused features, and N represents the batch size.

[0027] The data augmentation strategies in step 2.1 include any combination of different probabilities among random cropping, color dithering, random horizontal flipping, Gaussian blur, and exposure adjustment.

[0028] The loss function in step 3 includes an instance-level loss function and a cluster-level loss function;

[0029] The instance-level loss function is expressed as:

[0030]

[0031] Where N is the batch size, and τ1 is the temperature parameter of the instance layer. and For a set of fused feature pairs, for any two feature representations f i f j Using the cosine similarity formula Calculate their feature similarity;

[0032] The clustering-level loss function is expressed as:

[0033]

[0034] in, and y is the feature of the i-th column of the clustering soft label matrix, i.e., the feature of the i-th cluster, τ2 is the clustering layer temperature parameter, K is the number of ground truth classes, and for two arbitrary feature representations y i With y j Using the cosine similarity formula Calculate their feature similarity;

[0035] The loss function is expressed as:

[0036]

[0037] Where λ is the dynamic equilibrium parameter, For instance-level loss function, This is the clustering-level loss function.

[0038] Step 3 specifically involves:

[0039] Step 3.1: Take any pair of corresponding fusion feature pairs in the fusion feature matrix as positive samples, and the remaining fusion feature pairs as negative samples;

[0040] Step 3.2: Use the instance-level loss function to perform instance-level representation learning on positive and negative samples, that is, perform comparative learning on the rows of the fusion matrix to obtain cluster soft labels;

[0041] Step 3.3: Input the fused feature matrix into the clustering layer and perform cluster-level comparative learning using the clustering-level loss function, that is, perform comparative learning on the columns of the fused feature matrix to obtain the clustering soft label matrix;

[0042] Step 3.4: Repeat steps 3.1-3.4 for iterative training until convergence, obtaining the trained redundancy detection model. The trained redundancy detection model is represented by the following formula:

[0043] c i =argmaxMLP c (f i ),

[0044] In the formula, f i Let c represent the i-th sensing data point. i This represents the cluster label of the i-th perceptual data.

[0045] The clustering layer in step 3.3 is constructed by a multilayer perceptron mapping fused feature matrix pairs to a space with a dimension equal to the number of ground truth classes.

[0046] A crowd-sensing data dynamic redundancy detection system based on contrastive learning, comprising:

[0047] Data acquisition module: Acquires the perception dataset of the task objective;

[0048] Result acquisition module: Uses a pre-trained redundancy detection model to perform redundancy detection on the perceptual dataset of the task target and obtains the redundancy detection results;

[0049] The training steps of the pre-trained redundancy detection model include:

[0050] Construct a perception dataset for mobile swarm photography perception scenarios, wherein the perception dataset includes several perception data from the same task target or different task targets;

[0051] Data augmentation, feature extraction, and multimodal feature fusion are performed sequentially on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix;

[0052] Positive and negative samples are constructed based on the fused feature matrix, and instance-level and cluster-level comparative learning are performed. The loss function is then used for iterative training until convergence, resulting in a trained redundant detection model.

[0053] An electronic device, comprising:

[0054] At least one processor;

[0055] And, a memory communicatively connected to the at least one processor;

[0056] The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the dynamic redundancy detection method for crowd sensing data based on contrastive learning as described above.

[0057] A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the dynamic redundancy detection method for crowd-sensing data based on contrastive learning as described above.

[0058] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0059] 1. This invention proposes a dynamic redundancy detection method (MCPCC) for crowd-sensing image data based on contrastive learning, which aims to improve the accuracy of redundancy detection by utilizing the multimodal information of crowd-sensing data and reduce the burden on the sensing platform.

[0060] 2. This invention utilizes a contrastive learning method to solve the problem of redundancy detection in swarm intelligence data, exhibiting good flexibility and applicability to swarm intelligence image data under different tasks.

[0061] 3. This invention treats representation learning and multimodal clustering as two aspects of a problem. By constructing clustering layers to directly generate cluster labels, it simplifies the detection process and optimizes the clustering results to approach the ideal number of clusters, ensuring that diverse images can fully meet the task requirements.

[0062] In summary, this invention achieves single-stage end-to-end redundant clustering by integrating clustering tasks within a contrastive learning framework. For dynamic swarm intelligence sensing data, it enables accurate and flexible redundant data identification, effectively improving the accuracy of redundancy detection and data quality. Attached Figure Description

[0063] Figure 1 This is a flowchart of the method of the present invention.

[0064] Figure 2 This is a flowchart of the redundancy detection model trained according to the present invention.

[0065] Figure 3 This is a flowchart of the main stages of a complete crowd intelligence task.

[0066] Figure 4 This is a framework diagram of the dynamic redundancy detection method for crowd-aware sensing data based on contrastive learning, which is the subject of this invention. Detailed Implementation

[0067] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0068] A complete crowd intelligence task typically includes four main phases. For example... Figure 3 As shown, the process first involves a task allocation phase, where the requester defines, publishes, and assigns the task to the task executors (participants). Next comes the task execution phase, where participants collect and upload data according to the task requirements. Following this is the data aggregation phase, where the submitted data is processed and filtered based on the task requirements. Finally, the results are handed over to the task requester. This invention focuses on the data aggregation phase and improves this process using a dynamic redundancy detection method based on contrastive learning.

[0069] Once the platform collects a large amount of redundant data from different data sources, it performs redundancy detection. For example... Figure 3 Images capturing the same Point of Interest (POI) are grouped into clusters, and redundant data is filtered out based on the clustering results. For example, depending on the specific task requirements, the most representative samples or samples with high perceptual coverage from a particular cluster can be selected to reduce unnecessary transmission and computation.

[0070] This invention considers the impact of multimodal information from swarm intelligence data on the accuracy of redundant clustering. Furthermore, contrastive learning, as one of the most popular self-supervised learning methods, has achieved significant results in representation learning. Contrastive learning aims to learn highly discriminative representations by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs. Based on this, it is found that contrastive learning and clustering are very similar in concept. As swarm intelligence data becomes increasingly complex, traditional clustering algorithms have poor representation learning capabilities and struggle to handle high-dimensional data. This invention enhances the clustering algorithm's understanding of the similarities and differences between data by obtaining richer hidden information between modalities through multimodal feature fusion based on an attention mechanism. Moreover, within the contrastive learning framework, cluster labels are directly obtained by integrating clustering layers, jointly optimizing the representation learning and clustering processes to help obtain cluster-friendly modality-consistent representations, making the clustering results more accurate and interpretable.

[0071] The technical solution of the present invention is described in detail below:

[0072] A method for dynamic redundancy detection of crowd-sensing data based on contrastive learning, such as Figure 1 As shown, the method includes:

[0073] Obtain the perception dataset of the task objective;

[0074] A pre-trained redundancy detection model is used to perform redundancy detection on the perceptual dataset of the task target, and the redundancy detection results are obtained.

[0075] Among them, such as Figure 2 As shown, the training steps for the pre-trained redundancy detection model include:

[0076] Step 1, construct the perception dataset P for mobile crowdsourced photography perception scenarios, denoted as P. It is a set of n data points from multiple data sources (i.e., multiple participants), where each perceptual data point j consists of several attributes t. j (i.e., metadata) and image v j Composition. Among them, metadata t j Includes: timestamp (ts) j The camera's position coordinates log when the image is captured. j Shooting angle ang j .

[0077] Step 2: Perform data augmentation, feature extraction, and multimodal feature fusion on several pieces of perceived data in sequence to obtain fused feature pairs and construct a fused feature matrix.

[0078] In step 2, data augmentation, feature extraction, and multimodal feature fusion are sequentially performed on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix, including:

[0079] Step 2.1: Using different data augmentation strategies H, perform data augmentation on the image data in each piece of perceptual data. For the i-th image sample v... i The visual encoder f(·) with shared parameters extracts image feature pairs from its augmented sample pairs. And construct the image feature matrix of the perceptual dataset for V 1 V 2 , represented as:

[0080]

[0081] Among them, v i For the image data of the i-th perception data, H 1 and H 2 Different data augmentation strategies, For H 1 Image features obtained through data augmentation strategies For H 2 Image features obtained through data augmentation strategies and For a set of image feature pairs, Image features from all perceived data The constructed image feature matrix, Image features from all perceived data The constructed image feature matrix, V 1 and V 2 For the image feature matrix pairs of the perception dataset, d1 is the dimension of the image representation, N is the batch size, f(·) is the visual encoder with shared parameters, and f(·) is initialized by the pre-trained ResNet18.

[0082] Step 2.2: Use a Transformer architecture with random dropout to augment the metadata in each piece of perceived data, where for metadata sample t... i The text encoder g(·) is input twice to generate metadata feature pairs. And construct the metadata feature matrix of the perception dataset for T. 1 T 2 , represented as:

[0083]

[0084] Among them, t i This is the metadata for the i-th sensing data. The metadata features obtained after one forward propagation. Metadata features obtained after another forward propagation and For a set of metadata feature pairs, Metadata features from all perceived data The constructed metadata feature matrix, Metadata features from all perceived data The constructed metadata feature matrix, T 1 and T 2 d2 is the metadata feature matrix pair of the perception dataset, d2 is the dimension of the metadata representation, N is the batch size, and g(·) is the text encoder pre-trained by BERT.

[0085] Step 2.3: Perform multimodal feature fusion on the image feature pairs and metadata feature pairs in the same perceptual data to obtain fused feature pairs, represented as:

[0086]

[0087] in, Image features and metadata features The fusion characteristics after fusion Image features and metadata features The fusion characteristics after fusion and MHA(·) is a multimodal feature fusion layer containing 8 independent attention heads, which is a set of fused feature pairs.

[0088] Based on this, the differences between different data modalities can be effectively explored, and data from different perspectives and information sources can be integrated to generate a comprehensive data representation. This integrated data representation will provide a more accurate and comprehensive information foundation for subsequent clustering analysis. By introducing an attention mechanism, the model can automatically learn the correlation between features of different modalities according to task requirements and assign appropriate weights to each modality. Specifically, features that play a key role in the perception result will receive greater weights, while secondary or irrelevant features will have reduced weights. Through this dynamic adjustment of the contributions of features of different modalities, the model can more effectively utilize multimodal information, thereby improving the performance of downstream tasks.

[0089] Step 2.4: Perform multimodal feature fusion on the image feature matrix pairs of the perceptual dataset and the metadata feature matrix pairs of the perceptual dataset to obtain the fused feature matrix, represented as:

[0090]

[0091] Among them, F 1 For the image feature matrix V 1 and metadata feature matrix T1 The constructed fusion feature matrix, F 2 For the image feature matrix V 2 and metadata feature matrix T 2 The constructed fusion feature matrix, d represents the dimension of the fused features, and N represents the batch size.

[0092] The data augmentation strategies in step 2.1 include any combination of different probabilities among random cropping, color dithering, random horizontal flipping, Gaussian blur, and exposure adjustment.

[0093] Step 3: Construct positive and negative samples based on the fused feature matrix, perform instance-level and cluster-level comparative learning, and use the loss function for iterative training until convergence, thus obtaining the trained redundant detection model.

[0094] The central idea of ​​this step is to transform the problem of redundancy detection in crowdsourced sensing data into a clustering analysis problem. To ensure the completion rate of crowdsourced tasks and the flexibility of sensing, crowdsourced sensing platforms often adopt a distributed data collection method to collect image data from different data sources. This often results in the final collected dataset containing a large amount of redundant data, or occluded or incorrectly submitted data, seriously affecting data quality. The large size and multidimensionality of visual data further exacerbate the challenge of data transmission under limited resources.

[0095] Faced with a large amount of low-quality and redundant swarm intelligence data, clustering analysis can be used to group images that capture the same Point of Interest (POI) into clusters. The clustering results can then be used to effectively filter redundant data. For example, based on the specific needs of the task, the most representative samples or samples with high perceptual coverage from a particular cluster can be selected and retained to reduce unnecessary transmission and computation. This not only improves storage and processing efficiency and enhances the usability of the data, but also ensures the reliability of the data in subsequent applications.

[0096] Specifically:

[0097] The loss function in step 3 includes an instance-level loss function and a cluster-level loss function;

[0098] This invention introduces the concept of contrastive learning, aiming to learn highly discriminative representations by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs. For each perceived data sample in a batch, it and the data obtained by augmenting and fusing that sample with different modalities are considered positive sample pairs, while the remaining samples in the batch are considered negative sample pairs. Instance-level representation learning is performed in this way, and the instance-level loss function is constructed based on InfoNCE loss as follows:

[0099]

[0100] Where N is the batch size, and τ1 is the temperature parameter of the instance layer. and For a set of fused feature pairs, for any two feature representations f i f j Using the cosine similarity formula Calculate their feature similarity;

[0101] Secondly, representation learning and multimodal clustering are considered as two aspects of a problem and jointly optimized. In deep neural network (DNN) tasks, each point in the feature space can be regarded as a representation of a latent label, which allows the algorithm to effectively cluster data points without label guidance; that is, feature vectors can be regarded as corresponding soft labels. By constructing clustering layers using a multilayer perceptron within a contrastive learning framework, the fused feature matrix F... 1 F 2 Cluster-level representation learning is performed by mapping to a space with a dimension equal to the number of true ground-level classes. The clustering layer takes a multimodal consistent representation F as input and a cluster soft label Y as output. K represents the number of real-world classes. Since each sample belongs to only one class, ideally, the i-th row in Y... The soft label (corresponding to the i-th fused sample) should be in one-hot encoded form. The i-th column in Y This can be interpreted as the probability that the i-th fused sample is assigned to the i-th class. Y1 and Y2, obtained through data augmentation, should have the same columns for their corresponding soft labels. Similar to instance-level representation learning, within each batch, cluster labels obtained by augmenting and fusing the same sample are considered positive sample pairs, and the remaining samples are considered negative sample pairs. The clustering-level loss function is designed as follows:

[0102]

[0103] in, and y is the feature of the i-th column of the clustering soft label matrix, i.e., the feature of the i-th cluster, τ2 is the clustering layer temperature parameter, K is the number of ground truth classes, and for two arbitrary feature representations y i With y j Using the cosine similarity formula Calculate their feature similarity;

[0104] By minimizing the clustering layer loss, the aim is to maximize the similarity of the soft-label group corresponding to the i-th sample, while minimizing its similarity to other sample soft-label groups. Here, τ2 is the clustering layer temperature parameter, τ1 is the clustering layer temperature parameter, and K is the number of ground truth classes.

[0105] The loss function is expressed as:

[0106]

[0107] Where λ is the dynamic equilibrium parameter, For instance-level loss function, This is the clustering-level loss function.

[0108] Step 3 specifically involves:

[0109] Step 3.1: Take any pair of corresponding fusion feature pairs in the fusion feature matrix as positive samples, and the remaining fusion feature pairs as negative samples;

[0110] Step 3.2: Use the instance-level loss function to perform instance-level representation learning on positive and negative samples, that is, perform comparative learning on the rows of the fusion matrix to obtain cluster soft labels;

[0111] Step 3.3: Input the fused feature matrix into the clustering layer and perform cluster-level comparative learning using the clustering-level loss function, that is, perform comparative learning on the columns of the fused feature matrix to obtain the clustering soft label matrix;

[0112] Step 3.4: Repeat steps 3.1-3.4 for iterative training until convergence, obtaining the trained redundancy detection model. The trained redundancy detection model is represented by the following formula:

[0113] c i =argmaxMLP c (f i ),

[0114] In the formula, f i Let c represent the i-th sensing data point. i This represents the cluster label of the i-th perceptual data.

[0115] The clustering layer in step 3.3 is constructed by a multilayer perceptron (MLP) that maps fused feature matrix pairs to a space with a dimension equal to the number of ground truth classes.

[0116] MLPs have better generalization ability than fully connected (FC) networks, resulting in optimized network parameters and improved prediction accuracy. This is because dimensionality reduction through the projection layer decreases the size of cluster centers, leading to a smaller reduction in the total number of parameters compared to the original network. A two-layer MLP consists of a hidden layer connected to a ReLU activation function and an output layer connected to a softmax function, optimizing...

[0117] The rows and columns of the fused feature matrix are regarded as instance soft labels and cluster representations, respectively. That is, the rows can be interpreted as the probability of a sample corresponding to each cluster, and the columns are regarded as the distribution of the sample in a certain cluster.

[0118] The purpose of instance-level contrastive learning is to bring positive sample pairs closer together and push negative samples further apart using an instance-level loss function. This helps learn feature representations that distinguish different data points; similar feature representations are close in the feature space, while dissimilar ones are far apart. In other words, contrastive learning is performed on the rows of the fusion matrix to obtain a better feature representation called clustering soft labels. These soft labels are a special type of label representation, where the i-th row and j-th column of the matrix represents the probability of the i-th sample being assigned to the j-th class.

[0119] Cluster-level contrastive learning integrates the clustering task into a contrastive learning framework. Specifically, instance-level contrastive learning is the initial step, with an additional MLP clustering layer added. Contrastive learning is performed on the columns of the fused feature matrix to obtain the cluster representation; the j-th column of the matrix represents the j-th class.

[0120] Instance-level and cluster-level learning complement each other and proceed simultaneously. Under the influence of the total loss, end-to-end learning distinguishes the feature representations and cluster assignments of different data points.

[0121] A crowd-sensing data dynamic redundancy detection system based on contrastive learning, comprising:

[0122] Data acquisition module: Acquires the perception dataset of the task objective;

[0123] Result acquisition module: Uses a pre-trained redundancy detection model to perform redundancy detection on the perceptual dataset of the task target and obtains the redundancy detection results;

[0124] The training steps of the pre-trained redundancy detection model include:

[0125] Construct a perception dataset for mobile swarm photography perception scenarios, wherein the perception dataset includes several perception data from the same task target or different task targets;

[0126] Data augmentation, feature extraction, and multimodal feature fusion are performed sequentially on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix;

[0127] Positive and negative samples are constructed based on the fused feature matrix, and instance-level and cluster-level comparative learning are performed. The loss function is then used for iterative training until convergence, resulting in a trained redundant detection model.

[0128] An electronic device, comprising:

[0129] At least one processor;

[0130] And, a memory communicatively connected to the at least one processor;

[0131] The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the dynamic redundancy detection method for crowd sensing data based on contrastive learning as described above.

[0132] A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the dynamic redundancy detection method for crowd-sensing data based on contrastive learning as described above.

[0133] In summary, this invention enhances the clustering algorithm's understanding of the similarities and differences between data by acquiring richer hidden information between modalities through multimodal feature fusion based on an attention mechanism. Furthermore, within a contrastive learning framework, cluster labels are directly obtained by integrating clustering layers, jointly optimizing the representation learning and clustering processes to help obtain a cluster-friendly, modality-consistent representation, making the clustering results more accurate and interpretable.

[0134] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for dynamic redundancy detection of crowd-sensing data based on contrastive learning, characterized in that, The method includes: Obtain the perception dataset of the task objective; A pre-trained redundancy detection model is used to perform redundancy detection on the perceptual dataset of the task target, and the redundancy detection results are obtained. The training steps of the pre-trained redundancy detection model include: Step 1: Construct a perception dataset for mobile swarm photography perception scenarios. The perception dataset includes several perception data points from the same task target or different task targets. Step 2: Perform data augmentation, feature extraction, and multimodal feature fusion on several pieces of perceived data in sequence to obtain fused feature pairs and construct a fused feature matrix; Step 3: Construct positive and negative samples based on the fused feature matrix, perform instance-level contrastive learning and cluster-level contrastive learning, and use the loss function for iterative training until convergence to obtain the trained redundant detection model. The loss function in step 3 includes an instance-level loss function and a cluster-level loss function; The instance-level loss function is expressed as: ， Where N is the batch size. These are the temperature parameters of the instance layer. and For a set of fused feature pairs, for any two feature representations Using the cosine similarity formula Calculate their feature similarity; The clustering-level loss function is expressed as: ， in, and These are the features in the i-th column of the clustering soft label matrix, i.e., the features of the i-th cluster. For the temperature parameters of the clustering layer, Let be the number of ground truth classes, for any two feature representations. and Using the cosine similarity formula Calculate their feature similarity; The loss function is expressed as: ， in, It is a dynamic equilibrium parameter. For instance-level loss functions, The clustering-level loss function; Step 3 specifically involves: Step 3.1: Take any pair of corresponding fusion feature pairs in the fusion feature matrix as positive samples, and the remaining fusion feature pairs as negative samples; Step 3.2: Use the instance-level loss function to perform instance-level representation learning on positive and negative samples, that is, perform comparative learning on the rows of the fusion matrix to obtain cluster soft labels; Step 3.3: Input the fused feature matrix into the clustering layer and perform cluster-level comparative learning using the clustering-level loss function, that is, perform comparative learning on the columns of the fused feature matrix to obtain the clustering soft label matrix; Step 3.4: Repeat steps 3.1-3.4 for iterative training until convergence, obtaining the trained redundancy detection model. The trained redundancy detection model is represented by the following formula: ， In the formula, Indicates the first i Perceptual data, Indicates the first i Clustering labels for perceptual data.

2. The method for dynamic redundancy detection of crowd-sensing data based on contrastive learning according to claim 1, characterized in that, The sensing data in step 1 includes image data and metadata; the metadata includes timestamps, camera position coordinates when the image was captured, and shooting angle.

3. The method for dynamic redundancy detection of crowd-sensing data based on contrastive learning according to claim 2, characterized in that, In step 2, data augmentation, feature extraction, and multimodal feature fusion are sequentially performed on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix, including: Step 2.1: Apply different data augmentation strategies to the image data in each piece of perceptual data, extract image features, obtain image feature pairs, and construct the image feature matrix pair of the perceptual dataset, represented as: ， ， in, For the first i Image data of strip perception data, and Different data augmentation strategies, For the process Image features obtained through data augmentation strategies For the process Image features obtained through data augmentation strategies and For a set of image feature pairs, Image features from all perceived data The constructed image feature matrix, Image features from all perceived data The constructed image feature matrix, V 1 and V 2 For the image feature matrix pairs of the perception dataset, For the dimensions of image representation, N For batch size, A visual encoder with shared parameters; Step 2.2: Utilize the Transformer architecture to augment the metadata in each piece of sensing data and extract metadata features to obtain metadata feature pairs. Construct the metadata feature matrix of the sensing dataset, represented as follows: ， ， in, This is the metadata for the i-th sensing data. The metadata features obtained after one forward propagation. Metadata features obtained after another forward propagation and For a set of metadata feature pairs, Metadata features from all perceived data The constructed metadata feature matrix, Metadata features from all perceived data The constructed metadata feature matrix, T 1 and T 2 To perceive the metadata feature matrix pairs of the dataset, For the dimensions of metadata representation, N For batch size, For text encoders; Step 2.3: Perform multimodal feature fusion on the image feature pairs and metadata feature pairs in the same perceptual data to obtain fused feature pairs, represented as: in, Image features and metadata features The fusion characteristics after fusion Image features and metadata features The fusion characteristics after fusion and For a set of fused feature pairs, This is a multimodal feature fusion layer containing eight independent attention heads within a single attention layer; Step 2.4: Perform multimodal feature fusion on the image feature matrix pairs of the perceptual dataset and the metadata feature matrix pairs of the perceptual dataset to obtain the fused feature matrix, represented as: ， ， in, F 1 For the image feature matrix V 1 and metadata feature matrix T 1 The constructed fusion feature matrix, F 1 , F 2 For the image feature matrix V 2 and metadata feature matrix T 2 The constructed fusion feature matrix, F 2 , N represents the dimension of the fused features, and N is the batch size.

4. The method for dynamic redundancy detection of crowd-sensing data based on contrastive learning according to claim 3, characterized in that, The data augmentation strategies in step 2.1 include any combination of different probabilities among random cropping, color dithering, random horizontal flipping, Gaussian blur, and exposure adjustment.

5. The method for dynamic redundancy detection of crowd sensing data based on contrastive learning according to claim 1, characterized in that, The clustering layer in step 3.3 is constructed by a multilayer perceptron mapping fused feature matrix pairs to a space with a dimension equal to the number of ground truth classes.

6. A dynamic redundancy detection system for crowd-sensing data based on contrastive learning, characterized in that, include: Data acquisition module: Acquires the perception dataset of the task objective; Result acquisition module: Uses a pre-trained redundancy detection model to perform redundancy detection on the perceptual dataset of the task target and obtains the redundancy detection results; The training steps of the pre-trained redundancy detection model include: Construct a perception dataset for mobile swarm photography perception scenarios, wherein the perception dataset includes several perception data from the same task target or different task targets; Data augmentation, feature extraction, and multimodal feature fusion are performed sequentially on several pieces of perceived data to obtain fused feature pairs and construct a fused feature matrix; Positive and negative samples are constructed based on the fused feature matrix, and instance-level and cluster-level comparative learning are performed. The loss function is used for iterative training until convergence, resulting in a trained redundant detection model. The loss function includes an instance-level loss function and a cluster-level loss function; The instance-level loss function is expressed as: ， Where N is the batch size. These are the temperature parameters of the instance layer. and For a set of fused feature pairs, for any two feature representations Using the cosine similarity formula Calculate their feature similarity; The clustering-level loss function is expressed as: ， in, and These are the features in the i-th column of the clustering soft label matrix, i.e., the features of the i-th cluster. For the temperature parameters of the clustering layer, Let be the number of ground truth classes, for any two feature representations. and Using the cosine similarity formula Calculate their feature similarity; The loss function is expressed as: ， in, It is a dynamic equilibrium parameter. For instance-level loss functions, The clustering-level loss function; Step 3 specifically involves: Step 3.1: Take any pair of corresponding fusion feature pairs in the fusion feature matrix as positive samples, and the remaining fusion feature pairs as negative samples; Step 3.2: Use the instance-level loss function to perform instance-level representation learning on positive and negative samples, that is, perform comparative learning on the rows of the fusion matrix to obtain cluster soft labels; Step 3.3: Input the fused feature matrix into the clustering layer and perform cluster-level comparative learning using the clustering-level loss function, that is, perform comparative learning on the columns of the fused feature matrix to obtain the clustering soft label matrix; Step 3.4: Repeat steps 3.1-3.4 for iterative training until convergence, obtaining the trained redundancy detection model. The trained redundancy detection model is represented by the following formula: ， In the formula, Indicates the first i Perceptual data, Indicates the first i Clustering labels for perceptual data.

7. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the crowd intelligence sensing data dynamic redundancy detection method based on contrastive learning as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the dynamic redundancy detection method for crowd-aware data based on contrastive learning as described in any one of claims 1 to 5.

Images