Public opinion data key sample extraction method and system based on cross-modal alignment, medium and product

Abstract

The invention discloses a public opinion data key sample extraction method and system based on cross-modal alignment, a medium and a product, and relates to the field of public opinion data processing, and the method comprises the steps: obtaining a multi-modal public opinion data set of a to-be-processed public opinion event; carrying out cross-modal public opinion information alignment and coding on the multi-modal public opinion data set by utilizing an ImageBind cross-modal coding technology to obtain a joint representation of each public opinion sample; and according to the joint representation of each public opinion sample, performing modal perception key sample screening on the multi-modal public opinion data set through an improved K-Center-Greedy algorithm to obtain a multi-modal public opinion key sample data set. According to the method, accurate screening of key samples in mass multi-modal public opinion data can be realized, and the problems of redundancy of multi-modal data, difficulty in semantic alignment and low screening efficiency of mass information are solved.

Description

Technical Field

[0001] The present invention relates to the field of sample data processing technology, and in particular to a method, system, medium and product for extracting key samples of public opinion data based on cross-modal alignment. Background Art

[0002] With the rapid development of the internet and social media, public opinion data has become multimodal, massive, and significantly more complex. Public opinion data typically contains information in multiple modalities, including text, images, audio, and video. These diverse modalities together constitute a complete description of public opinion events. In the field of public opinion monitoring and analysis, selecting the most representative key samples from massive amounts of multimodal data has become a key challenge in improving the efficiency and accuracy of public opinion analysis. Effectively selected key public opinion samples have a wide range of applications: they can be used to analyze public opinion trends, rapidly constructing event development context and opinion distribution maps; support public opinion risk warnings by identifying early signs of public opinion risk through abnormal features in key samples; improve the efficiency of in-depth content analysis, significantly reducing analysis costs while ensuring quality; optimize response strategy formulation by tailoring differentiated responses to representative opinions; provide high-quality training data for intelligent public opinion analysis models, enabling efficient allocation of annotation resources; and support public opinion policy research and decision-making by identifying key factors influencing public perception.

[0003] At present, the core challenges facing multimodal public opinion analysis are: (1) how to effectively integrate public opinion information of different modalities in a unified semantic space; (2) in public opinion data, a small amount of hot information is often repeated in large quantities, while key details are sparsely distributed. How to select the most representative key samples from massive redundant data is a key challenge; (3) the uneven distribution of different modalities in the original data set affects the accuracy of downstream analysis tasks. Summary of the Invention

[0004] In order to solve the technical problems existing in the background technology, the present invention proposes a method, system, medium and product for extracting key samples of public opinion data based on cross-modal alignment.

[0005] In a first aspect, the present invention proposes a method for extracting key samples of public opinion data based on cross-modal alignment, comprising:

[0006] Obtain a multimodal public opinion dataset of the public opinion events to be processed;

[0007] ImageBind cross-modal encoding technology is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset to obtain a joint representation of each public opinion sample;

[0008] According to the joint representation of each public opinion sample, the multimodal public opinion dataset is screened for modality-aware key samples through the improved K-Center-Greedy algorithm to obtain a multimodal public opinion key sample dataset.

[0009] Preferably, the multimodal public opinion dataset includes multiple public opinion samples, each of which includes one or more modal information of text, image, audio and video content.

[0010] Preferably, the ImageBind cross-modal encoding technology is used to align and encode the multimodal public opinion data set to obtain a joint representation of each public opinion sample, specifically including:

[0011] ImageBind cross-modal encoding technology is used to map the public opinion information of different modalities of each public opinion sample in the multimodal public opinion dataset into a unified semantic space, thus achieving cross-modal public opinion information alignment of each public opinion sample;

[0012] The public opinion information of each public opinion sample in a unified semantic space is encoded through a pre-trained encoder group to generate a joint representation of each public opinion sample.

[0013] Preferably, Where, d i is the i-th public opinion sample, h i is the joint representation of the i-th public opinion sample, E is the encoder group, M i is the modal set of the i-th public opinion sample, k is the index of the modality, E k (·) is the pre-trained encoder of the k-th modality, and the representation of the i-th public opinion sample on the k-th modality.

[0014] Preferably, based on the joint representation of each public opinion sample, the multimodal public opinion dataset is subjected to modality-aware key sample screening by an improved K-Center-Greedy algorithm to obtain a key sample dataset, specifically including:

[0015] The modal combination of each public opinion sample is defined as a modal combination category label, and the modal distribution ratio of each modal combination category label in the multimodal public opinion dataset is statistically calculated;

[0016] Based on the preset initialization center calculation formula, the initialization center of the multimodal public opinion key sample set is calculated;

[0017] Iteratively update the multimodal public opinion key sample set;

[0018] Based on the updated multimodal public opinion key sample set, determine whether the preset termination conditions are met;

[0019] If so, we obtain a multimodal public opinion key sample set;

[0020] If not, continue to iteratively update the multimodal public opinion key sample set until the preset termination condition is met.

[0021] Preferably, the initialization center calculation formula is

[0022]

[0023] Where d1 is the initialization center, c(d i ) is the public opinion sample d i The modal combination category; α is the control modal balance strength, α≥0, when α=0, it degenerates to the original algorithm, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion sample, is the modal distribution ratio of the modal combination category c where the i-th public opinion sample belongs, and D is the multimodal public opinion dataset;

[0024] in,

[0025] In the formula, C is the category label of all modal combinations, c is the category label of a modal combination in C, and p c is the modal distribution ratio of c, d i is the i-th public opinion sample, and D is the multimodal public opinion dataset;

[0026] in,

[0027] Where, d t+1 is the multimodal public opinion key sample set S t The t+1th public opinion key sample, t=1,2,3,…,k-1, k <n; represents the modal compensation factor;

[0028] in, Where, is the multimodal public opinion key sample set S t Mode distribution ratio of mode combination c, S t is the multimodal public opinion key sample set, d j is the jth public opinion key sample in the multimodal public opinion key sample set.

[0029] Preferably, the preset termination condition is an adaptive termination condition or the number of iterations reaches a preset number of iterations.

[0030] Preferably, the adaptive termination condition is: when the distances of all public opinion samples in the multimodal public opinion data set to the closest public opinion key samples in the multimodal public opinion key sample set are less than or equal to the first adaptive threshold and the modal distribution similarity measure is less than or equal to the second adaptive threshold, the iterative update extraction process terminates.

[0031] Preferably, ∈ t =μ t +σ t ; In the formula, ∈ t is the first adaptive threshold, μ t is the average distance between all public opinion samples in the multimodal public opinion dataset and the nearest multimodal public opinion key sample in the multimodal public opinion key sample set, σ t It is the standard deviation of the distance between all samples in the multimodal public opinion dataset and the closest key public opinion sample in the multimodal public opinion key sample set.

[0032] Preferably,

[0033]

[0034] Where, dist i Represents the closest public opinion key sample d from the i-th public opinion sample in the multimodal public opinion dataset to the multimodal public opinion key sample set j The distance, S t represents the multimodal public opinion key sample set, D is the multimodal public opinion dataset, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion key sample, ‖ ‖2 represents the L2 norm;

[0035] in, Where D dist (t) is the modal distribution similarity measure, C is the category label of all modal combinations, and c is a modal combination category label in C. represents the modal compensation factor.

[0036] Preferably, Where δ0 is the initial threshold, δ t is the second adaptive threshold, |S t | represents the number of key public opinion samples in the multimodal key public opinion sample set.

[0037] In a second aspect, the present invention also proposes a system for extracting key samples of public opinion data based on cross-modal alignment, comprising:

[0038] The acquisition module is used to obtain the multimodal public opinion dataset of the public opinion event to be processed;

[0039] The alignment and encoding module is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset using ImageBind cross-modal encoding technology to obtain a joint representation of each public opinion sample;

[0040] The screening module is used to screen the modality-aware key samples of the multimodal public opinion dataset through the improved K-Center-Greedy algorithm based on the joint representation of each public opinion sample to obtain a multimodal public opinion key sample dataset.

[0041] In the third aspect, the present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of the first aspects are implemented.

[0042] In a fourth aspect, the present invention further proposes a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of the first aspects.

[0043] In the present invention, the proposed method, system, medium and product for extracting key samples of public opinion data based on cross-modal alignment, through the combination of ImageBind cross-modal encoding technology and the improved K-Center-Greedy algorithm, perform unified semantic space mapping and modal distribution representation on multimodal public opinion data (text, image, audio, video, etc.), so that the obtained multimodal public opinion key sample data set can cover the semantic space of the entire data set to the greatest extent, while maintaining modal distribution consistency, realizing accurate screening of key samples in massive multimodal public opinion data, and solving the problems of multimodal data redundancy, semantic alignment difficulties and low efficiency in screening massive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a method for extracting key samples of public opinion data based on cross-modal alignment proposed in the present invention.

[0045] Figure 2 This is a flow chart of the improved K-Center-Greedy algorithm for extracting key samples of public opinion data based on cross-modal alignment proposed in the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] First, as Figure 1 and Figure 2As shown, the present invention proposes a method for extracting key samples of public opinion data based on cross-modal alignment, including:

[0048] S1. Obtain a multimodal public opinion dataset of the public opinion event to be processed;

[0049] S2. Use ImageBind cross-modal encoding technology to align and encode the multimodal public opinion data set to obtain a joint representation of each public opinion sample;

[0050] S3. Based on the joint representation of each public opinion sample, the multimodal public opinion dataset is screened for modality-aware key samples using the improved K-Center-Greedy algorithm to obtain a multimodal public opinion key sample dataset.

[0051] In the specific implementation, the ImageBind cross-modal encoding technology is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset to obtain a joint representation of each public opinion sample, thereby achieving cross-modal semantic consistency; and based on the joint representation of each public opinion sample, the improved K-Center-Greedy algorithm is used to screen the modality-aware key samples of the multimodal public opinion dataset to obtain a multimodal public opinion key sample dataset, so that the multimodal public opinion key sample dataset can cover the semantic space of the entire dataset to the greatest extent while maintaining modal distribution consistency.

[0052] It's important to note that ImageBind primarily focuses on the correspondence between modalities, without considering how to extract key samples from massive amounts of data. In particular, it lacks a mechanism for preserving the modal distribution characteristics of the original data. While the K-Center-Greedy algorithm has excelled in many fields, its application in public opinion analysis, particularly in multimodal data processing, has yet to be fully explored. Furthermore, the traditional K-Center-Greedy algorithm doesn't consider the distribution characteristics of different modal combinations, which can lead to deviations in the modal distribution of the selected results from that of the original dataset. Furthermore, the algorithm lacks an adaptive termination mechanism, making it difficult to dynamically adjust the sample selection process based on data characteristics.

[0053] The present invention combines ImageBind cross-modal encoding technology with an improved K-Center-Greedy algorithm to perform unified semantic space mapping and modal distribution representation on multimodal public opinion data (text, images, audio, video, etc.), so that the obtained multimodal public opinion key sample data set covers the semantic space of the entire data set to the greatest extent while maintaining modal distribution consistency, realizing accurate screening of key samples in massive multimodal public opinion data, and solving the problems of multimodal data redundancy, difficulty in semantic alignment, and low efficiency in screening massive information.

[0054] It should be understood that the public opinion event E to be processed in this embodiment is, for example, a public emergency, a hot social topic, or a brand crisis event.

[0055] The multimodal public opinion dataset in this embodiment includes multiple public opinion samples. The multimodal public opinion dataset (denoted as D) can be expressed as D = {d1, d2, ..., d n}, i=1,2,3,…,n; These represent the different modalities of the text, image, audio, and video content of the i-th public opinion sample, respectively. It should be noted that in actual public opinion data, each piece of data typically only contains a subset of modalities, such as a plain text microblog, a combined text and image post, or a video comment.

[0056] Therefore, each public opinion sample d in this embodiment i Includes one or more modal information including text, images, audio and video content.

[0057] In this embodiment, in S2, the ImageBind cross-modal encoding technology is used to perform cross-modal public opinion information alignment and encoding on the multimodal public opinion dataset of the public opinion event to be processed, and a joint representation of each public opinion sample is obtained, which specifically includes:

[0058] The ImageBind cross-modal encoding technology is used to map the public opinion information of different modalities of each public opinion sample in the multimodal public opinion dataset into a unified semantic space, and the pre-trained encoder group E = {E t ,E i ,E a ,E v}Generate a joint representation of each public opinion sample.

[0059] in,

[0060] Where, d i is the i-th public opinion sample, h i is the joint representation of the i-th public opinion sample, E is the encoder group, M i Public opinion sample d i The existing modal set (e.g. a piece of public opinion information contains both {text, image} modalities), k is the index of the modality, E k (·) is the pre-trained encoder for the k-th modality, is the i-th public opinion sample d i Representation on the k-th mode.

[0061] All modal features of each public opinion sample in this embodiment It needs to undergo L2 normalization before participating in the fusion, which can ensure that different public opinion modalities have the same weight in the joint representation, thereby achieving cross-modal alignment and unified representation of public opinion information.

[0062] The pre-trained encoder in this embodiment is a ViT or CLIP text encoder.

[0063] In the process of mapping the public opinion information of different modes in the multimodal public opinion dataset into a unified semantic space, a unified semantic mapping function family is first established through the ImageBind technology; wherein, the semantic mapping function family includes: f t 、f g 、f a and f v ; Map different modal features to a unified semantic space S∈R d , making This achieves semantic consistency across modalities.

[0064] Compared with traditional single-modal or simple multimodal fusion methods, the ImageBind cross-modal encoding technology in this embodiment can more effectively capture cross-modal semantic associations, and by maintaining the original modal distribution, ensure that public opinion information in different forms of dissemination is represented according to its actual importance, significantly improving the accuracy of public opinion analysis results.

[0065] In this embodiment, based on the joint representation of each public opinion sample, the multimodal public opinion dataset is subjected to modality-aware key sample screening by the improved K-Center-Greedy algorithm to obtain a key sample dataset, specifically including:

[0066] The multimodal public opinion dataset is iteratively updated and extracted through the improved K-Center-Greedy algorithm until the preset termination conditions are met, and the multimodal public opinion key sample set is obtained. This enables the multimodal public opinion key sample set to cover the semantic space of the entire multimodal public opinion dataset to the greatest extent while maintaining the consistency of the modal distribution ratio.

[0067] When extracting key samples from multimodal public opinion, it's crucial to fully consider the modal distribution characteristics of the original data. Maintaining the modal distribution consistency between the key sample set and the original dataset not only ensures sample representativeness and avoids modal bias, where certain modalities are overrepresented or completely ignored, but also significantly improves the generalization ability of the extracted samples in downstream public opinion analysis tasks.

[0068] The preset termination condition in this embodiment is an adaptive convergence condition or the number of iterations reaches a preset number of iterations.

[0069] In order to ensure that the multimodal public opinion key sample set can cover the semantic space of the entire multimodal public opinion dataset to the greatest extent possible while maintaining the consistency of the modal distribution ratio, during the iterative update extraction process, the modal combination of each public opinion sample is defined as a modal combination category label, and the modal distribution ratio of each modal combination category label in the multimodal public opinion dataset is statistically calculated;

[0070] Then, based on the preset initialization center calculation formula, the initialization center of the multimodal public opinion key sample set is calculated;

[0071] Then, iteratively update the multimodal public opinion key sample set;

[0072] Based on the updated multimodal public opinion key sample set, determine whether the preset termination conditions are met;

[0073] If so, a multimodal public opinion key sample set is obtained; if not, the multimodal public opinion key sample set is iteratively updated until the preset termination condition is met.

[0074] In the online public opinion environment, information is transmitted in the form of multiple modal combinations. In this embodiment, the modal combination of each public opinion sample is defined as a category label (such as T for plain text microblogs, T+G for graphic comments, etc.), and the modal distribution ratio of the category label of each modal combination in the original multimodal public opinion dataset is calculated:

[0075] in,

[0076] In the formula, C is the category label of all modal combinations, c is the category label of a modal combination in C, and p c is the modal distribution ratio of c, d i is the i-th public opinion sample, and D is the multimodal public opinion dataset.

[0077] The improved K-Center-Greedy algorithm proposed in this example effectively addresses the long-tail effect of public opinion data. By dynamically adjusting the balancing factor, it allows the retention of small but potentially important public opinion signals in the long-tail region according to their actual importance. This provides a reliable data foundation for public opinion early warning and risk monitoring, significantly improving the foresight and comprehensiveness of public opinion analysis.

[0078] In the iterative update extraction, the modal distribution ratio of the multimodal public opinion key sample set is dynamically monitored, and the selected multimodal public opinion key sample set S is recorded in real time. t The modal distribution ratio of each modal combination in:

[0079] in, Where, is the multimodal public opinion key sample set S t Mode distribution ratio of mode combination c, S tis the multimodal public opinion key sample set, d j is the jth public opinion key sample in the multimodal public opinion key sample set.

[0080] The calculation formula for the initialization center is:

[0081]

[0082] Where d1 is the initialization center, c(d i ) is the public opinion sample d i The modal combination category; α is the control modal balance strength, α≥0, when α=0, it degenerates to the original algorithm, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion sample, is the modal distribution ratio of the modal combination category c where the i-th public opinion sample is located, and D is the multimodal public opinion dataset.

[0083] This embodiment is configured in such a way that public opinion samples with unique semantic features in high-influence modal categories (such as popular communication forms) can be preferentially selected as initial representative points, avoiding the concentration of key samples in niche communication forms and ensuring the capture of mainstream public opinion trends.

[0084] in,

[0085] Where, d t+1 is the multimodal public opinion key sample set S t The t+1th public opinion key sample, represents the modal compensation factor.

[0086] If the category c(d i ) is lower than the original proportion Automatically increase the selection priority of the public opinion samples of this modality to ensure that the key public opinion samples finally selected can fully reflect the voices of various communication channels.

[0087] This method uses modal distribution characteristics as a core constraint for key sample selection, breaking through the limitations of traditional methods that focus solely on feature space coverage. By introducing a modal balance factor and a dynamic compensation mechanism, it ensures that the selected samples accurately reflect the modal distribution characteristics of the original public opinion data while maintaining semantic coverage. This effectively addresses the modal bias problem in existing technologies and makes public opinion analysis more comprehensive and objective.

[0088] In a further embodiment, the adaptive termination condition is: if the distances of all public opinion samples in the multimodal public opinion data set to the closest public opinion key sample in the multimodal public opinion key sample set are less than or equal to the first adaptive threshold and the modal distribution similarity measure is less than or equal to the second adaptive threshold, then the adaptive termination condition is satisfied. and D dist (t)≤δ t When , all samples are considered to be effectively covered and the iterative process terminates; among them, dist i Represents the i-th public opinion sample in the multimodal public opinion dataset and the multimodal public opinion key sample set S t The closest public opinion key sample d j The distance between t represents the first adaptive threshold, D dist (t) is the modal distribution similarity measure, δ t is the second adaptive threshold.

[0089] The design of the adaptive termination condition in this embodiment integrates semantic coverage and modal distribution consistency, achieving dynamic determination of the sample size. Compared with the fixed-number sampling method, this mechanism can intelligently adjust the sampling amount based on the actual distribution characteristics of the public opinion data, avoiding information redundancy while ensuring that key public opinion signals are fully captured, greatly improving the efficiency of public opinion monitoring resource utilization. In other words, based on the dual indicators of semantic coverage and modal distribution consistency, the sample size is effectively controlled while ensuring screening quality.

[0090] The calculation process of the first adaptive threshold includes:

[0091] Calculate the distance set from all public opinion samples in the multimodal public opinion dataset to the closest public opinion key sample in the multimodal public opinion key sample set; Where, dist i Represents the i-th public opinion sample in the multimodal public opinion dataset D to the multimodal public opinion key sample set S t The closest public opinion key sample d j The distance, S t represents the multimodal public opinion key sample set, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion key sample, ‖ ‖2 represents the L2 norm;

[0092] Calculate the average distance between all public opinion samples in the multimodal public opinion dataset and the closest public opinion key sample in the multimodal public opinion key sample set; Where μ t All public opinion samples in the multimodal public opinion dataset D are converted to the multimodal public opinion key sample set St The average distance between the closest key public opinion samples in ;

[0093] Calculate the standard deviation of the distance between all public opinion samples in the multimodal public opinion dataset and the closest public opinion key sample in the multimodal public opinion key sample set; Where σ t All public opinion samples in the multimodal public opinion dataset D are converted to the multimodal public opinion key sample set S t The standard deviation of the distance between the closest key public opinion samples;

[0094] According to the mean value and standard deviation, the first adaptive threshold is calculated; where ∈ t =μ t +σ t ; In the formula, ∈ t is the first adaptive threshold.

[0095] In order to constrain the distribution consistency of public opinion dissemination forms, in this embodiment, the modal distribution similarity measure is defined as follows:

[0096] Where D dist (t) is the modal distribution similarity measure, C is the category label of all modal combinations, and c is a modal combination category label in C. represents the modal compensation factor.

[0097] The modality compensation factor in this embodiment may be a positive number (i.e., the proportion of the modality in the currently selected sample is lower than its proportion in the original data) (priority increases, more likely to be selected) or a negative number (the possibility of selection decreases).

[0098] During the update process of the multimodal public opinion key sample set, this embodiment dynamically adjusts the sample selection priority through the modal compensation factor, that is, according to the difference between the modal distribution of the current sample set and the modal distribution of the original data.

[0099] What you need to know is that D dist (t), which is the standardized L1 distance, is used to measure the degree of difference in the distribution of propagation forms between the screened samples and the original public opinion data. The range is between [0,1], where 0 means the distribution is completely consistent and 1 means completely different.

[0100] in, In the formula, δ0 is the initial threshold. As the number of samples increases, the required distribution similarity increases, and δ t is the second adaptive threshold, |S t | represents the number of key public opinion samples in the multimodal public opinion key sample set.

[0101] In one specific embodiment, δ0 is set to 0.5.

[0102] In summary, the present invention solves the modal imbalance problem in the existing extraction of key public opinion samples, provides a more reliable data basis for multimodal public opinion analysis, effectively improves the comprehensiveness, accuracy and efficiency of public opinion monitoring, analysis and early warning, and has technological innovation and practical application value.

[0103] In a second aspect, the present invention proposes a system for extracting key samples of public opinion data based on cross-modal alignment, comprising:

[0104] The acquisition module is used to obtain the multimodal public opinion dataset of the public opinion event to be processed;

[0105] The alignment and encoding module is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset using ImageBind cross-modal encoding technology to obtain a joint representation of each public opinion sample;

[0106] The screening module is used to screen the modality-aware key samples of the multimodal public opinion dataset through the improved K-Center-Greedy algorithm based on the joint representation of each public opinion sample to obtain a multimodal public opinion key sample dataset.

[0107] In the third aspect, the present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of the first aspects are implemented.

[0108] In a fourth aspect, the present invention further proposes a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of the first aspects.

[0109] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for extracting key samples from public opinion data based on cross-modal alignment, characterized by: include: Obtain a multimodal public opinion dataset of the public opinion events to be processed; ImageBind cross-modal encoding technology is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset to obtain a joint representation of each public opinion sample; According to the joint representation of each public opinion sample, the multimodal public opinion dataset is screened for modality-aware key samples through the improved K-Center-Greedy algorithm to obtain a multimodal public opinion key sample dataset.

2. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 1 is characterized in that: The multimodal public opinion dataset includes multiple public opinion samples, each of which includes one or more modal information in text, image, audio and video content.

3. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 1 is characterized in that: ImageBind cross-modal encoding technology is used to align and encode the multimodal public opinion data set to obtain a joint representation of each public opinion sample, including: ImageBind cross-modal encoding technology is used to map the public opinion information of different modalities of each public opinion sample in the multimodal public opinion dataset into a unified semantic space, thus achieving cross-modal public opinion information alignment of each public opinion sample; The public opinion information of each public opinion sample in the unified semantic space is encoded by the pre-trained encoder group to generate a joint representation of each public opinion sample: in, Where, d i is the i-th public opinion sample, h i is the joint representation of the i-th public opinion sample, E is the encoder group, M i is the modal set of the i-th public opinion sample, k is the index of the modality, E k (·) is the pre-trained encoder for the k-th modality, is the representation of the i-th public opinion sample on the k-th mode.

4. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 1 is characterized in that: According to the joint representation of each public opinion sample, the improved K-Center-Greedy algorithm is used to screen modality-aware key samples of the multimodal public opinion dataset to obtain the key sample dataset, which specifically includes: The modal combination of each public opinion sample is defined as a modal combination category label, and the modal distribution ratio of each modal combination category label in the multimodal public opinion dataset is calculated; Based on the preset initialization center calculation formula, the initialization center of the multimodal public opinion key sample set is calculated; Iteratively update the multimodal public opinion key sample set; Based on the updated multimodal public opinion key sample set, determine whether the preset termination conditions are met; If so, we obtain a multimodal public opinion key sample set; If not, continue to iteratively update the multimodal public opinion key sample set until the preset termination condition is met.

5. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 4 is characterized in that: The initialization center calculation formula is: Where d1 is the initialization center, c(d i ) is the public opinion sample d i The modal combination category; α is the control modal balance strength, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion sample, is the modal distribution ratio of the modal combination category c where the i-th public opinion sample belongs, and D is the multimodal public opinion dataset; in, In the formula, C is the category label of all modal combinations, c is the category label of a modal combination in C, and p c is the modal distribution ratio of c, d i is the i-th public opinion sample, and D is the multimodal public opinion dataset; in, Where, d t+1 is the multimodal public opinion key sample set S t The t+1th public opinion key sample, t=1,2,3,…,k-1, k <n; represents the modal compensation factor; in, Where, is the modal distribution ratio of the modal combination c in the multimodal public opinion key sample set, S t is the multimodal public opinion key sample set, d j is the jth public opinion key sample in the multimodal public opinion key sample set.

6. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 4 is characterized in that: The preset termination condition is an adaptive termination condition or the number of iterations reaches a preset number of iterations.

7. The method for extracting key samples of public opinion data based on cross-modal alignment according to claim 6 is characterized in that: The adaptive termination condition is: when the distances of all public opinion samples in the multimodal public opinion dataset to the closest public opinion key sample in the multimodal public opinion key sample set are less than or equal to the first adaptive threshold and the modal distribution similarity measure is less than or equal to the second adaptive threshold, the iterative update extraction process terminates; Preferably, ∈ t =μ t +σ t ; In the formula, ∈ t is the first adaptive threshold, μ t is the average distance between all public opinion samples in the multimodal public opinion dataset and their closest key public opinion samples, σ t The standard deviation of the distance between all public opinion samples in the multimodal public opinion dataset and their closest key public opinion samples; in, Where, dist i represents the distance between the i-th public opinion sample in the multimodal public opinion dataset and the closest public opinion key sample in the multimodal public opinion key sample set, S t represents the multimodal public opinion key sample set, D is the multimodal public opinion dataset, h i is the joint representation of the i-th public opinion sample, h j is the joint representation of the j-th public opinion key sample; ‖‖2 represents the L2 norm; in, Where D dist (t) is the modal distribution similarity measure, all modal combination category labels, c is a modal combination category label in C, represents the modal compensation factor. in, Where δ0 is the initial threshold, δ t is the second adaptive threshold, |S t | represents the number of key public opinion samples in the multimodal key public opinion sample set.

8. A system for extracting key samples of public opinion data based on cross-modal alignment, characterized by: include: The acquisition module is used to obtain the multimodal public opinion dataset of the public opinion event to be processed; The alignment and encoding module is used to align and encode the cross-modal public opinion information of the multimodal public opinion dataset using ImageBind cross-modal encoding technology to obtain a joint representation of each public opinion sample; The screening module is used to screen the modality-aware key samples of the multimodal public opinion dataset through the improved K-Center-Greedy algorithm based on the joint representation of each public opinion sample to obtain a multimodal public opinion key sample dataset.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for extracting key samples of public opinion data based on cross-modal alignment as described in any one of claims 1 to 7 are implemented.

Images