A privacy protection system and method in network multi-modal sentiment analysis based on quantum key distribution

Abstract

The application discloses a privacy protection system and method in network multi-modal sentiment analysis based on quantum key distribution, which comprises the following steps: 1) collecting multi-modal data, and performing feature extraction and sentiment state analysis to obtain a sentiment score reflecting the sentiment tendency of a user; 2) based on a symmetric encryption algorithm AES, using a quantum key to encrypt the multi-modal data and the sentiment score to generate ciphertext; 3) using a traditional communication channel to transmit the ciphertext; and 4) a receiver uses a shared quantum key to decrypt the ciphertext to restore the original data. The system comprises an authentication center management unit, a multi-modal sentiment analysis unit, a quantum key preparation unit, a data encryption transmission unit and a data decryption unit. The application encrypts data through a quantum key, guarantees the confidentiality and integrity of the data, effectively resists the security threat brought by quantum computing, and provides a highly secure privacy protection solution for network video sentiment analysis.

Description

Technical Field

[0001] This invention relates to the field of privacy protection technology in network multimodal sentiment analysis, specifically a privacy protection system and method for network multimodal sentiment analysis based on quantum key distribution. Background Technology

[0002] Multimodal sentiment analysis is a technology that uses data from multiple modalities (such as text, audio, and video) to analyze and identify users' emotional states. It accurately captures and interprets the emotional expressions of the observed individual by integrating, processing, and aligning data from different sources, using deep learning-based data analysis and prediction. This analytical method can be applied to various sub-scenarios requiring emotional needs analysis while maintaining data confidentiality, such as remote online work, online job and school interviews, classroom learning effectiveness evaluation, prison monitoring, and important government meetings in fields like enterprise management, education, judicial governance, and national defense. For example, in judicial interrogations, multimodal sentiment analysis can assist in analyzing the emotional state of suspects or witnesses, assessing the credibility of their statements through nonverbal cues; however, such scenarios also require strict confidentiality of the signals collected from each channel to protect the suspect's privacy.

[0003] Multimodal sentiment analysis is an end-to-end process. It first requires collecting multimodal data containing emotional expressions, including video, audio, and text. Specifically, this involves using a camera to capture real-time facial video of the user to obtain visual data, using a microphone to capture real-time speech to obtain speech data, and then using speech recognition technology to transcribe the speech into text to obtain text data. Next, facial features such as facial landmarks and head pose are extracted using tools like OpenFace; speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q-transform (CQT), and logarithmic fundamental frequency are extracted using tools like Librosa; the Whisper deep learning model is used to transcribe the speech into text information, and then the BERT model is used to convert the text information into vector representations and extract text features. These structurally dissimilar multidimensional feature data are then integrated and aligned to obtain a complete set of sentiment feature information that can be used to train a sentiment analysis neural network model. This set of sentiment feature information is transmitted to the host machine at the analysis end via the internet. Upon receiving the sentiment information set, the analysis end constructs a model, designing a multimodal sentiment recognition model that includes a feature embedding subnetwork, a multi-head self-attention mechanism, a tensor fusion-based feature fusion network, and a sentiment analysis network. The model analyzes the relationships between features across different modalities, concatenates and linearly transforms the attention distributions of multiple subspaces to obtain the final fused features. Finally, through a sentiment analysis network, the model outputs a sentiment score that reflects the emotional tendency of the observed individual, such as positive, negative, or neutral.

[0004] In the above process, the set of emotional feature information will be transmitted to the host of the analysis terminal via the Internet, which poses a challenge to privacy protection. Summary of the Invention

[0005] The purpose of this invention is to provide a privacy protection system for network multimodal sentiment analysis based on quantum key distribution, including a sender and a receiver;

[0006] The transmitting end includes an authentication center management unit, a multimodal emotion analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0007] The authentication center management unit is used for identity verification and data transmission security authentication. If the authentication is successful, it sends an authentication success signal to the multimodal sentiment analysis unit.

[0008] After receiving the authentication success signal, the multimodal sentiment analysis unit collects and processes multimodal data to obtain sentiment feature information. Then, it uses a multimodal sentiment recognition model to analyze the sentiment feature information and obtain a sentiment score that reflects the user's sentiment tendency.

[0009] The multimodal sentiment analysis unit transmits multimodal data and sentiment scores to the data encryption transmission unit;

[0010] The quantum key preparation unit generates a quantum key and transmits it to the quantum key verification unit at the receiving end;

[0011] The data encryption transmission unit uses quantum keys to encrypt multimodal data and sentiment scores, and then transmits the data to the receiving end through a traditional channel.

[0012] The receiving end includes a quantum key verification unit and a data decryption unit;

[0013] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0014] The data decryption unit decrypts the received data using a quantum key.

[0015] Furthermore, the certification center is used to issue digital certificates to both communicating parties to ensure the authenticity of their identities and to authenticate the communication channel, thereby guaranteeing the security of data transmission.

[0016] Furthermore, the multimodal data includes user facial video data and user voice data;

[0017] User facial video data is collected via camera, and user voice data is collected via microphone.

[0018] Furthermore, the steps for the multimodal sentiment analysis unit to process multimodal data include:

[0019] Visual features, including facial landmarks and head pose, were extracted using the OpenFace tool.

[0020] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0021] The Whisper deep learning model is used to transcribe user speech data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0022] Furthermore, the steps for analyzing emotional feature information using a multimodal emotion recognition model include:

[0023] a1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information, and transmit it to the analysis end;

[0024] a2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between various modal features, splices and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0025] Furthermore, the modal sentiment recognition model includes a feature embedding subnetwork, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

[0026] Furthermore, the steps for generating quantum keys in the quantum key preparation unit include:

[0027] b1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel; the quantum state of the qubit sequence is the polarization state of the photon; the polarization state of the photon is generated by generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon with a polarizer to obtain the polarization state of the photon.

[0028] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0029] b3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0030] b4) Determine if there is any eavesdropping. If so, discard the current key distribution process and regenerate a new key.

[0031] Furthermore, quantum states include horizontally polarized states, vertically polarized states, 45-degree polarized states, and 135-degree polarized states;

[0032] The horizontal polarization state is denoted as |0> and assigned a value of 0;

[0033] The vertical polarization state is denoted as |1> and assigned a value of 1;

[0034] A 45-degree polarization state is denoted as |+> and assigned a value of 1;

[0035] A polarization state of 135 degrees is denoted as |-> and assigned a value of 0.

[0036] The privacy protection method based on the system includes the following steps:

[0037] 1) Verify the identities of both communicating parties through an authentication center;

[0038] 2) Collect multimodal data, perform feature extraction and sentiment analysis, and obtain sentiment scores that reflect users' emotional tendencies;

[0039] The multimodal data includes user facial video data, user voice data, and text data converted from the voice data;

[0040] The steps for feature extraction include: using OpenFace tools to extract visual features, including facial key points and head pose;

[0041] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0042] The Whisper deep learning model is used to transcribe speech into text information, and then the BERT model is used to convert the text information into vector representation and extract text features.

[0043] 3) Generate a random bit sequence using a quantum true random number generator, and transmit the encoded quantum state through a quantum channel to prepare a quantum key. The steps include:

[0044] 3.1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel;

[0045] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received qubit sequence;

[0046] 3.3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0047] 3.4) Determine if eavesdropping occurs; if so, discard the current key and generate a new key.

[0048] 4) Based on the symmetric encryption algorithm AES, quantum key distribution is used to encrypt multimodal data and sentiment scores to generate ciphertext;

[0049] 5) Transmit encrypted messages using traditional communication channels;

[0050] 6) The receiver uses the shared quantum key to decrypt the ciphertext and recover the original data.

[0051] Furthermore, in step 3.3), the step of extracting the corresponding bit sequence according to the preset selection rules is as follows:

[0052] 3.3.1) Generate a binary true random number sequence using quantum mechanics principles, and use it as a measurement basis sequence;

[0053] 3.3.2) Randomly select from the measurement base sequence base or Basis, as a measurement basis;

[0054] Two sets of measurement bases measure the received quantum state, and the measurement bases allow photons with the same polarization direction to pass through;

[0055] If the basis chosen for preparing the quantum state is consistent with the basis chosen during measurement, it is recorded as a correct measurement basis bit; otherwise, the corresponding bit data is deleted. A subset of positions is randomly selected from the correct measurement basis bits and recorded as random bits. For each random bit, if the measurement result matches the sender's result, it is considered that there was no eavesdropping.

[0056] The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

[0057] The technical effectiveness of this invention is undeniable. It generates and securely distributes secure keys through QKD (Quantum Key Decoding), utilizes multimodal sentiment analysis technology to classify and assess emotional states by combining video content and voice information, and enhances the security of traditional communication channels through a Certificate Authority (CA). By encrypting data with quantum keys, even when transmitted over insecure networks, the confidentiality and integrity of the data are guaranteed, effectively resisting the security threats posed by quantum computing. This provides a highly secure privacy protection solution for online video sentiment analysis. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the overall invention;

[0059] Figure 2 This is an overall flowchart of the present invention;

[0060] Figure 3 This is a system block diagram of the multimodal sentiment analysis unit in this invention. It describes the data processing flow of multimodal sentiment analysis, including data acquisition, feature extraction, sentiment state classification, and result output.

[0061] Figure 4 The flowchart of a quantum key distribution unit includes the encoding, transmission, measurement, and key generation of quantum states. Detailed Implementation

[0062] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0063] Example 1:

[0064] See Figures 1 to 4 A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, comprising a sender and a receiver;

[0065] The transmitting end includes an authentication center management unit, a multimodal emotion analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0066] The authentication center management unit is used for identity verification and data transmission security authentication. If the authentication is successful, it sends an authentication success signal to the multimodal sentiment analysis unit.

[0067] After receiving the authentication success signal, the multimodal sentiment analysis unit collects and processes multimodal data to obtain sentiment feature information. Then, it uses a multimodal sentiment recognition model to analyze the sentiment feature information and obtain a sentiment score that reflects the user's sentiment tendency.

[0068] The multimodal sentiment analysis unit transmits multimodal data and sentiment scores to the data encryption transmission unit;

[0069] The quantum key preparation unit generates a quantum key and transmits it to the quantum key verification unit at the receiving end;

[0070] The data encryption transmission unit uses quantum keys to encrypt multimodal data and sentiment scores, and then transmits the data to the receiving end through a traditional channel.

[0071] The receiving end includes a quantum key verification unit and a data decryption unit;

[0072] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0073] The data decryption unit decrypts the received data using a quantum key.

[0074] The certification authority is used to issue digital certificates to both parties in a communication to ensure the authenticity of their identities and to authenticate the communication channel, thereby guaranteeing the security of data transmission.

[0075] The multimodal data includes user facial video data and user voice data;

[0076] User facial video data is collected via camera, and user voice data is collected via microphone.

[0077] The steps involved in processing multimodal data by the multimodal sentiment analysis unit include:

[0078] Visual features, including facial landmarks and head pose, were extracted using the OpenFace tool.

[0079] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0080] The Whisper deep learning model is used to transcribe user speech data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0081] The steps for analyzing sentiment feature information using a multimodal sentiment recognition model include:

[0082] a1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information, and transmit it to the analysis end;

[0083] a2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between various modal features, splices and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0084] Modal sentiment recognition models include feature embedding subnetworks, multi-head self-attention mechanisms, tensor fusion-based feature fusion networks, and sentiment analysis networks.

[0085] The steps of generating a quantum key in a quantum key preparation unit include:

[0086] b1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel; the quantum state of the qubit sequence is the polarization state of the photon; the polarization state of the photon is generated by generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon with a polarizer to obtain the polarization state of the photon.

[0087] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0088] b3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0089] b4) Determine if there is any eavesdropping. If so, discard the current key and regenerate a new key.

[0090] Quantum states include horizontally polarized states, vertically polarized states, 45-degree polarized states, and 135-degree polarized states;

[0091] The horizontal polarization state is denoted as |0> and assigned a value of 0;

[0092] The vertical polarization state is denoted as |1> and assigned a value of 1;

[0093] A 45-degree polarization state is denoted as |+> and assigned a value of 1;

[0094] A polarization state of 135 degrees is denoted as |-> and assigned a value of 0.

[0095] In online interviews, multimodal sentiment analysis helps interviewers assess candidates' emotions and reactions by analyzing their voice, facial expressions, and verbal expressions, leading to a more comprehensive and objective evaluation. In judicial interrogations, multimodal sentiment analysis can assist in analyzing the emotional state of suspects or witnesses, assessing the credibility of their statements through nonverbal cues. In classroom learning effectiveness analysis, multimodal sentiment analysis technology objectively analyzes and evaluates learning outcomes using sensor data from multiple modalities, including internal and external manifestations such as facial expressions, postures, movement frequencies, and brainwaves. In customer service, multimodal sentiment analysis can enhance customer experience by understanding customer emotions and needs, providing more personalized services. Based on the above considerations and the importance of online privacy protection, this invention proposes a privacy protection system and method for multimodal sentiment analysis in networks based on quantum key distribution. This method is based on specific quantum mechanical principles and properties, such as quantum superposition, Heisenberg uncertainty principle, and quantum no-cloning property. Through an effective eavesdropping detection mechanism, it achieves absolutely secure key distribution and unconditional security for secure communication based on the one-time pad principle of cryptography. This ensures the integrity and accuracy of sentiment analysis data, protects users' privacy rights to the greatest extent, and makes the application of network multimodal sentiment analysis more secure and reliable. It builds an unbreakable security defense for the data transmission process in the specific field of network sentiment analysis.

[0096] Example 2:

[0097] The privacy protection method based on the system described in Example 1 includes the following steps:

[0098] 1) Verify the identities of both communicating parties through an authentication center;

[0099] 2) Collect multimodal data, perform feature extraction and sentiment analysis, and obtain sentiment scores that reflect users' emotional tendencies;

[0100] The multimodal data includes user facial video data and user voice data;

[0101] The steps for feature extraction include: using OpenFace tools to extract visual features, including facial key points and head pose;

[0102] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0103] The Whisper deep learning model is used to transcribe speech into text information, and then the BERT model is used to convert the text information into vector representation and extract text features.

[0104] 3) Generate a random bit sequence using a quantum true random number generator, and transmit the encoded quantum state through a quantum channel to prepare a quantum key. The steps include:

[0105] 3.1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel;

[0106] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received qubit sequence;

[0107] 3.3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0108] 3.4) Determine if eavesdropping occurs; if so, discard the current key and generate a new key.

[0109] 4) Based on the symmetric encryption algorithm AES, quantum key distribution is used to encrypt multimodal data and sentiment scores to generate ciphertext;

[0110] 5) Transmit encrypted messages using traditional communication channels;

[0111] 6) The receiver uses the shared quantum key to decrypt the ciphertext and recover the original data.

[0112] In step 3.3), the step of extracting the corresponding bit sequence according to the preset selection rules is as follows:

[0113] 3.3.1) Generate a binary true random number sequence using quantum mechanics principles, and use it as a measurement basis sequence;

[0114] 3.3.2) Randomly select {|0>, |1>} from the measurement basis sequence (i.e., (base) or {|+>, |->} (i.e.) Two sets of measurement bases are used to measure the received quantum state. The measurement bases allow photons with the same polarization direction to pass through. If the base chosen for preparing the quantum state is the same as the base chosen for measurement, it is recorded as the correct measurement base bit; otherwise, the corresponding bit data is deleted. A subset of positions in the correct measurement base bits is randomly selected and recorded as random bits. For each random bit, if the measurement result matches the sender's result, it is considered that there is no eavesdropping. Finally, the remaining positions in the correct measurement base bits after removing the random bits are recorded as valid bits. The key (1 or 0) is determined based on the measurement result corresponding to the valid bits.

[0115] Example 3:

[0116] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, comprising a sender and a receiver;

[0117] The transmitting end includes an authentication center management unit, a multimodal emotion analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0118] The authentication center management unit is used for identity verification and data transmission security authentication. If the authentication is successful, it sends an authentication success signal to the multimodal sentiment analysis unit.

[0119] After receiving the authentication success signal, the multimodal sentiment analysis unit collects and processes multimodal data to obtain sentiment feature information. Then, it uses a multimodal sentiment recognition model to analyze the sentiment feature information and obtain a sentiment score that reflects the user's sentiment tendency.

[0120] The multimodal sentiment analysis unit transmits multimodal data and sentiment scores to the data encryption transmission unit;

[0121] The quantum key preparation unit generates a quantum key and transmits it to the quantum key verification unit at the receiving end;

[0122] The data encryption transmission unit uses quantum keys to encrypt multimodal data and sentiment scores, and then transmits the data to the receiving end through a traditional channel.

[0123] The receiving end includes a quantum key verification unit and a data decryption unit;

[0124] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0125] The data decryption unit decrypts the received data using a quantum key.

[0126] Example 4:

[0127] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution is provided. The technical content is the same as in Embodiment 3. Furthermore, the authentication center is used to issue digital certificates to both parties in the communication to ensure the authenticity of their identities and to authenticate the communication channel to ensure the security of data transmission.

[0128] Example 5:

[0129] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with the same technical content as any one of embodiments 3-4, further wherein the multimodal data includes user facial video data and user voice data;

[0130] User facial video data is collected via camera, and user voice data is collected via microphone.

[0131] Example 6:

[0132] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with technical content identical to any one of embodiments 3-5, further comprising the following steps for the multimodal sentiment analysis unit to process multimodal data:

[0133] Visual features, including facial landmarks and head pose, were extracted using the OpenFace tool.

[0134] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0135] The Whisper deep learning model is used to transcribe user speech data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0136] Example 7:

[0137] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with technical content identical to any one of embodiments 3-6, further comprising the following steps for analyzing sentiment feature information using a multimodal sentiment recognition model:

[0138] s1) Integrate and align the multimodal data after feature extraction to obtain a complete set of sentiment feature information, and transmit it to the analysis end;

[0139] s2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between various modal features, splices and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0140] Example 8:

[0141] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with the same technical content as any one of embodiments 3-7. Further, the modal sentiment recognition model includes a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

[0142] Example 9:

[0143] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with technical content identical to any one of embodiments 3-8, further comprising the following steps for the quantum key preparation unit to generate quantum keys:

[0144] b1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel; the quantum state of the qubit sequence is the polarization state of the photon; the polarization state of the photon is generated by generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon with a polarizer to obtain the polarization state of the photon.

[0145] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0146] b3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0147] b4) Determine if there is any eavesdropping. If so, discard the current key and regenerate a new key.

[0148] Example 10:

[0149] A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, with the same technical content as any one of embodiments 3-9, further wherein the quantum states include horizontal polarization state, vertical polarization state, 45-degree polarization state and 135-degree polarization state;

[0150] The horizontal polarization state is denoted as |0> and assigned a value of 0;

[0151] The vertical polarization state is denoted as |1> and assigned a value of 1;

[0152] A 45-degree polarization state is denoted as |+> and assigned a value of 1;

[0153] A polarization state of 135 degrees is denoted as |-> and assigned a value of 0.

[0154] Example 11:

[0155] The privacy protection method based on any one of the systems described in Embodiments 1-10 includes the following steps:

[0156] 1) Verify the identities of both communicating parties through an authentication center;

[0157] 2) Collect multimodal data, perform feature extraction and sentiment analysis, and obtain sentiment scores that reflect users' emotional tendencies;

[0158] The multimodal data includes user facial video data and user voice data;

[0159] The steps for feature extraction include: using OpenFace tools to extract visual features, including facial key points and head pose;

[0160] Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency.

[0161] The Whisper deep learning model is used to transcribe speech into text information, and then the BERT model is used to convert the text information into vector representation and extract text features.

[0162] 3) Generate a random bit sequence using a quantum true random number generator, and transmit the encoded quantum state through a quantum channel to prepare a quantum key. The steps include:

[0163] 3.1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel;

[0164] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received qubit sequence;

[0165] 3.3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key;

[0166] 3.4) Determine if eavesdropping occurs; if so, discard the current key and generate a new key.

[0167] 4) Based on the symmetric encryption algorithm AES, quantum key distribution is used to encrypt multimodal data and sentiment scores to generate ciphertext;

[0168] 5) Transmit encrypted messages using traditional communication channels;

[0169] 6) The receiver uses the shared quantum key to decrypt the ciphertext and recover the original data.

[0170] Example 12:

[0171] The privacy protection method based on any one of the systems described in Embodiments 1-10 has the same technical content as Embodiment 11. Further, in step 3.3), the step of extracting the corresponding bit sequence according to a preset selection rule is as follows:

[0172] 3.3.1) Generate a binary true random number sequence using quantum mechanics principles, and use it as a measurement basis sequence;

[0173] 3.3.2) Randomly select from the measurement base sequence base or Basis, as a measurement basis;

[0174] Two sets of measurement bases measure the received quantum state, and the measurement bases allow photons with the same polarization direction to pass through;

[0175] If the basis chosen for preparing the quantum state is consistent with the basis chosen during measurement, it is recorded as a correct measurement basis bit; otherwise, the corresponding bit data is deleted. A subset of positions is randomly selected from the correct measurement basis bits and recorded as random bits. For each random bit, if the measurement result matches the sender's result, it is considered that there was no eavesdropping.

[0176] The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

[0177] Example 13:

[0178] A privacy protection method for network multimodal sentiment analysis based on quantum key distribution, comprising the following steps:

[0179] S1: Authentication, verifying the identities of both communicating parties through a Certification Authority (CA);

[0180] S2: Multimodal sentiment analysis, which collects multimodal data such as video, audio and text through devices such as cameras and microphones, and performs feature extraction and sentiment state classification;

[0181] S3: Shared quantum key preparation, which uses a quantum true random number generator to generate a series of random bit sequences and transmits the encoded quantum state through a quantum channel;

[0182] S4: Data encryption uses the symmetric encryption algorithm AES and a key generated by quantum key distribution technology to encrypt the data and generate ciphertext;

[0183] S5: Data transmission transmits encrypted text through traditional communication channels and enhances transmission security using a Certificate Authority (CA) mechanism;

[0184] S6: Data decryption. The receiver uses the shared quantum key to decrypt the ciphertext and recover the original data.

[0185] The multimodal sentiment analysis includes the following steps:

[0186] S21: Use tools such as OpenFace to extract visual features such as facial key points and head posture;

[0187] S22: Use tools such as Librosa to extract speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q-transform (CQT), and logarithmic fundamental frequency; use the Whisper deep learning model to transcribe speech into text information; use the BERT model to convert text information into vector representation and extract text features;

[0188] S23: Integrate and align the extracted multidimensional feature data to obtain a complete set of sentiment feature information, and transmit the set of sentiment feature information to the host of the analysis terminal via the Internet;

[0189] S24: The analysis end has a multimodal emotion recognition model embedded in it. By analyzing the relationship between the features of each modality, the attention distribution of multiple subspaces is spliced ​​and linearly transformed to obtain the final fused features.

[0190] S25: Finally, through the sentiment analysis network, the model outputs a sentiment score, which can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0191] The shared quantum key preparation includes the following steps:

[0192] S31: The sender uses a quantum random number generator to generate a sequence of random bits; the sender transmits the encoded quantum state to the receiver through a quantum channel;

[0193] S32: The receiver performs random basis selection and measurement;

[0194] S33: The sender and receiver compare and select the quantum states that are measured using the same basis by both parties, and then extract the corresponding bit sequence according to the predetermined selection rules to form a shared key;

[0195] S34: Determine if there is any eavesdropping. If so, discard the current key and generate a new key.

[0196] Regarding the quantum states mentioned in S31, this patent requires the transmitting end to generate a single-photon source using a semiconductor quantum dot photon generator, and then process the photons using a polarizer to obtain their polarization states. The polarization characteristics of the photons are then used to represent key information. The four polarization states are represented by |0>, |1>, |+>, and |->, respectively. It is stipulated that polarization states |1> and |+> are assigned a value of 1, and polarization states |0> and |-> are assigned a value of 0.

[0197] Certification Authority (CA) issues digital certificates to both communicating parties to ensure the authenticity of their identities and that only authorized users can access the communication network. When ciphertext is transmitted over traditional channels, the CA mechanism authenticates the traditional communication channel, guaranteeing the security of data transmission.

[0198] Example 14:

[0199] A system that applies the method of any one of embodiments 1-13 includes:

[0200] a. The authentication center management unit is used for identity verification and data transmission security authentication;

[0201] b. Multimodal sentiment analysis unit, used for collecting and processing multimodal data;

[0202] c. Quantum key preparation unit, used to generate and distribute quantum keys;

[0203] d. A data encryption transmission unit, used to encrypt data and transmit it through a traditional channel;

[0204] e. Data decryption unit, used to decrypt received data.

[0205] Example 15:

[0206] A privacy-preserving system for network multimodal sentiment analysis based on quantum key distribution is proposed, and its principle is as follows:

[0207] Particles are inherently uncertain at the microscopic level: at the quantum level, particles can exist in more than one place at the same time, or be in more than one state of existence at the same time, and it is impossible to predict their exact quantum state (Heisenberg uncertainty principle).

[0208] Particles can be randomly measured at binary positions: a photon is one of the particles and is the smallest particle of light. It can be set to have a specific polarity, which can be directly mapped to the binary classical bits of "1" and "0" in the classical computing system.

[0209] Quantum systems cannot be measured without being altered: according to the laws of quantum physics, measuring the fundamental behavior of a quantum system will have an irreversible effect on the system.

[0210] Particles can be partially cloned, but not completely: while some properties of particles can be cloned, 100% cloning is considered impossible (quantum non-cloning).

[0211] The above four fundamental characteristics ensure that QKD is a method for secure digital key distribution over long distances protected by the microscopic physical properties of quantum mechanics.

[0212] The core of QKD is to ensure secure key distribution. If an eavesdropper appears during the key distribution process, QKD can ensure that the eavesdropper's presence is detected.

[0213] If eavesdropping is detected, it indicates that the current key may have been stolen. The QKD mechanism will then interrupt the current key distribution process, discard the current key, and restart a new QKD key distribution process to generate a new key, until the keys are finally exchanged securely.

[0214] A necessary condition for absolute security (or unconditional security) in cryptography is one-time pad.

[0215] One-time pad requirements: The key is completely random, the key length is the same as the plaintext length, and the key itself is used only once.

[0216] By utilizing the fundamental principles of quantum mechanics, one-time pad can be easily achieved, and the absolute security of the key distribution process can be guaranteed; it is a confidential communication that is unconditionally secure in principle.

[0217] The system mainly consists of the following units: authentication center management unit, multimodal sentiment analysis unit, quantum key preparation unit, data encryption and transmission unit, and data decryption unit.

[0218] The technical solutions adopted by this invention for the above-mentioned units are described below:

[0219] Certification Center Management Unit: This unit serves two main purposes.

[0220] 1. Certification Authority (CA) issues digital certificates to both parties in a communication to ensure the authenticity of their identities and that only authorized users can access the communication network.

[0221] 2. When transmitting encrypted data over a traditional network channel, a Certificate Authority (CA) mechanism is used to authenticate the traditional communication channel, ensuring the security of data transmission.

[0222] Multimodal sentiment analysis unit: The purpose of this unit is to output multimodal sentiment analysis results as encrypted data information.

[0223] The multimodal sentiment analysis process is as follows:

[0224] 1. Use a camera to capture the user's facial video in real time to obtain visual data, use a microphone to capture the user's voice in real time to obtain voice data, and use speech recognition technology to transcribe the speech into text to obtain text data.

[0225] 2. Use tools such as OpenFace to extract facial features such as facial landmarks, head pose, and eye gaze; use tools such as Librosa to extract speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q-transform (CQT), and logarithmic fundamental frequency (log F0); use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features.

[0226] 3. The extracted multimodal features with different structures are processed to unify and standardize their sequence lengths, thereby obtaining a complete set of sentiment feature information that can be used to train a sentiment analysis neural network model, thus facilitating model input.

[0227] 4. A multimodal sentiment recognition model was designed, including a feature embedding sub-network, a multi-head self-attention mechanism, a tensor fusion-based feature fusion network, and a sentiment analysis network. The model was trained using the CH-SIMS Chinese multimodal dataset, and evaluated using metrics such as accuracy and F1-Score.

[0228] 4. Through the sentiment analysis network, a sentiment score is output, which can reflect the sentiment tendency of the observed person.

[0229] Quantum key generation unit: The purpose of this unit is to obtain a key that can be used to encrypt data. The process consists of the following three steps:

[0230] 1. The sender uses a quantum random number generator to generate a sequence of random bits, and converts each random bit into a quantum bit. In this invention, the polarization state of a photon is used to represent this.

[0231] 2. The sender transmits the encoded quantum state to the receiver through the quantum channel, performs random basis selection and measurement, and compares the basis selection information to select the quantum states that are measured using the same basis by both parties, thus obtaining the corresponding bits to form a shared key.

[0232] 3. If an eavesdropper is discovered during the key acquisition process, the key distribution process is abandoned, and a new shared key is obtained.

[0233] 4. After screening, error correction, and privacy amplification techniques, the communicating parties obtain a final secure shared key for the encrypted protection of the multimodal sentiment analysis prediction results.

[0234] Data Encryption Transmission Unit: This unit is used to encrypt the prediction results output during multimodal sentiment analysis and transmit them securely through a traditional channel. The specific steps are as follows:

[0235] 1. Select the symmetric encryption algorithm AES to encrypt the data, i.e., encrypt the prediction result;

[0236] 2. Initialize the AES encryption algorithm using the key generated by QKD;

[0237] 3. Encrypt the prediction results to generate ciphertext;

[0238] 4. The encrypted text is transmitted over the network through a traditional channel, and the traditional communication channel is authenticated through a Certificate Authority (CA) mechanism to ensure the security of data transmission.

[0239] Data decryption unit: This unit is used to decrypt the received ciphertext, restore it to the original data, and read it. The process consists of two steps:

[0240] 1. When the receiver receives the ciphertext, it uses the same key to decrypt it, turning the ciphertext back into the original data;

[0241] 2. Receiver secure access to prediction results of multimodal sentiment analysis.

[0242] The key distribution process will now be explained in more detail.

[0243] First, prepare the channels: quantum channels and traditional channels.

[0244] Quantum channels (qubit swapping) are used for key exchange.

[0245] Traditional channels are authentication channels so that man-in-the-middle attacks can be detected and that potential attackers cannot modify the information exchange.

[0246] Alice (in cryptography, Alice usually refers to the sender) randomly generates a total of 2n qubits of photons in the following four states: |0>, |1>, |+>, and |->.

[0247] |0> and |1>: These are the standard basis of quantum computing, called... Basis or computational basis. Represents two orthogonal states of a qubit, corresponding to two orthogonal polarization states of a photon, such as horizontal and vertical polarization.

[0248] |+> and |->: are the basis obtained from the Hadamard transform, called... Basis or Hadamard basis. They are superpositions of |0> and |1>.

[0249] The four types of photons are defined as follows:

[0250]

[0251] To measure the polarization state of photons, a measurement basis is introduced. and The measurement basis allows photons aligned with its polarization direction to pass through.

[0252] Photon measurement basis in states |↑> and |→> During measurement, we can obtain |↑> and |→> respectively. This is the correct feedback result. If the state is... Photons being measured During measurement, its state is as follows:

[0253]

[0254] After being measured, There is a 50% probability that |↑>, and a 50% probability that |→>.

[0255] Status is and Photons being measured During measurement, the following can be obtained respectively and This is the correct feedback result. If the photon in the state |↑> is measured from the basis... During measurement, its state is as follows:

[0256]

[0257] After being measured, |↑> has a 50% probability of being 50% probability

[0258] Due to the measurement base and The non-orthogonal relationship means that the state of a photon may change after it is measured by a random measurement basis of the eavesdropper.

[0259] In practice, the polarization states |1> and |+> of photons are assigned a value of 1, while the polarization states |0> and |-> are assigned a value of 0. An example of this key generation process is shown in the table below.

[0260]

[0261] The operations performed on the quantum channel are as follows:

[0262] Alice uses a quantum true random number generator to generate a string of binary random classical bit sequences [A], such as (11001001...), as shown in the first row of the table above.

[0263] Alice randomly assigns a function to each digit of the sequence [A]. base or Base. For example, if a bit in [A] is 1, and Alice specifies that bit... If the base is used, then the 1 at that bit will be emitted using the quantum state |↑>, instead of using a quantum state. As shown in the second and third rows of the table above.

[0264] Alice sends these quantum states to Bob (in cryptography, Bob usually refers to the receiver) via a quantum channel.

[0265] Bob randomly selects a measurement basis to measure these quantum states, obtaining a series of binary strings associated with 0 and 1, as shown in the sixth row of the table above.

[0266] Perform the following operations on a traditional channel:

[0267] Bob transmits the type of the selected measurement basis to Alice, but not the measurement results. For example, Bob informs Alice: First choice... Base, second choice Base, third choice The base is shown in the fourth row of the table above.

[0268] After receiving the measurement basis information selected by Bob, Alice compares it and transmits the position information of the correctly selected measurement basis to Bob. If the set of correctly selected positions is called set [R], then Alice sends the set [R] information to Bob, as shown in the seventh row of the table above.

[0269] After receiving the information from set [R], Bob randomly selects a subset of positions from set [R], denoted as set [S], and sends the measurement results of the corresponding positions in set [S] to Alice. For example, if Bob learns from set [R] that his 1st, 2nd, 3rd, and 7th positions have chosen the correct measurement basis, Bob randomly selects a few positions, such as the 1st and 7th, and records the information for these two positions; this is set [S]. Bob then sends the measurement results (0 or 1) of the corresponding positions in set [S] to Alice, as shown in rows 8 and 9 of the table above.

[0270] After receiving the measurement results, Alice checks her own measurement results against the corresponding positions in set [S]. If the results match, Alice sends a message to Bob notifying him that he has not been eavesdropped on. Then, both parties remove the corresponding positions from set [S] in set [R] and select the bit information corresponding to the remaining positions as the shared key for their communication. As shown in the tenth row of the table above.

[0271] The security of this method is demonstrated as follows:

[0272] In this process, if Eve eavesdrops on a traditional channel, the following two scenarios may occur:

[0273] 1. When Bob transmits the type of the selected measurement base to Alice, Eve eavesdrops on Bob's measurement method, but Bob does not send the measurement results.

[0274] 2. Eve will eavesdrop on some of Bob's correct measurements, namely set [S] and the measured results, but Alice will completely discard set [S] when Alice and Bob choose their communication key at the end.

[0275] Therefore, Eve only knows which bits of [A] Alice and Bob used as their communication key, but she doesn't know the specific values ​​of those bits.

[0276] Therefore, even if traditional channels are eavesdropped on, security can still be guaranteed.

[0277] If Eve were to eavesdrop on a quantum channel, the following two scenarios would occur:

[0278] 1. When Alice sends the quantum state to Bob, Eve can eavesdrop, but cannot determine the type of measurement basis used.

[0279] 2. If Eve chooses a random set of measurement bases to measure the quantum state, she will inevitably choose an incorrect type of measurement base, leading to changes in some quantum states. For example, if [A] has a value of 1 at a certain bit, Alice sends a message using |↑>, but Eve incorrectly selects a measurement base. This quantum state may then become or If Bob happens to choose the measurement base at this time Measurements eavesdropped on by Eve (or ), quantum state results (or The bit becomes a quantum state (or a quantum state, where the information is correctly transmitted, and the bit will not attract Alice and Bob's attention). The original value was 1, but Bob correctly measured it using the measurement basis and got 0. When selecting set [S], if Bob happens to select this bit and send it to Alice, Alice will find that Bob used the correct measurement basis for this bit but did not get the correct value. When this happens more frequently than a certain threshold, Alice will realize that the quantum channel is being eavesdropped on.

[0280] Therefore, quantum key distribution protocols can generate keys known only to the communicating parties. This key is both random and secure because any attempt to eavesdrop on the key will be detected by both parties immediately during communication.

[0281] The following explains the specific operation of the multimodal sentiment analysis unit:

[0282] 1. Data Acquisition and Preprocessing: Visual data is obtained by using a camera to capture the user's facial video in real time, and voice data is obtained by using a microphone to capture the user's voice in real time. The voice is then transcribed into text using speech recognition technology to obtain text data.

[0283] 2. Multimodal Feature Extraction and Secure Processing: Facial features such as facial landmarks, head pose, and eye gaze are extracted using tools such as OpenFace; speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q-transform (CQT), and logarithmic fundamental frequency (log F0) are extracted using tools such as Librosa; the speech is transcribed into text using the Whisper deep learning model, and then the text is converted into a vector representation using the BERT model to extract text features.

[0284] 3. Feature Normalization: The extracted multimodal features with different structures are processed to unify and normalize their sequence lengths to facilitate model input. This results in a complete set of sentiment feature information that can be used to train the sentiment analysis neural network model. This set of sentiment feature information will be transmitted to the host computer at the analysis end via the Internet.

[0285] 4. Model Training and Evaluation: After receiving the sentiment information set, the analysis end constructs a multimodal sentiment recognition model, including a feature embedding sub-network, a multi-head self-attention mechanism, a tensor fusion-based feature fusion network, and a sentiment analysis network. The model is trained using the CH-SIMS Chinese multimodal dataset, and evaluated using metrics such as accuracy and F1-Score.

[0286] The functions of these different networks are as follows:

[0287] Feature embedding sub-network: Further embedding and dimensionality adjustment of features from different modalities.

[0288] Multi-head self-attention mechanism: Multiple independent attention mechanisms are run in parallel to obtain the attention distribution of different subspaces of the input sequence.

[0289] Tensor fusion-based feature fusion network: Construct a triple Cartesian product representation of the three-modal feature sequences to learn intra-modal and inter-modal dynamics.

[0290] Sentiment analysis network: The fused features are processed through fully connected layers, and finally the sentiment prediction result is obtained by using the Sigmoid activation function.

[0291] 4. Output prediction results: Through the sentiment analysis network, the model will output a sentiment score, which can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0292] Example 16:

[0293] The system flowchart for the application of quantum key distribution technology in the field of emotional needs analysis includes the following parts:

[0294] S1, Authentication; S2, Multimodal sentiment analysis; S3, Shared quantum key preparation; S4, Data encryption; S5, Data transmission; S6, Data decryption.

[0295] The details of each step are as follows:

[0296] S1. Identity Verification: The Certification Authority (CA) issues digital certificates to both parties in the communication to ensure the authenticity of their identities and that only legitimate users can access the communication network.

[0297] S2. Multimodal Sentiment Analysis: Network-based multimodal sentiment analysis is an end-to-end process. First, it collects and processes multimodal data including video, audio, and text containing emotional expressions using devices such as cameras and microphones. Next, facial features such as facial landmarks and head pose are extracted using tools like OpenFace; speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q-transform (CQT), and logarithmic fundamental frequency are extracted using tools like Librosa; the Whisper deep learning model is used to transcribe the speech into text information, and then the BERT model is used to convert the text information into vector representations and extract text features. These structurally dissimilar multidimensional feature data are then integrated and aligned to obtain a complete set of sentiment feature information that can be used to train the sentiment analysis neural network model. This set of sentiment feature information is transmitted to the host machine at the analysis end via the internet. Upon receiving the sentiment information set, the analysis end constructs a model, designing a multimodal sentiment recognition model that includes a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network. The model analyzes the relationships between features across different modalities, concatenates and linearly transforms the attention distributions of multiple subspaces to obtain the final fused features. Finally, through a sentiment analysis network, the model outputs a sentiment score that reflects the emotional tendency of the observed individual, such as positive, negative, or neutral.

[0298] S3. Shared Quantum Key Preparation: The sender uses a quantum true random number generator to generate a series of random bit sequences and converts these bit sequences into qubits. In this invention, the preparation of qubits utilizes the polarization state of photons. Subsequently, the sender transmits these encoded quantum states to the receiver via a quantum channel. Upon receiving the quantum states, the receiver performs random basis selection and measurement. By comparing the basis selection information of both parties, the sender and receiver jointly select quantum states measured under the same basis, thereby extracting the corresponding bit sequences according to a predetermined selection rule to form a shared key.

[0299] To ensure the security of the key distribution process, a security detection mechanism is introduced. If potential eavesdropping is detected during the key distribution process, a new QKD key distribution process is restarted to generate a new key, until the keys are finally securely exchanged.

[0300] Furthermore, to further enhance key security, privacy amplification technology was employed to reduce the risk of information leakage to potential eavesdroppers. Through a series of screening, error correction, and privacy amplification processes, the communicating parties ultimately obtained a highly secure shared key.

[0301] S4. Data Encryption: To ensure the security of data transmission during online video sentiment analysis, the Advanced Encryption Standard (AES) symmetric encryption algorithm is used to encrypt the data. A key generated using quantum key distribution (QKD) technology is used to initialize the AES algorithm's operating mode, encrypting the prediction results of multimodal sentiment analysis to generate corresponding ciphertext. This process ensures the confidentiality and integrity of the data during transmission.

[0302] S5. Data Transmission: During network transmission, encrypted data will be transmitted through traditional communication channels. To enhance data transmission security, a Certificate Authority (CA) mechanism is introduced. As a trusted third-party organization, the CA is responsible for authenticating the traditional communication channels, ensuring the security and reliability of data during transmission. Through the CA mechanism, the identities of both communicating parties can be verified, preventing unauthorized access and potential man-in-the-middle attacks, thus providing an additional layer of security for data transmission in network video sentiment analysis.

[0303] S6. Data Decryption: After successfully receiving the ciphertext, the receiver uses a pre-prepared quantum shared key for decryption. Through this process, the ciphertext is converted back to its original form. Thus, the receiver can securely retrieve and access the results of the sentiment analysis, ensuring the confidentiality and integrity of the data during transmission.

[0304] The specific steps in the shared quantum key preparation process are as follows:

[0305] The transmitting end generates a single-photon source through a semiconductor quantum dot photon generator, and then processes the photons using a polarizer to obtain the polarization state of the photons;

[0306] The key information is represented using the polarization properties of photons. Four polarization states are represented by |0>, |1>, |+>, and |->. It is stipulated that polarization states |1> and |+> are assigned a value of 1, and polarization states |0> and |-> are assigned a value of 0.

[0307] The sending end randomly assigns a function to each bit of sequence [A] in sequence. Measurement base or Measurement base.

[0308] The transmitter sends these quantum states, i.e., the sequence [A], to the receiver via a quantum channel.

[0309] The receiver randomly uses a series of measurement bases to measure the quantum state in [A] and obtains a series of binary bit information related to 0 and 1.

[0310] The receiving end transmits the information of the measurement base to the sending end, but does not transmit the measurement results.

[0311] After receiving the selected measurement base type information, the sending end immediately compares it with its own selected measurement base type, transmits the bit information with consistent measurement base selection to the receiving end, and the set of correctly measured bits is called set [R].

[0312] After receiving the information from set [R], the receiving end randomly selects a portion of positions from set [R] and denotes it as set [S]. The receiving end then transmits the measurement results of the corresponding positions in set [S] to the sending end.

[0313] The sending end checks the result of the corresponding position in set [S]. If it is correct, the sending end sends a message to notify the receiving end that there was no eavesdropping during this key distribution process.

[0314] Then both parties remove the corresponding position from set [S] in set [R] and select the bit information corresponding to the remaining position as the shared key for communication between the two parties.

[0315] Therefore, quantum key distribution protocols can generate keys known only to the communicating parties. This key is both random and secure because any attempt to eavesdrop on the key will be detected by both parties immediately during communication.

[0316] Through the detailed process and approach outlined above, this invention provides an effective method for protecting the privacy of online video sentiment analysis data in the quantum era. This method not only utilizes quantum key distribution technology to ensure data transmission security but also combines multimodal sentiment analysis and authentication center mechanisms, providing a comprehensive privacy protection solution for online video sentiment analysis.

Claims

1. A privacy protection system for network multimodal sentiment analysis based on quantum key distribution, characterized in that, This includes the sending end, receiving end, authentication center, and analysis end; The transmitting end includes an authentication center management unit, a multimodal emotion analysis unit, a quantum key preparation unit, and a data encryption transmission unit; The authentication center management unit is used for identity verification and data transmission security authentication. If the authentication is successful, it sends an authentication success signal to the multimodal sentiment analysis unit. After receiving the authentication success signal, the multimodal sentiment analysis unit collects and processes multimodal data to obtain sentiment feature information. Then, it uses a multimodal sentiment recognition model to analyze the sentiment feature information and obtain a sentiment score that reflects the user's sentiment tendency. The multimodal sentiment analysis unit transmits multimodal data and sentiment scores to the data encryption transmission unit; The quantum key preparation unit generates a quantum key and transmits it to the quantum key verification unit at the receiving end; The data encryption transmission unit uses quantum keys to encrypt multimodal data and sentiment scores, and transmits them to the receiving end through a traditional communication channel; The receiving end includes a quantum key verification unit and a data decryption unit; The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end; The data decryption unit decrypts the received data using a quantum key; The steps involved in processing multimodal data by the multimodal sentiment analysis unit include: Visual features, including facial landmarks and head pose, were extracted using the OpenFace tool. Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency. The Whisper deep learning model is used to transcribe user speech data into text information, and the BERT model is used to convert the text information into vector representation and extract text features. The steps for analyzing sentiment feature information using a multimodal sentiment recognition model include: a1) Integrate and align the multimodal data after feature extraction to obtain a complete set of sentiment feature information, and transmit it to the analysis end; a2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between various emotion features in the emotion feature information set, splices and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score reflecting the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral. The steps of generating a quantum key in a quantum key preparation unit include: b1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel; the quantum state of the qubit sequence is the polarization state of the photon; the polarization state of the photon is generated by generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon with a polarizer to obtain the polarization state of the photon. b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence; b3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key; b4) Determine if there is eavesdropping; if so, discard the current key distribution process and regenerate a new key. The certification authority is used to issue digital certificates to both parties in a communication to ensure the authenticity of their identities and to authenticate the communication channel, thereby guaranteeing the security of data transmission. The multimodal data includes user facial video data and user voice data; User facial video data is collected via camera, and user voice data is collected via microphone; The multimodal sentiment recognition model includes a feature embedding subnetwork, a multi-head self-attention mechanism, a tensor fusion-based feature fusion network, and a sentiment analysis network; Quantum states include horizontally polarized states, vertically polarized states, 45-degree polarized states, and 135-degree polarized states; Horizontal polarization state is denoted as The value is assigned to 0; Vertical polarization state is denoted as The value is assigned to 1; A 45-degree polarization state is denoted as The value is assigned to 1; A 135-degree polarization state is denoted as The value is assigned to 0.

2. A privacy protection method based on the system described in claim 1, characterized in that, Includes the following steps: Step 1) Verify the identities of both communicating parties through the authentication center; Step 2) Collect multimodal data, perform feature extraction and sentiment analysis, and obtain sentiment scores that reflect the user's emotional tendencies; The multimodal data includes user facial video data, user voice data, and text data converted from the voice data; The steps for feature extraction include: using OpenFace tools to extract visual features, including facial key points and head pose; Speech features were extracted using the Librosa tool, including Mel-frequency cepstral coefficients, constant Q-transform, and logarithmic fundamental frequency. The Whisper deep learning model is used to transcribe speech into text information, and then the BERT model is used to convert the text information into vector representation and extract text features. Step 3) Generate a random bit sequence using a quantum true random number generator, and transmit the encoded quantum state through a quantum channel to prepare a quantum key. The steps include: Step 3.1) The quantum key preparation unit at the transmitting end uses a quantum true random number generator to generate a sequence of random qubits and sends the sequence of qubits to the receiving end through a quantum channel; Step 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received qubit sequence; Step 3.3) Compare and select the quantum states that are measured using the same measurement basis by the quantum key preparation unit and the quantum key verification unit, and then extract the corresponding bit sequence according to the preset selection rules to form a quantum key; Step 3.4) Determine if there is any eavesdropping. If so, discard the current key and generate a new key. Step 4) Based on the symmetric encryption algorithm AES, use quantum key distribution to encrypt the multimodal data and sentiment scores to generate ciphertext; Step 5) Transmit the ciphertext using a traditional communication channel; Step 6) The receiving end uses the shared quantum key to decrypt the ciphertext and recover the original data; In step 3.3), the step of extracting the corresponding bit sequence according to the preset selection rules is as follows: Step 3.3.1) Generate a binary true random number sequence using the principles of quantum mechanics, as the measurement basis sequence; Step 3.3.2) Randomly select from the measurement base sequence base or Basis, as a measurement basis; The received quantum state is measured using two sets of measurement bases, which allow photons with the same polarization direction to pass through. If the basis chosen for preparing the quantum state is consistent with the basis chosen during measurement, it is recorded as the correct measurement basis bit; if they are inconsistent, the corresponding bit data is deleted. A portion of the correct measurement basis bits are randomly selected and recorded as random bits. For each random bit, if the measurement result is consistent with the sender's result, it is considered that there was no eavesdropping. The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

Images