Privacy protection system and method in network multi-mode sentiment analysis based on quantum key distribution

Abstract

The invention discloses a privacy protection system and method in network multi-mode sentiment analysis based on quantum key distribution, and the method comprises the following steps: 1) collecting multi-mode data, and carrying out feature extraction and sentiment state analysis to obtain a sentiment score reflecting the sentiment tendency of a user; 2) based on an AES (Advanced Encryption Standard) algorithm, encrypting the multi-modal data and the emotion score by using a quantum key to generate a ciphertext; (3) the ciphertext is transmitted through a traditional communication channel; and 4) the receiver decrypts the ciphertext by using the shared quantum key to recover the original data. The system comprises an authentication center management unit, a multi-mode sentiment analysis unit, a quantum key preparation unit, a data encryption transmission unit and a data decryption unit. According to the method, the data is encrypted through the quantum key, so that the confidentiality and integrity of the data are guaranteed, security threats caused by quantum computing are effectively resisted, and a high-security privacy protection solution is provided for network video sentiment analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy protection in network multimodal sentiment analysis scenarios, and specifically to a privacy protection system and method in network multimodal sentiment analysis based on quantum key distribution. Background Art

[0002] Multimodal sentiment analysis is a technology that uses data from multiple modalities (such as text, audio, video, etc.) to analyze and identify the user's emotional state. It accurately captures and interprets the inner emotional expression of the observed person by integrating, processing, and aligning data from different sources, thereby performing data analysis and prediction based on deep learning. This analysis method can be applied to remote online office, online job interviews, classroom learning effect evaluation, prison monitoring, important government meetings, and other scenarios in the fields of corporate management, education, judicial administration, national defense and military, which require emotional demand analysis but also require data confidentiality. For example, in judicial interrogations, multimodal sentiment analysis can assist in analyzing the emotional state of suspects or witnesses and assess the credibility of their statements through non-verbal clues; but such scenarios also require that the signals of each channel collected be kept strictly confidential to protect the privacy of the suspect from being leaked.

[0003] Network multimodal sentiment analysis is an end-to-end process. The process first needs to collect multimodal data such as video, audio and text containing emotional expressions. The specific methods include using a camera to collect the user's facial video in real time to obtain visual data, using a microphone to collect the user's voice in real time to obtain voice data, and using voice recognition technology to transcribe the voice into text to obtain text data. Next, facial features such as facial key points and head posture are extracted by using tools such as OpenFace; voice features such as Mel frequency cepstral coefficients (MFCCs), constant Q transform (CQT) and logarithmic fundamental frequency are extracted by using tools such as Librosa; the voice is transcribed into text information using the Whisper deep learning model, and then the text information is converted into vector representation using the BERT model to extract text features. Then, these multidimensional feature data with different structures are integrated and aligned to obtain a complete set of emotional feature information that can be used to train the emotional analysis neural network model. The emotional feature information set will be transmitted to the host of the analysis end via the Internet. After receiving the emotional information set, the analysis end builds the model and designs a multimodal emotion recognition model including a feature embedding subnetwork, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and an emotional analysis network. The model analyzes the relationship between each modal feature, concatenates and linearly transforms the attention distribution of multiple subspaces, and obtains the final fusion feature. Finally, through the sentiment analysis network, the model outputs a sentiment score that can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0004] In the above process, the set of emotional feature information will be transmitted to the host at the analysis end via the Internet, which poses a challenge to privacy protection. Summary of the invention

[0005] The purpose of the present invention is to provide a privacy protection system in network multimodal sentiment analysis based on quantum key distribution, including a transmitting end and a receiving end;

[0006] The sending end includes an authentication center management unit, a multimodal sentiment analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0007] The authentication center management unit is used to perform identity authentication and data transmission security authentication. If the authentication is successful, an authentication success signal is sent to the multimodal sentiment analysis unit;

[0008] After receiving the authentication success signal, the multimodal emotion analysis unit collects and processes the multimodal data to obtain emotion feature information, and then uses the multimodal emotion recognition model to analyze the emotion feature information to obtain an emotion score reflecting the user's emotional tendency;

[0009] The multimodal sentiment analysis unit transmits the multimodal data and the sentiment score to the data encryption transmission unit;

[0010] The quantum key preparation unit generates a quantum key and transmits it to a quantum key verification unit at the receiving end;

[0011] The data encryption transmission unit encrypts the multimodal data and the emotion score using a quantum key, and transmits the data to a receiving end through a traditional channel;

[0012] The receiving end includes a quantum key verification unit and a data decryption unit;

[0013] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0014] The data decryption unit decrypts the received data using a quantum key.

[0015] Furthermore, the authentication center is used to issue digital certificates to both communicating parties to ensure the authenticity of their identities, as well as to authenticate the communication channel to ensure the security of data transmission.

[0016] Further, the multimodal data includes user facial video data and user voice data;

[0017] User facial video data is collected through the camera, and user voice data is collected through the microphone.

[0018] Further, the steps of processing the multimodal data by the multimodal sentiment analysis unit include:

[0019] Use OpenFace tools to extract visual features, including facial key points and head posture;

[0020] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0021] The Whisper deep learning model is used to transcribe user voice data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0022] Further, the steps of analyzing the emotion feature information using the multimodal emotion recognition model include:

[0023] a1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information and transmit it to the analysis end;

[0024] a2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between the features of each modality, concatenates and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0025] Furthermore, the modal emotion recognition model includes a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

[0026] Further, the step of generating a quantum key by the quantum key preparation unit includes:

[0027] b1) The quantum key preparation unit at the transmitting end generates a string of random quantum bit sequences using a quantum true random number generator, and sends the quantum bit sequence to the receiving end through a quantum channel; the quantum state of the quantum bit sequence is the polarization state of the photon; the polarization state of the photon is generated by: generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon using a polarizer to obtain the polarization state of the photon;

[0028] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0029] b3) comparing and selecting the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extracting the corresponding bit sequence according to a preset selection rule to form a quantum key;

[0030] b4) Determine whether there is eavesdropping. If so, abandon the current key distribution process and regenerate a new key.

[0031] Further, the quantum state includes a horizontal polarization state, a vertical polarization state, a 45 degree polarization state, and a 135 degree polarization state;

[0032] The horizontal polarization state is denoted as |0> and is assigned a value of 0;

[0033] The vertical polarization state is denoted as |1> and is assigned a value of 1;

[0034] The 45-degree polarization state is denoted as |+> and is assigned a value of 1;

[0035] The 135-degree polarization state is denoted as |-> and is assigned a value of 0.

[0036] The privacy protection method based on the system includes the following steps:

[0037] 1) Verify the identities of both parties in communication through the authentication center;

[0038] 2) Collect multimodal data, perform feature extraction and emotional state analysis, and obtain an emotional score that reflects the user's emotional tendency;

[0039] The multimodal data includes user facial video data, user voice data, and text data converted from the voice data;

[0040] The steps of feature extraction include: extracting visual features, including facial key points and head posture, using OpenFace tools;

[0041] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0042] Use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features;

[0043] 3) Using a quantum true random number generator to generate a random bit sequence, and transmitting the encoded quantum state through a quantum channel to prepare a quantum key, the steps include:

[0044] 3.1) The quantum key preparation unit at the sending end uses a quantum true random number generator to generate a string of random quantum bit sequences, and sends the quantum bit sequence to the receiving end through the quantum channel;

[0045] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0046] 3.3) Compare and select the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extract the corresponding bit sequence according to the preset selection rules to form the quantum key;

[0047] 3.4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key;

[0048] 4) Based on the symmetric encryption algorithm AES, the multimodal data and sentiment scores are encrypted using quantum keys to generate ciphertext;

[0049] 5) Using traditional communication channels to transmit ciphertext;

[0050] 6) The receiver uses the shared quantum key to decrypt the ciphertext and restore the original data.

[0051] Further, in step 3.3), the step of extracting the corresponding bit sequence according to the preset selection rule is:

[0052] 3.3.1) Generate a binary true random number sequence using the principles of quantum mechanics as the measurement basis sequence;

[0053] 3.3.2) Randomly select from the measurement basis sequence Base or Basis, as a measurement basis;

[0054] Two sets of measurement bases measure the received quantum states, and the measurement bases allow photons that are consistent with their own polarization directions to pass through;

[0055] If the basis selected for preparing the quantum state is consistent with the basis selected for measurement, it is recorded as the correct measurement basis. If not, the corresponding bit data is deleted. A portion of the positions in the correct measurement basis are randomly selected as random bits. For each random bit, if the measurement result is consistent with the sender, it is considered that there is no eavesdropping.

[0056] The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

[0057] The technical effect of the present invention is unquestionable. The present invention generates and securely distributes security keys through QKD, classifies and evaluates emotional states using multimodal sentiment analysis technology combined with video content and voice information, and enhances the security of traditional communication channels through the certification center (CA). By encrypting data with quantum keys, the confidentiality and integrity of data can be guaranteed even when transmitted in an insecure network, effectively resisting the security threats brought by quantum computing, and providing a highly secure privacy protection solution for network video sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is an overall schematic diagram of the present invention;

[0059] Figure 2 It is the overall flow chart of the present invention;

[0060] Figure 3 This is a system block diagram of the multimodal sentiment analysis unit in the present invention. It describes the data processing flow of multimodal sentiment analysis, including data collection, feature extraction, sentiment state classification and result output;

[0061] Figure 4 This is a flowchart of the quantum key distribution unit, including the encoding, transmission, measurement and key generation of quantum states. DETAILED DESCRIPTION

[0062] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0063] Embodiment 1:

[0064] See also Figures 1 to 4 ,A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, including a sending end and a receiving end;

[0065] The sending end includes an authentication center management unit, a multimodal sentiment analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0066] The authentication center management unit is used to perform identity authentication and data transmission security authentication, and if the authentication is successful, sends an authentication success signal to the multimodal sentiment analysis unit;

[0067] After receiving the authentication success signal, the multimodal emotion analysis unit collects and processes the multimodal data to obtain emotion feature information, and then uses the multimodal emotion recognition model to analyze the emotion feature information to obtain an emotion score reflecting the user's emotional tendency;

[0068] The multimodal sentiment analysis unit transmits the multimodal data and the sentiment score to the data encryption transmission unit;

[0069] The quantum key preparation unit generates a quantum key and transmits it to a quantum key verification unit at the receiving end;

[0070] The data encryption transmission unit encrypts the multimodal data and the emotion score using a quantum key, and transmits the data to a receiving end through a traditional channel;

[0071] The receiving end includes a quantum key verification unit and a data decryption unit;

[0072] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0073] The data decryption unit decrypts the received data using a quantum key.

[0074] The authentication center is used to issue digital certificates to both communicating parties to ensure the authenticity of their identities, as well as to authenticate the communication channels to ensure the security of data transmission.

[0075] The multimodal data includes user facial video data and user voice data;

[0076] User facial video data is collected through the camera, and user voice data is collected through the microphone.

[0077] The steps of processing multimodal data by the multimodal sentiment analysis unit include:

[0078] Use OpenFace tools to extract visual features, including facial key points and head posture;

[0079] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0080] The Whisper deep learning model is used to transcribe user voice data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0081] The steps of analyzing emotional feature information using a multimodal emotion recognition model include:

[0082] a1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information and transmit it to the analysis end;

[0083] a2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between the features of each modality, concatenates and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0084] The modal emotion recognition model includes a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

[0085] The steps of generating a quantum key by the quantum key preparation unit include:

[0086] b1) The quantum key preparation unit at the transmitting end generates a string of random quantum bit sequences using a quantum true random number generator, and sends the quantum bit sequence to the receiving end through a quantum channel; the quantum state of the quantum bit sequence is the polarization state of the photon; the polarization state of the photon is generated by: generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon using a polarizer to obtain the polarization state of the photon;

[0087] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0088] b3) comparing and selecting the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extracting the corresponding bit sequence according to a preset selection rule to form a quantum key;

[0089] b4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key.

[0090] The quantum states include horizontal polarization state, vertical polarization state, 45 degree polarization state and 135 degree polarization state;

[0091] The horizontal polarization state is denoted as |0> and is assigned a value of 0;

[0092] The vertical polarization state is denoted as |1> and is assigned a value of 1;

[0093] The 45-degree polarization state is denoted as |+> and is assigned a value of 1;

[0094] The 135-degree polarization state is denoted as |-> and is assigned a value of 0.

[0095] In the network interview of the present invention, multimodal sentiment analysis can help the interviewer evaluate the candidate's emotions and reactions by analyzing the candidate's voice, facial expressions and language expressions, so as to make a more comprehensive and objective evaluation; in judicial interrogation, multimodal sentiment analysis can assist in analyzing the emotional state of suspects or witnesses, and evaluate the credibility of their statements through non-verbal clues; in classroom learning effect analysis, multimodal sentiment analysis technology objectively analyzes and evaluates the learning effect through various internal and external performances of multiple students such as facial expressions, postures, movement frequencies, brain waves and other multi-modal sensor data; in the field of customer service, multimodal sentiment analysis can enhance customer experience and provide more personalized services by understanding customers' emotions and needs. Based on the above starting point and the importance of network privacy protection, the present invention proposes a privacy protection system and method in network multimodal sentiment analysis based on quantum key distribution. This method is based on specific quantum mechanics principles and characteristics, such as quantum superposition, Heisenberg uncertainty principle and quantum non-cloning. Through an effective eavesdropping detection mechanism, it can achieve absolutely safe key distribution and realize unconditional security of confidential communication based on the one-time pad principle of cryptography, thereby ensuring the integrity and accuracy of sentiment analysis data, maintaining the privacy rights of users to the greatest extent, making the application of network multimodal sentiment analysis more secure and reliable, and building an indestructible security line for the data transmission process in the specific field of network sentiment analysis.

[0096] Embodiment 2:

[0097] The privacy protection method based on the system described in Example 1 includes the following steps:

[0098] 1) Verify the identities of both parties in communication through the authentication center;

[0099] 2) Collect multimodal data, perform feature extraction and emotional state analysis, and obtain an emotional score that reflects the user's emotional tendency;

[0100] The multimodal data includes user facial video data and user voice data;

[0101] The steps of feature extraction include: extracting visual features, including facial key points and head posture, using OpenFace tools;

[0102] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0103] Use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features;

[0104] 3) Using a quantum true random number generator to generate a random bit sequence, and transmitting the encoded quantum state through a quantum channel to prepare a quantum key, the steps include:

[0105] 3.1) The quantum key preparation unit at the sending end uses a quantum true random number generator to generate a string of random quantum bit sequences, and sends the quantum bit sequence to the receiving end through the quantum channel;

[0106] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0107] 3.3) Compare and select the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extract the corresponding bit sequence according to the preset selection rules to form the quantum key;

[0108] 3.4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key;

[0109] 4) Based on the symmetric encryption algorithm AES, the multimodal data and sentiment scores are encrypted using quantum keys to generate ciphertext;

[0110] 5) Using traditional communication channels to transmit ciphertext;

[0111] 6) The receiver uses the shared quantum key to decrypt the ciphertext and restore the original data.

[0112] In step 3.3), the steps of extracting the corresponding bit sequence according to the preset selection rule are:

[0113] 3.3.1) Generate a binary true random number sequence using the principles of quantum mechanics as the measurement basis sequence;

[0114] 3.3.2) Randomly select {|0>, |1>} from the measurement basis sequence (i.e. base) or {|+>,|->} (i.e. The two measurement bases (base) measure the received quantum state. The measurement base allows photons with the same polarization direction to pass through. If the base selected for preparing the quantum state is consistent with the base selected for measurement, it is recorded as the correct measurement base. If not, the corresponding bit data is deleted. A portion of the positions in the correct measurement base are randomly selected and recorded as random bits. For each random bit, if the measurement result is consistent with that of the sender, it is considered that there is no eavesdropping. Finally, the remaining positions in the correct measurement base after removing the random bits are recorded as valid bits. The key is determined to be 1 or 0 based on the measurement result corresponding to the valid bit.

[0115] Embodiment 3:

[0116] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, including a sending end and a receiving end;

[0117] The sending end includes an authentication center management unit, a multimodal sentiment analysis unit, a quantum key preparation unit, and a data encryption transmission unit;

[0118] The authentication center management unit is used to perform identity authentication and data transmission security authentication, and if the authentication is successful, sends an authentication success signal to the multimodal sentiment analysis unit;

[0119] After receiving the authentication success signal, the multimodal emotion analysis unit collects and processes the multimodal data to obtain emotion feature information, and then uses the multimodal emotion recognition model to analyze the emotion feature information to obtain an emotion score reflecting the user's emotional tendency;

[0120] The multimodal sentiment analysis unit transmits the multimodal data and the sentiment score to the data encryption transmission unit;

[0121] The quantum key preparation unit generates a quantum key and transmits it to a quantum key verification unit at the receiving end;

[0122] The data encryption transmission unit encrypts the multimodal data and the emotion score using a quantum key, and transmits the data to a receiving end through a traditional channel;

[0123] The receiving end includes a quantum key verification unit and a data decryption unit;

[0124] The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end;

[0125] The data decryption unit decrypts the received data using a quantum key.

[0126] Embodiment 4:

[0127] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content of which is the same as that of Example 3. Furthermore, the authentication center is used to issue digital certificates to both communicating parties to ensure the authenticity of the identities of both parties, and to authenticate the communication channel to ensure the security of data transmission.

[0128] Embodiment 5:

[0129] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content of which is the same as any one of Embodiments 3-4, and further, the multimodal data includes user facial video data and user voice data;

[0130] User facial video data is collected through the camera, and user voice data is collected through the microphone.

[0131] Embodiment 6:

[0132] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content is the same as any one of embodiments 3-5, further, the step of processing multimodal data by the multimodal sentiment analysis unit includes:

[0133] Use OpenFace tools to extract visual features, including facial key points and head posture;

[0134] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0135] The Whisper deep learning model is used to transcribe user voice data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

[0136] Embodiment 7:

[0137] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content is the same as any one of embodiments 3-6, and further, the step of analyzing the sentiment feature information using the multimodal sentiment recognition model includes:

[0138] s1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information and transmit it to the analysis end;

[0139] s2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between each modal feature, concatenates and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

[0140] Embodiment 8:

[0141] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content of which is the same as any one of Examples 3-7. Furthermore, the modal sentiment recognition model includes a feature embedding subnetwork, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

[0142] Embodiment 9:

[0143] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content is the same as any one of embodiments 3-8, and further, the step of generating a quantum key by a quantum key preparation unit includes:

[0144] b1) The quantum key preparation unit at the transmitting end generates a string of random quantum bit sequences using a quantum true random number generator, and sends the quantum bit sequence to the receiving end through a quantum channel; the quantum state of the quantum bit sequence is the polarization state of the photon; the polarization state of the photon is generated by: generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon using a polarizer to obtain the polarization state of the photon;

[0145] b2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0146] b3) comparing and selecting the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extracting the corresponding bit sequence according to a preset selection rule to form a quantum key;

[0147] b4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key.

[0148] Embodiment 10:

[0149] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, the technical content of which is the same as any one of Embodiments 3-9, and further, the quantum state includes a horizontal polarization state, a vertical polarization state, a 45-degree polarization state, and a 135-degree polarization state;

[0150] The horizontal polarization state is denoted as |0> and is assigned a value of 0;

[0151] The vertical polarization state is denoted as |1> and is assigned a value of 1;

[0152] The 45-degree polarization state is denoted as |+> and is assigned a value of 1;

[0153] The 135-degree polarization state is denoted as |-> and is assigned a value of 0.

[0154] Embodiment 11:

[0155] The privacy protection method of the system according to any one of embodiments 1 to 10 comprises the following steps:

[0156] 1) Verify the identities of both parties in communication through the authentication center;

[0157] 2) Collect multimodal data, perform feature extraction and emotional state analysis, and obtain an emotional score that reflects the user's emotional tendency;

[0158] The multimodal data includes user facial video data and user voice data;

[0159] The steps of feature extraction include: extracting visual features, including facial key points and head posture, using OpenFace tools;

[0160] Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency;

[0161] Use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features;

[0162] 3) Using a quantum true random number generator to generate a random bit sequence, and transmitting the encoded quantum state through a quantum channel to prepare a quantum key, the steps include:

[0163] 3.1) The quantum key preparation unit at the sending end uses a quantum true random number generator to generate a string of random quantum bit sequences, and sends the quantum bit sequence to the receiving end through the quantum channel;

[0164] 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence;

[0165] 3.3) Compare and select the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extract the corresponding bit sequence according to the preset selection rules to form the quantum key;

[0166] 3.4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key;

[0167] 4) Based on the symmetric encryption algorithm AES, the multimodal data and sentiment scores are encrypted using quantum keys to generate ciphertext;

[0168] 5) Using traditional communication channels to transmit ciphertext;

[0169] 6) The receiver uses the shared quantum key to decrypt the ciphertext and restore the original data.

[0170] Embodiment 12:

[0171] Based on the privacy protection method of the system described in any one of Examples 1-10, the technical content is the same as that of Example 11. Further, in step 3.3), the step of extracting the corresponding bit sequence according to the preset selection rule is:

[0172] 3.3.1) Generate a binary true random number sequence using the principles of quantum mechanics as the measurement basis sequence;

[0173] 3.3.2) Randomly select from the measurement basis sequence Base or Basis, as a measurement basis;

[0174] Two sets of measurement bases measure the received quantum states, and the measurement bases allow photons that are consistent with their own polarization directions to pass through;

[0175] If the basis selected for preparing the quantum state is consistent with the basis selected for measurement, it is recorded as the correct measurement basis. If not, the corresponding bit data is deleted. A portion of the positions in the correct measurement basis are randomly selected as random bits. For each random bit, if the measurement result is consistent with the sender, it is considered that there is no eavesdropping.

[0176] The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

[0177] Embodiment 13:

[0178] A privacy protection method in network multimodal sentiment analysis based on quantum key distribution, the steps are as follows:

[0179] S1: Identity verification, verifying the identities of both parties in communication through a certification authority (CA);

[0180] S2: Multimodal sentiment analysis, which collects multimodal data such as video, audio, and text through cameras, microphones, and other devices, and performs feature extraction and sentiment state classification;

[0181] S3: Shared quantum key preparation, using a quantum true random number generator to generate a series of random bit sequences, and transmitting the encoded quantum state through a quantum channel;

[0182] S4: Data encryption, using the symmetric encryption algorithm AES, using the key generated by quantum key distribution technology to encrypt the data and generate ciphertext;

[0183] S5: Data transmission, the ciphertext is transmitted through traditional communication channels, and the certification authority (CA) mechanism is used to enhance transmission security;

[0184] S6: Data decryption: The receiver uses the shared quantum key to decrypt the ciphertext and restore the original data.

[0185] The method according to claim 2, characterized in that the multimodal sentiment analysis comprises the following steps:

[0186] S21: Use tools such as OpenFace to extract visual features such as facial key points and head posture;

[0187] S22: Use tools such as Librosa to extract speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q transform (CQT), and logarithmic fundamental frequency; use the Whisper deep learning model to transcribe speech into text information; use the BERT model to convert text information into vector representation and extract text features;

[0188] S23: Integrate and align the extracted multi-dimensional feature data to obtain a complete set of emotional feature information, and transmit the emotional feature information set to a host at the analysis end via the Internet;

[0189] S24: The analysis end is embedded with a multi-modal emotion recognition model, which analyzes the relationship between the features of each modality, concatenates and linearly transforms the attention distribution of multiple subspaces, and obtains the final fusion feature;

[0190] S25: Finally, through the sentiment analysis network, the model outputs a sentiment score, which can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0191] The method according to claim 2, characterized in that the shared quantum key preparation comprises the following steps:

[0192] S31: The sender generates a random bit sequence using a quantum random number generator; the sender sends the encoded quantum state to the receiver through a quantum channel;

[0193] S32: The receiver performs random base selection and measurement;

[0194] S33: The sender and the receiver compare and select the quantum states measured by both parties using the same basis, and then extract the corresponding bit sequence according to a predetermined selection rule to form a shared key;

[0195] S34: Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key.

[0196] Regarding the quantum state mentioned in S31, this patent requires that the transmitter generates a single photon source through a semiconductor quantum dot photon generator, and then processes the photons using a polarizer to obtain the polarization state of the photons, and uses the polarization characteristics of the photons to represent the key information. The four polarization states are represented by |0>, |1>, |+>, and |->, respectively. It is also stipulated that the polarization states |1> and |+> are assigned to 1, and the polarization states |0> and |-> are assigned to 0.

[0197] The method of claim 1 is characterized in that a certification center (CA) issues digital certificates to both communicating parties to ensure the authenticity of the identities of both parties and to ensure that only legitimate users can access the communication network. When the ciphertext is transmitted over the network through a traditional channel, the traditional communication channel is authenticated through the certification center (CA) mechanism to ensure the security of data transmission.

[0198] Embodiment 14:

[0199] A system using the method described in any one of embodiments 1 to 13, comprising:

[0200] a. Authentication center management unit, used for identity authentication and data transmission security authentication;

[0201] b. Multimodal sentiment analysis unit, used to collect and process multimodal data;

[0202] c. Quantum key preparation unit, used to generate and distribute quantum keys;

[0203] d. Data encryption transmission unit, used to encrypt data and transmit it through traditional channels;

[0204] e. Data decryption unit, used to decrypt received data.

[0205] Embodiment 15:

[0206] A privacy protection system in network multimodal sentiment analysis based on quantum key distribution. The principle is as follows:

[0207] Particles at the microscopic level are inherently uncertain: at the quantum level, particles can be in more than one place at the same time, or in more than one state of existence at the same time, and it is impossible to predict their exact quantum state (Heisenberg uncertainty principle).

[0208] Particles can be measured randomly in binary positions: a photon is one type of particle, the smallest particle of light, which can be set to have a specific polarity, which can directly correspond to the binary classical bits of "1" and "0" in classical computing systems.

[0209] Quantum systems cannot be measured without being altered: according to the laws of quantum physics, measuring the fundamental behavior of a quantum system will have irreversible effects on that system.

[0210] Particles can be partially cloned, but not completely: While some properties of particles can be cloned, 100% cloning is considered impossible (quantum non-cloning).

[0211] The above four basic characteristics ensure that QKD is a method of distributing secure digital keys over long distances protected by the microscopic physical properties of quantum mechanics.

[0212] The core of QKD is to ensure secure key distribution. If there is an eavesdropper during the key distribution process, QKD can ensure that the eavesdropper's presence is detected.

[0213] If eavesdropping is found, it means that the current key may be stolen. The QKD mechanism will interrupt the current key distribution process, abandon the current key, restart a new QKD key distribution process, generate a new key, and finally exchange the key securely.

[0214] A necessary condition for absolute security (or unconditional security) in cryptography is one-time pad.

[0215] One-time pad requirements: the key is completely random, the key length is the same as the plaintext length, and the key itself is only used once.

[0216] By utilizing the basic principles of quantum mechanics, we can easily achieve one-time one-pad and ensure that the key distribution process is absolutely secure; it is a confidential communication that is unconditionally secure in principle.

[0217] The system is mainly composed of the following units: authentication center management unit, multimodal sentiment analysis unit, quantum key preparation unit, data encryption transmission unit, and data decryption unit.

[0218] The technical solutions adopted by the present invention for the above units are described as follows:

[0219] Certification center management unit: The functions of this unit are divided into the following two points.

[0220] 1. The certification authority (CA) issues digital certificates to both communicating parties to ensure the authenticity of their identities and that only legitimate users can access the communication network.

[0221] 2. When the ciphertext is transmitted over the network through a traditional channel, the traditional communication channel is authenticated through the authentication center (CA) mechanism to ensure the security of data transmission.

[0222] Multimodal sentiment analysis unit: The purpose of this unit is to output multimodal sentiment analysis results as encrypted data information.

[0223] The multimodal sentiment analysis process is as follows:

[0224] 1. Use the camera to collect the user's facial video in real time to obtain visual data, use the microphone to collect the user's voice in real time to obtain voice data, and use voice recognition technology to transcribe the voice into text to obtain text data.

[0225] 2. Use tools such as OpenFace to extract facial features such as facial key points, head posture, and eye gaze. Use tools such as Librosa to extract speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q transform (CQT), and logarithmic fundamental frequency (log F0). Use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into a vector representation and extract text features.

[0226] 3. Unify and normalize the sequence length of these extracted multimodal features with different structures, so as to obtain a complete set of emotional feature information that can be used to train the emotional analysis neural network model, so as to facilitate the input of the model.

[0227] 4. Design a multimodal emotion recognition model including a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network. Use the CH-SIMS Chinese multimodal dataset for model training and use accuracy, F1-Score and other evaluation indicators to evaluate the model.

[0228] 4. Through the sentiment analysis network, an emotion score is output, which can reflect the emotional tendency of the observed person.

[0229] Quantum key preparation unit: The purpose of this unit is to obtain the key that can be used to encrypt data. The process is divided into the following three steps:

[0230] 1. The sender uses a quantum random number generator to generate a string of random bit sequences and converts each random bit into a quantum bit, which is represented by the polarization state of the photon in the present invention.

[0231] 2. The sender sends the encoded quantum state to the receiver through the quantum channel, and performs random basis selection and measurement. By comparing the basis selection information, the sender and receiver filter out the quantum states measured by both parties using the same basis, and obtain the corresponding bits to form a shared key.

[0232] 3. If an eavesdropper is found during the key distribution process, the key distribution process will be abandoned and a new shared key will be obtained.

[0233] 4. After screening, error correction and privacy amplification technology, the communicating parties obtain a final secure shared key, which is used to encrypt and protect the prediction results of multimodal sentiment analysis.

[0234] Data encryption transmission unit: This unit is used to encrypt the prediction results output during the multimodal sentiment analysis process and transmit them securely through traditional channels. The specific steps are divided into the following four steps:

[0235] 1. Select the symmetric encryption algorithm AES to encrypt the data, that is, to encrypt the prediction results;

[0236] 2. Initialize the encryption algorithm AES using the key generated by QKD;

[0237] 3. Encrypt the prediction results to generate ciphertext;

[0238] 4. Transmit the ciphertext over the network through traditional channels, and authenticate the traditional communication channels through the certification center (CA) mechanism to ensure the security of data transmission.

[0239] Data decryption unit: This unit is used to decrypt the received ciphertext, restore it to the original data and read it. The steps are divided into two steps:

[0240] 1. When the receiver receives the ciphertext, it uses the same key to decrypt it and convert the ciphertext back to the original data;

[0241] 2. The recipient securely accesses the prediction results of multimodal sentiment analysis.

[0242] A more detailed explanation of the key distribution process is now given.

[0243] First, channel preparation is carried out: quantum channel and traditional channel.

[0244] Quantum channels (quantum bit exchange) are used for key exchange.

[0245] The traditional channel is an authenticated channel so that man-in-the-middle attacks can be detected and a potential attacker is guaranteed not to be able to modify this information exchange.

[0246] Alice (Alice in cryptography usually refers to the sender) randomly generates photons totaling 2n qubits in the following four states: |0>, |1>, |+> and |->.

[0247] |0> and |1>: are the standard basis for quantum computing, called Basis or computational basis. Represents two orthogonal states of a qubit, corresponding to two orthogonal polarization states of a photon, such as horizontal polarization and vertical polarization.

[0248] |+> and |->: The basis obtained by Hadamard transformation is called Basis or Hadamard basis. They are superposition states of |0> and |1>.

[0249] The four types of photons are defined as follows:

[0250]

[0251] In order to measure the polarization state of photons, the measurement basis is introduced. and The measurement basis allows photons that are consistent with its polarization direction to pass through.

[0252] The photons in the states |↑> and |→> are measured based on During measurement, we can get |↑> and |→> respectively. This is the correct feedback result. If the state is The photons are measured based on When measuring, its status is:

[0253]

[0254] After being measured, There is a 50% probability of |↑> and a 50% probability of |→>.

[0255] Status is and The photons are measured based on When measuring, we can get and This is the correct feedback result. When measuring, its status is:

[0256]

[0257] After being measured, |↑> has a 50% probability of being 50% probability is

[0258] Since the measurement base and It is a non-orthogonal relationship, so the state of the photon may change after being measured by the random measurement basis of the listener.

[0259] In actual operation, the polarization states |1> and |+> of photons are assigned a value of 1, and the polarization states |0> and |-> are assigned a value of 0. The key generation process is illustrated by an example, as shown in the following table.

[0260]

[0261] The operations performed on the quantum channel are as follows:

[0262] Alice uses a quantum true random number generator to generate a binary random classical bit sequence [A], such as (11001001…), as shown in the first row of the table above.

[0263] Alice randomly assigns each digit of sequence [A] a Base or For example, if a bit of [A] is 1, if Alice specifies basis, then the 1 in this bit will be emitted using the quantum state |↑> instead of the quantum state As shown in the second and third rows of the table above.

[0264] Alice sends these quantum states to Bob (in cryptography, Bob usually refers to the recipient) through a quantum channel.

[0265] Bob randomly selects a measurement basis to measure these quantum states and obtains a series of binary strings related to 0 and 1, as shown in the sixth row of the table above.

[0266] Perform the following operations on the traditional channel:

[0267] Bob transmits the type of measurement basis he chooses to Alice. But he does not transmit the measurement result. For example, Bob informs Alice that the first choice Base, second choice Base, third choice Base..., as shown in the fourth row of the table above.

[0268] After receiving the measurement base information selected by Bob, Alice compares it and transmits the position information of the correctly selected measurement base to Bob. If the set of correctly selected positions is called set [R], Alice sends the set [R] information to Bob, as shown in the seventh row of the above table.

[0269] After receiving the information from set [R], Bob randomly selects a part of the positions in set [R] and records it as set [S], and sends the measurement results of the corresponding positions in set [S] to Alice. For example, Bob knows from set [R] that he has selected the correct measurement basis for the 1st, 2nd, 3rd, and 7th positions, then Bob randomly selects a few positions, for example, the 1st and 7th positions, and records the information of these two positions, which is set [S]. Bob sends the measurement results (0 or 1) of the corresponding positions in set [S] to Alice. This is shown in the eighth and ninth rows of the table above.

[0270] After Alice receives the measurement result, she checks her own measurement result of the corresponding position in set [S]. If the results are consistent, Alice sends a message to inform Bob that he has not been eavesdropped. Then both parties remove the position corresponding to set [S] in set [R] and select the bit information corresponding to the remaining position as the shared key for communication between the two parties. As shown in the tenth row of the table above.

[0271] The safety proof of this method is as follows:

[0272] In this process, if the eavesdropper Eve eavesdrops on the traditional channel, the following two situations will occur:

[0273] 1. When Bob transmits the type of measurement basis he chooses to Alice, Eve eavesdrops on Bob’s measurement method, but Bob does not send the measurement results.

[0274] 2. Eve will eavesdrop on some of Bob’s correctly measured bits, namely set [S] and the measured results, but in the end Alice and Bob will completely discard set [S] when choosing the communication key.

[0275] Therefore, Eve only knows which bits of [A] Alice and Bob use as communication keys, but does not know the specific values ​​of these bits.

[0276] Therefore, security can be guaranteed even if the traditional channel is eavesdropped.

[0277] If the eavesdropper Eve eavesdrops on the quantum channel, the following two situations will occur:

[0278] 1. When Alice sends the quantum state to Bob, Eve can eavesdrop but cannot determine the type of measurement basis chosen.

[0279] 2. If Eve chooses a random measurement basis to measure the quantum state, then Eve will inevitably choose the wrong type of measurement basis, which will cause some quantum states to change. For example, the value of [A] in a certain position is 1, Alice uses |↑> to send information, but Eve chooses the wrong measurement basis Then this quantum state may become or If Bob happens to choose the measurement basis Measuring Eve's eavesdropping (or ), quantum state results (or ) becomes a |→> quantum state (or |↑> quantum state, if the information happens to be correctly transmitted, then this bit will not attract the attention of Alice and Bob), and the value that was originally 1 is measured by Bob's correct measurement basis but is 0. When selecting the set [S], if Bob happens to select this bit and send it to Alice, Alice will find that Bob used the correct measurement basis for this bit but did not get the correct value. When the frequency of this situation exceeds a certain threshold, Alice will realize that the quantum channel has been eavesdropped.

[0280] Therefore, the quantum key distribution protocol can generate a key that only the communicating parties know. This key is random and secure, because during the communication process, any attempt to eavesdrop on the key will be discovered by both parties in a timely manner.

[0281] The specific operation of the multimodal sentiment analysis unit is described below:

[0282] 1. Data collection and preprocessing: Use the camera to collect the user's facial video in real time to obtain visual data, use the microphone to collect the user's voice in real time to obtain voice data, and use speech recognition technology to transcribe the voice into text to obtain text data.

[0283] 2. Multimodal feature extraction and security processing: Use tools such as OpenFace to extract facial features such as facial key points, head posture, and eye gaze status; use tools such as Librosa to extract speech features such as Mel-frequency cepstral coefficients (MFCCs), constant Q transform (CQT), and logarithmic fundamental frequency (log F0); use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features.

[0284] 3. Feature normalization: The extracted multimodal features with different structures are normalized and processed to unify the sequence length for the input of the model. Thus, a complete set of emotional feature information that can be used to train the emotional analysis neural network model is obtained. The emotional feature information set will be transmitted to the host of the analysis end via the Internet.

[0285] 4. Model training and evaluation: After receiving the set of emotional information, the analysis end constructs the model and designs a multimodal emotion recognition model including a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and an emotion analysis network. The CH-SIMS Chinese multimodal dataset is used for model training, and the model is evaluated using evaluation indicators such as accuracy and F1-Score.

[0286] The functions of these networks are as follows:

[0287] Feature embedding subnetwork: further embeds and adjusts the dimensions of features of different modalities.

[0288] Multi-head self-attention mechanism: Run multiple independent attention mechanisms in parallel to obtain attention distributions in different subspaces of the input sequence.

[0289] Tensor fusion based feature fusion network: Constructing a triple Cartesian product representation of tri-modal feature sequences to learn intra-modal and inter-modal dynamics.

[0290] Sentiment analysis network: The fused features are processed through the fully connected layer, and finally the Sigmoid activation function is used to obtain the sentiment prediction result.

[0291] 4. Output prediction results: Through the sentiment analysis network, the model will output a sentiment score that can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0292] Embodiment 16:

[0293] The system flow chart of the application of quantum key distribution technology in the field of emotional demand analysis includes the following parts:

[0294] S1, identity authentication; S2, multimodal sentiment analysis; S3, shared quantum key preparation; S4, data encryption; S5, data transmission; S6, data decryption.

[0295] The contents of each step are as follows:

[0296] S1. Identity verification: The certification authority (CA) issues digital certificates to both communicating parties to ensure the authenticity of their identities and to ensure that only legitimate users can access the communication network.

[0297] S2. Multimodal sentiment analysis: Network multimodal sentiment analysis is an end-to-end process. The process first needs to collect and process multimodal data such as video, audio and text containing emotional expressions through devices such as cameras and microphones. Next, facial features such as facial key points and head posture are extracted by using tools such as OpenFace; speech features such as Mel frequency cepstral coefficients (MFCCs), constant Q transform (CQT) and logarithmic fundamental frequency are extracted by using tools such as Librosa; the speech is transcribed into text information using the Whisper deep learning model, and then the text information is converted into vector representation using the BERT model to extract text features. Then these multidimensional feature data with different structures are integrated and aligned to obtain a complete set of emotional feature information that can be used to train the emotional analysis neural network model. The emotional feature information set will be transmitted to the host of the analysis end via the Internet. After receiving the emotional information set, the analysis end constructs the model and designs a multimodal emotion recognition model including a feature embedding subnetwork, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and an emotional analysis network. The model analyzes the relationship between each modal feature, concatenates and linearly transforms the attention distribution of multiple subspaces, and obtains the final fusion feature. Finally, through the sentiment analysis network, the model outputs a sentiment score that can reflect the emotional tendency of the observed person, such as positive, negative, neutral, etc.

[0298] S3, preparation of shared quantum key: The sender generates a series of random bit sequences using a quantum true random number generator and converts these bit sequences into quantum bits. In the implementation of the present invention, the preparation of quantum bits uses the polarization state of photons. Subsequently, the sender transmits these encoded quantum states to the receiver through a quantum channel. After receiving the quantum state, the receiver selects and measures the random basis. By comparing the basis selection information of both parties, the sender and the receiver jointly screen out the quantum states measured under the same basis, and then extract the corresponding bit sequence according to the predetermined selection rules to form a shared key.

[0299] In order to ensure the security of the key distribution process, a security detection mechanism is introduced. If potential eavesdropping is detected during the key distribution process, a new QKD key distribution process is restarted to generate a new key until the key is finally exchanged securely.

[0300] In addition, to further enhance the security of the key, privacy amplification technology is used to reduce the risk of information leakage to potential eavesdroppers. Through a series of screening, error correction and privacy amplification processing, the communicating parties finally obtain a highly secure shared key.

[0301] S4. Data encryption: To ensure the security of data transmission during the online video sentiment analysis process, the symmetric encryption algorithm, Advanced Encryption Standard (AES), is used to encrypt the data. The key generated by the application of quantum key distribution (QKD) technology is used to initialize the working mode of the AES algorithm, and the prediction results of the multimodal sentiment analysis are encrypted to generate the corresponding ciphertext. This process can ensure the confidentiality and integrity of the data during transmission.

[0302] S5. Data transmission: During network transmission, ciphertext will be transmitted through traditional communication channels. In order to enhance the security of data transmission, the certification authority (CA) mechanism is introduced. As a trusted third-party organization, the certification authority is responsible for authenticating traditional communication channels to ensure the security and reliability of data during transmission. Through the CA mechanism, the identities of both parties in communication can be verified to prevent unauthorized access and potential man-in-the-middle attacks, thus providing an additional layer of security for data transmission of network video sentiment analysis.

[0303] S6. Data decryption: When the receiver successfully receives the ciphertext, it uses the pre-prepared quantum shared key to decrypt it. Through this process, the ciphertext is converted back to its original form of data. In this way, the receiver can safely retrieve and access the results of the emotional needs analysis, ensuring the confidentiality and integrity of the data during transmission.

[0304] In the process of preparing a shared quantum key, the specific steps are as follows:

[0305] The transmitter generates a single photon source through a semiconductor quantum dot photon generator, and then processes the photons using a polarizer to obtain the polarization state of the photons;

[0306] The key information is represented by the polarization characteristics of photons. The four polarization states are represented by |0>, |1>, |+> and |->. It is stipulated that the polarization states |1> and |+> are assigned to 1, and the polarization states |0> and |-> are assigned to 0.

[0307] The sender randomly assigns each bit of sequence [A] to its The measurement base is Measurement base.

[0308] The sender sends these quantum states, i.e., sequence [A], to the receiver through the quantum channel.

[0309] The receiving end randomly uses a series of measurement bases to measure the quantum state in [A] and obtains a series of binary bit information related to 0 and 1.

[0310] The receiving end transmits the information of the measurement basis to the sending end, but does not transmit the measurement results.

[0311] After receiving the information of the selected measurement basis type, the sender immediately compares it with the measurement basis type selected by itself, transmits the bit order information of the measurement basis selected in a consistent manner to the receiver, and calls the set of correctly measured bits set [R].

[0312] After receiving the set [R] information, the receiving end randomly selects a part of the positions in the set [R] and records it as the set [S], and transmits the measurement results of the corresponding positions in the set [S] to the sending end.

[0313] The sender checks the result of the corresponding position of the set [S]. If it is correct, the sender sends a message to notify the receiver that there is no eavesdropping in the key distribution process.

[0314] The two parties then remove the bits corresponding to set [S] in set [R] and select the bit information corresponding to the remaining bits as the shared key for communication between the two parties.

[0315] Therefore, the quantum key distribution protocol can generate a key that only the communicating parties know. This key is random and secure, because during the communication process, any attempt to eavesdrop on the key will be discovered by both parties in a timely manner.

[0316] Through the above detailed process and ideas, the present invention provides an effective method for protecting the privacy of network video sentiment analysis data in the quantum era. This method not only uses quantum key distribution technology to ensure the security of data transmission, but also combines multimodal sentiment analysis and authentication center mechanism to provide a comprehensive privacy protection solution for network video sentiment analysis.

Claims

1. A privacy protection system in network multimodal sentiment analysis based on quantum key distribution, characterized in that: Including the sending end and the receiving end. The sending end includes an authentication center management unit, a multimodal sentiment analysis unit, a quantum key preparation unit, and a data encryption transmission unit. The authentication center management unit is used to perform identity authentication and data transmission security authentication. If the authentication is successful, an authentication success signal is sent to the multimodal sentiment analysis unit; After receiving the authentication success signal, the multimodal emotion analysis unit collects and processes the multimodal data to obtain emotion feature information, and then uses the multimodal emotion recognition model to analyze the emotion feature information to obtain an emotion score reflecting the user's emotional tendency; The multimodal sentiment analysis unit transmits the multimodal data and the sentiment score to the data encryption transmission unit; The quantum key preparation unit generates a quantum key and transmits it to a quantum key verification unit at the receiving end; The data encryption transmission unit encrypts the multimodal data and the emotion score using a quantum key, and transmits the data to a receiving end through a traditional channel; The receiving end includes a quantum key verification unit and a data decryption unit; The quantum key verification unit receives and verifies the quantum key and feeds it back to the sending end; The data decryption unit decrypts the received data using a quantum key.

2. According to the privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1, it is characterized in that: The authentication center is used to issue digital certificates to both communicating parties to ensure the authenticity of their identities, as well as to authenticate the communication channels to ensure the security of data transmission.

3. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1 is characterized in that: The multimodal data includes user facial video data and user voice data; User facial video data is collected through the camera, and user voice data is collected through the microphone.

4. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1 is characterized in that: The steps of processing multimodal data by the multimodal sentiment analysis unit include: Use OpenFace tools to extract visual features, including facial key points and head posture; Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency; The Whisper deep learning model is used to transcribe user voice data into text information, and the BERT model is used to convert the text information into vector representation and extract text features.

5. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1, characterized in that: The steps of analyzing emotional feature information using a multimodal emotion recognition model include: 1) Integrate and align the multimodal data after feature extraction to obtain a complete set of emotional feature information and transmit it to the analysis end; 2) The analysis end uses a multimodal emotion recognition model to analyze the relationship between the features of each modality, concatenates and linearly transforms the attention distribution of multiple subspaces to obtain fused features, and processes the fused features to obtain an emotion score that reflects the user's emotional tendency; the user's emotional tendency includes positive, negative, and neutral.

6. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1, characterized in that: The modal emotion recognition model includes a feature embedding sub-network, a multi-head self-attention mechanism, a feature fusion network based on tensor fusion, and a sentiment analysis network.

7. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 1 is characterized in that: The steps of generating a quantum key by the quantum key preparation unit include: 1) The quantum key preparation unit at the transmitting end generates a string of random quantum bit sequences using a quantum true random number generator, and sends the quantum bit sequence to the receiving end through a quantum channel; the quantum state of the quantum bit sequence is the polarization state of the photon; the polarization state of the photon is generated by: generating a single photon source through a semiconductor quantum dot photon generator, and then processing the photon with a polarizer to obtain the polarization state of the photon; 2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence; 3) Compare and select the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extract the corresponding bit sequence according to the preset selection rules to form the quantum key; 4) Determine whether there is eavesdropping. If so, abandon the current key distribution process and regenerate a new key.

8. The privacy protection system in network multimodal sentiment analysis based on quantum key distribution according to claim 7 is characterized in that: The quantum states include horizontal polarization state, vertical polarization state, 45 degree polarization state and 135 degree polarization state; The horizontal polarization state is denoted as |0> and is assigned a value of 0; The vertical polarization state is denoted as |1> and is assigned a value of 1; The 45-degree polarization state is denoted as |+> and is assigned a value of 1; The 135-degree polarization state is denoted as |-> and is assigned a value of 0.

9. A privacy protection method based on the system according to any one of claims 1 to 8, characterized in that: The following steps are involved: 1) Verify the identities of both parties in communication through the authentication center; 2) Collect multimodal data, perform feature extraction and emotional state analysis, and obtain emotional scores that reflect the user's emotional tendencies; The multimodal data includes user facial video data, user voice data, and text data converted from the voice data; The steps of feature extraction include: using OpenFace tools to extract visual features, including facial key points and head posture; Use Librosa tools to extract speech features, including Mel-frequency cepstral coefficients, constant Q transform, and logarithmic fundamental frequency; Use the Whisper deep learning model to transcribe speech into text information, and then use the BERT model to convert the text information into vector representation and extract text features; 3) Using a quantum true random number generator to generate a random bit sequence, and transmitting the encoded quantum state through a quantum channel to prepare a quantum key, the steps include: 3.1) The quantum key preparation unit at the sending end uses a quantum true random number generator to generate a string of random quantum bit sequences, and sends the quantum bit sequence to the receiving end through the quantum channel; 3.2) The quantum key verification unit at the receiving end randomly selects a measurement basis to measure the received quantum bit sequence; 3.3) Compare and select the quantum states measured by the quantum key preparation unit and the quantum key verification unit using the same measurement basis, and then extract the corresponding bit sequence according to the preset selection rules to form the quantum key; 3.4) Determine whether there is eavesdropping. If so, discard the current key and regenerate a new key; 4) Based on the symmetric encryption algorithm AES, the multimodal data and sentiment scores are encrypted using quantum keys to generate ciphertext; 5) Using traditional communication channels to transmit ciphertext; 6) The receiver uses the shared quantum key to decrypt the ciphertext and restore the original data.

10. The privacy protection method in network multimodal sentiment analysis based on quantum key distribution according to claim 9 is characterized in that: In step 3.3), the steps of extracting the corresponding bit sequence according to the preset selection rule are: 3.3.1) Generate a binary true random number sequence using the principles of quantum mechanics as the measurement basis sequence; 3.3.2) Randomly select from the measurement basis sequence Base or Basis, as a measurement basis; Two sets of measurement bases measure the received quantum states, and the measurement bases allow photons that are consistent with their own polarization directions to pass through; If the basis selected for preparing the quantum state is consistent with the basis selected for measurement, it is recorded as the correct measurement basis. If not, the corresponding bit data is deleted. A portion of the positions in the correct measurement basis are randomly selected as random bits. For each random bit, if the measurement result is consistent with the sender, it is considered that there is no eavesdropping. The positions remaining after removing the random bits from the correctly measured base bits are recorded as valid bits, and the key is determined based on the measurement results corresponding to the valid bits.

Images