A personalized content recommendation method and system based on multi-modal perception

Abstract

The application relates to the technical field of personalized content recommendation, and discloses a personalized content recommendation method and system based on multi-modal perception, which comprises the following steps: acquiring multi-modal data and preliminarily compressing the multi-modal data into low-dimensional potential representations, extracting cross-dimension correlation features and performing feature fusion to generate a unified feature vector, acquiring potential modes, calculating the correlation coefficients of each dimension and sensitive attributes, starting data disturbance processing if the correlation coefficients exceed a threshold value, obtaining an encoding representation of hidden sensitive attributes, simultaneously monitoring the core feature retention rate, dynamically adjusting the disturbance intensity if the retention rate is too low, performing iterative optimization, acquiring optimized encoding features, performing matching detection on the optimized encoding features and an encoding feature library, determining a personalized content list, obtaining an original recommendation result, performing privacy protection detection and optimization on the original recommendation result, and obtaining a final recommendation result. The method solves the imbalance between privacy protection and recommendation accuracy in personalized content recommendation.

Description

Technical Field

[0001] This invention relates to the field of personalized content recommendation technology, and in particular to a personalized content recommendation method and system based on multimodal perception. Background Technology

[0002] In the digital age, the importance of personalized content recommendation is becoming increasingly prominent. It's not only closely related to improving user experience, but also directly impacts a platform's user appeal and business profitability. By analyzing user behavior and preferences, personalized content recommendation systems can push content that matches users' interests, thereby optimizing the efficiency of information acquisition.

[0003] Current personalized recommendation systems rely solely on single-dimensional user data. When processing multi-dimensional user data, they lack the ability to integrate information from different perceptual dimensions. This fragmented information makes it difficult for the system to capture the full picture of user preferences, resulting in recommendation results that fail to comprehensively capture user needs. Furthermore, existing methods often directly expose sensitive information when processing user data and lack effective encoding mechanisms to hide sensitive attributes.

[0004] Therefore, there is an imbalance between privacy protection and recommendation accuracy in the multimodal data processing of personalized content recommendation, which directly affects the credibility of the personalized content recommendation system and users' trust in the platform. Summary of the Invention

[0005] This invention provides a personalized content recommendation method and system based on multimodal perception, which can solve the problem of the imbalance between privacy protection and recommendation accuracy in multimodal data processing for personalized content recommendation.

[0006] Firstly, to address the aforementioned technical problems, this invention provides a personalized content recommendation method based on multimodal perception, comprising:

[0007] Acquire multimodal data, perform preliminary compression processing on the multimodal data, and obtain a low-dimensional latent representation;

[0008] Based on the low-dimensional latent representation, cross-dimensional related features are obtained and feature fusion processing is performed to obtain a unified feature vector.

[0009] Based on the unified feature vector, a latent pattern is obtained. If the correlation coefficient of the sensitive attribute in the latent pattern exceeds a preset sensitivity threshold, data perturbation processing is performed to obtain the encoded representation of the hidden sensitive attribute.

[0010] Based on the unified feature vector, the core feature retention rate is calculated. If the core feature retention rate is lower than the preset retention rate threshold, the encoding representation of the hidden sensitive attribute is adjusted to obtain the adjusted encoding version.

[0011] The adjusted encoding version is iteratively optimized to obtain optimized encoding features;

[0012] The optimized coding features are matched and detected with a preset coding feature library to determine a personalized content list and obtain the original recommendation result;

[0013] The original recommendation results are then subjected to privacy protection checks and optimizations to obtain the final recommendation results.

[0014] In one optional implementation, the step of acquiring multimodal data and performing preliminary compression processing on the multimodal data to obtain a low-dimensional latent representation includes:

[0015] We acquire interaction records and extract text, image, and audio data from social media platforms, sensors, and log files to obtain the raw multimodal dataset; we then perform regularization using a Sigmund distribution sampling method.

[0016] The original multimodal data is standardized to obtain a multimodal dataset with a uniform format;

[0017] The multimodal dataset is encoded using a preset variational autoencoder to extract features from each modality and obtain a low-dimensional latent representation.

[0018] In one optional implementation, the step of obtaining cross-dimensional correlation features based on the low-dimensional latent representation and performing feature fusion processing to obtain a unified feature vector includes:

[0019] Based on the low-dimensional latent representation, features of each modality are extracted to obtain feature sets of each modality;

[0020] Principal component analysis is used to reduce the dimensionality of each modal feature set to obtain a cross-dimensional feature set.

[0021] An attention mechanism is used to assign weights and perform weighted fusion on the cross-dimensional feature set to obtain a unified feature vector.

[0022] In one optional implementation, the step of obtaining a latent pattern based on the unified feature vector, and if the correlation coefficient of sensitive attributes in the latent pattern exceeds a preset sensitivity threshold, then performing data perturbation processing to obtain an encoded representation of the hidden sensitive attributes, includes:

[0023] The unified feature vector is decomposed using principal component analysis to obtain a set of latent patterns;

[0024] Calculate the correlation coefficient of the sensitive attributes of any pattern in the set of potential patterns;

[0025] Gaussian noise is injected into the pattern data whose correlation coefficient of the sensitive attribute exceeds a preset sensitivity threshold to obtain perturbed pattern data.

[0026] The perturbed pattern data is encoded using a preset variational autoencoder to obtain an encoded representation of the hidden sensitive attributes.

[0027] In one optional implementation, the step of calculating the core feature retention rate based on the unified feature vector, and if the core feature retention rate is lower than a preset retention rate threshold, adjusting the encoded representation of the hidden sensitive attributes to obtain an adjusted encoded version, includes:

[0028] For the unified feature vector set, the initial representation of the core features is determined by principal component analysis, and the core feature retention rate is calculated.

[0029] If the core feature retention rate is lower than the preset retention rate threshold, the preset variational autoencoder parameters are adjusted using the gradient descent algorithm to obtain the adjusted parameter configuration.

[0030] The modified parameter configuration is used to adjust the encoding representation of the hidden sensitive attributes to obtain a preliminary encoding representation;

[0031] Verify the consistency of the business data represented by the preliminary encoding, and further process the noise perturbation of the part related to sensitive attributes to obtain an adjusted encoding version that hides the sensitive attributes.

[0032] In one optional implementation, the iterative optimization of the adjusted encoding version to obtain optimized encoding features includes:

[0033] Based on the adjusted encoding version, a preset variational autoencoder is used to perform preliminary encoding processing on the multimodal data to obtain preliminary optimized feature representations;

[0034] To verify the consistency of the feature distribution of the preliminary optimized feature representation, regularization processing is performed through Gaussian distribution sampling to obtain a regularized feature representation;

[0035] Based on the regularized feature representation, the preset variational autoencoder parameters are iteratively optimized using the gradient descent algorithm to obtain the optimized parameter configuration.

[0036] Using the optimized parameter configuration, the multimodal data is processed by the variational autoencoder to obtain the optimized coding features.

[0037] In one optional implementation, the step of matching and detecting the optimized coding features with a preset coding feature library to determine a personalized content list and obtain the original recommendation result includes:

[0038] The optimized coding features are matched with a preset coding feature library, and a recommendation candidate set is obtained through collaborative filtering;

[0039] Content filtering is used to sort the content in the recommendation candidate set to obtain a personalized content list;

[0040] The personalized content list is encrypted to obtain the original accurate recommendation results.

[0041] In one optional implementation, the step of performing privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results includes:

[0042] Calculate the matching degree between the optimized encoding features and the pre-stored historical recommendation data. If the matching degree is lower than a preset sensitivity threshold, enhance the cross-dimensional integration strength through an attention mechanism to obtain an improved recommendation candidate set.

[0043] The enhanced recommendation candidate set is subjected to protection detection, and the core feature retention rate judgment result is integrated. If the retention rate index meets the privacy sensitivity threshold after being correlated with the matching degree, privacy protection optimization processing is performed, the weight distribution is adjusted, and a privacy-enhanced candidate version is obtained. Then, the privacy-enhanced candidate version is compared with the encoded representation of the hidden sensitive attribute. If the retention rate is optimized, the final recommendation set with privacy protection optimization is obtained.

[0044] The final recommendation set optimized for privacy protection is compared with the user's current behavior data to obtain the final recommendation result.

[0045] Secondly, the present invention provides a personalized content recommendation system based on multimodal perception, comprising:

[0046] The data feature extraction module is used to acquire raw multimodal data, perform preliminary compression processing on the raw multimodal data, and generate a low-dimensional latent representation;

[0047] The data feature fusion module is used to acquire cross-dimensional related features and perform feature fusion processing to obtain a unified feature vector.

[0048] The data privacy protection module is used to detect potential patterns where the correlation coefficient of sensitive attributes exceeds a preset sensitivity threshold, and to perform data perturbation processing to obtain the encoded representation of the hidden sensitive attributes;

[0049] The data adjustment module is used to calculate the core feature retention rate of the encoded representation of hidden sensitive attributes. If the core feature retention rate is lower than the retention rate threshold, the encoding parameters are adjusted and the encoded version of the multimodal data is regenerated to obtain the adjusted encoded version.

[0050] The data iteration and optimization module is used to iteratively optimize the adjusted encoding version and obtain optimized encoding features;

[0051] The recommendation generation module is used to match and detect the optimized coding features with a preset coding feature library to determine a personalized content list and obtain the original recommendation result;

[0052] The privacy protection detection and optimization module is used to perform privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) This invention acquires multimodal data, initially compresses the multimodal data into a low-dimensional latent representation, extracts cross-dimensional correlation features, and performs feature fusion to generate a unified feature vector. This invention realizes the transformation from single-dimensional analysis to multi-dimensional analysis, breaks down information barriers between perceptual dimensions through feature fusion, and constructs a unified feature expression that can comprehensively reflect the complex interest patterns of users. This processing method not only significantly improves the completeness and accuracy of feature expression, but also lays a solid data foundation for subsequent accurate recommendations.

[0055] (2) This invention obtains latent patterns based on a unified feature vector, calculates the correlation coefficients between each dimension and sensitive attributes, and initiates data perturbation processing if the correlation coefficients exceed a preset sensitivity threshold to obtain an encoded representation of the hidden sensitive attributes. Simultaneously, it monitors the retention rate of core features; if the retention rate is too low, it dynamically adjusts the perturbation intensity. The adjusted encoded version is then iteratively optimized to obtain optimized encoded features. This approach only applies minimal interference to data patterns that genuinely pose privacy risks, thereby effectively obfuscating sensitive attributes while maximizing the retention of core features for recommendation. This allows the system to maintain high-quality personalized recommendation performance while strictly protecting user sensitive information from leakage.

[0056] (3) This invention determines a personalized content list by matching the optimized coding features with a preset coding feature library to obtain the original recommendation results. Then, the original recommendation results are subjected to privacy protection detection and optimization to obtain the final recommendation results. After generating the initial recommendation results, the system does not output them directly, but instead adds a dedicated privacy protection detection and optimization step to perform a final security review and calibration of the recommendation results. This design forms an effective secondary protection barrier, capable of identifying and eliminating the risk of sensitive information association that may remain in the previous processing, ensuring that the final output recommendation content accurately matches user interests and fully complies with privacy protection standards. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a personalized content recommendation method based on multimodal perception provided in the first embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of a personalized content recommendation system based on multimodal perception provided in the second embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Reference Figure 1 The first embodiment of the present invention provides a schematic diagram of a personalized content recommendation method based on multimodal perception, including steps S101 to S107, as follows:

[0061] Step S101: Acquire multimodal data, perform preliminary compression processing on the multimodal data, and obtain a low-dimensional latent representation;

[0062] Step S102: Based on the low-dimensional latent representation, obtain cross-dimensional related features and perform feature fusion processing to obtain a unified feature vector;

[0063] Step S103: Based on the unified feature vector, obtain the latent pattern. If the correlation coefficient of the sensitive attribute in the latent pattern exceeds the preset sensitivity threshold, perform data perturbation processing to obtain the encoded representation of the hidden sensitive attribute.

[0064] Step S104: Calculate the core feature retention rate based on the unified feature vector. If the core feature retention rate is lower than the preset retention rate threshold, adjust the encoding representation of the hidden sensitive attribute to obtain the adjusted encoding version.

[0065] Step S105: Iteratively optimize the adjusted encoding version to obtain optimized encoding features;

[0066] Step S106: Match the optimized coding features with the preset coding feature library to determine the personalized content list and obtain the original recommendation result;

[0067] Step S107: Perform privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results.

[0068] In step S101, multimodal data is acquired, and the multimodal data is subjected to preliminary compression processing to obtain a low-dimensional latent representation.

[0069] In one optional implementation, the step of acquiring multimodal data and performing preliminary compression processing on the multimodal data to obtain a low-dimensional latent representation includes:

[0070] The interaction records were obtained, and text, image, and audio data were extracted from social platforms, sensors, and log files to obtain the raw multimodal dataset.

[0071] The original multimodal data is standardized to obtain a multimodal dataset with a uniform format;

[0072] The multimodal dataset is encoded using a preset variational autoencoder to extract features from each modality and obtain a low-dimensional latent representation.

[0073] The acquisition of interaction records from multimodal data sources can be achieved through pre-designed data collection tools. For example, for social media platforms, web crawlers can be used to extract user comments, images, and voice messages from Weibo or WeChat; sensor data can be obtained from smartwatches, such as user heart rate and movement trajectory; and log files can be extracted from servers to record user actions. These data sources together constitute the original multimodal dataset.

[0074] It's worth noting that social media platform data may contain text in JSON format, images in BMP format, and audio in WAV format, while sensor data may be in CSV format and log files in TXT format. Such format differences can lead to difficulties in subsequent processing. Data preprocessing can standardize these data. For example, text data can be uniformly converted to UTF-8 encoding, image data can be adjusted to a uniform resolution such as 256x256 pixels and converted to JPEG format, and audio data can be uniformly converted to MP3 format with a 16kHz sampling rate. Preferably, text data is normalized through word segmentation and stop word removal; image data is standardized in size and color space through cropping and grayscale processing; and audio data is processed through noise reduction and volume standardization. This standardization ensures that different modalities of data have a consistent input format during encoding.

[0075] It is worth noting that the pre-defined variational autoencoder is used to encode a consistent multimodal dataset to extract latent features. The consistent multimodal dataset serves as the input to the variational autoencoder. In each training iteration, mini-batch samples sampled from the data distribution are input to the encoder. First, a modal encoding strategy is employed: convolutional neural networks process image data, recurrent neural networks process text sequences, and VGGish processes audio signals, mapping the original data of each modality to a probability distribution space. The encoder outputs a mean vector and a variance vector for each modality sample. Through reparameterization techniques, smooth and continuous latent feature vectors are generated by sampling from this distribution. The decoder reconstructs these latent feature vectors into the data for each modality and optimizes them using a dual-objective loss function. The objective function consists of a reconstruction loss and a KL divergence regularization term: the reconstruction loss measures the negative log-likelihood of the decoder output to the original input, while the KL divergence regularization optimizes the latent representations of text, images, and audio, aligning them within the same latent space, facilitating unified modeling of multimodal data. Then, backpropagation is used to calculate the gradient of the objective function with respect to the parameters, and optimization methods such as adaptive moment estimation are employed to update the network weights.

[0076] In step S102, cross-dimensional related features are obtained based on the low-dimensional latent representation and feature fusion processing is performed to obtain a unified feature vector.

[0077] Based on the low-dimensional latent representation, features of each modality are extracted to obtain feature sets of each modality;

[0078] Principal component analysis is used to reduce the dimensionality of each modal feature set to obtain a cross-dimensional feature set.

[0079] An attention mechanism is used to assign weights and perform weighted fusion on the cross-dimensional feature set to obtain a unified feature vector.

[0080] This process involves acquiring interaction records from multimodal data sources using pre-established acquisition tools to extract visual, auditory, and tactile signals from social media platforms, sensors, and log files. For example, efficient acquisition of interaction records from multimodal data sources can be achieved through customized tools. For social media platforms, web crawlers can be designed to extract user-uploaded video clips, bullet screen text, and likes from short video platforms. Sensor data can be collected via smart bracelets to capture tactile signals such as user cadence and skin conductance. Log files can be used to extract user click paths and dwell times from application servers. These data represent the feature sets of each modality of visual, auditory, and tactile signals, respectively.

[0081] It's worth noting that due to the dimensionality differences between the feature sets of different modalities, direct fusion can lead to an imbalance in feature weights. Principal component analysis (PCA) can be used for dimensionality reduction. For example, all features can be uniformly reduced to 256 dimensions to generate a cross-dimensional feature set. This process can retain more than 90% of the data variance, ensuring minimal information loss. The dimensionality-unified feature set facilitates cross-modal fusion.

[0082] It's worth noting that the attention mechanism can assign weights to cross-dimensional feature sets. After heterogeneous feature vectors extracted from different sensory channels (visual, auditory, and tactile) across 256 dimensions enter the fusion layer, a channel-level attention map is obtained by calculating the correlation score between any two feature dimensions using a self-attention unit. This score is then used to weight and label the original features, assigning high weights to dimensions highly correlated with the current user's interests, ensuring their information dominates in subsequent fusion, while weakly correlated or redundant dimensions are suppressed to low weights. Finally, the weighted features interact through a cross-modal attention module, generating a unified feature vector with interpretable weights. For example, human actions in a video might contribute more to sentiment analysis, with a weight of 0.6; the rhythm of background music has a weight of 0.3; and tactile signals have a weight of 0.1. The softmax function normalizes these weights, generating a uniform weight distribution. Ultimately, the fused unified feature vector comprehensively reflects the user's interactive behavior.

[0083] In step S103, a latent pattern is obtained based on the unified feature vector. If the correlation coefficient of the sensitive attribute in the latent pattern exceeds a preset sensitivity threshold, data perturbation processing is performed to obtain the encoded representation of the hidden sensitive attribute.

[0084] The unified feature vector is decomposed using principal component analysis to obtain a set of latent patterns;

[0085] Calculate the correlation coefficient of the sensitive attributes of any pattern in the set of potential patterns;

[0086] Gaussian noise is injected into the pattern data whose correlation coefficient of the sensitive attribute exceeds a preset sensitivity threshold to obtain perturbed pattern data.

[0087] The perturbed pattern data is encoded using a preset variational autoencoder to obtain an encoded representation of the hidden sensitive attributes.

[0088] In multimodal data fusion scenarios, Principal Component Analysis (PCA) can be used to decompose a unified feature vector to extract a set of latent patterns. PCA maps high-dimensional features to a low-dimensional space through linear transformation, preserving the main data variance. For example, a 512-dimensional unified feature vector of a short video platform user can be decomposed into several latent patterns using PCA. Each pattern reflects different aspects of user behavior, such as viewing preferences, interaction frequency, or emotional responses. Assuming the decomposition yields 10 latent patterns, each represented by a set of feature vectors, 95% of the original data variance is preserved. This decomposition helps reveal deeper patterns in user behavior, facilitating subsequent analysis.

[0089] It's worth noting that determining whether the correlation coefficient of sensitive attributes for any pattern in the latent pattern set exceeds a preset threshold requires first defining the sensitive attributes. Sensitive attributes refer to personal privacy characteristics that require special protection and can be potentially inferred from multimodal behavioral data. These can be preset through business rules or learned from labeled data. Specifically, these attributes are not directly stored raw data fields, but rather refer to information related to a user's identity, state, or inherent characteristics. For example, in short video platforms, sensitive attributes include, but are not limited to, a user's emotional state or bias. Emotional state can be defined based on positive / negative scores derived from sentiment analysis of user comment text, and bias can be defined by calculating the standard deviation of click-through rates for different content categories. The correlation coefficient can be obtained by calculating the Pearson correlation coefficient between the latent pattern vector and the sensitive attribute vector. Referring to historical sensitive data correlation patterns, by analyzing the correlation coefficient between the pattern vector and the sensitive attribute vector in past sensitive events, when this value exceeds 0.7, the success rate of inferring sensitive attributes from the pattern vector increases significantly. Therefore, the preset sensitivity threshold is 0.7. If the coefficient of a pattern reaches 0.8, it indicates that the pattern is highly correlated with sensitive attributes and may leak user privacy. This judgment helps identify data patterns that need protection and ensures the security of subsequent processing.

[0090] It is worth noting that perturbing pattern data exceeding a threshold using Gaussian noise injection can effectively hide sensitive information. Typically, the noise mean is set to 0 to avoid introducing systematic bias; the variance is dynamically adjusted based on the data sensitivity. For highly sensitive pattern data exceeding the threshold, a standard deviation of 1.5-2 times the data standard deviation is used to ensure the original pattern is difficult to reconstruct after perturbation. For less sensitive but still protectable pattern data, a smaller standard deviation of 0.5-1 times the data standard deviation is used to preserve the overall data distribution characteristics while hiding sensitive information. For example, for a pattern with a correlation coefficient of 0.8, Gaussian noise with a mean of 0 and a standard deviation of 0.5-1 times the data standard deviation can be introduced. The perturbed pattern data still retains its main behavioral characteristics, but the identifiability of sensitive attributes is significantly reduced. This method can balance data usability and privacy protection in practical applications.

[0091] It's worth noting that a variational autoencoder (VAE) is used to encode the perturbed pattern data, generating an encoded representation that hides sensitive attributes. The VAE compresses the input data into a low-dimensional representation using an encoder, and then reconstructs the data using a decoder. For example, when perturbed 256-dimensional pattern data is input into the VAE, the encoder generates a 128-dimensional hidden representation, and the decoder attempts to recover the original pattern. During training, the hidden representation effectively preserves non-sensitive features, such as user preferences for video types, while further weakening sensitive attributes, such as specific emotional intensity. This encoded representation facilitates subsequent tasks, such as recommender system optimization, while reducing privacy risks.

[0092] In step S104, the core feature retention rate is calculated based on the unified feature vector. If the core feature retention rate is lower than a preset retention rate threshold, the encoding representation of the hidden sensitive attribute is adjusted to obtain the adjusted encoding version.

[0093] For the unified feature vector set, the initial representation of the core features is determined by principal component analysis, and the core feature retention rate is calculated.

[0094] If the core feature retention rate is lower than the preset retention rate threshold, the preset variational autoencoder parameters are adjusted using the gradient descent algorithm to obtain the adjusted parameter configuration.

[0095] The modified parameter configuration is used to adjust the encoding representation of the hidden sensitive attributes to obtain a preliminary encoding representation;

[0096] Verify the consistency of the business data represented by the preliminary encoding, and further process the noise perturbation of the part related to sensitive attributes to obtain an adjusted encoding version that hides the sensitive attributes.

[0097] It is worth noting that core feature evaluation is performed on the unified feature vector set. Principal component analysis (PCA) is used to extract the principal components with the greatest information contribution from the high-dimensional feature vectors; these components constitute the initial representation of the core features. Subsequently, the core feature retention rate is calculated. For example, for a user's 1024-dimensional feature vector, including viewing time, likes, and comment frequency, PCA can extract the top 20 principal components, generating a unified feature vector set that retains 90% of the original information. The core feature retention rate can be obtained by calculating the cumulative variance contribution rate of the principal components.

[0098] It is worth noting that the preset retention rate threshold is a critical standard for judging whether privacy-processed data impairs the usability of personalized content recommendations, determined through historical datasets. Analysis of historical unified feature vector sets reveals that when the core feature retention rate is above 85%, sufficient user-related information can be obtained through feature extraction to generate accurate and personalized recommendations; therefore, the retention rate threshold is set at 85%. Those skilled in the art will understand that the 85% retention rate threshold is an empirically calculated value based on general recommendation scenarios. In practical applications, this threshold can be adjusted within the range of 80% to 90% depending on the service type or performance requirements. For example, for recommendation services with high accuracy requirements, the threshold can be increased to 90%; for lightweight recommendation scenarios, the threshold can be decreased to 80%. When the core feature retention rate is detected to be lower than the preset retention rate threshold, it indicates that the current privacy protection processing is too aggressive and has impaired the usability of the recommendations. At this point, an adaptive optimization mechanism is activated, using a gradient descent algorithm to fine-tune the parameters of the preset variational autoencoder. Its optimization objective is to adjust the internal weights of the variational encoder. The specific implementation is as follows: First, calculate the loss function based on the features output by the current model and the key recommendation features in the original data. This includes reconstruction loss and regularization loss. Then, solve for the gradient of the encoder layer weights with respect to the objective function through backpropagation. Next, update the parameters along the negative gradient direction according to a certain proportion of the initial learning rate. For example, enhance the convolutional kernel weights in the encoder that retain video category features, and weaken the weights of fully connected layers that over-compress key information. After each round of fine-tuning, monitor the core feature retention rate in real time. If it rises for three consecutive rounds and stabilizes above the threshold, stop the iteration.

[0099] It is worth noting that the adjusted variational autoencoder parameters are used to encode multimodal data and generate a preliminary encoded representation. For example, inputting a 512-dimensional feature vector into the variational autoencoder generates a 256-dimensional preliminary encoded representation.

[0100] After generating the initial encoded representation, business data consistency verification must be performed. This verification method compares the similarity between the encoded representation and the original data, such as calculating the cosine similarity between the initial encoded representation and the original data. If this coefficient reaches a preset threshold of 0.7, the encoded representation is deemed to meet the consistency requirements. Referring to historical data, a cosine similarity of 0.6 essentially preserves the core user behavior characteristics; above 0.7, the overlap in feature directions exceeds 70%, fully preserving the core user behavior characteristics. Furthermore, even with Gaussian noise added to the encoding at a similarity of 0.7, the overall similarity remains within the 0.6-0.7 range, still meeting the requirement of preserving core user behavior characteristics and reserving information buffer space for subsequent Gaussian noise perturbation of sensitive attributes, balancing business usability and privacy protection feasibility. Therefore, a preset threshold of 0.7 is chosen. Subsequently, Gaussian noise perturbation should be injected into the parts related to sensitive attributes to achieve user privacy protection. Typically, the noise mean is set to 0 to avoid introducing systematic bias. The variance is dynamically adjusted based on the data sensitivity. For highly sensitive pattern data exceeding a threshold, a larger standard deviation (1.5-2 times the data standard deviation) is used to ensure the original pattern is difficult to recover after perturbation. For low-sensitivity pattern data that still requires protection, a smaller variance (0.5-1 times the data standard deviation) is used to preserve the overall data distribution characteristics while hiding sensitive information. The sensitive attributes include the user's age distribution or emotional tendencies.

[0101] For example, after inputting a 512-dimensional user behavior feature vector into an autoencoder to generate a 256-dimensional preliminary encoded representation, its consistency with the original business data needs to be verified using cosine similarity. If the similarity reaches 0.9, it indicates that the encoding can reflect the overall behavioral trends of users, such as viewing preferences and interaction habits, and meets the consistency requirements. After the encoding meets the consistency requirements, Gaussian noise perturbation needs to be applied to the parts related to sensitive attributes (such as age distribution and emotional tendencies) to protect privacy. For example, a subset of vectors related to emotional tendencies is identified from the 256-dimensional preliminary encoding, and the cosine similarity is 0.75 (exceeding the preset threshold of 0.7). At this point, Gaussian noise with a mean of 0 and a standard deviation of 0.05 is introduced for perturbation. The cosine similarity of the perturbed encoded version is calculated again to be 0.68. Therefore, the perturbed encoded version retains non-sensitive information such as user preferences for video types while reducing the identifiability of emotional tendencies. This method ensures a balance between data availability and privacy protection. It should be noted that the perturbation and encoding processes need to be dynamically adjusted according to business needs. For example, by regularly updating the noise standard deviation, a balance between privacy protection and data utility can be ensured.

[0102] In one possible implementation, the adjusted encoding version can be further used to optimize the recommendation system. The encoding representation retains core behavioral features, such as a user's preference for funny videos, while downplaying sensitive information. This encoding can be directly input into the recommendation model, improving the accuracy of content matching. For example, the recommendation system can prioritize pushing entertainment short videos that users frequently watch based on the encoded representation, thereby improving the user experience.

[0103] In step S105, the adjusted encoding version is iteratively optimized to obtain optimized encoding features.

[0104] Based on the adjusted encoding version, a preset variational autoencoder is used to perform preliminary encoding processing on the multimodal data to obtain preliminary optimized feature representations;

[0105] To verify the consistency of the feature distribution of the preliminary optimized feature representation, regularization processing is performed through Gaussian distribution sampling to obtain a regularized feature representation;

[0106] Based on the regularized feature representation, the parameters of the preset variational autoencoder are iteratively optimized using the gradient descent algorithm to obtain the optimized parameter configuration.

[0107] Using the optimized parameter configuration, the multimodal data is processed by variational autoencoder to obtain the optimized coding features.

[0108] It's worth noting that variational autoencoders (VAEs) are used for preliminary encoding of multimodal data, aiming to extract more representative feature representations from complex, high-dimensional data. For example, in user behavior analysis scenarios on short video platforms, VAEs, by introducing probability distribution modeling, compress user multimodal data such as viewing time, sharing behavior, and search keywords into a low-dimensional latent feature space. Assuming the original data is a 768-dimensional feature vector, a VAE can generate a 128-dimensional preliminary optimized feature representation.

[0109] It's worth noting that the consistency verification of the feature distribution for the preliminary optimized feature representation involves checking whether it conforms to the expected business data distribution. Specifically, consistency can be assessed by comparing the statistical properties of the preliminary optimized feature representation with the original data, such as mean and variance. Assuming consistency requires the KL divergence of the feature distribution to be less than 0.1, if the calculated result is 0.08, it indicates that the preliminary optimized feature representation meets the requirement. In this case, regularization processing of the feature representation can be performed using Gaussian distribution sampling.

[0110] It is worth noting that, based on regularized feature representation, a gradient descent algorithm is used to iteratively optimize the parameters of the variational autoencoder. The optimization objective is to minimize the loss, including reconstruction loss and regularization loss. The iterative optimization loop uses a gradient descent-based optimization algorithm, aiming to adjust the encoder and decoder weights multiple times to make the latent feature space more accurately capture user behavior patterns, thus obtaining the optimized parameter configuration. The core process is as follows: Initialize encoder parameters. and decoder parameters Set the maximum number of iterations N=100, and input randomly sampled user behavior samples. The mean of the latent variables is obtained. and variance Select Latent features are generated by sampling that follows a normal distribution and is obtained through reparameterization techniques. The input is fed into the decoder to obtain the reconstructed data. ', Calculate user behavior samples With reconstructing data 'Comprehensive Loss' The overall loss is the sum of the reconstruction loss and the regularization loss. The reconstruction loss is calculated using the numerical difference method to determine the mean square error, and the regularization loss is calculated using the formula... Realization, i.e., the mean of each dimension of the latent features. ,variance The gradient ∇ of the encoder is calculated by summing according to this formula. and the gradient of the decoder∇ Update the parameters of the input encoder along the negative gradient direction. Decoder parameters Learning rate Set the value to 0.005 and repeat the above steps for iterative optimization until the maximum number of iterations is reached, thus obtaining the optimized encoder and decoder parameter configurations.

[0111] It is worth noting that when encoding multimodal data using the optimized parameter configuration, the original data is re-input into the variational autoencoder to generate the final optimized encoded features. These optimized features retain core behavioral information while reducing redundancy between features through regularization.

[0112] In step S106, the optimized coding features are matched and detected with a preset coding feature library to determine the personalized content list and obtain the original recommendation result.

[0113] The optimized coding features are matched with a preset coding feature library, and a recommendation candidate set is obtained through collaborative filtering;

[0114] Content filtering is used to sort the content in the recommendation candidate set to obtain a personalized content list;

[0115] The personalized content list is encrypted to obtain the original accurate recommendation results.

[0116] It's worth noting that the matching detection operation calculates the cosine similarity between the optimized encoded features and the features corresponding to each tag in the pre-set encoded feature library, and selects the tag with the highest cosine similarity as the matching result. Collaborative filtering is used to filter the initial recommendation candidate set. For example, based on the matched "entertainment preference" tag, other sets of videos liked by users with similar behavioral patterns are filtered out, such as recently popular funny short videos or variety show clips, forming a candidate set. The construction of the pre-set encoded feature library can be advanced step-by-step from multiple dimensions. First, a basic interest domain framework is determined based on the platform's core business scenarios (such as short videos, e-commerce, news, etc.). For example, a short video platform can first divide into primary interest categories such as entertainment, education, lifestyle, fitness, and technology. Then, deep mining is performed using historical user behavior data (including clicks, viewing time, favorites, comment keywords, purchase records, etc.) to extract key behavioral indicators under each interest domain. For example, the "entertainment preference" category can extract core features such as "funny video viewing rate ≥ 60%" and "variety show content like rate ≥ 40%", transforming these indicators into standardized feature vectors. Subsequently, an industry-standard interest tag system and manual... The annotation and validation process involves filtering and optimizing the initially extracted feature vectors, removing redundant or low-relevance features, and supplementing features corresponding to emerging interest directions (such as the recently emerging "AI-generated content preferences" and "immersive experience content preferences"). Finally, a dynamic update mechanism for the feature library is established, which adjusts the dimensional weights of the feature vectors and adds or merges interest categories (such as subdividing "fitness" into "home fitness" and "outdoor fitness") regularly (e.g., weekly or monthly) based on new user behavior data and changes in the platform's content ecosystem. The feature vectors are also normalized using methods such as cosine similarity to ensure that the features in the library can accurately match user behavior encoding and flexibly adapt to the dynamic changes in user interests and platform business.

[0117] In one feasible approach, collaborative filtering combines historical user data and real-time interaction data to generate a more timely candidate set. Specifically: First, the similarity between the user and other users is calculated using historical and real-time interaction data. Common similarity calculation methods include cosine similarity and Pearson correlation coefficient. Next, the n most similar users to the target user are identified, and their behaviors are used to generate recommendations. Based on the behavior of these similar users, content that the target user has not accessed and has a click-through rate higher than 10% within the past seven days is extracted to form the candidate set.

[0118] It's worth noting that the content filtering and ranking method sorts the initial recommendation candidate set to generate a personalized content list. For example, it analyzes the metadata of videos in the candidate set, such as video duration, content tags, and user interaction history, prioritizing videos highly relevant to the user's preferences. For instance, if a user frequently watches funny videos under 5 minutes, these videos would be ranked higher in the list. This ranking method ensures that recommended content better matches the user's interests.

[0119] It's worth noting that the personalized content list is encrypted by adding noise. For example, noise is added to the metadata of the recommendation list, such as adding a random perturbation with a mean of 0 and a standard deviation of 0.01 to the recommendation weight of videos. This approach protects user privacy while ensuring the accuracy of the recommendation results. The encrypted list can still accurately push funny videos, but specific user behavior data is not directly exposed.

[0120] In one feasible approach, the noise standard deviation can be dynamically adjusted, such as increasing it from 0.01 to 0.02, to balance privacy protection and recommendation effectiveness.

[0121] In step S107, the original recommendation results are subjected to privacy protection detection and optimization to obtain the final recommendation results.

[0122] Calculate the matching degree between the optimized encoding features and the pre-stored historical recommendation data. If the matching degree is lower than the preset matching degree threshold, enhance the cross-dimensional integration strength through an attention mechanism to obtain an improved recommendation candidate set.

[0123] The enhanced recommendation candidate set is subjected to protection detection, and the core feature retention rate judgment result is integrated. If the retention rate index is lower than the preset privacy sensitivity threshold after being correlated with the matching degree, privacy protection optimization processing is performed, the weight distribution is adjusted, and a privacy-enhanced candidate version is obtained. Then, the privacy-enhanced candidate version is compared with the encoded representation of the hidden sensitive attribute. If the retention rate is optimized, the final recommendation set with privacy protection optimization is obtained.

[0124] The final recommendation set optimized for privacy protection is compared with the user's current behavior data to obtain the final recommendation result.

[0125] It's worth noting that when the matching degree between the optimized encoded features and the pre-stored historical recommendation data is lower than a preset matching degree threshold, it indicates that the user's interests may have shifted. The preset matching degree threshold is determined by analyzing historical user behavior data and calculating the similarity distribution between the behavioral vectors of the user's recent behavior features during periods of stable and shifting interests. This threshold ensures that it can effectively distinguish between fluctuations and migrations in user interests. To this end, an attention mechanism is activated to dynamically enhance the integration strength of multi-dimensional user behavior data and re-evaluate the weight allocation of each behavioral dimension. For example, the optimized encoded features are matched with the pre-stored historical recommendation data to determine if they meet the preset threshold, such as a cosine similarity of 0.85. If the matching degree is lower than this threshold, it indicates that the user's interests may have changed, and further feature optimization is needed. For example, a user's optimized encoded features have a similarity of only 0.7 with the "funny videos" tag in the pre-stored historical recommendation data, which is lower than the threshold. In this case, an attention mechanism is introduced to assign weights to the optimized encoded features. Attention mechanisms analyze recent user interaction data, such as frequent pausing to watch food videos, and assign higher weights to food-related feature dimensions to generate enhanced cross-dimensional integrated features, such as a 256-dimensional comprehensive interest vector. This approach can more accurately reflect the user's current interest tendencies.

[0126] It is worth noting that privacy protection detection is initiated on the improved candidate set, and a correlation analysis is performed between the core feature retention rate and user matching degree. Specifically, the correlation analysis involves using the information entropy of the feature vector as the core feature retention rate metric for the improved recommendation candidate set, and comparing the result of multiplying the core feature retention rate by the user matching degree with a preset privacy threshold of 0.8. This preset privacy threshold is a critical standard for determining whether data in personalized content recommendation poses a risk of leaking user privacy. A common calculation method is to first statistically analyze the baseline range of information entropy of user preference feature vectors in normal scenarios. Through analysis of a large number of samples, it is found that when the information entropy of the feature vectors in the recommendation candidate set is higher than 0.8, the distribution of user preferences contained in the vectors is more dispersed, making it difficult to pinpoint specific users; when the information entropy is lower than 0.8, the preference features are highly concentrated, and the risk of leaking user identity or privacy needs increases significantly. Therefore, a privacy threshold of 0.8 is calculated to ensure that this threshold can cover the preference feature distribution of the vast majority of normal users, while accurately identifying high-risk data with excessively concentrated preferences. Those skilled in the art will understand that the privacy threshold of 0.8 is an empirically calculated value based on a sample of a general user group. In practical applications, this threshold can be adjusted within the range of 0.75 to 0.85 according to the characteristics of the user group or the sensitivity of the scenario. For example, for recommendation scenarios targeting a specific user group, the threshold can be raised to 0.85; for recommendation scenarios targeting general users, the threshold can be lowered to 0.75.

[0127] If the retention rate of the core features of the initial recommendation candidate set is lower than a preset privacy threshold after being correlated with the matching degree, privacy protection optimization processing is performed on the candidate set. The weight distribution of the initial recommendation candidate set is adjusted using a logistic regression algorithm to obtain an adjusted weight distribution. Based on the adjusted weight distribution, the initial recommendation candidate set is reordered to generate a privacy-enhanced candidate version. Subsequently, the optimized candidate set is compared and verified with a fully anonymized encoding representation to ensure that, under the premise of optimizing the retention rate, a final recommendation set that balances privacy protection and recommendation utility is ultimately generated. The fully anonymized encoding refers to an encoding that, after being perturbed by injecting Gaussian noise, cannot be directly or indirectly associated with or restored to the original personal information or original sensitive attributes. To achieve full anonymization, corresponding Gaussian noise needs to be injected using a differential privacy algorithm. The core of this method lies in: firstly, determining the global sensitivity of the query function (such as gradient, summation function), i.e., the maximum value of the function output change. ; then through the formula Calculate the standard deviation σ of the noise. This is a privacy parameter, typically set between 0.1 and 10. To mitigate the risk of minor violations of strict differential privacy, it is typically set to 10. -5 Inject a mean of 0 and a standard deviation of into the original data or intermediate results. The output contains Gaussian noise. After this processing, it is impossible to deduce user information from the output.

[0128] For example, the video platform performs privacy protection checks on the improved food recommendation candidate set: First, core features are extracted, and their retention rate is calculated to be 85%; the product of 90% user matching degree is lower than the preset privacy sensitivity threshold of 0.8. Privacy protection optimization is then initiated, adjusting the weight distribution of the candidate set through a logistic regression algorithm, determining the weights of sensitive and non-sensitive features, setting the maximum number of iterations N=100, and the learning rate... Set to 0.1, and based on the current weights, use the Sigmoid activation function to predict the input sample. Compare the predicted value y' with the target value y, and adjust the gradient of each weight using the cross-entropy loss function. Use gradient descent algorithm to update the weights. , For each iteration, new weight values ​​are provided. The weight values ​​are defined for each iteration. The above steps are repeated iteratively until the maximum number of iterations is reached. For example, the weight of sensitive features is decreased from 0.3 to 0.1, while the weight of non-sensitive features is increased from 0.2 to 0.4, generating a privacy-enhanced candidate version. Further testing shows that the core feature retention rate has increased to 89%. Multiplying the core feature retention rate by the user matching degree (90%) yields 0.801, reaching the preset privacy sensitivity threshold of 0.8. After comprehensive evaluation, this is determined as the final recommendation set for privacy protection optimization.

[0129] It's worth noting that before outputting the final result, the privacy-optimized final recommendation set is compared with the user's current behavioral data to further filter or adjust the recommended content, ensuring that the recommendation results not only meet privacy protection requirements but also accurately match the user's current real needs. For example, the platform has completed privacy optimization processing, generating final recommendation candidate sets for three major categories. Sensitive fields in this set have been anonymized, and the retention rate of core features and user matching degree both meet the balance standard between privacy protection and recommendation usability. Based on the behavioral comparison mechanism, user interaction data from the past hour is extracted, identifying short videos clicked three times consecutively and content quickly skipped. The categories of the continuously clicked short videos and the quickly skipped content are extracted. By matching the optimized recommendation set with the above behavioral features, content strongly correlated with the category of continuously clicked short videos is filtered out, while the category of quickly skipped content is simultaneously removed. Finally, a privacy-optimized final recommendation set that matches the user's current behavior is output.

[0130] Reference Figure 2 The second embodiment of the present invention provides a personalized content recommendation system based on multimodal perception, comprising:

[0131] The data feature extraction module is used to acquire raw multimodal data, perform preliminary compression processing on the raw multimodal data, and generate a low-dimensional latent representation;

[0132] The data feature fusion module is used to acquire cross-dimensional related features and perform feature fusion processing to obtain a unified feature vector.

[0133] The data privacy protection module is used to detect potential patterns where the correlation coefficient of sensitive attributes exceeds a preset sensitivity threshold, and to perform data perturbation processing to obtain the encoded representation of the hidden sensitive attributes;

[0134] The data adjustment module is used to calculate the core feature retention rate of the encoding table for hidden sensitive attributes. If the core feature retention rate is lower than the retention rate threshold, the encoding parameters are adjusted and the encoding version of the multimodal data is regenerated to obtain the adjusted encoding version.

[0135] The data iteration and optimization module is used to iteratively optimize the adjusted encoding version and obtain optimized encoding features;

[0136] The recommendation generation module is used to match and detect the optimized coding features with a preset coding feature library to determine a personalized content list and obtain the original recommendation result;

[0137] The privacy protection detection and optimization module is used to perform privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results.

[0138] It should be noted that the personalized content recommendation system based on multimodal perception provided in this embodiment of the invention is used to execute all the process steps of the personalized content recommendation method based on multimodal perception in the above embodiments. The working principles and beneficial effects of the two correspond one-to-one, so they will not be described again.

[0139] In summary, this invention discloses a personalized content recommendation method based on multimodal perception. The method includes acquiring multimodal data, initially compressing the multimodal data into a low-dimensional latent representation, extracting cross-dimensional correlation features, and performing feature fusion to generate a unified feature vector. Based on the unified feature vector, latent patterns are obtained, and the correlation coefficients between each dimension and sensitive attributes are calculated. If these coefficients exceed a preset sensitivity threshold, data perturbation processing is initiated to obtain an encoded representation of hidden sensitive attributes. Simultaneously, the retention rate of core features is monitored; if the retention rate is too low, the perturbation intensity is dynamically adjusted. The adjusted encoded version is then iteratively optimized to obtain optimized encoded features, which are then matched and detected to determine a personalized content list, resulting in an initial recommendation result. Finally, the initial recommendation result undergoes privacy protection detection and optimization to obtain the final recommendation result. This invention achieves a shift from single-dimensional to multi-dimensional analysis, constructing a unified feature expression that comprehensively reflects users' complex interest patterns through feature fusion. Furthermore, after generating the initial recommendation result, it is not directly output but instead includes a dedicated privacy protection detection and optimization step for final security review and calibration of the recommendation result. This design forms an effective secondary protection barrier, which can identify and eliminate the risk of sensitive information association that may remain in the previous processing, ensuring that the final output of recommended content is both accurately matched with user interests and fully compliant with privacy protection standards.

[0140] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A personalized content recommendation method based on multimodal perception, characterized in that, include: Acquire multimodal data, perform preliminary compression processing on the multimodal data, and obtain a low-dimensional latent representation; Based on the low-dimensional latent representation, cross-dimensional related features are obtained and feature fusion processing is performed to obtain a unified feature vector. Based on the unified feature vector, a latent pattern is obtained. If the correlation coefficient of the sensitive attribute in the latent pattern exceeds a preset sensitivity threshold, data perturbation processing is performed to obtain the encoded representation of the hidden sensitive attribute. Based on the unified feature vector, the core feature retention rate is calculated. If the core feature retention rate is lower than the preset retention rate threshold, the encoding representation of the hidden sensitive attribute is adjusted to obtain the adjusted encoding version. The adjusted encoding version is iteratively optimized to obtain optimized encoding features; The optimized coding features are matched and detected with a preset coding feature library to determine a personalized content list and obtain the original recommendation result; The original recommendation results are subjected to privacy protection checks and optimizations to obtain the final recommendation results; The step of calculating the core feature retention rate based on the unified feature vector, and adjusting the encoding representation of the hidden sensitive attributes to obtain the adjusted encoding version if the core feature retention rate is lower than a preset retention rate threshold, includes: For a unified set of feature vectors, the initial representation of the core features is determined by principal component analysis, and the retention rate of the core features is calculated. If the core feature retention rate is lower than the preset retention rate threshold, the preset variational autoencoder parameters are adjusted using the gradient descent algorithm to obtain the adjusted parameter configuration. The modified parameter configuration is used to adjust the encoding representation of the hidden sensitive attributes to obtain a preliminary encoding representation; Verify the consistency of the business data represented by the preliminary encoding, and further process the noise perturbation of the part related to sensitive attributes to obtain an adjusted encoding version that hides the sensitive attributes.

2. The personalized content recommendation method based on multimodal perception according to claim 1, characterized in that, The process of acquiring multimodal data and performing preliminary compression processing on the multimodal data to obtain a low-dimensional latent representation includes: The interaction records were obtained, and text, image, and audio data were extracted from social platforms, sensors, and log files to obtain the raw multimodal dataset. The original multimodal data is standardized to obtain a multimodal dataset with a uniform format; The multimodal dataset is encoded using a preset variational autoencoder to extract features from each modality and obtain a low-dimensional latent representation.

3. The personalized content recommendation method based on multimodal perception according to claim 1, characterized in that, The step of obtaining cross-dimensional related features based on the low-dimensional latent representation and performing feature fusion processing to obtain a unified feature vector includes: Based on the low-dimensional latent representation, features of each modality are extracted to obtain feature sets of each modality; Principal component analysis is used to reduce the dimensionality of each modal feature set to obtain a cross-dimensional feature set. An attention mechanism is used to assign weights and perform weighted fusion on the cross-dimensional feature set to obtain a unified feature vector.

4. The personalized content recommendation method based on multimodal perception according to claim 1, characterized in that, The step of obtaining a latent pattern based on the unified feature vector, and if the correlation coefficient of the sensitive attribute in the latent pattern exceeds a preset sensitivity threshold, then performing data perturbation processing to obtain the encoded representation of the hidden sensitive attribute, includes: The unified feature vector is decomposed using principal component analysis to obtain a set of latent patterns; Calculate the correlation coefficient of the sensitive attributes of any pattern in the set of potential patterns; Gaussian noise is injected into the pattern data whose correlation coefficient of the sensitive attribute exceeds a preset sensitivity threshold to obtain perturbed pattern data. The perturbed pattern data is encoded using a preset variational autoencoder to obtain an encoded representation of the hidden sensitive attributes.

5. The personalized content recommendation method based on multimodal perception according to claim 1, characterized in that, The iterative optimization of the adjusted encoding version to obtain optimized encoding features includes: Based on the adjusted encoding version, a preset variational autoencoder is used to perform preliminary encoding processing on the multimodal data to obtain preliminary optimized feature representations; To verify the consistency of the feature distribution of the preliminary optimized feature representation, regularization processing is performed through Gaussian distribution sampling to obtain a regularized feature representation; Based on the regularized feature representation, the preset variational autoencoder parameters are iteratively optimized using the gradient descent algorithm to obtain the optimized parameter configuration. Using the optimized parameter configuration, the multimodal data is processed by the variational autoencoder to obtain the optimized coding features.

6. The personalized content recommendation method based on multimodal perception according to claim 1, characterized in that, The step of matching and detecting the optimized coding features with a preset coding feature library to determine a personalized content list and obtain the original recommendation result includes: The optimized coding features are matched with a preset coding feature library, and a recommendation candidate set is obtained through collaborative filtering; Content filtering is used to sort the content in the recommendation candidate set to obtain a personalized content list; The personalized content list is encrypted to obtain the original accurate recommendation results.

7. The personalized content recommendation method based on multimodal perception according to claim 6, characterized in that, The step of performing privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results includes: Calculate the matching degree between the optimized encoding features and the pre-stored historical recommendation data. If the matching degree is lower than the preset matching degree threshold, enhance the cross-dimensional integration strength through an attention mechanism to obtain an improved recommendation candidate set. The enhanced recommendation candidate set is subjected to protection detection, and the core feature retention rate judgment result is integrated. If the retention rate index is lower than the preset privacy sensitivity threshold after being correlated with the matching degree, privacy protection optimization processing is performed, the weight distribution is adjusted, and a privacy-enhanced candidate version is obtained. Then, the privacy-enhanced candidate version is compared with the encoded representation of the hidden sensitive attribute. If the retention rate is optimized, the final recommendation set with privacy protection optimization is obtained. The final recommendation set optimized for privacy protection is compared with the user's current behavior data to obtain the final recommendation result.

8. A personalized content recommendation system based on multimodal perception, characterized in that, A method for implementing a personalized content recommendation method based on multimodal perception as described in any one of claims 1 to 7, comprising: The data feature extraction module is used to acquire raw multimodal data, perform preliminary compression processing on the raw multimodal data, and generate a low-dimensional latent representation; The data feature fusion module is used to acquire cross-dimensional related features and perform feature fusion processing to obtain a unified feature vector. The data privacy protection module is used to detect potential patterns where the correlation coefficient of sensitive attributes exceeds a preset sensitivity threshold, and to perform data perturbation processing to obtain the encoded representation of the hidden sensitive attributes; The data adjustment module is used to calculate the core feature retention rate of the encoded representation of hidden sensitive attributes. If the core feature retention rate is lower than the retention rate threshold, the encoding parameters are adjusted and the encoded version of the multimodal data is regenerated to obtain the adjusted encoded version. The data iteration and optimization module is used to iteratively optimize the adjusted encoding version and obtain optimized encoding features; The recommendation generation module is used to match and detect the optimized coding features with a preset coding feature library to determine a personalized content list and obtain the original recommendation result; The privacy protection detection and optimization module is used to perform privacy protection detection and optimization on the original recommendation results to obtain the final recommendation results.

Images