Data analysis processing method and system based on multi-source data fusion

Abstract

The invention discloses a data analysis processing method and system based on multi-source data fusion, and relates to the technical field of data analysis fusion processing. Comprising the following steps: data acquisition and preprocessing: acquiring multi-source data, including user behavior data, content data and social data, related to APP pushing and cultural transmission from a plurality of data sources such as social software and an internet platform, after acquisition, preprocessing the multi-source data, generating preprocessed data, and establishing a dynamic credibility evaluation model; according to the method, the dynamic credibility evaluation model is established to distribute the weight for the data source, so that the interference of low-credibility data on the analysis result is avoided, and the information analysis of the user interest intensity, the content credibility and the cultural value based on the fused data is more accurate; therefore, a more reliable basis is provided for making an APP push strategy and a cultural transmission scheme.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis and fusion processing, and in particular to a data analysis and processing method and system based on multi-source data fusion. Background Art

[0002] In today's digital age, apps like Douyin, Kuaishou, and Xiaohongshu play a vital role in information push and cultural dissemination. To achieve accurate content push and effective cultural dissemination, it's necessary to integrate and analyze multi-source data.

[0003] However, existing technologies have many problems in the process of multi-source data fusion:

[0004] For example, the credibility or reliability of different data sources vary. For example, the accuracy and reliability of user behavior data and social data on different platforms may vary due to data collection methods or data quality control issues, making it difficult to accurately integrate and utilize this data during data fusion.

[0005] The huge differences in the intrinsic representations and features of highly heterogeneous data modalities greatly increase the difficulty of fusion. For example, text data and image data have different feature representations, making it difficult to effectively fuse different feature representations.

[0006] The goals or priorities of integration may change dynamically with time or context. For example, users' demand for different types of content may change in different time periods, and the push strategy of user application apps also needs to be adjusted accordingly with changes in dynamic content. However, existing technologies are difficult to adapt to such dynamic changes in real time.

[0007] The existing technology has the following shortcomings: Therefore, in the process of APP content push and cultural communication of user applications, facing the problems of data source credibility differences, difficulty in heterogeneous data modal fusion, and dynamic changes in fusion targets in multi-source data fusion in existing technologies, there is an urgent need for a data analysis and processing method and system based on multi-source data fusion to achieve more accurate and efficient content push and cultural communication.

[0008] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0009] The purpose of the present invention is to provide a data analysis and processing method and system based on multi-source data fusion. The present invention solves the problems in the above-mentioned background technology by establishing a dynamic credibility assessment model, constructing a cross-modal decoupler-alignment architecture, adopting a dynamic fusion strategy and combining it with a built-in multi-objective reinforcement learning decision engine and a PPO strategy optimization algorithm to formulate an APP push strategy and cultural communication plan.

[0010] In order to achieve the above object, the present invention provides the following technical solution: a data analysis and processing method based on multi-source data fusion, comprising the following steps:

[0011] S1. Data Collection and Preprocessing: Collect multi-source data related to app push and cultural communication from multiple data sources of social software and internet platforms, including user behavior data, content data, and social data. After collection, preprocess the multi-source data to generate preprocessed data, establish a dynamic credibility assessment model, and assign weights to data sources based on the assessment results.

[0012] S2. Feature Extraction and Representation for Heterogeneous Data: We build a cross-modal decoupler-alignment architecture, utilizing visual CNN models, LSTM models, BERT models, and adversarial training deep learning models to extract and separate unique features for video frames, audio, and text modalities from preprocessed data. We then map features from different modalities into the same space and unify their representation.

[0013] S3. Multi-source data fusion: A dynamic fusion strategy is used to fusion the multimodal features of unified representation. According to the dynamic changes of fusion objectives and priorities, the fusion method and parameters are adjusted in real time to obtain fused data.

[0014] S4. Data Analysis and Dynamic Decision-Making: Analyze the integrated data to determine user interest, content credibility, and cultural value. Combined with the built-in multi-objective reinforcement learning decision-making engine, the PPO strategy optimization algorithm is used to develop app push strategies and cultural communication plans.

[0015] S5. Feedback and Optimization: Collect user feedback on the content pushed by the APP and the effectiveness of cultural communication, including user click behavior, comments and sharing times, and use online reinforcement learning algorithms to continuously optimize and improve the accuracy of APP push and the effectiveness of cultural communication.

[0016] Optionally, the steps of acquiring and preprocessing the multi-source data are as follows:

[0017] Using the Scrapy crawler framework and Kafka real-time stream collection tools, we crawl multi-source data, including text, visual, and audio data, from social apps like WeChat / Weibo, content platforms like Douyin / Kuaishou / Xiaohongshu, and third-party knowledge bases on the internet.

[0018] Data cleaning includes denoising text data using regular expressions, normalizing the resolution of visual data using OpenCV, and denoising audio data using filters;

[0019] Time linear interpolation method is used to fill missing values ​​in multi-source data;

[0020] and use the IQR interquartile range algorithm to handle outliers in multi-source data;

[0021] After preprocessing multi-source data through data cleaning, missing value filling and outlier processing, preprocessed data is generated, and the expression of preprocessed data is , where It represents pre-processed data after data cleaning, missing value filling and outlier processing of multi-source data collected in real time. Represented as text data, Represented as visual data, Represented as audio data.

[0022] Optionally, the steps for establishing the dynamic credibility assessment model are as follows:

[0023] Assign credibility to the data source, assign initial credibility to the type of data source, and mark it as , including official certified accounts, the initial credibility is assigned to 0.95; ordinary user UGC, the initial credibility is assigned to 0.75; third-party unverified data sources, the initial credibility is assigned to 0.60;

[0024] The attenuation factor of the credibility of real-time user feedback is obtained by the number of times the user reports the data source information in a unit time. Among them, the attenuation factor of credibility is The calculation formula is , where Expressed as a natural exponential function, Expressed as the attenuation coefficient, Expressed as the number of reports per unit time;

[0025] Jaccard similarity is used to verify the cross-platform consistency of data sources. The calculation formula for Jaccard similarity verification is: , where Expressed as similarity, Indicates that this platform, Indicates a third-party platform. Represented as the intersection symbol, It is represented by the union symbol;

[0026] According to the attenuation factor and similarity verification of the new collected data and user feedback information, the dynamic weight synthesis method is used to obtain the final credibility weight of the data source. The calculation formula of the final credibility weight is: ,and , where Expressed as the final credibility weight.

[0027] Optionally, the steps of extracting the unique features are as follows:

[0028] Use the visual CNN model to separate video frames, extract visual features of video frames by processing frame data, and convert visual data in preprocessed data into The row temporal spatial feature processing is performed and the visual feature vector is output, where The calculation formula for extracting visual features is: 、 ,and 、 , where Represented as the output visual feature vector, Expressed as The convolution calculation of the model, Expressed as CNN weight parameters, Represented as 1024 dimensions, Represented as the input video frame sequence, Represented as the input video frame sequences and the total number of video frame sequences, Represented as a sequence of video frames A 4D tensor consisting of the number of frames, frame height, frame width, and number of RGB channels, and The size is ;

[0029] The LSTM model processes audio data, mines the time series features in the audio data, and converts the audio data in the preprocessed data into , and output the audio feature vector of the audio data. The calculation formula for extracting audio features by the LSTM model is: 、 ,and 、 , where Represented as the output audio feature vector, Represents the last time step output by the LSTM model The hidden state of Represented as the three gating mechanisms and hidden state update calculations of the LSTM model, Represented as iteratively processing each sequence of audio frames by time step, and , The last audio frame number of the audio frame sequence. Expressed as Mel frequency dimension, Represented as 256 dimensions;

[0030] The BERT model processes text data, obtains the semantic features of the text, and converts the text data in the preprocessed data into , and output the semantic features of the text. The calculation formula for the BERT model to extract the [CLS] tag vector as the semantic feature of the text is: 、 ,and 、 ,as well as , where Represented as the semantic features of the output, The hidden state matrix represented as [CLS] notation, Represented as 768 dimensions, Represented as the BERT model output hidden state matrix, Represented as the calculation of the multi-head attention mechanism of the BERT model, Expressed as BERT weight parameters, Expressed as times the sequence length dimension, Represented as the sequence length of text data, It represents the input text sequence fed into the BERT model, and [CLS] represents the token added at the beginning of the input text sequence.

[0031] Optionally, the feature mapping step of the adversarial training deep learning model is as follows:

[0032] Using anti-training deep learning model to transform visual features , audio features and semantic features Different modal features are mapped to the same space And unified expression, where the projection expression of the same space is ,and 、 、 , where 、 、 Represented as shared features after projection of visual features, audio features and semantic features, 、 、 They are respectively represented as visual feature projection weight matrix, audio feature projection weight matrix, and text semantic feature projection weight matrix, 、 、 They are respectively represented as the visual feature projection bias vector, the audio feature projection bias vector, and the text semantic feature projection bias vector;

[0033] The generator G of the adversarial training deep learning model shares the same spatial projection layer and the mode of the discriminator D predicting the feature source. The calculation formula of the adversarial training deep learning model is ,and , where Expressed as the minimization loss of the generator, Expressed as the expected operation on the same spatial mapping feature, It is expressed as the probability of the discriminator outputting the target distribution, Denoted as the minimization loss of the discriminator, Represented as shared features after visual feature projection The expectation operation of Represented as the shared feature after projection of the given visual feature , the output of the conditional discriminator belongs to the visual feature The probability of Represented as the shared features after audio feature projection The expectation operation of Represented as the shared feature after projection of the given audio feature , the output of the conditional discriminator belongs to the audio feature The probability of Represented as shared features after semantic feature projection The expectation operation of Represented as the shared feature after projection of given semantic features , the conditional discriminator outputs semantic features probability;

[0034] Then, the semantic features, audio features, and visual features are aligned using the mean square error (MSE) to minimize the inter-modal distance. The formula for minimizing the inter-modal distance is: , where It is expressed as taking semantic features as anchors to minimize the inter-modal distance between audio features and visual features. Expressed as the square of the L2 norm, Expressed as the Euclidean distance between visual features and semantic features, Expressed as the Euclidean distance between audio features and semantic features;

[0035] And the adversarial loss is optimized to the minimum and maximum of the total goal, where the calculation formula for the optimization total goal is: , where Expressed as the total loss of minimax optimization, Denoted as adversarial loss weight, Denoted as the alignment loss weight, Represented as a generator, Denoted as the discriminator.

[0036] Optionally, the steps of acquiring the fused data are as follows:

[0037] Obtaining unified representation of multimodal features , according to the dynamic fusion strategy, the target weight of each modal feature is determined as , where Expressed as a multi-objective weight vector, Expressed as the target weight of the cultural goal, Expressed as the target weight for the click-through rate goal, The target weight representing the user retention goal;

[0038] The target gated attention mechanism is used for weighted calculation to adjust the importance of each modal feature and obtain the gated weighted feature. The calculation formula of the gated weighted feature is: ,and , where Represented as gated weighted features, Expressed as activation function, Expressed as a gating weight matrix, is represented as a gate bias vector, Expressed as the Hadamard product, The shared features of different modalities represented as semantic features, audio features, and visual features are mapped into the same space;

[0039] The credibility attention mechanism is used to perform weighted calculation of the fusion weight of each modal feature to obtain the attention weight, where the calculation formula of the attention weight is: 、 ,and 、 ,as well as , where Expressed as the multi-target raw attention score, Expressed as activation function, Represented as an attention weight vector, Expressed as the query transformation matrix, Expressed as a key transformation matrix, Expressed as the multimodal feature mean, Denoted as the normalized attention weight, Expressed as a natural exponential function, Expressed as a credibility weighted term;

[0040] The gated weighted features and attention weights of each modality are weighted and summed again to calculate the fusion feature, where the calculation formula of the fusion feature is: , where Represented as weighted fusion features;

[0041] Perform L2 normalization on the fusion features and output the normalized fusion data, where the expression of the fusion data is: , where Represented as the fusion data output after normalization, Expressed as L2 norm.

[0042] Optionally, the dynamic decision-making steps of the PPO strategy optimization algorithm are as follows:

[0043] Based on the analysis results, the state space is constructed and the action space of the APP push strategy and cultural communication plan is defined, including the type of push content, push time, and communication channels. , action space ;

[0044] Design a reward function, use the policy network to generate the action probability distribution, use the value network to evaluate the value of the state, and combine the GAE algorithm to estimate the action-state advantage. The advantage function expression of the GAE algorithm is ,and , where Represented as time step The advantage function estimate of Expressed as a discount factor, Expressed as the smoothing coefficient of GAE, It represents the next step length. Expressed as time steps The time difference error, Represented as time step Rewards received, Represented as time step state The estimated value of Represented as time step state estimated value of;

[0045] Select an action based on the action probability distribution generated by the current policy network, and observe the reward and next state of the APP environment interaction feedback after executing the action;

[0046] Store the information of status, action, and reward to form trajectory data;

[0047] Then use the PPO algorithm to optimize the strategy network and value network to improve the performance of the strategy.

[0048] The data analysis and processing system based on multi-source data fusion includes a data collection and preprocessing module: using crawler tools to crawl multi-source data such as user behavior data, content data, and social data from social software and Internet platforms in real time, and then perform data cleaning, missing value filling and outlier processing to obtain preprocessed data, and then transfer the preprocessed data to the preprocessed data. By establishing a dynamic credibility assessment model, the credibility of each data source is assessed, the reliability of the data source is quantified, the assessment results are generated, and dynamic weights are assigned to the data sources;

[0049] Heterogeneous Data Feature Extraction and Unified Representation Module: This module receives preprocessed data, uses a visual CNN model to separate video frame data and extract visual features, employs an LSTM model to process audio data and mine temporal features, and employs a BERT model to obtain semantic features of text data. After combining visual, temporal, and semantic features into multimodal feature data, it uses an adversarial training deep learning model to map the multimodal feature data into the same space, unify the representation, and pass it to the multi-source data cross-modal fusion module.

[0050] Multi-source data cross-modal fusion module: Receives multi-modal feature data in a unified representation, uses a dynamic gating mechanism algorithm and attention mechanism to adjust the fusion process in real time based on the dynamic changes in fusion targets and priorities, outputs the fused data, and passes it to the data analysis and decision module;

[0051] Data analysis and decision-making module: Receives integrated data and uses classification and regression algorithms in machine learning to analyze and quantify multi-dimensional information on user interest intensity, content credibility, and cultural value. It also combines the PPO strategy optimization algorithm to generate personalized app push strategies and cultural communication plans.

[0052] Information feedback optimization and adjustment module: collects user feedback on click behavior, comments and sharing times of APP push content and cultural communication effects, uses online reinforcement learning algorithm to process feedback information, and optimizes the parameters and target weights of the dynamic fusion strategy online, so as to continuously adjust the accuracy of APP push and the effect of cultural communication.

[0053] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned data analysis and processing method based on multi-source data fusion when executing the computer program.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned data analysis and processing method based on multi-source data fusion.

[0055] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0056] This paper assigns weights to data sources by establishing a dynamic credibility assessment model, avoiding the interference of low-credibility data on analysis results. This makes the information analysis of user interest intensity, content credibility, and cultural value based on fused data more accurate, thereby providing a more reliable basis for formulating APP push strategies and cultural communication programs. By constructing a cross-modal decoupler-alignment architecture, the features of different modalities are successfully extracted and uniformly represented, overcoming the difficulties of heterogeneous data modality fusion, enabling the full utilization of multi-source data and mining more comprehensive and valuable information.

[0057] By adopting a dynamic fusion strategy, the fusion method and parameters can be adjusted in real time according to the dynamic changes of fusion goals and priorities, so that the system can quickly adapt to various changes, ensuring that the APP push strategy and cultural communication plan always meet actual needs, and improving the adaptability and flexibility of the system; and the APP push strategy and cultural communication plan formulated by combining the built-in multi-objective reinforcement learning decision engine and PPO strategy optimization algorithm can better meet users' interests and needs, and improve user participation and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0059] Figure 1 The figure is a flow chart of the data analysis and processing method based on multi-source data fusion of the present invention.

[0060] Figure 2 This is a module block diagram of the data analysis and processing system based on multi-source data fusion of the present invention. DETAILED DESCRIPTION

[0061] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0062] Example 1

[0063] The present invention provides Figure 1 The data analysis and processing method based on multi-source data fusion shown includes the following steps:

[0064] S1. Data Collection and Preprocessing: Collect multi-source data related to app push and cultural communication from multiple data sources on social software and internet platforms, including user behavior data, content data, and social data. After collection, preprocess the multi-source data to generate preprocessed data, and establish a dynamic credibility assessment model to evaluate the credibility and reliability of each data source, and assign weights to the data sources based on the assessment results.

[0065] Specifically, the steps for acquiring and preprocessing multi-source data are as follows:

[0066] Using the Scrapy crawler framework and Kafka real-time stream collection tools, we crawl multi-source data, including text, visual, and audio data, from social apps like WeChat / Weibo, content platforms like Douyin / Kuaishou / Xiaohongshu, and third-party knowledge bases on the internet.

[0067] Data cleaning includes denoising text data using regular expressions, normalizing the resolution of visual data using OpenCV, and denoising audio data using filters;

[0068] Time linear interpolation method is used to fill missing values ​​in multi-source data;

[0069] and use the IQR interquartile range algorithm to handle outliers in multi-source data;

[0070] After preprocessing multi-source data through data cleaning, missing value filling and outlier processing, preprocessed data is generated, and the expression of preprocessed data is , where It represents pre-processed data after data cleaning, missing value filling and outlier processing of multi-source data collected in real time. Represented as text data, Represented as visual data, Represented as audio data.

[0071] Specifically, the steps for establishing the dynamic credibility evaluation model are as follows:

[0072] Assign credibility to the data source, assign initial credibility to the type of data source, and mark it as , including official certified accounts, the initial credibility is assigned to 0.95; ordinary user UGC, the initial credibility is assigned to 0.75; third-party unverified data sources, the initial credibility is assigned to 0.60;

[0073] The attenuation factor of the credibility of real-time user feedback is obtained by the number of times the user reports the data source information in a unit time. Among them, the attenuation factor of credibility is The calculation formula is , where Expressed as a natural exponential function, Expressed as the attenuation coefficient, Expressed as the number of reports per unit time;

[0074] Jaccard similarity is used to verify the cross-platform consistency of data sources. The calculation formula for Jaccard similarity verification is: , where Expressed as similarity, Indicates that this platform, Indicates a third-party platform. Represented as the intersection symbol, It is represented by the union symbol;

[0075] According to the attenuation factor and similarity verification of the new collected data and user feedback information, the dynamic weight synthesis method is used to obtain the final credibility weight of the data source. The calculation formula of the final credibility weight is: ,and , where Expressed as the final credibility weight.

[0076] S2. Feature Extraction and Representation for Heterogeneous Data: We build a cross-modal decoupler-alignment architecture, utilizing visual CNN models, LSTM models, BERT models, and adversarial training deep learning models to extract and separate unique features for video frames, audio, and text modalities from preprocessed data. We then map features from different modalities into the same space and unify their representation.

[0077] Specifically, the steps for extracting unique features are as follows:

[0078] Use the visual CNN model to separate video frames, extract visual features of video frames by processing frame data, and convert visual data in preprocessed data into The row temporal spatial feature processing is performed and the visual feature vector is output, where The calculation formula for extracting visual features is: 、 ,and 、 , where Represented as the output visual feature vector, Expressed as The convolution calculation of the model, Expressed as CNN weight parameters, Represented as 1024 dimensions, Represented as the input video frame sequence, Represented as the input video frame sequences and the total number of video frame sequences, Represented as a sequence of video frames A 4D tensor consisting of the number of frames, frame height, frame width, and number of RGB channels, and The size is ;

[0079] The LSTM model processes audio data, mines the time series features in the audio data, and converts the audio data in the preprocessed data into , and output the audio feature vector of the audio data. The calculation formula for extracting audio features by the LSTM model is: 、 ,and 、 , where Represented as the output audio feature vector, Represents the last time step output by the LSTM model The hidden state of Represented as the three gating mechanisms and hidden state update calculations of the LSTM model, Represented as iteratively processing each sequence of audio frames by time step, and , The last audio frame number of the audio frame sequence. Expressed as Mel frequency dimension, Represented as 256 dimensions;

[0080] The BERT model processes text data, obtains the semantic features of the text, and converts the text data in the preprocessed data into , and output the semantic features of the text. The calculation formula for the BERT model to extract the [CLS] tag vector as the semantic feature of the text is: 、 ,and 、 ,as well as , where Represented as the semantic features of the output, The hidden state matrix represented as [CLS] notation, Represented as 768 dimensions, Represented as the BERT model output hidden state matrix, Represented as the calculation of the multi-head attention mechanism of the BERT model, Expressed as BERT weight parameters, Expressed as times the sequence length dimension, Represented as the sequence length of text data, It represents the input text sequence fed into the BERT model, and [CLS] represents the token added at the beginning of the input text sequence.

[0081] Specifically, the feature mapping steps of the adversarial training deep learning model are as follows:

[0082] Using anti-training deep learning model to transform visual features , audio features and semantic features Different modal features are mapped to the same space And unified expression, where the projection expression of the same space is ,and 、 、 , where 、 、 Represented as shared features after projection of visual features, audio features and semantic features, 、 、 They are respectively represented as visual feature projection weight matrix, audio feature projection weight matrix, and text semantic feature projection weight matrix, 、 、 They are respectively represented as the visual feature projection bias vector, the audio feature projection bias vector, and the text semantic feature projection bias vector;

[0083] The generator G of the adversarial training deep learning model shares the same spatial projection layer and the mode of the discriminator D predicting the feature source. The calculation formula of the adversarial training deep learning model is ,and , where Expressed as the minimization loss of the generator, Expressed as the expected operation on the same spatial mapping feature, It is expressed as the probability of the discriminator outputting the target distribution, Denoted as the minimization loss of the discriminator, Represented as shared features after visual feature projection The expectation operation of Represented as the shared feature after projection of the given visual feature , the output of the conditional discriminator belongs to the visual feature The probability of Represented as the shared features after audio feature projection The expectation operation of Represented as the shared feature after projection of the given audio feature , the output of the conditional discriminator belongs to the audio feature The probability of Represented as shared features after semantic feature projection The expectation operation of Represented as the shared feature after projection of given semantic features , the conditional discriminator outputs semantic features probability;

[0084] Then, the semantic features, audio features, and visual features are aligned using the mean square error (MSE) to minimize the inter-modal distance. The formula for minimizing the inter-modal distance is: , where It is expressed as taking semantic features as anchors to minimize the inter-modal distance between audio features and visual features. Expressed as the square of the L2 norm, Expressed as the Euclidean distance between visual features and semantic features, Expressed as the Euclidean distance between audio features and semantic features;

[0085] And the adversarial loss is optimized to the minimum and maximum of the total goal, where the calculation formula for the optimization total goal is: , where Expressed as the total loss of minimax optimization, Denoted as adversarial loss weight, Denoted as the alignment loss weight, Represented as a generator, Denoted as the discriminator.

[0086] S3. Multi-source data fusion: A dynamic fusion strategy is used to fusion the multimodal features of unified representation. According to the dynamic changes of fusion objectives and priorities, the fusion method and parameters are adjusted in real time to obtain fused data.

[0087] Specifically, the fusion rules of the dynamic fusion policy are as follows:

[0088] Target Adaptive Rule: Adjust the assigned feature weights based on the importance of different modal data in the current cultural goal, click-through rate goal, and user retention goal tasks;

[0089] Data source credibility priority principle: Features of highly credible data sources are given higher weights;

[0090] Real-time feedback modal complementarity principle: Different modal features enhance expression capabilities through nonlinear combination, and the weight of each modal data is adjusted in real time.

[0091] Specifically, the steps for obtaining fusion data are as follows:

[0092] Obtaining unified representation of multimodal features , according to the dynamic fusion strategy, the target weight of each modal feature is determined as , where Expressed as a multi-objective weight vector, Expressed as the target weight of the cultural goal, Expressed as the target weight for the click-through rate goal, The target weight representing the user retention goal;

[0093] The target gated attention mechanism is used for weighted calculation to adjust the importance of each modal feature and obtain the gated weighted feature. The calculation formula of the gated weighted feature is: ,and , where Represented as gated weighted features, Expressed as activation function, Expressed as a gating weight matrix, is represented as a gate bias vector, Expressed as the Hadamard product, The shared features of different modalities represented as semantic features, audio features, and visual features are mapped into the same space;

[0094] The credibility attention mechanism is used to perform weighted calculation of the fusion weight of each modal feature to obtain the attention weight, where the calculation formula of the attention weight is: 、 ,and 、 ,as well as , where Expressed as the multi-target raw attention score, Expressed as activation function, Represented as an attention weight vector, Expressed as the query transformation matrix, Expressed as a key transformation matrix, Expressed as the multimodal feature mean, Denoted as the normalized attention weight, Expressed as a natural exponential function, Expressed as a credibility weighted term;

[0095] The gated weighted features and attention weights of each modality are weighted and summed again to calculate the fusion feature, where the calculation formula of the fusion feature is: , where Represented as weighted fusion features;

[0096] Perform L2 normalization on the fusion features and output the normalized fusion data, where the expression of the fusion data is: , where Represented as the fusion data output after normalization, Expressed as L2 norm.

[0097] S4. Data Analysis and Dynamic Decision-Making: Analyze the integrated data to determine user interest, content credibility, and cultural value. Combined with the built-in multi-objective reinforcement learning decision-making engine, the PPO strategy optimization algorithm is used to develop app push strategies and cultural communication plans.

[0098] Specifically, the steps for obtaining the analysis results are as follows:

[0099] Construct a user click-through rate behavior sequence, combine it with the target attention mechanism to calculate the activation weight, use the weighted aggregation method to generate the user interest vector, concatenate the interest vector with the target feature, and use normalization to quantify the interest intensity score to obtain the user's interest intensity information;

[0100] Using the logistic regression enhancement algorithm, the Jaccard similarity verification of the credibility of the data source is performed to determine the dynamic adjustment of feature weights and obtain content credibility information;

[0101] By linking cultural entities, using the TransE algorithm to calculate cultural value and standardize the output, cultural value information is generated;

[0102] The analysis results include information on user interest intensity, content credibility, and cultural value.

[0103] Specifically, the dynamic decision-making steps of the PPO strategy optimization algorithm are as follows:

[0104] Based on the analysis results, the state space is constructed and the action space of the APP push strategy and cultural communication plan is defined, including the type of push content, push time, and communication channels. , action space ;

[0105] Design a reward function, use the policy network to generate the action probability distribution, use the value network to evaluate the value of the state, and combine the GAE algorithm to estimate the action-state advantage. The advantage function expression of the GAE algorithm is ,and , where Represented as time step The advantage function estimate of Expressed as a discount factor, Expressed as the smoothing coefficient of GAE, It represents the next step length. Expressed as time steps The time difference error, Represented as time step Rewards received, Represented as time step state The estimated value of Represented as time step state estimated value of;

[0106] Select an action based on the action probability distribution generated by the current policy network, and observe the reward and next state of the APP environment interaction feedback after executing the action;

[0107] Store the information of status, action, and reward to form trajectory data;

[0108] Then use the PPO algorithm to optimize the strategy network and value network to improve the performance of the strategy.

[0109] S5. Feedback and Optimization: Collect user feedback on the content pushed by the APP and the effectiveness of cultural communication, including user click behavior, comments and sharing times, and use online reinforcement learning algorithms to continuously optimize and improve the accuracy of APP push and the effectiveness of cultural communication.

[0110] Specifically, the steps of online reinforcement learning of feedback information are as follows:

[0111] Collect feedback from users on click behavior, comments, and sharing times, and design a reward function based on this feedback;

[0112] Initialize the action selection and evaluation state value parameters of the policy network and value network, and use the online PPO algorithm to update the policy network parameters;

[0113] This enables the DNQ policy network to learn the optimal push strategy.

[0114] Example 2

[0115] The present invention provides Figure 2 The data analysis and processing system based on multi-source data fusion shown in the figure includes a data acquisition and preprocessing module: using crawler tools to crawl multi-source data of user behavior data, content data, and social data from social software and Internet platforms in real time, and perform data cleaning, missing value filling and outlier processing to obtain preprocessed data, and transmit the preprocessed data to the preprocessed data. By establishing a dynamic credibility evaluation model, the credibility of each data source is evaluated, the reliability of the data source is quantified, the evaluation results are generated, and dynamic weights are assigned to the data sources;

[0116] Heterogeneous Data Feature Extraction and Unified Representation Module: This module receives preprocessed data, uses a visual CNN model to separate video frame data and extract visual features, employs an LSTM model to process audio data and mine temporal features, and employs a BERT model to obtain semantic features of text data. After combining visual, temporal, and semantic features into multimodal feature data, it uses an adversarial training deep learning model to map the multimodal feature data into the same space, unify the representation, and pass it to the multi-source data cross-modal fusion module.

[0117] Multi-source data cross-modal fusion module: Receives multi-modal feature data in a unified representation, uses a dynamic gating mechanism algorithm and attention mechanism to adjust the fusion process in real time based on the dynamic changes in fusion targets and priorities, outputs the fused data, and passes it to the data analysis and decision module;

[0118] Data analysis and decision-making module: Receives integrated data and uses classification and regression algorithms in machine learning to analyze and quantify multi-dimensional information on user interest intensity, content credibility, and cultural value. It also combines the PPO strategy optimization algorithm to generate personalized app push strategies and cultural communication plans.

[0119] Information feedback optimization and adjustment module: collects user feedback on click behavior, comments and sharing times of APP push content and cultural communication effects, uses online reinforcement learning algorithm to process feedback information, and optimizes the parameters and target weights of the dynamic fusion strategy online, so as to continuously adjust the accuracy of APP push and the effect of cultural communication.

[0120] The data analysis and processing system based on multi-source data fusion provided in an embodiment of the present invention is realized by the above-mentioned data analysis and processing method based on multi-source data fusion. The specific methods and processes of the data analysis and processing system based on multi-source data fusion are detailed in the embodiment of the above-mentioned data analysis and processing method based on multi-source data fusion, which will not be repeated here.

[0121] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0122] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0123] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data analysis and processing method based on multi-source data fusion, characterized in that: The steps include: S1. Data Collection and Preprocessing: Collect multi-source data related to app push and cultural communication from multiple data sources of social software and internet platforms, including user behavior data, content data, and social data. After collection, preprocess the multi-source data to generate preprocessed data, establish a dynamic credibility assessment model, and assign weights to data sources based on the assessment results. S2. Feature Extraction and Representation for Heterogeneous Data: We build a cross-modal decoupler-alignment architecture, utilizing visual CNN models, LSTM models, BERT models, and adversarial training deep learning models to extract and separate unique features for video frames, audio, and text modalities from preprocessed data. We then map features from different modalities into the same space and unify their representation. S3. Multi-source data fusion: A dynamic fusion strategy is used to fusion the multimodal features of unified representation. According to the dynamic changes of fusion objectives and priorities, the fusion method and parameters are adjusted in real time to obtain fused data. S4. Data Analysis and Dynamic Decision-Making: Analyze the integrated data to determine user interest, content credibility, and cultural value. Combined with the built-in multi-objective reinforcement learning decision-making engine, the PPO strategy optimization algorithm is used to develop app push strategies and cultural communication plans. S5. Feedback and Optimization: Collect user feedback on the content pushed by the APP and the effectiveness of cultural communication, including user click behavior, comments and sharing times, and use online reinforcement learning algorithms to continuously optimize and improve the accuracy of APP push and the effectiveness of cultural communication.

2. The data analysis and processing method based on multi-source data fusion according to claim 1 is characterized in that: The steps for acquiring and preprocessing the multi-source data are as follows: Use the Scrapy crawler framework and Kafka real-time stream collection tool to crawl multi-source data including text data, visual data, and audio data in real time; After the multi-source data undergoes preprocessing operations such as data cleaning, missing value filling, and outlier processing, preprocessed data is generated, and the expression of the preprocessed data is , where It represents pre-processed data after data cleaning, missing value filling and outlier processing of multi-source data collected in real time. Represented as text data, Represented as visual data, Represented as audio data.

3. The data analysis and processing method based on multi-source data fusion according to claim 2 is characterized in that: The steps for establishing the dynamic credibility evaluation model are as follows: Assign credibility to the data source, assign initial credibility to the type of data source, and mark it as , including official certified accounts, the initial credibility is assigned to 0.95; ordinary user UGC, the initial credibility is assigned to 0.75; third-party unverified data sources, the initial credibility is assigned to 0.60; The attenuation factor of the credibility of real-time user feedback is obtained by the number of times the user reports the data source information in a unit time. Among them, the attenuation factor of credibility is The calculation formula is , where Expressed as a natural exponential function, Expressed as the attenuation coefficient, Expressed as the number of reports per unit time; Jaccard similarity is used to verify the cross-platform consistency of data sources. The calculation formula for Jaccard similarity verification is: , where Expressed as similarity, Indicates that this platform, Indicates a third-party platform. Represented as the intersection symbol, It is represented by the union symbol; According to the attenuation factor and similarity verification of the new collected data and user feedback information, the dynamic weight synthesis method is used to obtain the final credibility weight of the data source. The calculation formula of the final credibility weight is: ,and , where Expressed as the final credibility weight.

4. The data analysis and processing method based on multi-source data fusion according to claim 3 is characterized in that: The steps for extracting the unique features are as follows: Use the visual CNN model to separate video frames, extract visual features of video frames by processing frame data, and convert visual data in preprocessed data into The row temporal spatial feature processing is performed and the visual feature vector is output, where The calculation formula for extracting visual features is: 、 ,and 、 , where Represented as the output visual feature vector, Expressed as The convolution calculation of the model, Expressed as CNN weight parameters, Represented as 1024 dimensions, Represented as the input video frame sequence, Represented as the input video frame sequences and the total number of video frame sequences, Represented as a sequence of video frames A 4D tensor consisting of the number of frames, frame height, frame width, and number of RGB channels, and The size is ; The LSTM model processes audio data, mines the time series features in the audio data, and converts the audio data in the preprocessed data into , and output the audio feature vector of the audio data. The calculation formula for extracting audio features by the LSTM model is: 、 ,and 、 , where Represented as the output audio feature vector, Represents the last time step output by the LSTM model The hidden state of Represented as the three gating mechanisms and hidden state update calculations of the LSTM model, Represented as iteratively processing each sequence of audio frames by time step, and , The last audio frame number of the audio frame sequence. Expressed as Mel frequency dimension, Represented as 256 dimensions; The BERT model processes text data, obtains the semantic features of the text, and converts the text data in the preprocessed data into , and output the semantic features of the text. The calculation formula for the BERT model to extract the [CLS] tag vector as the semantic feature of the text is: 、 ,and 、 ,as well as , where Represented as the output semantic features, The hidden state matrix represented as [CLS] notation, Represented as 768 dimensions, Represented as the BERT model output hidden state matrix, Represented as the calculation of the multi-head attention mechanism of the BERT model, Expressed as BERT weight parameters, Expressed as times the sequence length dimension, Represented as the sequence length of text data, Represented as the input text sequence for the BERT model, [CLS] represents the input text sequence The tag added at the beginning.

5. The data analysis and processing method based on multi-source data fusion according to claim 4 is characterized in that: The feature mapping steps of the adversarial training deep learning model are as follows: Using anti-training deep learning model to transform visual features , audio features and semantic features Different modal features are mapped to the same space And unified expression, where the projection expression of the same space is ,and 、 、 , where 、 、 Represented as shared features after projection of visual features, audio features and semantic features, 、 、 They are respectively represented as visual feature projection weight matrix, audio feature projection weight matrix, and text semantic feature projection weight matrix, 、 、 They are respectively represented as the visual feature projection bias vector, the audio feature projection bias vector, and the text semantic feature projection bias vector; The generator G of the adversarial training deep learning model shares the same spatial projection layer and the mode of the discriminator D predicting the feature source. The calculation formula of the adversarial training deep learning model is ,and , where Expressed as the minimization loss of the generator, Expressed as the expected operation on the same spatial mapping feature, It is expressed as the probability of the discriminator outputting the target distribution, Denoted as the minimization loss of the discriminator, Represented as shared features after visual feature projection The expectation operation of Represented as the shared feature after projection of the given visual feature , the output of the conditional discriminator belongs to the visual feature The probability of Represented as the shared features after audio feature projection The expectation operation of Represented as the shared feature after projection of the given audio feature , the output of the conditional discriminator belongs to the audio feature The probability of Represented as shared features after semantic feature projection The expectation operation of Represented as the shared feature after projection of given semantic features , the conditional discriminator outputs semantic features probability; Then, the semantic features, audio features, and visual features are aligned using the mean square error (MSE) to minimize the inter-modal distance. The formula for minimizing the inter-modal distance is: , where It is expressed as taking semantic features as anchors to minimize the inter-modal distance between audio features and visual features. Expressed as the square of the L2 norm, Expressed as the Euclidean distance between visual features and semantic features, Expressed as the Euclidean distance between audio features and semantic features; And the adversarial loss is optimized to the minimum and maximum overall goal, where the calculation formula for the optimization overall goal is: , where Expressed as the total loss of minimax optimization, Denoted as adversarial loss weight, Denoted as the alignment loss weight, Represented as a generator, Denoted as the discriminator.

6. The data analysis and processing method based on multi-source data fusion according to claim 5 is characterized in that: The steps for obtaining the fusion data are as follows: Obtaining unified representation of multimodal features , according to the dynamic fusion strategy, the target weight of each modal feature is determined as , where Expressed as a multi-objective weight vector, Expressed as the target weight of the cultural target, Expressed as the target weight for the click-through rate goal, The target weight representing the user retention goal; The target gated attention mechanism is used for weighted calculation to adjust the importance of each modal feature and obtain the gated weighted feature. The calculation formula of the gated weighted feature is: ,and , where Represented as gated weighted features, Expressed as activation function, Expressed as a gating weight matrix, is represented as a gate bias vector, Expressed as the Hadamard product, The shared features of different modalities represented as semantic features, audio features, and visual features are mapped into the same space; The credibility attention mechanism is used to perform weighted calculation of the fusion weight of each modal feature to obtain the attention weight, where the calculation formula of the attention weight is: 、 ,and 、 ,as well as , where Expressed as the multi-target raw attention score, Expressed as activation function, Represented as an attention weight vector, Expressed as the query transformation matrix, Expressed as a key transformation matrix, Expressed as the multimodal feature mean, Denoted as the normalized attention weight, Expressed as a natural exponential function, Expressed as a credibility weighted term; The gated weighted features and attention weights of each modality are weighted and summed again to calculate the fusion feature, where the calculation formula of the fusion feature is: , where Represented as weighted fusion features; Perform L2 normalization on the fusion features and output the normalized fusion data, where the expression of the fusion data is: , where Represented as the fused data output after normalization, Expressed as L2 norm.

7. The data analysis and processing method based on multi-source data fusion according to claim 6 is characterized in that: The dynamic decision-making steps of the PPO strategy optimization algorithm are as follows: Based on the analysis results, the state space is constructed and the action space of the APP push strategy and cultural communication plan is defined, including the type of push content, push time, and communication channels. , action space ; Design a reward function, use the policy network to generate the action probability distribution, use the value network to evaluate the value of the state, and combine the GAE algorithm to estimate the action-state advantage. The advantage function expression of the GAE algorithm is ,and , where Represented as time step The advantage function estimate of Expressed as a discount factor, Expressed as the smoothing coefficient of GAE, It represents the next step length. Expressed as time steps The time difference error, Represented as time step Rewards received, Represented as time step state The estimated value of Represented as time step state estimated value of; Select an action based on the action probability distribution generated by the current policy network, and observe the reward and next state of the APP environment interaction feedback after executing the action; Store the information of status, action, and reward to form trajectory data; Then use the PPO algorithm to optimize the strategy network and value network to improve the performance of the strategy.

8. A data analysis and processing system based on multi-source data fusion, implemented by the data analysis and processing method based on multi-source data fusion according to any one of claims 1 to 8, characterized in that: It includes a data collection and preprocessing module: using crawler tools to crawl multi-source data such as user behavior data, content data, and social data from social software and Internet platforms in real time, and then perform data cleaning, missing value filling and outlier processing to obtain preprocessed data, and then transfer the preprocessed data to the preprocessed data. By establishing a dynamic credibility assessment model, the credibility of each data source is assessed, the reliability of the data source is quantified, the assessment results are generated, and dynamic weights are assigned to the data sources. Heterogeneous Data Feature Extraction and Unified Representation Module: This module receives preprocessed data, uses a visual CNN model to separate video frame data and extract visual features, employs an LSTM model to process audio data and mine temporal features, and employs a BERT model to obtain semantic features of text data. After combining visual, temporal, and semantic features into multimodal feature data, it uses an adversarial training deep learning model to map the multimodal feature data into the same space, unify the representation, and pass it to the multi-source data cross-modal fusion module. Multi-source data cross-modal fusion module: Receives multi-modal feature data in a unified representation, uses a dynamic gating mechanism algorithm and attention mechanism to adjust the fusion process in real time based on the dynamic changes in fusion targets and priorities, outputs the fused data, and passes it to the data analysis and decision module; Data analysis and decision-making module: Receives integrated data and uses classification and regression algorithms in machine learning to analyze and quantify multi-dimensional information on user interest intensity, content credibility, and cultural value. It also combines the PPO strategy optimization algorithm to generate personalized app push strategies and cultural communication plans. Information feedback optimization and adjustment module: collects user feedback on click behavior, comments and sharing times of APP push content and cultural communication effects, uses online reinforcement learning algorithm to process feedback information, and optimizes the parameters and target weights of the dynamic fusion strategy online, so as to continuously adjust the accuracy of APP push and the effect of cultural communication.

9. A computer device comprising: memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the data analysis and processing method based on multi-source data fusion according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data analysis and processing method based on multi-source data fusion according to any one of claims 1 to 7 are implemented.

Images