A health knowledge video character IP intelligent generation method

Abstract

This invention provides a method for intelligently generating health knowledge video personalities (IPs). It involves extracting features from acquired multimodal behavioral data to obtain user feature vectors; constructing a candidate pool based on textual, visual, behavioral, and professional qualification data of multiple health knowledge video personalities (IPs), and calculating the attribute feature vectors of all IPs in the candidate pool; inputting the user feature vectors and the attribute feature vectors of all IPs in the candidate pool into a trained bimodal Transformer fusion model for prediction, obtaining a fit score for each feature vector; sorting all fit scores in descending order, selecting the top few IP recommendation schemes, and using all IP recommendation schemes to fine-tune the attribute feature vectors of all IPs in the candidate pool to generate health knowledge video personality (IP) schemes; thereby improving the accuracy of IP matching with user needs and enhancing the dissemination effect of health knowledge.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of multimedia technology, and in particular to a method for intelligently generating IP characters for health knowledge videos. Background Technology

[0002] With the rapid development of short video platforms, the way health knowledge is disseminated is undergoing profound changes. Personal branding, as an important vehicle for health knowledge dissemination, directly impacts the effectiveness of knowledge dissemination due to its compatibility with the target audience. Currently, the creation of health knowledge video personal branding relies primarily on manual planning and experience-based judgment, which has the following technical limitations:

[0003] 1. Lack of data-driven approach in task IP design: Existing technologies rely heavily on the subjective experience of the production and directing team to design health knowledge video character IPs, failing to effectively utilize the massive behavioral data generated by users on short video platforms. This approach leads to a deviation between IP design and users' actual needs.

[0004] 2. Low utilization rate of multimodal user behavior data: User behavior data on short video platforms includes multimodal information such as textual (comments, search terms), behavioral (viewing time, likes, reposts) and preference (content theme distribution), but existing technologies often only focus on single-dimensional data (such as viewership) and lack systematic integration and in-depth mining of multimodal data;

[0005] 3. Lack of Quantitative Standards for User-IP Compatibility: In existing technologies, the compatibility between users and IPs is mainly measured through qualitative descriptions (e.g., the target users are middle-aged and elderly groups) rather than quantitative indicators, resulting in a lack of clear direction for IP optimization. Although some platforms have attempted to use collaborative filtering algorithms for content recommendation, this method is based solely on the user-content interaction history and does not consider the multi-dimensional attributes of the IP itself (such as professional background, language style, and visual features) and their correlation with users' deeper needs, thus limiting recommendation accuracy.

[0006] 4. Immature multimodal feature fusion technology: When it comes to the fusion of multimodal data such as text, vision and phase, existing technologies often use simple splicing or weighted summation, which fails to effectively capture the complex relationships between modalities.

[0007] 5. Insufficient dynamic adaptation capability: Users' health needs and content preferences have dynamic characteristics (such as seasonal health problems and phased fitness goals), while existing IP generation solutions are mostly static designs, lacking a real-time update mechanism, and the correlation between new data and historical data is not fully considered during the update process;

[0008] 6. Insufficient integration of professional knowledge: Health knowledge is highly specialized, and the professional background requirements for character IPs vary significantly across different sub-fields (such as chronic disease management, sports rehabilitation, and nutrition). Existing technologies have failed to systematically integrate professional knowledge in the health field into the IP generation process, resulting in some IPs having problems such as inappropriate use of professional terminology and insufficient scientific content.

[0009] 7. Lack of targeted integration of multimodal features with IP generation: Although existing technologies have involved multimodal user feature extraction and attention mechanism fusion, they have not been deeply integrated with the core attributes of health knowledge video personalities (such as professional credibility, communication affinity, and scene adaptability), resulting in a disconnect between technology application and the actual needs of IP generation, and failing to form a dedicated technical solution system.

[0010] In summary, existing technologies for generating health knowledge video character IPs suffer from several problems, including low data utilization, vague adaptation standards, immature integration technology, insufficient dynamic adjustment capabilities, inadequate integration of professional knowledge, and a lack of targeted combination of technology and specific scenarios. Summary of the Invention

[0011] This invention provides a method for intelligently generating IP characters in health knowledge videos, aiming to improve the accuracy of IP matching with user needs and enhance the dissemination effect of health knowledge.

[0012] To achieve the above objectives, this invention provides a method for intelligently generating IP characters in health knowledge videos, comprising:

[0013] Step 1: Obtain multimodal behavioral data of the target user;

[0014] Step 2: Extract features from the multimodal behavioral data to obtain user feature vectors;

[0015] Step 3: Construct a candidate pool containing various health knowledge video IPs based on text data, visual data, behavioral data, and professional qualification data of multiple health knowledge video IPs, and calculate the attribute feature vectors of all IPs in the candidate pool.

[0016] Step 4: Input the user feature vector and the attribute feature vectors of all the IPs in the candidate pool into the trained bimodal Transformer fusion model for prediction, and obtain the fitting score of each feature vector.

[0017] Step 5: Sort all the matching scores in descending order, select the top few IP recommendation schemes, and use all the IP recommendation schemes to fine-tune the attribute feature vectors of all the IPs in the candidate pool to generate health knowledge video IP schemes.

[0018] Furthermore, prior to step 2, the following steps are also included:

[0019] The multimodal behavioral data is preprocessed to obtain preprocessed multimodal behavioral data.

[0020] Furthermore, the preprocessed multimodal behavioral data includes:

[0021] Preprocessed text data, preprocessed behavioral data, and preprocessed preference data.

[0022] Furthermore, step 2 includes:

[0023] Semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modal features;

[0024] The preprocessed behavioral data is subjected to behavioral sequence feature extraction to obtain the total behavioral modality features.

[0025] Sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total features of the preference modality;

[0026] The user feature vector is obtained by fusing the total features of the text modality, the total features of the behavioral modality, and the total features of the preference modality.

[0027] Furthermore, semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modality features, including:

[0028] The pre-trained BERT model is used to encode the pre-processed text data to obtain semantic feature vectors;

[0029] A three-layer Bayesian probability model is used to extract topic distribution from the preprocessed text data to obtain topic distribution probabilities.

[0030] The semantic feature vector is concatenated with the topic distribution probability to obtain the total text modality feature.

[0031] Furthermore, behavioral sequence features are extracted from the preprocessed behavioral data to obtain the total behavioral modality features, including:

[0032] Gated recurrent neural networks are used to model behavioral sequences from preprocessed behavioral data to obtain behavioral sequence features.

[0033] The preprocessed behavioral data is flattened according to channel priority to obtain the flattened vector.

[0034] The flattened vector is concatenated with the behavioral sequence features to obtain the total behavioral modality features.

[0035] Furthermore, sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total features of the preference modality, including:

[0036] The VADER model was used to extract sentiment features from the preprocessed preference data to obtain the sentiment features of the comments.

[0037] The preprocessed preference data is subjected to keyword matching and style recognition using a classification model to obtain style tags.

[0038] The complexity preference is calculated based on the distribution of the duration of target users watching videos of different complexities.

[0039] By concatenating the sentiment features, style tags, and complexity preferences of the comments, the total feature of the preference modality is obtained.

[0040] Furthermore, the bimodal Transformer fusion model includes an intramodal encoder, a knowledge integration module, a cross-modal fusion module, a cross-modal attention calibration module, and an adaptation score prediction module;

[0041] Intramodal encoders are used to perform modal self-attention modeling on user feature vectors and attribute feature vectors to obtain user context vectors and IP context vectors.

[0042] The professional knowledge integration module is used to fuse user context vectors and IP context vectors with health knowledge graph features respectively, to obtain user graph fusion features and IP graph fusion features;

[0043] The cross-modal fusion module is used to perform cross-attention fusion of user graph fusion features and IP graph fusion features to obtain cross-modal fusion features;

[0044] The cross-modal attention calibration module is used to introduce calibration loss to optimize the cross-modal fusion features, resulting in optimized fusion features;

[0045] The adaptation score prediction module is used to predict the adaptation score based on the optimized fusion features.

[0046] Furthermore, by fusing user context vectors and IP context vectors with health knowledge graph features respectively, user graph fusion features and IP graph fusion features are obtained, including:

[0047] Construct a health knowledge graph that includes entities and relationships in the health domain, and integrate user feature vectors and attribute feature vectors into the health knowledge graph;

[0048] In the health knowledge graph, entity association queries are performed on the text modal features in the user feature vector and the professional qualification modal in the attribute feature vector to obtain the user health knowledge graph features and IP health knowledge graph features.

[0049] By fusing user health knowledge graph features with user context vectors, user graph fusion features are obtained.

[0050] By fusing the IP health knowledge graph features with the IP context vector, we obtain the IP graph fusion features.

[0051] Furthermore, the loss function of the bimodal Transformer fusion model consists of a main loss function, an auxiliary loss function, an attention regularization loss function, and a expertise calibration loss function.

[0052] The above-described solution of the present invention has the following beneficial effects:

[0053] This invention extracts features from acquired multimodal behavioral data to obtain user feature vectors; it constructs a candidate pool containing various types of health knowledge video IPs based on textual, visual, behavioral, and professional qualification data of multiple IPs, and calculates the attribute feature vectors of all IPs in the candidate pool; it inputs the user feature vectors and the attribute feature vectors of all IPs in the candidate pool into a trained bimodal Transformer fusion model for prediction, obtaining a fit score for each feature vector; it sorts all fit scores in descending order, selects the top few IP recommendation schemes, and uses all IP recommendation schemes to fine-tune the attribute feature vectors of all IPs in the candidate pool to generate health knowledge video IP schemes; compared with existing technologies, this invention constructs a candidate pool containing various types of health knowledge video IPs based on textual, visual, behavioral, and professional qualification data of multiple IPs to improve the accuracy of IP matching with user needs; and by proposing a bimodal Transformer fusion model to capture the complex relationship between user feature vectors and attribute feature vectors, it enhances the effect of health knowledge dissemination.

[0054] Other beneficial effects of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0056] Figure 2 This is a structural diagram of the dual-modal Transformer fusion model in an embodiment of the present invention. Detailed Implementation

[0057] To make the technical problems, solutions, and advantages of this invention clearer, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0058] In the description of this invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0059] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a locking connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0061] This invention addresses existing problems by providing a method for intelligently generating IP characters for health knowledge videos.

[0062] like Figure 1 As shown, an embodiment of the present invention provides a method for intelligently generating health knowledge video character IPs, including:

[0063] Step 1: Obtain multimodal behavioral data of the target user;

[0064] Step 2: Extract features from the multimodal behavioral data to obtain user feature vectors;

[0065] Step 3: Construct a candidate pool containing various health knowledge video IPs based on text data, visual data, behavioral data, and professional qualification data of multiple health knowledge video IPs, and calculate the attribute feature vectors of all IPs in the candidate pool.

[0066] Step 4: Input the user feature vector and the attribute feature vectors of all the IPs in the candidate pool into the trained bimodal Transformer fusion model for prediction, and obtain the fitting score of each feature vector.

[0067] Step 5: Sort all the matching scores in descending order, select the top few IP recommendation schemes, and use all the IP recommendation schemes to fine-tune the attribute feature vectors of all the IPs in the candidate pool to generate health knowledge video IP schemes.

[0068] Specifically, before step 2, the following is also included:

[0069] The multimodal behavioral data is preprocessed to obtain preprocessed multimodal behavioral data.

[0070] In this embodiment of the invention, preprocessing of multimodal behavioral data includes:

[0071] Data cleaning was performed on the multimodal behavioral data to remove duplicate and outlier values, resulting in the first multimodal behavioral data.

[0072] The first multimodal behavior data is format-converted to obtain the second multimodal behavior data, which is then used as the preprocessed multimodal behavior data.

[0073] Specifically, the preprocessed multimodal behavioral data includes:

[0074] Preprocessed text data, preprocessed behavioral data, and preprocessed preference data.

[0075] This invention utilizes the open API data collection interface of short video platforms to select 1000 typical users, covering 15 sub-groups with different health needs (such as fitness beginners, people with high blood pressure, high cholesterol, and high blood sugar, prenatal care, etc.), and collects their multimodal behavioral data over the past 30 days as a training dataset. Including text data Behavioral data and preference data The data collected included textual data such as comments and search terms, behavioral data such as viewing history, likes, and shares, and preference data such as content theme distribution and interaction frequency distribution. Fifty candidate IPs in the health field were selected, covering different professional backgrounds (medicine, nutrition, sports, etc.), language styles (serious, lively, humorous, etc.), and visual characteristics (age, gender, scene, etc.). Textual data was collected from each IP. Visual data Behavioral data and professional qualification data As multimodal data for each IP The text data includes IP introduction, time-frequency script, title, etc.; the visual data includes video frame screenshots and cover images; the behavioral data includes audience profiles and interaction rates of historical videos; and the professional qualification data includes years of experience in related fields, professional certification certificates, and academic achievements.

[0076] Specifically, step 2 includes:

[0077] Semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modal features;

[0078] The preprocessed behavioral data is subjected to behavioral sequence feature extraction to obtain the total behavioral modality features.

[0079] Sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total features of the preference modality;

[0080] The user feature vector is obtained by fusing the total features of the text modality, the total features of the behavioral modality, and the total features of the preference modality.

[0081] In this embodiment of the invention, the preprocessed text data is ,in, Indicates the first One user comment, This indicates the number of user comments. Indicates the first 1 search term, Indicates the number of search terms;

[0082] Preprocessed behavioral data ,in, Indicates the number of content categories. It represents the number of behavior categories, specifically including three-dimensional statistical data of 10 health areas, 5 content formats, and 4 interactive behaviors;

[0083] Preprocessed preference data This includes sentiment in comments, time distribution of content interaction, and preference for content complexity.

[0084] Specifically, semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modality features, including:

[0085] The pre-trained BERT model is used to encode the pre-processed text data to obtain semantic feature vectors;

[0086] A three-layer Bayesian probability model is used to extract topic distribution from the preprocessed text data to obtain topic distribution probabilities.

[0087] The semantic feature vector is concatenated with the topic distribution probability to obtain the total text modality feature.

[0088] In this embodiment of the invention, the pre-trained BERT model is specifically fine-tuned on a health-related corpus (including medical literature, health science popularization texts, etc.). The input is the concatenation of all user text data, and the output is the hidden state vector of the [CLS] token, i.e., the semantic feature vector.

[0089] ;

[0090] in, This indicates that the pre-trained BERT model was used for encoding;

[0091] A three-layer Bayesian probabilistic model was used to extract topic distributions from the preprocessed text data, and the expression for the topic distribution probability was obtained as follows:

[0092] ;

[0093] This invention provides 10 preset health themes (such as weight loss, diabetes management, and sports rehabilitation), therefore... ;

[0094] By concatenating the semantic feature vector with the topic distribution probability, the expression for the total text modality feature is obtained as follows:

[0095] ;

[0096] in, Represents the overall features of the text modality. This represents the probability distribution of the topic.

[0097] Specifically, behavioral sequence features are extracted from the preprocessed behavioral data to obtain the total behavioral modality features, including:

[0098] Gated recurrent neural networks are used to model behavioral sequences from preprocessed behavioral data to obtain behavioral sequence features.

[0099] The preprocessed behavioral data is flattened according to channel priority to obtain the flattened vector.

[0100] The flattened vector is concatenated with the behavioral sequence features to obtain the total behavioral modality features.

[0101] In this embodiment of the invention, before using a gated recurrent neural network to model the behavior sequence of the preprocessed behavior data, the method further includes:

[0102] The normalized frequency of a user on each combination is calculated to obtain the behavioral frequency. The calculation expression is as follows:

[0103] ;

[0104] in, Indicates the user in the category Behavior occurring above The number of times.

[0105] In this embodiment of the invention, a gated recurrent neural network is used to model the behavior sequence of the preprocessed behavior data, and the expression for the behavior sequence features is obtained as follows:

[0106] ;

[0107] in, Represents behavioral sequence features;

[0108] Preprocessed behavioral data Flattening the vector according to channel priority yields the following expression:

[0109] ;

[0110] The flattened vector is concatenated with the behavioral sequence features to obtain the expression for the total behavioral modality features:

[0111] ;

[0112] in, Represents the overall characteristics of behavioral modalities. This represents the flattened vector.

[0113] Specifically, sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total features of the preference modality, including:

[0114] The VADER model was used to extract sentiment features from the preprocessed preference data to obtain the sentiment features of the comments.

[0115] The preprocessed preference data is subjected to keyword matching and style recognition using a classification model to obtain style tags.

[0116] The complexity preference is calculated based on the distribution of the duration of target users watching videos of different complexities.

[0117] By concatenating the sentiment features, style tags, and complexity preferences of the comments, the total feature of the preference modality is obtained.

[0118] In this embodiment of the invention, the VADER model is used to extract sentiment features from the preprocessed preference data, and the expression for the sentiment features of the comments is as follows:

[0119] ;

[0120] in, The sentiment characteristics of the comments are represented by five dimensions: positivity, negativity, neutrality, emotional intensity, and polarity.

[0121] The preprocessed preference data is analyzed using a classification model to perform keyword matching and style identification, resulting in style tags. The style tags include three binary tags: data-driven, story-driven, and expert-driven.

[0122] Based on the distribution of viewing time for videos of different complexities by target users, the expression for complexity preference is calculated from the preprocessed preference data:

[0123] ;

[0124] in, Indicates the viewing duration. Indicates the content complexity score;

[0125] By concatenating the sentiment features, style tags, and complexity preferences of the comments, the expression for the total feature of the preference modality is obtained as follows:

[0126] ;

[0127] in, This represents the overall characteristics of the preference mode. Indicates style tags, This indicates a preference for complexity.

[0128] To address the issues of dimensional differences and noise in features across different modalities, this embodiment of the invention requires preprocessing before inputting the user feature vector and the attribute feature vectors of all IPs in the candidate pool into the trained bimodal Transformer fusion model for prediction. Specifically, this preprocessing includes:

[0129] use The behavioral modality features and preference modality features are z-score standardized to obtain the standardized features, where, Represents the standardized features. This represents the features before standardization, either behavioral modal features or preference modal features. , represent the mean and standard deviation of the features in the training set, respectively;

[0130] L2 regularization is used to suppress noise in text modal features. The expression is:

[0131] ;

[0132] in, Text modal features representing noise suppression To prevent division by zero of small constants;

[0133] The weights are dynamically adjusted based on the signal-to-noise ratio of each modality feature, as expressed by:

[0134] ;

[0135] in, Represents modal weights, , Indicates the modal signal-to-noise ratio. , Refers to the first Effective signals within modal data, that is, core information within that modality that truly reflects user needs, preferences, or IP attributes. Refers to the first Noise information in modal data, that is, redundant data in the modality that is irrelevant to the core requirements and interferes with feature extraction. Variance is a statistical measure of the dispersion of data, used to quantify the range of fluctuations in signals or noise.

[0136] Specifically, the attribute feature vectors of all IPs in the candidate pool are calculated, including:

[0137] The semantic vector of each IP's text data is extracted using a pre-trained BERT model, ensuring it is in the same semantic space as the user's text vector. The expression is as follows:

[0138] ;

[0139] in, Textual features representing IP addresses;

[0140] Extract the one-hot encoded vectors from the IP professional background. This indicates that the IP has the third Class of qualifications: This includes 10 health-related professional fields, therefore ;

[0141] A style classifier is used to identify the language style of an IP, including four dimensions: professionalism, accessibility, humor, and motivation. Each dimension has a value range of [0,1].

[0142] ;

[0143] in, Indicates stylistic features of language;

[0144] The FaceNet model is used to extract facial feature vectors, which include attributes such as age, gender, and expression.

[0145] ;

[0146] Use ResNet-50 to extract scene style features and capture the scene style of IP videos (such as laboratory, home, gym, etc.):

[0147] ;

[0148] A visual attention model trained using eye-tracking experimental data is used to extract visual focus from IP videos.

[0149] ;

[0150] in, Represents behavioral data of IP addresses. Represents facial feature vectors, Indicates the stylistic features of the scene. Indicates the visual focus;

[0151] Calculate the overlap between the IP's existing audience and the target user group, and generate user profile vectors. IP audience profile vector Both vectors are 10-dimensional feature vectors (containing age distribution (3-dimensional), gender ratio (1-dimensional), and health-related interest distribution (6-dimensional)). Cosine similarity is calculated for each corresponding dimension of the two vectors, and then these similarity values ​​are weighted and fused into a single scalar.

[0152] ;

[0153] in, For the first Weights of each dimension Profiling IP Audiences dimensional features, Profiling the target user group Dimensional features;

[0154] Extract the standardized interaction rate of IP videos:

[0155] ;

[0156] in, ;

[0157] The expressions for normalizing years of professional experience (0-1 range), numerically mapping professional certification levels (primary / intermediate / advanced) (1 / 2 / 3), and extending the number of academic achievements and qualification certificate types using one-hot encoding are as follows:

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] in, Indicates years of professional experience. Indicates the level of professional certification. Indicates academic achievements, Indicates the type of qualification certificate;

[0163] The expression for calculating the attribute feature vector is:

[0164] ;

[0165] in, Textual features representing all character IPs, Represents the visual characteristics of all character IPs. This represents the behavioral characteristics of all character IPs. This indicates the professional qualifications and characteristics of all character IPs.

[0166] Specifically, such as Figure 2 As shown, the bimodal Transformer fusion model includes an intramodal encoder, a knowledge integration module, a cross-modal fusion module, a cross-modal attention calibration module, and an adaptation score prediction module.

[0167] Intramodal encoders are used to perform modal self-attention modeling on user feature vectors and attribute feature vectors to obtain user context vectors and IP context vectors.

[0168] The professional knowledge integration module is used to fuse user context vectors and IP context vectors with health knowledge graph features respectively, to obtain user graph fusion features and IP graph fusion features;

[0169] The cross-modal fusion module is used to perform cross-attention fusion of user graph fusion features and IP graph fusion features to obtain cross-modal fusion features;

[0170] The cross-modal attention calibration module is used to introduce calibration loss to optimize the cross-modal fusion features, resulting in optimized fusion features;

[0171] The adaptation score prediction module is used to predict the adaptation score based on the optimized fusion features.

[0172] In this embodiment of the invention, the intramodal encoder includes a user feature encoder and an IP feature encoder;

[0173] The user feature encoder uses a 3-layer Transformer encoder to model the user feature vector using self-attention. The 3-layer Transformer encoder includes:

[0174] The input layer, used to map user feature vectors to the same dimension, is expressed as follows:

[0175] ;

[0176] ;

[0177] ;

[0178] in, The dimensions of the mapping matrix are respectively , and Ensure that the dimensions of matrix multiplication are matched;

[0179] The position coding layer is used to add position coding to distinguish different modes. It uses sine and cosine position coding, and the expression is:

[0180] ;

[0181] in, Indicates the modal number. Indicates a dimension index;

[0182] The location code is added as follows:

[0183] ;

[0184] ;

[0185] ;

[0186] in, The first character of the user feature encoder Modal position encoding;

[0187] The Transformer encoder consists of 3 Transformer layers, each with 8 self-attention heads. The feedforward hidden layer has a dimension of 1024. The input sequence has a length of 3, corresponding to 3 modal features, each with a dimension of 256, forming a 3×256 input matrix. The calculation expression is as follows:

[0188] ;

[0189] in, Parameters representing the user feature encoder;

[0190] The pooling layer is used to pool the features output by the Transformer encoder to obtain the user context vector, expressed as:

[0191] ;

[0192] in, Represents the user context vector;

[0193] The IP feature encoder employs a 3-layer Transformer encoder symmetrical to the user feature encoder, and also includes professional qualification modality encoding processing for modeling attribute feature vectors, specifically including:

[0194] The input layer, used to map attribute feature vectors to the same dimension, is expressed as follows:

[0195] ;

[0196] ;

[0197] ;

[0198] ;

[0199] in, The dimensions of the mapping matrix are respectively , , and Ensure that the dimensions of matrix multiplication are matched;

[0200] The location coding layer is used for independent IP modality location coding. It contains codes corresponding to four modalities, and the location codes are added as follows:

[0201] ;

[0202] ;

[0203] ;

[0204] ;

[0205] in, The first characteristic encoder of the IP feature encoder Each modal position is encoded independently of the user encoder position encoding parameters;

[0206] The Transformer encoder contains four Transformer layers. The input sequence has a length of 4, corresponding to four modal features. After mapping each of the four modal features to a 256-dimensional matrix, they are concatenated in modal order to form a 4×256 input matrix. The calculation expression is as follows:

[0207] ;

[0208] in, Parameters representing the user feature encoder;

[0209] The pooling layer is used to pool the features output by the Transformer encoder to obtain the IP context vector, expressed as:

[0210] ;

[0211] in, This represents the IP context vector.

[0212] Specifically, user context vectors and IP context vectors are fused with health knowledge graph features to obtain user graph fusion features and IP graph fusion features, including:

[0213] Construct a health knowledge graph that includes entities in the health field (diseases, symptoms, nutrients, exercise methods, etc.) and relationships (causal relationships, treatment relationships, recommendation relationships, etc.), and integrate user feature vectors and attribute feature vectors into the health knowledge graph;

[0214] In the health knowledge graph, entity association queries are performed on the text modality features in the user feature vector and the professional qualification modality in the attribute feature vector to obtain the user health knowledge graph features and IP health knowledge graph features. The expression is as follows:

[0215] ;

[0216] ;

[0217] in, This represents the characteristics of a user's health knowledge graph. This represents the characteristics of the IP health knowledge graph;

[0218] By fusing user health knowledge graph features with user context vectors, the user graph fusion feature is obtained, expressed as:

[0219] ;

[0220] in, Indicates user graph fusion features, , Indicates learnable parameters;

[0221] By fusing the IP health knowledge graph features with the IP context vector, the IP graph fusion feature is obtained, expressed as:

[0222] ;

[0223] in, This indicates the characteristics of IP map fusion.

[0224] In this embodiment of the invention, the cross-modal fusion module employs a dual-interaction cross-attention mechanism combined with health domain attention guidance to achieve dimensional alignment and fusion of user and IP characteristics, specifically including:

[0225] The first cross-attention step uses the user context vector as the query and IP modality features as key-value pairs to learn the IP features that the user follows. A health-themed attention mask is introduced to strengthen the weights of health-related features. The expression is as follows:

[0226] ;

[0227] in, This represents an attention mask based on a health knowledge graph, which suppresses the weights of IP features that are irrelevant to the user's health needs.

[0228] The second cross-attention process uses the IP context vector as the query and user modality features as key-value pairs to learn the matching between IP and user features. It introduces a professional qualification attention mask to strengthen the matching between IP professional attributes and user needs. The expression is:

[0229] ;

[0230] in, This represents an attention mask based on IP professional qualifications, highlighting the connection between IP professional background and user needs;

[0231] Multiple interaction features are combined through element-wise multiplication and concatenation, with the concatenation order being [user context vector]. IP Context Vector User-guided IP characteristics IP-guided user characteristics [Product of user IP elements], each concatenated vector has a dimension of There are 5 vectors in total, with a total concatenation dimension of . ;

[0232] Finally, a linear layer is used to map the 1280-dimensional concatenated vector to 256 dimensions, as shown in the expression:

[0233] ;

[0234] in, Represents element-wise product. The five pieces are spliced ​​together. dimensional feature mapping back dimension.

[0235] To address the issues of uneven attention distribution and inaccurate modal correlation capture in cross-modal fusion modules, this embodiment of the invention utilizes a cross-modal attention calibration module for calibration. The specific process is as follows:

[0236] Attention weight normalization: The weight matrix of the cross-attention output is row-normalized to ensure a reasonable weight distribution. The expression is as follows:

[0237] ;

[0238] in, This is the cross-attention weight matrix. To query the sequence length, The length of the key-value sequence;

[0239] Calibration Loss Calculation: A calibration loss is introduced to bring the attention weights closer to the optimal matching distribution in the health domain. The expression is as follows:

[0240] ;

[0241] in, This represents the optimal attention distribution matrix based on the experience of health experts.

[0242] Feature calibration update: Backpropagate the calibration loss to update the attention weight parameters and optimize the fused features. The expression is:

[0243] ;

[0244] in, This is the calibration coefficient.

[0245] Specifically, the adaptation score prediction module predicts the adaptation score based on fused features, as expressed by:

[0246] ;

[0247] in, This represents the correction factor. , To determine the health professionalism score, the degree of match between the IP's professional qualifications and the user's health needs is calculated. This represents a multilayer perceptron, consisting of two hidden layers (512→256), using the ReLU activation function, and the output layer scaled to the [0,10] range using the Simgiod function. .

[0248] In this embodiment of the invention, the adaptation score of each feature vector is calculated as follows:

[0249] Constructing a judgment matrix: Five experts in the field of health communication conducted pairwise comparisons of the importance of primary indicators to construct a 3×3 judgment matrix. :

[0250] ;

[0251] Calculate the weight vector: Find the largest eigenvalue of the judgment matrix using the eigenvalue method. And its corresponding feature vectors, after normalization, yield the first-level indicator weight row vector. ,in ;

[0252] Consistency check: The consistency index is calculated as follows:

[0253] ;

[0254] when At that time, it was assumed that the judgment matrix had satisfactory consistency.

[0255] Using the same method to determine the weights of the secondary indicators relative to the primary indicators, we obtain:

[0256] User preference matching degree: (Overlap of health themes) (Content format preference);

[0257] Trust perception fit: (Professional background matching degree) (Friendliness of expression);

[0258] Behavioral conversion fit: (Content depth adaptation) (Interaction rate overlap);

[0259] Total compatibility score calculation:

[0260] ;

[0261] in: Indicates the degree of matching of user preferences. , Indicates the degree of trust perception fit. , Indicates the adaptability of behavior transformation. , Indicates the degree of overlap in health-related topics. Indicates preference for content format. Indicates the degree of matching of professional background. Indicates the friendliness of the expression style. Indicates the depth of content adaptation. This indicates the degree of overlap in interaction rates.

[0262] Specifically, the loss function of the bimodal Transformer fusion model consists of a main loss function, an auxiliary loss function, an attention regularization loss function, and a expertise calibration loss function.

[0263] In this embodiment of the invention, the main loss is a smoothed L1 loss, which is more robust to outliers than MSE. The calculation expression is as follows:

[0264] ;

[0265] in, ;

[0266] The auxiliary loss is the contrast loss, used to enhance the discriminative power of positive samples. Its calculation expression is:

[0267] ;

[0268] in, This represents the prediction score for positive samples. This represents the prediction score for negative samples. Indicates the boundary threshold. ;

[0269] Attention regularization loss is used to prevent model overfitting and encourage uniform attention distribution. Let the first... Layer attention module contains There are 1 attention head, and each attention head generates an attention weight matrix. ,in To query the sequence length, Given the length of the key-value sequence, the attention weight matrix for this layer... Attention weight normalization step: Perform row normalization on the weight matrix of each attention head: ,in This is used to ensure that the weights in each row are uniformly distributed reference matrices with a sum of 1; a uniformly distributed reference matrix of the same dimension is used for all attention head weight matrices, where... The length of the key-value sequence (in the user encoder) This corresponds to three modal features; in the IP encoder (corresponding to 4 modal features) ,in, It is an array where all elements are 1. matrix, To query the sequence length (in the user / IP encoder) For values ​​of 3 and 4 respectively, the expression for calculating the attention regularization loss is:

[0270] ;

[0271] in, (3 layers of user encoder + 3 layers of IP encoder) is the total number of Transformer layers. The number of attention heads per layer, Here is the formula for calculating KL divergence;

[0272] The professionalism calibration loss is used to ensure an accurate match between the professional attributes of the IP and the user's health needs. The professionalism matching loss is calculated based on a health knowledge graph.

[0273] ;

[0274] in, The score matches the level of professionalism of the manually labeled data.

[0275] The total loss function of the dual-modal Transformer fusion model in this embodiment of the invention is: ;

[0276] Among them, hyperparameters , and It is the optimal value determined on the validation set through grid search; specifically, in , and In the parameter space, select the parameter combination that minimizes the MAE of the validation set.

[0277] Specifically, the attribute feature vectors of all IP personalities in the candidate pool are fine-tuned using all IP recommendation schemes to generate IP personalities for health knowledge videos, including:

[0278] Adjust the IP attributes of all IPs in the candidate pool using all IP recommendation schemes: adjust IP attributes based on key matching dimensions, with a focus on optimizing professional-related attributes;

[0279] The visual characteristics of all IPs in the candidate pool are adjusted using all IP recommendation schemes, such as age perception, facial expression style, and scene setting (balancing professional and everyday scenes).

[0280] Adjust the language style of all IPs in the candidate pool using all IP recommendation schemes: such as the proportion of professional terminology (adjusted according to the user's health literacy level), speech speed, and tone;

[0281] Adjust the content direction of all IPs in the candidate pool using all IP recommendation schemes: such as theme focus (based on the degree of overlap with health themes), case selection, and presentation format;

[0282] Adjust the professional attributes of all IPs in the candidate pool using all IP recommendation schemes: such as highlighting the core professional field of the IP and showcasing relevant professional qualifications;

[0283] Generate IP Scheme: Outputs a complete IP scheme for the individual, including:

[0284] IP positioning: core audience, value proposition, and differentiated advantages (including professional advantages).

[0285] Image design: visual feature description, clothing style, scene setting;

[0286] Language style: tone, speaking speed, common expressions, and standard use of professional terminology (by user group);

[0287] Content planning: theme distribution, content format, update frequency, and professional review process;

[0288] Professional endorsement: A scheme to showcase the professional qualifications of the IP, and an explanation of the source of the health knowledge;

[0289] Sample video scripts: 3-5 typical scenario video script examples (including professional content validation).

[0290] The embodiments of the present invention further verify the provided method through experiments, the process of which is as follows:

[0291] Data sample description:

[0292] User data: 1500 real users were selected from a mainstream short video platform, covering 5 health-related tags (fitness beginners, people trying to lose weight, diabetics, people with high blood pressure, and pregnant women), with 300 people in each group. User behavior data over the past 30 days was collected, including text data (comments, search terms), behavioral data (viewing history, likes, shares), and preference data (content theme distribution).

[0293] IP Data: Sixty candidate IPs in the health field were selected, covering different professional backgrounds (medicine, nutrition, sports, etc.), language styles (serious, lively, humorous, etc.), and visual characteristics (age, gender, scene, etc.). Their text data, visual data, behavioral data, and professional qualification data were collected.

[0294] Data annotation: A team of 3 health management experts and 5 user experience researchers was formed to annotate 1500×60=90000 (user, IP) sample pairs with adaptation scores (1-10 points) and professional matching scores (1-5 points).

[0295] Experimental setup:

[0296] Baseline methods: BERT+MLP, traditional collaborative filtering, single-modal SVM, simple concatenation + MLP, general Transformer fusion model;

[0297] Evaluation metrics: MAE, RMSE, Pearson correlation coefficient, Spearman rank correlation coefficient, and professionalism matching accuracy.

[0298] Experimental environment: NVIDIA Tesla V100 GPU, PyTorch 1.9.0 framework;

[0299] The experimental results are as follows:

[0300] The performance of the adaptation score prediction is shown in Table 1 below:

[0301] Table 1 Comparison of Adaptation Score Prediction Performance

[0302]

[0303] Experimental results show that the method of this invention is significantly better than the baseline method on all evaluation metrics, and the MAE is reduced by 22.4% compared with the best baseline method (general Transformer fusion model), which proves the accurate prediction capability of user-IP adaptability.

[0304] The results of the cross-modal fusion performance analysis are shown in Table 2 below:

[0305] Table 2 Analysis Results of Cross-Modal Fusion Effect

[0306]

[0307] The results show that the model that integrates four modal features (including professional qualifications) has the best performance, with a 22.4% reduction in MAE compared to the combination of three modal features, thus verifying the effectiveness of the professional qualification modality and cross-modal fusion architecture.

[0308] The evaluation results of the model's dynamic adaptability, simulating the change in user needs from "fat loss" to "muscle gain," are shown in Table 3 below:

[0309] Table 3 Evaluation results of the model's dynamic adaptation capability

[0310]

[0311] The results show that the incremental update mechanism of this invention can effectively track changes in user interests. After 4 weeks, the MAE remains at a low level of 0.55, which is better than the traditional static model and the general incremental update model.

[0312] The performance evaluation of professionalism matching is shown in Table 4 below:

[0313] Table 4. Performance Evaluation Results of Professionalism Matching

[0314]

[0315] The results show that the professional knowledge integration module and professionalism calibration loss in the embodiments of the present invention significantly improve the accuracy of professionalism matching, with an improvement of 21.1% compared with the model without professional knowledge integration.

[0316] Experimental results fully demonstrate that the method of this invention can accurately predict user-IP adaptation scores, with an MAE of 0.45 and a Pearson correlation coefficient of 0.87. The multimodal fusion architecture (including the professional qualification modality) is significantly superior to single modality or other modal combinations, reducing MAE by 22.4%, thus verifying the effectiveness of the cross-modal fusion architecture. The dynamic update mechanism can shorten the user interest drift response time from 6 months in traditional methods to 1 day. The model interpretation module improves the IP solution adoption rate by 46.2% and user satisfaction by 31.4%. The professional knowledge integration mechanism enables the professional matching accuracy to reach 89.3%, ensuring the professional accuracy of the IP.

[0317] This invention extracts features from acquired multimodal behavioral data to obtain user feature vectors. A candidate pool containing various types of health knowledge video personality IPs is constructed based on textual, visual, behavioral, and professional qualification data from multiple IPs. Attribute feature vectors for all IPs in the candidate pool are calculated. The user feature vectors and the attribute feature vectors of all IPs in the candidate pool are input into a trained bimodal Transformer fusion model for prediction, yielding a fitting score for each feature vector. All fitting scores are sorted from largest to smallest, and the top few IP recommendation schemes are selected. These IP recommendation schemes are then used to fine-tune the attribute feature vectors of all IPs in the candidate pool, generating health knowledge video personality IP schemes. Compared to existing technologies, this invention constructs a candidate pool containing various types of health knowledge video personality IPs based on textual, visual, behavioral, and professional qualification data from multiple IPs to improve the accuracy of IP matching with user needs. By proposing a bimodal Transformer fusion model to capture the complex correlation between user feature vectors and attribute feature vectors, the effectiveness of health knowledge dissemination is enhanced.

[0318] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligently generating IP characters for health knowledge videos, characterized in that, include: Step 1: Obtain multimodal behavioral data of the target user; Step 2: Extract features from the multimodal behavior data to obtain user feature vectors; Step 3: Construct a candidate pool containing various health knowledge video IPs based on text data, visual data, behavioral data, and professional qualification data of multiple health knowledge video IPs, and calculate the attribute feature vector of all IPs in the candidate pool. Step 4: Input the user feature vector and the attribute feature vectors of all characters' IPs in the candidate pool into the trained bimodal Transformer fusion model for prediction, and obtain the fitting score of each feature vector. Step 5: Sort all the matching scores in descending order, select the top few IP recommendation schemes, and use all the IP recommendation schemes to fine-tune the attribute feature vectors of all the IPs in the candidate pool to generate health knowledge video IP schemes. The bimodal Transformer fusion model includes an intramodal encoder, a professional knowledge integration module, a cross-modal fusion module, a cross-modal attention calibration module, and an adaptation score prediction module; The intramodal encoder is used to perform modal self-attention modeling on the user feature vector and the attribute feature vector to obtain the user context vector and the IP context vector. The expertise integration module is used to fuse the user context vector and the IP context vector with health knowledge graph features respectively, to obtain user graph fusion features and IP graph fusion features, including: Construct a health knowledge graph containing entities and relationships in the health domain, and integrate the user feature vector and the attribute feature vector into the health knowledge graph; In the health knowledge graph, entity association queries are performed on the text modal features in the user feature vector and the professional qualification modal features in the attribute feature vector to obtain user health knowledge graph features and IP health knowledge graph features; The user health knowledge graph features are fused with the user context vector to obtain the user graph fusion features; The IP health knowledge graph features are fused with the IP context vector to obtain the IP graph fusion features; The cross-modal fusion module is used to perform cross-attention fusion of the user graph fusion features and the IP graph fusion features to obtain cross-modal fusion features; The cross-modal attention calibration module is used to introduce calibration loss to optimize the cross-modal fusion features, thereby obtaining optimized fusion features; The adaptation score prediction module is used to predict the adaptation score based on the optimized fusion features.

2. The method for intelligently generating health knowledge video character IPs according to claim 1, characterized in that, Before step 2, the following is also included: The multimodal behavior data is preprocessed to obtain preprocessed multimodal behavior data.

3. The method for intelligently generating health knowledge video character IPs according to claim 2, characterized in that, The preprocessed multimodal behavioral data includes: Preprocessed text data, preprocessed behavioral data, and preprocessed preference data.

4. The intelligent generation method for health knowledge video character IP according to claim 3, characterized in that, Step 2 includes: Semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modal features; The preprocessed behavioral data is subjected to behavioral sequence feature extraction to obtain the total behavioral modality features. Sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total features of the preference modality; The total features of the text modality, the total features of the behavioral modality, and the total features of the preference modality are fused to obtain the user feature vector.

5. The method for intelligently generating health knowledge video character IPs according to claim 4, characterized in that, Semantic vector extraction and topic distribution extraction are performed on the preprocessed text data to obtain the total text modality features, including: The pre-trained BERT model is used to encode the pre-processed text data to obtain semantic feature vectors; A three-layer Bayesian probability model is used to extract topic distribution from the preprocessed text data to obtain topic distribution probabilities. The semantic feature vector is concatenated with the topic distribution probability to obtain the total text modality feature.

6. The method for intelligently generating health knowledge video character IPs according to claim 4, characterized in that, Behavioral sequence features are extracted from the preprocessed behavioral data to obtain the total behavioral modality features, including: Gated recurrent neural networks are used to model behavioral sequences from preprocessed behavioral data to obtain behavioral sequence features. The preprocessed behavioral data is flattened according to channel priority to obtain a flattened vector. The flattened vector is concatenated with the behavioral sequence features to obtain the total behavioral modality features.

7. The method for intelligently generating health knowledge video character IPs according to claim 4, characterized in that, Sentiment features, style tags, and complexity preferences are extracted from the preprocessed preference data to obtain the total preference modality features, including: The VADER model was used to extract sentiment features from the preprocessed preference data to obtain the sentiment features of the comments. The preprocessed preference data is subjected to keyword matching and style recognition using a classification model to obtain style tags. The complexity preference is calculated based on the distribution of the duration of target users watching videos of different complexities. The comment sentiment features, style tags, and complexity preferences are concatenated to obtain the overall preference modality features.

8. The method for intelligently generating health knowledge video character IPs according to claim 1, characterized in that, The loss function of the bimodal Transformer fusion model consists of a main loss function, an auxiliary loss function, an attention regularization loss function, and a professionalism calibration loss function.

Images