Digital human live broadcast strategy generation method and device, computer equipment and storage medium

Abstract

The invention discloses a digital human live broadcast strategy generation method and device, computer equipment and a storage medium. According to the specific implementation scheme, the method comprises the following steps: calculating the association degree of an associated live video and a current live video in a preset time interval based on a multi-modal association degree calculation mode, and obtaining an association degree calculation result; determining a dynamic digital human model corresponding to the associated live video according to a predetermined basic digital human model and the association degree calculation result; and determining a fusion live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model. According to the method, the fusion live broadcast strategy is determined according to the digital human models corresponding to the multi-dimensional data, dynamic changes in a live broadcast scene can be well adapted, a comprehensive live broadcast decision is made in time, and differentiation of the digital human models is further considered through multi-modal correlation degree calculation, so that the live broadcast effect is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for generating a digital human live broadcast strategy. Background Art

[0002] Digital humans are character models that use computer vision or computer graphics to generate real-life or cartoon-like characters. Users can interact with digital humans through voice, text, and other forms. Digital humans use algorithms to drive facial expressions, mouth shapes, and body movements, and interact with users and respond to voice. Currently, digital humans have been widely used in government affairs, finance, scenic spots, e-commerce, and other scenarios, such as providing tour guide services in scenic spots and customer consulting services on e-commerce websites.

[0003] Digital human live broadcast strategy refers to a series of systematic plans and methods developed to achieve the best results and business goals of digital human live broadcast. In terms of content strategy, according to the interests, preferences and needs of the target audience, choose appropriate live broadcast content themes, such as game commentary, music performances, emotional consultation, product recommendations, etc., to attract and retain audiences; in terms of interactive strategy, support multiple interactive methods such as text, voice, and images, so that the audience can interact with digital humans in real time through bullet screens, voice commands, and click buttons, etc., to enhance the audience's sense of participation and immersion; in terms of operational strategy, through regular interactions, holding events, and conducting community operations, establish close ties with the audience, increase audience loyalty and participation, and cultivate a stable community and fan base.

[0004] In the related technologies, on the one hand, it is difficult for general digital human models to quickly and accurately understand and adapt to new live content in live broadcast application scenarios. For example, when the live broadcast content involves knowledge in highly professional or niche fields, the general model may not be able to accurately grasp the key information and details, resulting in inaccurate or in-depth explanations, and unable to meet the audience's needs for professional knowledge. Due to the lack of targeted training and optimization, it is difficult for general digital human models to achieve dynamic interaction in the true sense. On the other hand, proprietary digital human models are usually customized for specific live broadcast content or specific audience groups. When the live broadcast situation changes, such as changing the live broadcast theme, adjusting the target audience, etc., these models are often difficult to adapt quickly. For example, a digital human model customized for the live broadcast of beauty products, whose knowledge base and interactive methods are mainly centered on the field of beauty, may need to be retrained and adjusted a lot if it suddenly switches to live broadcasting electronic products. Therefore, the above-mentioned general digital human models and proprietary digital human models are obviously insufficient in flexibility and adaptability, and cannot make the best decision as quickly as real anchors. Summary of the invention

[0005] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0006] To this end, the first purpose of the present application is to propose a method for generating a digital human live broadcast strategy. The present application discloses a method, device, computer equipment and storage medium for generating a digital human live broadcast strategy. By acquiring the associated live broadcast video in real time and calculating the correlation between the associated live broadcast video and the current live broadcast video within a preset time interval, it is possible to continuously pay attention to the changes in the live broadcast content and other video content associated with it, so as to timely capture various dynamic changes in the live broadcast scene, provide real-time data support for generating accurate live broadcast strategies, and enable live broadcast to flexibly respond to various emergencies and content changes. Based on multi-dimensional data such as the current live broadcast content, the characteristics of the real-time live broadcast audience, and the associated live broadcast videos associated with the current live broadcast content, a fusion live broadcast strategy is determined, thereby avoiding the limitations of single-factor decision-making, making it possible to make more comprehensive and integrated live broadcast decisions, and improving the overall effect and quality of the live broadcast. In addition, through the calculation of multi-modal correlation, the live broadcast videos associated with the current live broadcast content are further considered, and the differentiation of the digital human model is used to improve the live broadcast effect, which enables the live broadcast content to better echo the associated videos and enrich the connotation and expression of the live broadcast. At the same time, the application of differentiated digital human models can bring more novel and diverse live broadcast experiences to the audience, thereby effectively enhancing the attractiveness and influence of live broadcasts and strengthening the audience's willingness and loyalty to watch.

[0007] The second purpose of this application is to propose a digital human live broadcast strategy generation device.

[0008] The third objective of the present application is to provide a computer device.

[0009] A fourth objective of the present application is to provide a computer-readable storage medium.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a method for generating a digital human live broadcast strategy, comprising: Determine the basic digital human model based on the current live broadcast content and historical live broadcast data; Acquire live audience groups in real time, extract and classify the live audience groups, and evaluate and screen based on the classification results, match each digital human model in the personalized digital human model library, and determine the corresponding personalized digital human model; Acquire the associated live video in real time, and calculate the correlation between the associated live video and the current live video within a preset time interval based on a multimodal correlation calculation method to obtain a correlation calculation result; Determining a dynamic digital human model corresponding to the associated live video according to the basic digital human model and the correlation calculation result; A fusion live broadcast strategy is determined according to the basic digital human model, the personalized digital human model and the dynamic digital human model.

[0011] In some embodiments of the present application, the method for generating a digital human live broadcast strategy, the method for calculating the correlation between the associated live broadcast video and the current live broadcast video within a preset time interval based on a multimodal correlation calculation method, includes: Acquire multimodal data in the associated live video and the current live video, the multimodal data including: scene recognition data, live metadata, multimodal feature data, and activity data; Extracting target features corresponding to the multimodal data and performing feature fusion to generate a multimodal feature vector corresponding to the associated live video and the current live video; According to the multimodal feature vector, the correlation between the associated live video and the current live video within a preset time interval is calculated.

[0012] In some embodiments of the present application, the method for generating a digital human live broadcast strategy, the step of extracting target features corresponding to the multimodal data, includes: Identifying a target scene in the scene recognition data, and extracting scene features in the target scene; According to the type of the target scene and the preset weight allocation rule, a corresponding target weight is allocated to the scene feature; According to the scene characteristics, respectively calculating the correlation coefficients between the associated live video and the current live video within a preset time interval; According to the target weights corresponding to the scene features, weighted summation is performed on the correlation coefficients to obtain a comprehensive correlation coefficient; According to the comprehensive correlation coefficient, a first correlation degree between the associated live video and the current live video within a preset time interval is calculated as the target feature.

[0013] In some embodiments of the present application, the extracting target features corresponding to the multimodal data further includes: According to the metadata features of the live broadcast metadata and user behavior habits, initial weights are assigned to the metadata features; Monitor user interaction data in real time, and adjust the weight of each metadata feature according to the user's interaction focus; For each of the metadata features, respectively calculating the similarity between the associated live video and the current live video within a preset time interval; According to the weight of each metadata feature, a weighted sum is performed on the similarity of each metadata feature to obtain a second correlation between the associated live video and the current live video within a preset time interval as the target feature.

[0014] In some embodiments of the present application, the extracting target features corresponding to the multimodal data further includes: Performing fusion processing on the multimodal feature data to form a unified feature representation; Performing time series analysis on the unified feature representation to identify feature pattern changes within the preset time interval; Performing association rule mining on the unified feature representation to obtain association patterns between multimodal features; According to the feature pattern change and the association pattern, a third degree of association between the associated live video and the current live video within a preset time interval is calculated as the target feature.

[0015] In some embodiments of the present application, the extracting target features corresponding to the multimodal data further includes: Identifying and obtaining an activity indicator in the activity data; According to the activity index, converting the activity data into an activity feature vector; Calculate the similarity between the associated live video and the current live video corresponding to the activity feature vector within a preset time interval; Adjusting the weight of the activity feature vector according to the similarity corresponding to the activity feature vector; According to the adjusted weight of the activity feature vector, the similarities corresponding to the activity feature vector are weighted summed to obtain the fourth correlation between the associated live video and the current live video within a preset time interval as the target feature.

[0016] In some embodiments of the present application, determining the integrated live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model includes: Combine the live broadcast script and the current live broadcast video to determine the current live broadcast node; Determine a fusion live broadcast strategy according to the current live broadcast node, wherein the fusion live broadcast strategy includes: a digital human model fusion sub-strategy and a digital human model switching sub-strategy; The digital human model fusion sub-strategy includes: fusing the basic digital human model, the personalized digital human model and the dynamic digital human model into a target digital human model; or, performing role task allocation and parameter configuration based on a multi-digital human model live broadcast of the basic digital human model, the personalized digital human model and the dynamic digital human model; The digital human model switching sub-strategy includes: judging the difference between the current digital human model and the digital human model to be switched in the live broadcast scene, determining whether transition animation processing is required based on the difference judgment result, if so, generating a transition animation, performing registration and stitching of the current digital human model, the transition animation, and the digital human model to be switched, and completing the switching, if not, performing feature alignment and real-time deformation based on the current digital human model and the digital human model to be switched.

[0017] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a digital human live broadcast strategy generation device, comprising: Basic model determination module: used to determine the basic digital human model based on the current live broadcast content and historical live broadcast data; Matching module: used to obtain live audience groups in real time, extract and classify the live audience groups, and evaluate and screen based on the classification results, match the digital human models in the personalized digital human model library, and determine the corresponding personalized digital human model; A correlation calculation module; used to obtain the associated live video in real time, and based on the multimodal correlation calculation method, calculate the correlation between the associated live video and the current live video within a preset time interval to obtain the correlation calculation result; A dynamic model determination module: used to determine the dynamic digital human model corresponding to the associated live video according to the basic digital human model and the correlation calculation result; Strategy generation module: used to determine the integrated live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model.

[0018] To achieve the above-mentioned purpose, the third aspect of the present application provides a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the digital human live broadcast strategy generation method described in the first aspect above.

[0019] In order to achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium, and the computer instructions are used to enable the computer to execute the digital human live broadcast strategy generation method described in the first aspect.

[0020] According to the technical solution of the present application, by obtaining the associated live video in real time and calculating its correlation with the current live video within a preset time interval, it is possible to continuously pay attention to the changes in the live content and other video content associated with it, so as to timely capture various dynamic changes in the live scene, provide real-time data support for the generation of accurate live strategies, and enable live broadcasts to flexibly respond to various emergencies and content changes. Based on multi-dimensional data such as the current live content, the characteristics of the real-time live audience, and the associated live videos associated with the current live content, a fusion live strategy is determined, thereby avoiding the limitations of single-factor decision-making, making it possible to make more comprehensive and integrated live decisions, and improving the overall effect and quality of the live broadcast. In addition, through the calculation of multimodal correlation, the live video associated with the current live content is further considered, and the differentiation of the digital human model is used to improve the live effect, which enables the live content to better echo the associated video and enrich the connotation and expression of the live broadcast. At the same time, the application of differentiated digital human models can bring a more novel and diverse live experience to the audience, thereby effectively improving the attractiveness and influence of the live broadcast, and enhancing the audience's willingness and loyalty to watch.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application. Figure 1 A flowchart of a method for generating a digital human live broadcast strategy provided in this application; Figure 2 This is a structural block diagram of a digital human live broadcast strategy generation device provided in this application. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0024] In the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The following describes the digital human live broadcast strategy generation method, device, computer equipment and storage medium of the embodiment of this application with reference to the accompanying drawings.

[0025] Figure 1 This is a flow chart of a method for generating a digital human live broadcast strategy provided in the first embodiment of the present application. It should be noted that the method for generating a digital human live broadcast strategy in the embodiment of the present application can be applied to the digital human live broadcast strategy generating device in the embodiment of the present application, and the digital human live broadcast strategy generating device can be configured on a computer device. Figure 1 As shown, the digital human live broadcast strategy generation method may include the following steps: Step 101, determining a basic digital human model according to current live broadcast content and historical live broadcast data.

[0026] It should be noted that the current live broadcast content is various content information in the current live broadcast process, such as audience interaction information (bullet screen, likes, comments, etc.), audio and video information in the live broadcast room, and scene change information in the live broadcast screen. For example, in an e-commerce live broadcast, the audience has many inquiries about a certain product, and these bullet screen information will be collected. Natural language processing (NLP) technology (Natural Language Processing (NLP) is a technology in the field of computer science and artificial intelligence that aims to enable computers to understand and process human natural language) can be used to analyze the conversation content, topic direction and emotional tendency in the current live broadcast in real time. Historical live broadcast data is data from previous live broadcasts, including audience viewing time, retention rate, purchase conversion rate, feedback on different live broadcast content, etc. This application does not make specific restrictions on this. Among them, the basic digital human model has a specific appearance, action and expression characteristics, sound characteristics, and a prototype of the corresponding basic functions of a conventional digital human.

[0027] Optionally, in an embodiment of the present application, a basic digital human model library may be pre-constructed, and the digital human models in the basic digital human model library have different appearances, voices, professional expression capabilities, and corresponding behavioral characteristics.

[0028] As an example, a brand is conducting an e-commerce live broadcast with the goal of promoting high-end skin care products. By analyzing the current live broadcast content (such as product introduction, user consultation, etc.), a professional and realistic digital human anchor is needed.

[0029] By analyzing historical live broadcast data (such as audience preferences for professional images, feedback on interactive forms, etc.), it is found that audiences are more receptive to digital human hosts with professional images and real-time interactions. Based on the above analysis, a high-fidelity, high-simulation 3D basic digital human model (a virtual character model created using computer graphics, artificial intelligence, and deep learning technology, whose goal is to highly simulate the appearance, behavior, and emotional expression of real humans) is selected from the basic digital human model library.

[0030] Optionally, in some embodiments of the present application, the appearance of the digital human model can be determined based on the style of the current live content and the preferences of the target audience, combined with the audience's acceptance of different images in historical data. For example, if the live content is fashion and beauty for young people, then a basic digital human model with trendy elements and delicate facial features is selected. This application does not make specific restrictions on this.

[0031] As an example, in an e-commerce live broadcast, a brand plans to launch a new beauty product and hopes to promote it through digital human live broadcast. First, collect information about the audience's focus on beauty products in the current live broadcast, such as product efficacy, usage methods, etc., and sort out historical data of previous beauty live broadcasts. It is found that the audience has better feedback on anchors with professional explanation capabilities and friendly images. Therefore, a basic digital human model with professional beauty knowledge explanation capabilities, a friendly and natural appearance, and a gentle and patient voice characteristics is determined from the basic digital human model library.

[0032] Step 102, obtaining live audience groups in real time, extracting and classifying the live audience groups, and evaluating and screening based on the classification results, matching the digital human models in the personalized digital human model library, and determining the corresponding personalized digital human model.

[0033] It should be noted that real-time acquisition of live audience groups includes real-time data collection and behavior monitoring of the current live audience groups. For example, the API (Application Programming Interface) of the live platform can be used to collect audience interaction data in real time, such as bullet screens, likes, comments, etc., and basic information of the audience, such as age, gender, region, etc. At the same time, the audience's stay time in the live broadcast room, viewing frequency, interaction frequency and other behavioral data are monitored to understand the audience's participation and interests.

[0034] As an example, audience data from different channels and platforms can be integrated to form a complete audience database. For example, audience data from mobile phones, computers, and different live broadcast rooms can be aggregated to ensure the comprehensiveness and accuracy of the data.

[0035] Optionally, in some embodiments of the present application, feature extraction classification specifically includes: numerical feature extraction, that is, standardizing and normalizing numerical data such as the number of interactions and residence time of the audience, so as to facilitate better understanding and learning of the model; category feature extraction, that is, independently encoding or label encoding category data such as the gender and age group of the audience, converting it into a numerical form text feature extraction that can be used by the model, and processing text data such as the audience's barrage and comments to extract keywords, emotional tendencies and other features; feature classification, that is, dividing the audience into different groups based on the above-mentioned extracted features. For example, the audience is divided into groups such as young men, young women, middle-aged men, and middle-aged women according to age and gender, and the audience is divided into groups such as high participation, medium participation, and low participation according to the frequency of interaction and residence time. This application does not make specific limitations on this.

[0036] As an example, valuable features are extracted from the integrated audience data. These features can be basic attributes of the audience, such as age, gender, region, etc.; they can also be behavioral features, such as viewing frequency, preferred content type, interactive activity, etc. For example, by analyzing the audience's likes and comments, their preferences for live content can be extracted. After extracting many features, you can use a feature selection algorithm such as SelectKBest (SelectKBest is a feature selection algorithm based on statistical methods, which is used to select the k features that have the greatest impact on the target variable from the feature set). For example, through the F-test classification method (F-test is a statistical method used to evaluate the strength of the linear relationship between a feature and a target variable. In SelectKBest, the F-test measures the correlation between a feature and a target variable by calculating the F-statistic. The higher the F-statistic, the stronger the relationship between the feature and the target variable), select features that are highly correlated with audience classification, such as age, region, and interactive activity.

[0037] Optionally, in some embodiments of the present application, a machine learning algorithm, such as a decision tree, a clustering algorithm, etc., is used to classify the audience according to the selected features. For example, the audience groups are specifically divided into different groups such as young active users, middle-aged stable users, and elderly occasional viewing users. The classified audience groups are evaluated and screened, and factors such as the proportion, stability, and value of the groups can be mainly considered. Among them, audience groups with a high proportion have a greater impact on the overall effect of the live broadcast; groups with high stability can pay attention to the live broadcast for a long time and have high loyalty; groups with high value are audiences with strong consumption power or high interactive enthusiasm. According to the above evaluation indicators, the audience groups that are most critical to the generation of live broadcast strategies are screened out.

[0038] As an example, the audience is divided into three levels: high, medium and low according to the viewing time of the current live audience group, and the audience corresponding to the above three levels is divided into first-tier cities and non-first-tier cities according to the region. For example, the audience is divided into the following two categories: high-time audience (watching time > 90 minutes): Beijing (120 minutes); low-time audience (watching time < 90 minutes): Shanghai (30 minutes), Guangzhou (90 minutes).

[0039] Optionally, in some embodiments of the present application, a library containing a variety of personalized digital human models can be pre-built. These models have their own characteristics in appearance, voice, action, expression, etc., which can meet the preferences of different audiences. For example, some digital human models have a young and fashionable image, suitable for attracting young audiences; some models have a gentle and friendly voice, suitable for elderly audiences. This application does not make specific restrictions on this.

[0040] Among them, the personalized digital human model library can use machine learning algorithms (such as Support Vector Machine, eXtreme Gradient Boosting) to extract and classify different specific features of the audience during the live broadcast, and use deep learning technology (such as Generative Adversarial Network, Neural Radiance Fields) to train and optimize the corresponding personalized digital human models for the features such as expressions and movements of digital humans. For example, collect a large amount of human video data containing different expressions and movements, use OpenFace (an open source face recognition and expression analysis toolkit, based on deep neural network technology, supports real-time face analysis, and is suitable for a variety of application scenarios) to extract facial feature points and expression labels, use NeRF (Neural Radiance Fields) to build a general 3D digital human model, and use MimicTalk (Mimicking a personalized and expressive 3D talking face, that is, personalized and expressive 3D talking face generation technology) to fine-tune the solution to conduct personalized training for specific roles. In addition, the appearance, clothing, voice, etc. of the digital human can be customized according to the platform style and the core needs of the live audience, and the corresponding intelligent interaction system can be designed to enable the digital human to make real-time and correct responses according to the different questions or feedback of the live audience. The trained digital human model can be stored in the model library, and the appropriate digital human can be dynamically matched according to the characteristics of the live audience (such as age, gender, and preferences).

[0041] As an example, for young active user groups with a high proportion and stability, digital human models with fashionable images and strong interactivity are prioritized to increase the attractiveness of live broadcasts and the audience's participation. For another example, based on the evaluation and screening results, for high-time viewers, more professional and interactive digital human models are selected; for low-time viewers, more entertaining and eye-catching digital human models are selected.

[0042] Step 103, obtaining the associated live video in real time, and calculating the correlation between the associated live video and the current live video within a preset time interval based on a multimodal correlation calculation method to obtain a correlation calculation result.

[0043] It should be noted that the associated live video is the same type of live content or the same type of live grouping / model as the current live content, or a related video recommended based on a model, etc. For example, the API interface of the live platform can be used to obtain the stream data of the current live video in real time (WebRTC, i.e., web real-time communication, etc. can be used to process the video stream in real time to ensure the real-time and smoothness of the data), and at the same time collect the stream data of one or more other live videos related to the current live video, which may include multi-modal information such as audio, video frames, text (such as bullet screen, comments), etc.

[0044] As an example, if the current live broadcast is about beauty tutorials, the associated live broadcast videos can be tutorials from other beauty bloggers, or the algorithm can be used to analyze the current live broadcast content and audience behavior to recommend related live broadcasts or videos. For example, an e-commerce platform is conducting a live broadcast of beauty products, and the anchor is introducing a lipstick. The platform analyzes the current live broadcast content in real time, identifies keywords such as "beauty" and "lipstick", and then retrieves lipstick reviews or usage tutorials from other beauty bloggers from the database.

[0045] Optionally, in some embodiments of the present application, the multimodal correlation degree calculation method calculates the correlation degree including: extracting the multimodal data of each live video, including relevant features such as audio features, video frame features, and text features. For example, an audio encoder is used to extract audio features, an image encoder is used to extract video frame features, and a text encoder is used to extract text features. Secondly, the features of different modes are aligned using methods such as optimal transmission (OT) and maximum mean difference (MMD) to ensure that the features between different modes are in the same feature space. Among them, optimal transport (OT) is a mathematical theory and calculation method used to describe the optimal transmission strategy between two probability distributions. The core idea is to convert one probability distribution into another probability distribution by minimizing the transmission cost. Specifically, OT describes how to transfer the quality in the source distribution to the target distribution by constructing a transfer matrix to minimize the total cost. Maximum mean discrepancy (MMD) is a non-parametric statistical method for measuring the difference between two probability distributions. It compares their mean differences by mapping the distributions into the reproducing kernel Hilbert space (RKHS).

[0046] For example, aligning audio and video modalities to the language modality as anchors generates aligned embedding sequences. Then, a multimodal fusion network (such as Mamba Backbone Multi-Modality Fusion Network, a multimodal feature extraction and fusion network based on the Mamba architecture, used to efficiently process and integrate data from different modalities (such as RGB images, depth images, audio, etc.)) is used to fuse the aligned features and generate a comprehensive feature identifier. The relevance is obtained by calculating the similarity (such as cosine similarity) between the comprehensive features of the current live video and the associated live video.

[0047] As an example, multimodal data is extracted from the associated live video and the current live video, including video content (images, actions), audio (voice, tone), text (barrage, comments), etc. Image recognition technology can be used to extract key frame images in the video, and speech recognition technology can be used to extract text content in the audio. Deep learning models such as Transformer (a deep learning architecture based on self-attention mechanism) or Bidirectional Encoder Representations from Transformers are used to combine image, audio and text features to calculate the similarity between the two. Further, by calculating the cosine similarity of image features, the Euclidean distance of audio features, and the semantic similarity of text content, a correlation score is obtained.

[0048] Among them, the preset time interval is a pre-set time interval for continuously calculating the correlation between the current live video and the related live videos to ensure the real-time and accuracy of the results, such as the past 10 minutes, 30 minutes, etc., and no specific restrictions are made here.

[0049] Step 104: Determine the dynamic digital human model corresponding to the associated live video according to the basic digital human model and the correlation calculation result.

[0050] In the embodiment of the present application, it can be determined whether the digital human model needs to be differentiated based on the above-mentioned basic digital human model and the result of the correlation calculation. If the correlation is high, it means that the associated live video is similar to the current live video in content and style, and then the relevant digital human model that matches the content and style will continue to be selected to maintain the coherence and consistency of the live broadcast. If the correlation is low, a differentiated digital human model is selected to highlight the uniqueness of the current live broadcast. Among them, the dynamic digital human model can be obtained through open source projects and pre-trained model libraries.

[0051] As an example, the current live broadcast is a press conference on the theme of "high-end technology products", and a new smart watch is about to be introduced. Through real-time analysis, the system obtains related live videos similar to the current live content. These videos can also be product press conferences or smart device reviews of other technology brands. If the related live video is also a press conference about smart watches, and the style is also professional and high-end, then the correlation is high. In this case, select the relevant digital human model that is consistent with the current live content and style. If the related live video is a "game live broadcast" with the theme of high-end technology, it is completely different from the style of the current high-end technology press conference and has a low correlation. In this case, select the corresponding differentiated digital human model to highlight the uniqueness of the current live broadcast. For example, choose a digital human model with a more technological and futuristic feel.

[0052] Optionally, in some embodiments of the present application, the corresponding dynamic digital human model can also be determined based on the live broadcast effect change curve of the associated live broadcast video. If the live broadcast effect (such as audience interaction rate, viewing time, etc.) of the associated live broadcast video changes significantly within a certain period of time, the reason for the change can be analyzed and the digital human model of the current live broadcast can be adjusted accordingly. For example, if the audience interaction rate of the associated live broadcast video is significantly improved after introducing a specific digital human model, then a similar model is also introduced in the current live broadcast.

[0053] Optionally, in some embodiments of the present application, the correlation calculation result and the live broadcast effect change curve can also be comprehensively considered to determine the dynamic digital human model that best matches the current live broadcast video. For example, the system monitors the live broadcast effect change curve of the associated live broadcast video in real time, including the changes in indicators such as the number of viewers, interaction frequency, number of likes, and purchase conversion rate. If the live broadcast effect of the associated live broadcast video shows an upward trend, it means that its live broadcast strategy and digital human model are relatively successful. At this time, the system will tend to determine a related digital human model similar to the associated live broadcast video, and learn from its successful experience, such as the image style, explanation method, and interactive topics of the digital human, to improve the effect of the current live broadcast. If the live broadcast effect of the associated live broadcast video shows a downward trend, it means that there may be some problems with its live broadcast strategy and digital human model, which cannot effectively attract viewers or promote sales. At this time, the system will determine a differentiated digital human model to avoid being affected by the adverse effects of the associated live broadcast video. For example, the digital human in the associated live broadcast video is too serious and the explanation is too professional, resulting in audience loss. The digital human model of the current live broadcast can use a more relaxed and humorous way to explain the product in simple and easy-to-understand language, increase the fun and entertainment, attract the attention of the audience, and enhance the attractiveness and effect of the live broadcast.

[0054] As an example, constructing a live broadcast effect change curve associated with a live broadcast video may include the following steps: 1. Determine key indicators: audience engagement indicators, such as the number of real-time online users, the number of people entering the live broadcast room, the number of people leaving the live broadcast room, etc. These indicators can intuitively reflect the attractiveness of the live broadcast and the retention of the audience; interactive data indicators, such as the number of likes, comments, barrages, shares, etc. Interactive data can reflect the audience's participation and interest in the live broadcast content; sales conversion indicators, including product sales, conversion rate (number of buyers / number of viewers), number of add-ons, etc.

[0055] 2. Data collection and recording: Most live broadcast platforms provide data screens or data analysis tools, such as Douyin's Huliang Qianchuan and Taobao's Business Advisor. Through these tools, you can obtain the data of the above key indicators in real time; or during the live broadcast, record the data of key indicators at regular intervals (such as every 5 minutes, every 10 minutes). For example, record the number of online users, number of likes, sales, etc. at each time point for subsequent analysis.

[0056] 3. Draw a curve chart: Use data visualization tools such as Excel (Microsoft Excel), Tableau (a data visualization and business intelligence software tool that converts complex data into intuitive visualization charts, dashboards and reports, supports the connection and integration of multiple data sources, and simplifies the data analysis process through a drag-and-drop interface), and Power BI (Power Business Intelligence, a business intelligence (BI) and data visualization tool developed by Microsoft, used to convert data from various data sources into interactive dashboards and analytical reports, supports multiple data sources from Excel spreadsheets to cloud-based and local hybrid data warehouses, and can achieve data acquisition, cleaning, modeling and visualization) to draw the collected data into a curve chart. The horizontal axis represents time, and the vertical axis represents the value of each key indicator. Specifically, the curve chart includes: Audience participation curve: draw a curve chart of the real-time online number, the number of people entering the live broadcast room, and the number of people leaving the live broadcast room. By observing these curves, you can understand the trend of audience traffic changes. For example, if the online number curve presents a "single peak" shape, it means that the live broadcast attracted a large number of viewers in a certain period of time, but then the audience gradually lost. Interaction data curve: Draw curves of the number of likes, comments, bullet comments, and shares. Through these curves, you can understand the peaks and troughs of audience interaction. For example, if the number of likes and comments suddenly rises during a product introduction session, it means that the content of this session attracted audience interaction. Comprehensive trend chart: Integrate the curves of multiple key indicators into a comprehensive trend chart to more comprehensively understand the changes in the live broadcast effect. For example, you can superimpose the curves of the number of online users, the number of likes, and sales to observe the correlation and change trends between them.

[0057] Step 105: Determine a fusion live broadcast strategy based on the basic digital human model, the personalized digital human model and the dynamic digital human model.

[0058] It should be noted that in the embodiments of the present application, the basic digital human model provides the core framework for the integrated live broadcast strategy, which ensures that the digital human in the live broadcast can operate normally in various situations and complete basic live broadcast tasks. The personalized digital human model makes the integrated live broadcast strategy more targeted, better establishes an emotional connection with the audience, and improves the audience's participation and interactivity. At the same time, the personalized digital human model can also optimize and adjust the basic digital human model according to the audience characteristics. The dynamic digital human model further adds flexibility and timeliness to the integrated live broadcast strategy, further considers differentiation, and enables the digital human to respond promptly to changes in external information associated with the current live broadcast content, bringing a more comprehensive and in-depth live broadcast experience to the audience.

[0059] Optionally, in some embodiments of the present application, first, the basic digital human model is used as a blueprint to ensure that the basic process and functions of the live broadcast can operate normally. Then, the image, language style, interaction method, etc. of the digital human are adjusted in a targeted manner according to the personalized digital human model, so that the live broadcast can better attract and meet the needs of specific audience groups. Finally, combined with the dynamic digital human model, according to the correlation between the associated live video and the current live video, the associated content and elements are dynamically introduced to enrich the connotation of the live broadcast and enhance the fun and value of the live broadcast. Therefore, through the organic combination of the above three models, a set of integrated live broadcast strategies that comprehensively consider the live broadcast content, audience characteristics and related information are finally determined, so that the digital human live broadcast can more accurately adapt to different live broadcast scenarios and audience needs, and achieve the best live broadcast effect.

[0060] According to the method for generating a digital human live broadcast strategy in the embodiment of the present application, by acquiring the associated live broadcast video in real time and calculating the correlation between the associated live broadcast video and the current live broadcast video within a preset time interval, it is possible to continuously pay attention to the changes in the live broadcast content and other video content associated with it, thereby timely capturing various dynamic changes in the live broadcast scene, providing real-time data support for generating accurate live broadcast strategies, and enabling live broadcasts to flexibly respond to various emergencies and content changes. At the same time, based on multi-dimensional data such as the current live broadcast content, the characteristics of the real-time live broadcast audience, and the associated live broadcast videos associated with the current live broadcast content, a fusion live broadcast strategy is determined, thereby avoiding the limitations of single-factor decision-making, making it possible to make more comprehensive and integrated live broadcast decisions, and improving the overall effect and quality of the live broadcast. In addition, through the calculation of multi-modal correlation, the live broadcast videos associated with the current live broadcast content are further considered, and the differentiation of the digital human model is used to improve the live broadcast effect, which enables the live broadcast content to better echo with the associated videos and enrich the connotation and expression of the live broadcast. Thereby, the flexibility and adaptability of the digital human in the live broadcast process are improved, so that the digital human can quickly make the best decision based on the dynamically changing live broadcast scenario.

[0061] Based on the above implementation, in order to better adapt to the dynamic changes in the live broadcast scene and make comprehensive live broadcast decisions, it is necessary to accurately evaluate the correlation between the associated live video and the current live video. Traditional correlation calculation methods often only consider data of a single modality, such as text or images, which limits the ability to fully understand the live content. Therefore, the present application provides a correlation calculation method that comprehensively considers multimodal data to more comprehensively evaluate the similarity and relevance of live content. Specifically, in the embodiment of the present application, based on the multimodal correlation calculation method, the correlation between the associated live video and the current live video within a preset time interval is calculated, which may include the following steps: Step 201, obtain multimodal data in the associated live video and the current live video, the multimodal data including: scene recognition data, live metadata, multimodal feature data, and activity data.

[0062] It should be noted that scene recognition data is data obtained by identifying scenes in live broadcasts through image recognition technology, such as product display and chat interaction in live broadcasts; live broadcast metadata includes basic information such as the title, tags, and duration of the live broadcast; multimodal feature data includes visual features such as images, acoustic features of audio, and semantic features of text; activity data includes interactive data such as the number of likes, comments, and bullet comments from viewers, etc. This application does not make specific restrictions here.

[0063] Step 202: extract target features corresponding to the multimodal data and perform feature fusion to generate a multimodal feature vector corresponding to the associated live video and the current live video.

[0064] Optionally, in some embodiments of the present application, specifically in the feature extraction process, a preset multi-layer perceptual neural network can be used to extract features from multimodal data to obtain target features of each modality. For example, a convolutional neural network is used to extract image features, and a recurrent neural network is used to extract text features. In the feature fusion process, the extracted multimodal features can be fused to generate a comprehensive multimodal feature vector, wherein the fusion method includes splicing, weighted summation, or an adapted attention mechanism.

[0065] As an example, in feature representation: Text features, use a pre-trained language model (such as BERT, Bidirectional Encoder Representations from Transformers, a pre-trained language model that can learn rich language representations through bidirectional context modeling and Transformer architecture) to encode the live text (title, description, text content in the live broadcast, etc.) to obtain a text feature vector T with a dimension of dT. Audio features, extract features such as the Mel frequency cepstral coefficients of the audio to form an audio feature vector A with a dimension of dA. Visual features, extract features from live video frames through a convolutional neural network to generate a visual feature vector V with a dimension of dv. Scene features, independently encode scene types (chatting, selling goods, teaching, performances, etc.) to obtain a scene feature vector S with a dimension of the number of scene types ds. Activity features, represented by the activity value A, can be expanded to a vector O=[A] with a dimension of 1. During the feature fusion process: the above feature vectors are concatenated in sequence to form a fused feature vector F: F=[T;A;V;S;O]. The dimension of the fused feature vector F is d=dr+dA+dv+ds+1.

[0066] Step 203, calculating the correlation between the associated live video and the current live video within a preset time interval according to the multimodal feature vector.

[0067] It should be noted that, in the embodiments of the present application, the degree of association can be calculated using methods such as cosine similarity and Euclidean distance.

[0068] As an example, the correlation calculation may include: assuming that the fusion feature vector of the current live video is F1, and the fusion feature vector of the associated live video is F2, and calculating the correlation R using a method based on a combination of cosine similarity and Mahalanobis distance.

[0069] Specifically, the cosine similarity C is expressed as follows: Mahalanobis distance: First, calculate the covariance matrix ∑ of the two feature vectors. Assume that there are n samples similar to the current live video and the associated live video (which can be selected from historical live data). The fusion feature vectors of these samples form the matrix X = [F1, F2, ..., Fn]. The calculation of the covariance matrix ∑ is as follows: in is the mean vector of all sample fusion feature vectors.

[0070] The Mahalanobis distance M is expressed as follows: Comprehensive correlation: Introduce weight λ to balance the influence of cosine similarity and Mahalanobis distance, and get the correlation: , where λ∈[0,1], can be tuned through experiments or machine learning methods (such as cross-validation) to determine the most appropriate value to accurately reflect the degree of correlation between live videos.

[0071] According to the method for generating a digital human live broadcast strategy in the embodiment of the present application, it is possible to achieve a comprehensive and accurate calculation of the correlation between the associated live broadcast video and the current live broadcast video, which not only improves the accuracy of feature extraction and fusion, but also enhances the adaptability and generalization ability of the model through a dynamic weight adjustment mechanism. Ultimately, it is possible to more reliably evaluate the correlation between two live broadcast videos, providing more powerful support for the generation of digital human live broadcast strategies.

[0072] Furthermore, in the above process of extracting target features corresponding to multimodal data, the present application specifically includes: Step 301 : identifying a target scene in scene recognition data, and extracting scene features in the target scene.

[0073] It should be noted that the target scene is the actual application scene of live video, such as chatting, selling goods, teaching, performance and other scenes. Scene features refer to the key information or attributes in a specific target scene, which can reflect the core content, behavior pattern or user interaction method of the target scene. For example, the scene features in the selling goods scene include product visual features, interactive text features, order quantity features, etc.

[0074] Step 302: assign corresponding target weights to the scene features according to the type of the target scene and a preset weight assignment rule.

[0075] It should be noted that in the process of calculating the similarity between the associated live video and the current live video, the similarity calculation accuracy corresponding to different scene features in the same target scene is different. Therefore, in order to improve the accuracy of the similarity calculation between the associated live video and the current live video, the preset weight allocation rule can allocate different weights to the scene features in the target scene according to the type of the target scene, so as to reflect the importance level of different scene features in calculating the similarity between the associated live video and the current live video.

[0076] As an example, in a product promotion scenario, according to the preset weight assignment rules, the importance level of product visual features is determined to be higher than that of interactive text features, and therefore a higher weight is assigned to the product visual feature data.

[0077] Step 303, according to the scene characteristics, respectively calculate the correlation coefficients between the associated live video and the current live video within a preset time interval.

[0078] It should be noted that the correlation coefficient refers to the similarity or correlation between the corresponding feature data of the associated live video and the current live video based on the same specific target scene. The method for calculating the correlation coefficient can adopt the grey correlation analysis method.

[0079] As an example, the calculation of the correlation coefficient specifically includes: Data preprocessing: De-dimensionalize scene features, for example, by initializing the data to normalize it to between 0 and 1.

[0080] Calculate the correlation coefficient: Use the formula in the grey correlation analysis method to calculate the correlation coefficient. The formula is as follows: , Among them, ξ i,0 : association coefficient, indicating the degree of association between subsequence i and the parent sequence; Δmin: the minimum absolute value of the difference between the two sequences (subsequence and parent sequence); Δmax: the maximum absolute value of the difference between the values ​​of two sequences (subsequence and parent sequence); Δ t(i,0) : The absolute value of the difference between the observation values ​​of the subsequence and the parent sequence at the same observation time (same row); ρ: resolution coefficient, used to adjust the significance of differences between correlation coefficients, usually with a value of 0.5.

[0081] Step 304 , performing weighted summation on the correlation coefficients according to the target weights corresponding to the scene features to obtain a comprehensive correlation coefficient.

[0082] As an example, the specific steps for calculating the comprehensive correlation coefficient are as follows: multiply the correlation coefficient of each scene feature by its corresponding weight, and then sum them up to obtain the comprehensive correlation coefficient. The formula is as follows: Where n is the number of scene features, correlation coefficient i is the correlation coefficient of the i-th scene feature, and weight i is the weight of the i-th scene feature.

[0083] Step 305, based on the comprehensive correlation coefficient, calculate the first correlation degree between the associated live video and the current live video within a preset time interval as the target feature.

[0084] It should be noted that the first correlation refers to the average value of the comprehensive correlation coefficient within a preset time interval, which is used to represent the overall correlation between the two live videos within the time interval.

[0085] The specific steps are as follows: Calculate the first correlation: average the comprehensive correlation coefficient within a preset time interval to obtain the first correlation. The formula is as follows: Wherein, T is the length of the preset time interval, and the comprehensive correlation coefficient t is the comprehensive correlation coefficient at the tth time point.

[0086] According to the method for generating a digital human live broadcast strategy in the embodiment of the present application, it is possible to perform a comprehensive and accurate correlation calculation on the scene features corresponding to the associated live broadcast video and the current live broadcast video, which not only improves the accuracy of scene feature extraction and fusion, but also enhances the adaptability and generalization ability of the model through a dynamic weight adjustment mechanism, thereby more reliably evaluating the correlation between the two live broadcast videos and providing more powerful support for the generation of digital human live broadcast strategies.

[0087] Furthermore, in the above process of extracting target features corresponding to multimodal data, the present application specifically also includes: Step 401, assigning initial weights to metadata features according to metadata features of live broadcast metadata and user behavior habits.

[0088] It should be noted that in the embodiment of the present application, the live broadcast metadata includes basic data such as the title, label, duration, and release time of the live broadcast. The metadata feature is the specific characteristic content of the live broadcast metadata in the live broadcast scenario. For example, the title is an introduction to beauty products, and the metadata feature of the title is an introduction to women's beauty products. For another example, the label is "beauty and skin care", and the metadata feature of the label is the product details, price, and discount information of "beauty and skin care". Among them, user behavior habits can be obtained through log file analysis, such as page browsing, clicking, and dwell time in the live broadcast room. By analyzing these log files, users' behavior patterns and habits can be extracted.

[0089] Optionally, in an embodiment of the present application, initial weights are assigned to metadata features in order to highlight important features in the live broadcast metadata while taking into account user preferences to optimize the processing effect of multimodal data.

[0090] As an example, in the e-commerce live broadcast scenario, the live broadcast metadata includes the following: Live broadcast title: Introduction to beauty products; Live broadcast duration: 60 minutes; Category tags: beauty, skin care. At the same time, log file analysis shows that viewers have a high frequency of interaction with the discount information of beauty product A in the introduction of women's beauty products, and prefer to watch live broadcasts with a shorter viewing time (30-45 minutes). Based on the above content, the initial weights are assigned to the live broadcast metadata: women's beauty products are highly matched with user preferences, so a higher weight (such as 0.4) is assigned; the live broadcast duration is 60 minutes, which is slightly longer than the user preference (30-45 minutes), so a medium weight (such as 0.2) is assigned; the discount information of beauty product A is consistent with the user preference, so a higher weight (such as 0.4) is assigned.

[0091] Step 402 , monitor the user's interactive data in real time, and adjust the weight of each metadata feature according to the user's interactive focus.

[0092] It should be noted that the user's interactive data includes likes, comments, shares, bullet comments, etc. By monitoring these interactive data in real time, we can understand the user's focus and interests. For example, if a user frequently likes and comments on the content of a specific tag in a live broadcast, it means that the tag is highly attractive to the user, and the specific tag is the user's interactive focus. Therefore, the weight of the specific tag is further adjusted to make it occupy a more important position in the correlation calculation. Among them, the user's interactive behavior pattern can be analyzed through a machine learning algorithm to identify the user's current focus content.

[0093] As an example, in an e-commerce live broadcast, three beauty products (A, B, C) are being introduced, and an interactive session is set up. It is monitored that the number of user comments on the product details of product A is significantly higher than that of other products, and users frequently ask questions about the details of product A during the interactive session. Due to the high interactive data, the weight of the tag metadata feature of product A's product details is increased.

[0094] Step 403: For each metadata feature, the similarity between the associated live video and the current live video within a preset time interval is calculated.

[0095] It should be noted that similarity calculation is to evaluate the similarity between the associated live video and the current live video in a certain metadata feature. Similarity can be calculated using methods such as cosine similarity and Jaccard (Jaccard similarity coefficient, a statistical method for measuring the similarity between two sets, defined as the ratio of the intersection size to the union size of the two sets) similarity. For example, for the metadata feature of the live title, the text similarity calculation method can be used; for the metadata feature of the tag, the Jaccard similarity calculation method can be used.

[0096] Step 404: perform weighted summation on the similarity of each metadata feature according to the weight of each metadata feature, and obtain the second correlation between the associated live video and the current live video within a preset time interval as the target feature.

[0097] It should be noted that the second correlation is the result of comprehensively considering the similarity and weight of each metadata feature. Through weighted summation, a comprehensive correlation value of the live metadata feature can be obtained to evaluate the overall similarity of two live videos within a preset time interval. The comprehensive correlation value can be used as a target feature for subsequent feature fusion and correlation calculation in the live metadata feature.

[0098] According to the digital human live broadcast strategy generation method of the embodiment of the present application, by considering multiple aspects of the live broadcast metadata, it is possible to more comprehensively and accurately evaluate the similarity between the associated live broadcast video and the current live broadcast video. At the same time, the weights are dynamically adjusted according to the user interaction data, which can better adapt to the real-time interest changes of users, thereby improving the robustness and generalization ability of the model.

[0099] Furthermore, in the above process of extracting target features corresponding to multimodal data, the present application specifically also includes: Step 501, fusing the multimodal feature data to form a unified feature representation.

[0100] It should be noted that multimodal data fusion is to map different modal data (such as images, text, audio, etc.) in live video into a shared low-dimensional space for unified feature representation. Fusion methods can include embedded fusion, serial fusion, and target-level fusion.

[0101] As an example, data of different modalities are mapped into the same low-dimensional space through an autoencoder or a generative adversarial network, or data of different modalities are sequentially input into different sub-networks for feature extraction and then fused in subsequent network layers.

[0102] Step 502 , performing time series analysis on the unified feature representation to identify feature pattern changes within a preset time interval.

[0103] It should be noted that in the embodiments of the present application, time series analysis is used to identify the change pattern of multimodal features in the time series. Long short-term memory networks or Transformer models (neural network architecture based on self-attention mechanism) can be used to analyze the dynamic changes of features. Feature pattern changes specifically refer to the dynamic change rules or trends shown by multimodal feature data within a preset time interval. These changes reflect the interaction and internal connection between different modal data. For example, in the current live video, the expression of the digital human anchor changes from smiling to serious, and the audience's barrage changes from positive comments to inquiries about the product effect. In the associated live video, the change pattern of the digital human anchor's expression and voice intonation is similar to the current live video.

[0104] As an example, for image features (expressions and movements of digital people during live broadcasts), CNN (Convolutional Neural Network, a deep learning architecture widely used in computer vision, image processing, natural language processing and other fields, whose core idea is to extract local features of input data through convolutional layer, and perform feature dimension reduction and invariance processing through pooling layer) can capture the changes in the host's expression at different time points; for text features (the frequency and content of audience comments and comments), RNN (Recurrent Neural Network, a neural network architecture for processing sequence data, suitable for processing data with time dependency or sequential relationship, such as time series, natural language and voice signals, whose core feature is the presence of feedback connections in the network, which enables the network to remember previous information and use it for current calculations) can be used to extract text features, which can capture changes in audience interest; audio features (changes in the voice intonation of the digital human host) can be extracted through LSTM (Long Short-Term Long Short-Term Memory (LSTM) is a special recurrent neural network architecture designed to solve the gradient vanishing and gradient exploding problems that are prone to occur when standard RNN processes long sequence data. LSTM can effectively capture long-term dependencies in the sequence by introducing a "gating mechanism". The extracted audio features can capture the emotional changes in the voice.

[0105] Step 503: perform association rule mining on the unified feature representation to obtain association patterns between multimodal features.

[0106] It should be noted that association rule mining is a mining method based on frequent item sets, which aims to discover association relationships in a data set. In the present application, the data set includes a unified feature representation set corresponding to the associated live video and the current live video within a preset time interval. Among them, Apriori (Apriori Algorithm, Apriori algorithm is a classic algorithm for mining frequent item sets, widely used in association rule mining, it is based on the a priori property that "all non-empty subsets of frequent item sets must also be frequent", by generating candidate item sets layer by layer and scanning the database to determine the frequent item sets) or FP-Growth (Frequent Pattern Growth Algorithm, an improved frequent item set mining algorithm, by constructing a compact data structure-FP tree (Frequent Pattern Tree), avoiding the overhead of multiple database scans and generation of a large number of candidate item sets in the traditional Apriori algorithm, FP-Growth algorithm only needs to scan the database twice to efficiently mine frequent item sets) algorithm to mine association rules between multimodal features. For example, in both the associated live video and the current live video, when the digital human host smiled, the audience's positive evaluation increased. Based on this association pattern, it shows that smiling expressions have a positive impact on the audience's positive feedback.

[0107] Optionally, in some embodiments of the present application, the association patterns between multimodal features include: the association pattern between images and texts, that is, the association between the expression of the digital human host and the content of the audience's barrage, for example, when the host smiles, the audience's positive comments gradually increase; the association pattern between audio and images, that is, the synchronous changes in the voice tone and facial expressions of the digital human host, for example, when the host's tone increases, the expression will be more excited; the association pattern between interactive data and live broadcast content, that is, the association between the audience's likes, comments and purchasing behaviors and the live broadcast content, for example, when the live broadcast content involves a detailed introduction to a certain type of product, the audience's interaction frequency gradually increases.

[0108] Step 504: Calculate, based on the feature pattern change and the association pattern, a third degree of association between the associated live video and the current live video within a preset time interval as the target feature.

[0109] It should be noted that the third correlation degree calculated by comprehensively analyzing the above-mentioned feature pattern changes and association patterns can more comprehensively and accurately reflect the corresponding multimodal feature similarity or correlation between the associated live video and the current live video within a preset time interval.

[0110] As an example, the current live video is a live broadcast of a digital human host introducing a beauty product, and the associated live video is a previous live broadcast of a digital human host introducing a different beauty product of the same brand. The specific steps for calculating the third correlation between the above two live videos within a preset time interval (for example, 10 minutes) are as follows: 1. Data preparation: Multimodal feature data includes: image features (expressions and movements of the digital human anchor), text features (live scripts, audience comments and comments), audio features (voice and intonation of the digital human), and interactive features (audience likes, comments, and purchase behaviors). Use deep learning models (such as CNN to extract image features, RNN to extract text features, and LSTM to extract audio features) to map the above multimodal data into a unified low-dimensional feature space.

[0111] 2. Analysis of characteristic pattern changes: Use LSTM or Transformer models to perform time series analysis on unified feature representations and identify feature pattern changes within a preset time interval. For example, image feature pattern changes: the digital human anchor’s expression changes from smiling to serious; text feature pattern changes: audience comments change from “I like this product very much” to “How is the effect of this product”; Audio feature pattern changes: The tone of the digital human's voice changes from calm to excited.

[0112] 3. Association rule mining: Use Apriori or FP-Growth algorithms to mine association patterns between multimodal features. For example, the association pattern between images and text: when the digital human host smiles, the audience's positive comments increase; the association pattern between audio and interaction: when the digital human's voice tone increases, the audience's interaction frequency increases.

[0113] 4. Calculate the third degree of association: According to the above-mentioned feature pattern changes and association patterns, the third correlation between the associated live video and the current live video within a preset time interval is calculated. Among them, the similarity calculation can use cosine similarity or Euclidean distance to calculate the similarity between unified feature representations, such as calculating the similarity of image features, text features and audio features of the current live video and the associated live video. Then, the importance level is calculated based on the similarity of the feature pattern changes and the association pattern, and the similarity is weighted and summed. For example, if the change of image features has a greater impact on the live broadcast effect, a higher weight ratio is assigned.

[0114] According to the method for generating a digital human live broadcast strategy in the embodiment of the present application, the live broadcast strategy can be dynamically adjusted to adapt to changes in audience interests through time series analysis and association rule mining. Personalized live broadcast content can be provided to different audiences based on changes in feature patterns and association patterns. By identifying the association pattern between audience interaction data and live broadcast content, the interaction strategy can be adjusted in real time to improve audience participation.

[0115] Furthermore, in the above process of extracting target features corresponding to multimodal data, the present application specifically also includes: Step 601, identifying and acquiring activity indicators in activity data.

[0116] It should be noted that activity data includes user interactive behaviors, such as likes, comments, shares, bullet comments, etc. These indicators can reflect the user's participation and interest in the live content. For example, the number of likes and comments can be used as activity indicators.

[0117] Step 602: convert the activity data into an activity feature vector according to the activity index.

[0118] Optionally, as an example, the activity index is converted into a feature vector for subsequent similarity calculation. For example, the number of likes, the number of comments, the number of shares, and other indicators can be normalized to form a feature vector.

[0119] Step 603, calculating the similarity between the activity feature vectors of the associated live video and the current live video within a preset time interval.

[0120] In this embodiment, the cosine similarity method is used to calculate the similarity between the activity feature vectors of the associated live video and the current live video within a preset time interval. The cosine similarity formula is: Among them, in PyTorch, the F.cosine_similarity function can be used to calculate cosine similarity, where A and B are two vectors corresponding to the activity features of the above-mentioned associated live video and the current live video respectively, A•B represents the dot product (inner product) of vectors A and B, and ‖A‖ and ‖B‖ represent the modulus (length) of vectors A and B respectively.

[0121] Step 604: adjust the weight of the activity feature vector according to the similarity corresponding to the activity feature vector.

[0122] As an example, the weight of the activity feature vector can be dynamically adjusted according to the similarity. If the similarity of a feature vector is high, it means that the feature is more important in the correlation calculation, and its weight can be increased.

[0123] Step 605: perform weighted summation on the similarities corresponding to the activity feature vectors according to the weights of the adjusted activity feature vectors, and obtain a fourth correlation between the associated live video and the current live video within a preset time interval as a target feature.

[0124] It should be noted that the adjusted weight is multiplied by the similarity, and then weighted sum is performed to obtain the corresponding fourth correlation as the target feature, which can be used for the subsequent correlation calculation of the correlation between the live video and the current live video in terms of activity feature fusion.

[0125] In this embodiment, the fourth correlation degree between the current live video and the associated live video within a preset time interval is calculated through the following steps.

[0126] Example data: Current live video activity data: Likes: 1000; Comments: 500; Shares: 300. Related live video activity data: Likes: 800; Comments: 400; Shares: 200.

[0127] Convert to a feature vector: After normalizing the activity data, a feature vector is formed: current live video feature vector: 0.5, 0.25, 0.15; associated live video feature vector: 0.4, 0.2, 0.1.

[0128] Calculate similarity: The similarity between two feature vectors is calculated using cosine similarity.

[0129] Adjust weights: Adjust the weights based on the similarity: if the similarity is high, increase the weight; if the similarity is low, decrease the weight.

[0130] Weighted sum: The adjusted weight is multiplied by the similarity, and then a weighted sum is performed to obtain a fourth correlation degree.

[0131] According to the digital human live broadcast strategy generation method of the embodiment of the present application, by converting the activity data into a feature vector, similarity calculation can be performed more effectively, and the activity similarity of two live videos can be accurately evaluated using methods such as cosine similarity. At the same time, by dynamically adjusting the weight according to the similarity, the importance of different features can be more accurately reflected, and a comprehensive correlation value can be obtained through weighted summation, which is used to evaluate the overall similarity of the associated live videos and the current live video.

[0132] Furthermore, in the process of determining the integrated live broadcast strategy based on the basic digital human model, the personalized digital human model and the dynamic digital human model, the present application specifically includes: Step 701, determine the current live broadcast node by combining the live broadcast script and the current live broadcast video.

[0133] It should be noted that during the live broadcast process, the current live broadcast node can be determined by combining the live broadcast script and the content of the current live broadcast video. The live broadcast script contains various links and time nodes of the live broadcast, such as product introduction, interactive links, promotional activities, etc. By analyzing the content of the current live broadcast video in real time, it is possible to determine which link the current live broadcast is in, thereby providing a basis for subsequent strategy adjustments.

[0134] Step 702: Determine a fusion live broadcast strategy according to the current live broadcast node. The fusion live broadcast strategy includes: a digital human model fusion sub-strategy and a digital human model switching sub-strategy.

[0135] Optionally, in the embodiment of the present application, the digital human model fusion sub-strategy includes the following two methods: by fusing multiple digital human models, a comprehensive digital human model is generated to meet the needs of the current live broadcast node. For example, using multimodal feature fusion technology, the features of different models are weighted and summed to generate a target digital human model. And in the live broadcast of multiple digital human models, roles and tasks are assigned to different digital human models according to the needs of the current live broadcast node, and parameter configuration is performed. For example, one digital human model can be responsible for product introduction, and another digital human model can be responsible for the interactive link.

[0136] As an example, in the "on-site trial and interaction" stage, the system needs to integrate the basic digital human model and the personalized digital human model to achieve the following functions: the basic digital human model is used to provide general interactive capabilities, such as answering common questions from the audience and guiding the audience to participate in the interaction. The personalized digital human model is used to show the brand's unique style and language style to enhance brand recognition. Based on the above model, the corresponding integration method includes: integrating the general interactive capabilities of the basic digital human model with the brand style of the personalized digital human model. For example, the basic digital human model can provide a standard interactive template, while the personalized digital human model can adjust the language style to match the brand tone. At the same time, the adaptive weight mechanism is used to dynamically adjust the weights of the two models according to the complexity of the current interaction. For example, when the audience's questions are more complex, the weight of the basic digital human model is increased; when it is necessary to show the brand characteristics, the weight of the personalized digital human model is increased.

[0137] Secondly, the digital human model switching sub-strategy specifically includes the following steps: By comparing the features of the current digital human model and the digital human model to be switched, the difference between them is determined. If the difference is large, a transition animation is performed. If transition animation processing is required, the corresponding transition animation is generated, and the current digital human model, transition animation, and digital human model to be switched are aligned and stitched to complete the switch. Among them, the transition animation can smoothly connect the two digital human models to avoid the abruptness during the switching process. If the difference is small, feature alignment and real-time deformation are performed based on the current digital human model and the digital human model to be switched. Among them, feature alignment ensures that the two models are aligned in the feature space, and real-time deformation dynamically adjusts the features of the digital human model according to the needs of the current live broadcast node.

[0138] As an example, at a certain moment, the system needs to switch to a dynamic digital human model to meet the needs of real-time interaction. For example, when a viewer asks a sudden question in the barrage that is not related to the current product, the system needs to switch to a dynamic digital human model to generate an answer in real time. In the "Fan Q&A Session" (15:00-20:00), the system can completely switch to a dynamic digital human model to achieve a more natural interaction. Specific switching logic: The system monitors the barrage and audience interaction in real time, and triggers the switching mechanism when content that requires a real-time response is detected. During the switching process, the system will transition smoothly to ensure that the audience will not notice obvious signs of switching. For example, a dynamic digital human model can generate an answer immediately after receiving a question, and switch back to a basic or personalized digital human model after the answer is completed.

[0139] According to the digital human live broadcast strategy generation method of the embodiment of the present application, a comprehensive digital human model is generated through the digital human model fusion sub-strategy, thereby improving the diversity and adaptability of the live broadcast. And through the digital human model switching sub-strategy, smooth switching between digital human models is ensured, thereby improving the audience's viewing experience. That is, by fusing and switching multiple digital human models, dynamic adjustment and optimization of the live broadcast content is achieved, and the interactivity, personalization and naturalness of the live broadcast are enhanced. At the same time, through intelligent transition animation processing and real-time deformation technology, the smooth transition and continuity of the live broadcast picture are ensured, thereby significantly improving the overall quality of the live broadcast and the audience experience. As a result, the digital human live broadcast strategy can be managed and adjusted more effectively, and the interactivity of the live broadcast and the audience's participation can be improved.

[0140] A structural block diagram of a digital human live broadcast strategy generation device provided in an embodiment of the present application. Figure 2 As shown, the digital human live broadcast strategy generation device provided in the embodiment of the present application may include: a basic model determination module 1, a matching module 2, a correlation calculation module 3, a dynamic model determination module 4 and a strategy generation module 5.

[0141] Among them, the basic model determination module 1 is used to determine the basic digital human model according to the current live broadcast content and historical live broadcast data.

[0142] Matching module 2: used to obtain live audience groups in real time, extract and classify the live audience groups, and evaluate and screen based on the classification results, match the digital human models in the personalized digital human model library, and determine the corresponding personalized digital human model.

[0143] The correlation calculation module 3 is used to obtain the related live video in real time, and calculate the correlation between the related live video and the current live video within a preset time interval based on the multimodal correlation calculation method to obtain the correlation calculation result.

[0144] Dynamic model determination module 4: used to determine the dynamic digital human model corresponding to the associated live video based on the basic digital human model and the correlation calculation result.

[0145] Strategy generation module 5: used to determine the integrated live broadcast strategy based on the basic digital human model, the personalized digital human model and the dynamic digital human model.

[0146] According to the digital human live broadcast strategy generation device of the embodiment of the present application, by acquiring the associated live broadcast video in real time and calculating the correlation between the associated live broadcast video and the current live broadcast video within a preset time interval, it can continuously pay attention to the changes in the live broadcast content and other video content associated with it, so as to timely capture various dynamic changes in the live broadcast scene, provide real-time data support for the generation of accurate live broadcast strategies, and enable the live broadcast to flexibly respond to various emergencies and content changes. At the same time, based on multi-dimensional data such as the current live broadcast content, the characteristics of the real-time live broadcast audience, and the associated live broadcast videos associated with the current live broadcast content, a fusion live broadcast strategy is determined, thereby avoiding the limitations of single-factor decision-making, making it possible to make more comprehensive and integrated live broadcast decisions, and improving the overall effect and quality of the live broadcast. In addition, through the calculation of multi-modal correlation, the live broadcast videos associated with the current live broadcast content are further considered, and the differentiation of the digital human model is used to improve the live broadcast effect, which enables the live broadcast content to better echo with the associated videos and enrich the connotation and expression of the live broadcast. Thereby, the flexibility and adaptability of the digital human in the live broadcast process are improved, so that the digital human can quickly make the best decision based on the dynamically changing live broadcast scenario.

[0147] Based on the embodiments of the present application, the present application also provides a computer device, at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the digital human live broadcast strategy generation method of any of the aforementioned embodiments.

[0148] It should be noted that, when the method for generating a digital human live broadcast strategy proposed in this application is executed by a client, the computer device may be a device with interactive functions such as an intelligent robot or a mobile device; when the method for generating a digital human live broadcast strategy proposed in this application is executed by a client and a server, the computer device may include a server and a device with interactive functions (such as an intelligent robot). As an example, when the computer device includes a server and an intelligent robot, the server sends the generated synthesized audio and video data of the digital human to the intelligent robot so that the intelligent robot plays the synthesized audio and video data of the digital human on the human-computer interaction interface.

[0149] Based on the embodiments of the present application, the present application also provides a computer-readable storage medium, wherein computer instructions are used to enable a computer to execute a method for generating a digital human live broadcast strategy according to any of the aforementioned embodiments provided in the embodiments of the present application.

[0150] Computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0151] Among them, the computer device includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0152] A number of components in a computer device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0153] The computing unit may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing units include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit performs the various methods and processes described above, such as the digital human live broadcast strategy generation method. For example, in some embodiments, the digital human live broadcast strategy generation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the computing unit, one or more steps of the digital human live broadcast strategy generation method described above may be executed. Alternatively, in other embodiments, the computing unit may be configured to execute the digital human live broadcast strategy generation method in any other appropriate manner (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0156] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0158] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0159] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0160] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0161] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for generating a digital human live broadcast strategy, characterized in that: include: Determine the basic digital human model based on the current live broadcast content and historical live broadcast data; Acquire live audience groups in real time, extract and classify the live audience groups, and evaluate and screen based on the classification results, match each digital human model in the personalized digital human model library, and determine the corresponding personalized digital human model; Acquire the associated live video in real time, and calculate the correlation between the associated live video and the current live video within a preset time interval based on a multimodal correlation calculation method to obtain a correlation calculation result; Determining a dynamic digital human model corresponding to the associated live video according to the basic digital human model and the correlation calculation result; Determining a fusion live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model; The method of calculating the correlation between the associated live video and the current live video within a preset time interval based on the multimodal correlation calculation method includes: Acquire multimodal data in the associated live video and the current live video, the multimodal data including: scene recognition data, live metadata, multimodal feature data, and activity data; Extracting target features corresponding to the multimodal data and performing feature fusion to generate a multimodal feature vector corresponding to the associated live video and the current live video; According to the multimodal feature vector, the correlation between the associated live video and the current live video within a preset time interval is calculated.

2. The method for generating a digital human live broadcast strategy according to claim 1, characterized in that: The extracting target features corresponding to the multimodal data includes: Identifying a target scene in the scene recognition data, and extracting scene features in the target scene; According to the type of the target scene and the preset weight allocation rule, a corresponding target weight is allocated to the scene feature; According to the scene characteristics, respectively calculating the correlation coefficients between the associated live video and the current live video within a preset time interval; According to the target weights corresponding to the scene features, weighted summation is performed on the correlation coefficients to obtain a comprehensive correlation coefficient; According to the comprehensive correlation coefficient, a first correlation degree between the associated live video and the current live video within a preset time interval is calculated as the target feature.

3. The method for generating a digital human live broadcast strategy according to claim 1, characterized in that: The extracting target features corresponding to the multimodal data further includes: According to the metadata features of the live broadcast metadata and user behavior habits, initial weights are assigned to the metadata features; Monitor user interaction data in real time, and adjust the weight of each metadata feature according to the user's interaction focus; For each of the metadata features, respectively calculating the similarity between the associated live video and the current live video within a preset time interval; According to the weight of each metadata feature, a weighted sum is performed on the similarity of each metadata feature to obtain a second correlation between the associated live video and the current live video within a preset time interval as the target feature.

4. The method for generating a digital human live broadcast strategy according to claim 1, characterized in that: The extracting target features corresponding to the multimodal data further includes: Performing fusion processing on the multimodal feature data to form a unified feature representation; Performing time series analysis on the unified feature representation to identify feature pattern changes within the preset time interval; Performing association rule mining on the unified feature representation to obtain association patterns between multimodal features; According to the feature pattern change and the association pattern, a third degree of association between the associated live video and the current live video within a preset time interval is calculated as the target feature.

5. The method for generating a digital human live broadcast strategy according to claim 1, characterized in that: The extracting target features corresponding to the multimodal data further includes: Identifying and obtaining an activity indicator in the activity data; According to the activity index, converting the activity data into an activity feature vector; Calculate the similarity between the associated live video and the current live video corresponding to the activity feature vector within a preset time interval; Adjusting the weight of the activity feature vector according to the similarity corresponding to the activity feature vector; According to the adjusted weight of the activity feature vector, the similarities corresponding to the activity feature vector are weighted summed to obtain the fourth correlation between the associated live video and the current live video within a preset time interval as the target feature.

6. The method for generating a digital human live broadcast strategy according to claim 1, characterized in that: The step of determining the fusion live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model includes: Combine the live broadcast script and the current live broadcast video to determine the current live broadcast node; Determine a fusion live broadcast strategy according to the current live broadcast node, wherein the fusion live broadcast strategy includes: a digital human model fusion sub-strategy and a digital human model switching sub-strategy; The digital human model fusion sub-strategy includes: fusing the basic digital human model, the personalized digital human model and the dynamic digital human model into a target digital human model; or, performing role task allocation and parameter configuration based on a multi-digital human model live broadcast of the basic digital human model, the personalized digital human model and the dynamic digital human model; The digital human model switching sub-strategy includes: judging the difference between the current digital human model and the digital human model to be switched in the live broadcast scene, determining whether transition animation processing is required based on the difference judgment result, if so, generating a transition animation, performing registration and stitching of the current digital human model, the transition animation, and the digital human model to be switched, and completing the switching, if not, performing feature alignment and real-time deformation based on the current digital human model and the digital human model to be switched.

7. A digital human live broadcast strategy generation device, characterized in that: include: Basic model determination module: used to determine the basic digital human model based on the current live broadcast content and historical live broadcast data; Matching module: used to obtain live audience groups in real time, extract and classify the live audience groups, and evaluate and screen based on the classification results, match the digital human models in the personalized digital human model library, and determine the corresponding personalized digital human model; Relevance calculation module; Used to obtain the associated live video in real time, and based on the multimodal correlation calculation method, calculate the correlation between the associated live video and the current live video within a preset time interval to obtain the correlation calculation result; A dynamic model determination module: used to determine the dynamic digital human model corresponding to the associated live video according to the basic digital human model and the correlation calculation result; Strategy generation module: used to determine the fusion live broadcast strategy according to the basic digital human model, the personalized digital human model and the dynamic digital human model; The method of calculating the correlation between the associated live video and the current live video within a preset time interval based on the multimodal correlation calculation method includes: Acquire multimodal data in the associated live video and the current live video, the multimodal data including: scene recognition data, live metadata, multimodal feature data, and activity data; Extracting target features corresponding to the multimodal data and performing feature fusion to generate a multimodal feature vector corresponding to the associated live video and the current live video; According to the multimodal feature vector, the correlation between the associated live video and the current live video within a preset time interval is calculated.

8. A computer device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the digital human live broadcast strategy generation method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer instructions are used to enable the computer to execute the digital human live broadcast strategy generation method according to any one of claims 1 to 6.

Images