An automatic adaptation method of multi-national digital human image and multi-language lip-synch

Abstract

The present application relates to a kind of multi-national digital figure and the automatic adaptation method of multi-language oral broadcast, belong to the field of text-to-speech synthesis, more belong to the field of text-to-speech synthesis method.The method analyzes the multi-dimensional data of digital person corresponding to the target country or region of oral broadcast video delivery;Oral broadcast video script, cultural compliance rules, forbidden word database and platform play parameters matching associated product, play platform and oral broadcast language are obtained;Obtain the audience feedback information and oral broadcast video of the previous time segment The matching mark of lip subtitle;And using 3DGS model matching target country or region and matching current play platform, according to the above each content, automatically generate the oral broadcast video of next time segment.The present application realizes the simultaneous matching of model parameters and delivery platform and delivery country or region by the customized structure design of each place of 3DGS model, and continuously adjusts the generation strategy according to audience feedback and lip synchronization state, improves the adaptability and propagation effect of delivery.

Description

Technical Field

[0001] This invention belongs to the field of text-to-speech synthesis, and more specifically to the field of text-to-speech synthesis methods, particularly relating to an automatic adaptation method for multinational digital human figures and multilingual voiceovers. Background Technology

[0002] With the rapid development of cross-border e-commerce, social media marketing, and global content dissemination, digital avatars coupled with multilingual narrated videos have become an important vehicle for product promotion and brand communication. The same product or brand message typically needs to be targeted at multiple countries and regions, and audiences in different countries or regions have significant differences in facial aesthetic preferences, language habits, cultural taboos, and the technical specifications of the video playback platforms they use. Therefore, the need for localized and platform-specific adaptation of digital avatars and narrated video content is becoming increasingly prominent.

[0003] For example, Chinese invention patent publication CN113411655A discloses a method, apparatus, electronic device, and storage medium for generating spoken-video content. The method includes: displaying a video creation page, the video creation page including an editing operation area for a target template spoken-video, the editing operation area displaying at least one video element corresponding to the target template spoken-video; the at least one video element including a subtitle / dubbing element; updating the subtitle / dubbing element displayed in the editing operation area corresponding to the subtitle / dubbing element in response to a subtitle / dubbing editing instruction triggered based on the editing operation area corresponding to the subtitle / dubbing element; and generating a target spoken-video content based on the target template spoken-video and the updated subtitle / dubbing element in response to a video generation instruction. Using embodiments of this application can effectively reduce the production cost of spoken-video content and improve its production efficiency.

[0004] For example, Chinese invention patent publication CN119835491A discloses a method, apparatus, device, and storage medium for generating spoken video, relating to the field of computer technology. The method includes: receiving spoken audio samples and current text content; learning from the spoken audio samples using an initial character model to generate a target character model corresponding to the timbre of the spoken audio samples; and generating a spoken video based on the current text content using the target character model. Because this application generates a target character model corresponding to the timbre of the spoken audio samples by learning from the spoken audio samples using an initial character model, compared to existing technologies that can only generate spoken videos with default timbre based on text, this application can adjust the timbre of the initial character model using spoken audio samples, thereby meeting user needs and effectively improving the user experience.

[0005] Therefore, it is evident that existing methods for generating multinational digital human avatars and multilingual narrated videos have the following main shortcomings: First, the existing voice-over video generation mechanism cannot adaptively match the target country or region, nor can it adaptively match the current playback platform. This results in the voice-over video generation being incompatible with the platform resolution and the detail of digital human faces, and cannot dynamically meet the needs of the audience in the target country or region. Second, existing methods cannot target specific countries or regions, playback platforms, related products, and spoken language when filtering input data for generating spoken videos. They also cannot obtain multi-dimensional matching inputs such as digital human multi-dimensional data, industrialized, platformized, and localized spoken video scripts, cultural compliance rules, prohibited word databases, and platform playback parameters, resulting in the generated spoken video content being inconsistent with local cultural habits and platform rules. Third, existing methods cannot use audience feedback information and lip-syncing subtitle matching identifiers from previous time segments as feedback input content. They lack the ability to select the type of input data for generating spoken videos based on feedback data such as play count, completion rate, number of likes, and lip-syncing subtitle mismatch detection. This makes it impossible to continuously optimize subsequent generation based on the delivery effect and lip-syncing synchronization status, resulting in difficulty in timely adjustments to spoken videos based on actual dissemination effects after delivery.

[0006] Therefore, a technical solution is needed that can adopt a dynamic voice-over video generation model that matches the target country or region and the current playback platform. This model can selectively filter the basic data for generating voice-over videos based on the target country or region, playback platform, related products, and voice-over language. At the same time, it can use audience feedback information and lip-syncing subtitle matching status from the previous time segment as feedback-based basic data. This will generate voice-over videos that are adapted to the platform resolution and digital facial detail, conform to local cultural habits, and can be continuously adjusted based on the performance of the campaign. Summary of the Invention

[0007] The purpose of this invention is to provide an automatic adaptation method for multinational digital human images and multilingual narration. By employing a 3DGS model that matches the target country or region and the current playback platform, the total number of Gaussian pixels is proportional to the total number of pixels on the target playback platform, and the density control threshold is correlated with the number of digital human facial features. The method also performs targeted filtering of multiple input contents for the model based on the target country or region, playback platform, related products, and narration language. Furthermore, it incorporates audience feedback information from previous time segments and lip-sync matching identifiers as two feedback inputs to the model. This solves various technical problems in existing technologies, such as the incompatibility between the narration video generation mechanism and platform resolution and digital human facial detail, inconsistencies between input content and local cultural habits and platform rules, and the difficulty in continuously adjusting the narration video based on its performance and lip-sync matching status after generation.

[0008] According to a first aspect of the present invention, an automatic adaptation method for multinational digital human avatars and multilingual voiceovers is provided, the method comprising: Analyze the multidimensional data of digital humans corresponding to the target countries or regions for the delivery of voice-over videos; Acquire related products that match the spoken video, the target playback platform for the spoken video, and the localized spoken video script for the parsed digital human's spoken language; Obtain a database of cultural compliance rules and prohibited words for the target countries or regions where the audio-visual videos are to be delivered; Collect various platform playback parameters of the voice-over video corresponding to the current playback platform where the voice-over video is being played; The various audience feedback information collected during the delivery of the spoken video on the current playback platform within any time segment will be used as the feedback-related content corresponding to the time segment. Detect the lip-sync matching identifier for any given time segment; Using a 3DGS model that matches the target country or region and the current playback platform, the system generates lip-synced videos for the next time segment of the target time segment based on the multi-dimensional data of digital humans corresponding to the target country or region, the localized scripts of the lip-synced videos, the cultural compliance rules and prohibited word databases of the target country or region, the various platform playback parameters of the lip-synced videos corresponding to the current playback platform, the feedback-related content corresponding to the set time segments, the lip-sync matching identifiers corresponding to the set time segments, and the duration of each time segment. These videos are then used to deliver the videos to the current playback platform in the target country or region. The 3DGS model is a 3D Gaussian sputtering model.

[0009] Furthermore, in the method: The analysis of digital human multidimensional data corresponding to the target country or region of the broadcast video includes: analyzing the facial features, clothing, posture, broadcast background, broadcast language, broadcast timbre, broadcast speed and broadcast emotion of the digital human matched to the target country or region of the broadcast video, so as to serve as the digital human multidimensional data corresponding to the target country or region of the broadcast video. The spoken video script is the text script matched with the spoken video, which includes an opening speech, key content points, transitional phrases and closing instructions organized in chronological order; Among them, the platform playback parameters of the voice-over video corresponding to the current playback platform where the voice-over video is played include: the total number of pixels, playback duration, style and color of the voice-over video. The detection of lip-sync matching identifiers for any time segment includes: when a mismatch between lip-sync and subtitles is detected in the broadcast video during the current playback platform within any time segment, the lip-sync matching identifier for the time segment is set to a mismatch value; otherwise, it is set to a matching value.

[0010] Furthermore, in the method: The process involves acquiring relevant products that match the spoken video, the target playback platform for the spoken video, and the industrialized, platform-based, and localized spoken video scripts that parse the spoken language of the digital human. This includes: the acquired industrialized, platform-based, and localized spoken video scripts that match industry terminology, product selling points, brand tone, platform rules, and local expression habits. Among them, the core content points in the spoken video script refer to breaking down the main information of the spoken video script into independent points in a parallel or progressive manner, with each independent point corresponding to a complete unit of information. Among them, the audience feedback information corresponding to any time segment includes the number of plays, completion rate, average viewing time, number of likes, number of comments, number of shares, number of favorites, number of clicks on the product page, and number of orders converted within the time segment. Moreover, all audience feedback information before the first broadcast of the video on the current video platform is empty data. Among them, facial features include facial contour features, eye features, nose features, lip features, skin color and texture features, hair features, and grooming features. Facial contour features include face shape, cheekbone height, cheekbone width, jawline, and facial three-dimensionality. Eye features include eye shape, eye fissure length, eye fissure opening and closing, eye distance, iris color, eyebrow shape, and eyebrow color. Nose features include nasal bridge height and nasal tip shape. Lip features include lip shape and lip color. Skin color and texture features include skin color type and skin texture data. Hair features include hair color, hairstyle, hair texture data, and beard shape. Grooming features include facial makeup type and facial accessory type.

[0011] Furthermore, in the method: The total number of Gaussian points in the 3DGS model that matches the target country or region and the current playback platform is directly proportional to the total number of pixels corresponding to the target playback platform. Furthermore, the density control threshold of the 3DGS model that matches the target country or region and the current playback platform is positively correlated with the number of facial features of the digital human that are parsed and matched in the target country or region of the voice-over video. Among them, the 3DGS model that matches the target country or region and the current playback platform includes an initialization unit, a density control unit, a rendering unit, and a parameter update unit; The initialization unit sets the total number of reference Gaussians to 50,000, which corresponds to a total number of reference pixels of 2 million. The total number of Gaussians is equal to the total number of reference Gaussians multiplied by the ratio of the total number of pixels on the current playback platform to the total number of reference pixels. The higher the total number of pixels, the larger the total number of Gaussians. When the number of facial features is 20, the density control threshold is set to a baseline value of 0.01. For each additional facial feature, the density control threshold increases by 5%. The more facial features there are, the more precise the density control becomes.

[0012] Furthermore, in the method: The initialization unit generates a Gaussian point set based on the multidimensional data of the digital human corresponding to the target country or region. The Gaussian points are preferentially distributed in the face, lips and clothing areas of the digital human, and each Gaussian point carries five types of network parameters: position, scaling, rotation, opacity and color. The color is represented by spherical harmonic coefficients. Among them, the density control unit clones or splits Gaussian points according to the density control threshold: when the position adjustment of a Gaussian point exceeds the density control threshold, it is cloned or split into smaller Gaussian points, and Gaussian points with an opacity lower than the removal threshold of 0.005 are removed. The rendering unit projects the cloned or split Gaussian point set into image frames in a differentiable rasterization manner, and synthesizes the next time segment of the spoken video frame by frame along the time axis of the spoken video script under the constraints of the industrial platform localized spoken video script, cultural compliance rules and prohibited word database, platform playback parameters, feedback related content and lip-sync subtitle matching identifier. When the lip-syncing subtitle matching identifier is a mismatch value, Gaussian point reprojection correction is performed on the lip area to align the lip shape with the subtitle. The parameter update unit adjusts each network parameter in reverse according to the deviation between the synthesized frame and the constraint. The model learning rate is adjusted according to the feedback-related content. The higher the completion rate, the number of likes, and the number of order conversions, the smaller the learning rate, and vice versa.

[0013] Furthermore, before analyzing the digital human multidimensional data corresponding to the target country or region for the broadcast video, the method also includes: Construct a 3DGS model that matches any country or region and any playback platform; Among them, the 3DGS model that matches any country or region and matches any playback platform includes the 3DGS model that matches the target country or region and matches the current playback platform. Among them, the model parameters of the 3DGS model that matches the target country or region and the current playback platform represent the 3DGS model that matches the target country or region and the current playback platform.

[0014] Furthermore, after generating lip-sync captioned videos for the next time segment of a set time segment using a 3DGS model that matches the target country or region and the current playback platform, and then deploying these videos to the current playback platform in the target country or region, the method further includes: Receive the lip-sync video for the next time segment of the set time segment, and transmit it to the current playback platform in the target country or region for distribution via the network transmission interface. The process of receiving lip-synced video for the next time segment of a set time segment and transmitting it to the current playback platform in the target country or region via a network transmission interface includes: embedding video data of the lip-synced video for the next time segment of a set time segment into the API function of the network transmission interface, so as to transmit it to the current playback platform in the target country or region via the network transmission interface.

[0015] Furthermore, before analyzing the digital human multidimensional data corresponding to the target country or region for the broadcast video, the method also includes: Obtain the target country or region for the user-input verbal video delivery, and perform different country or region-specific identification by using the encoding values ​​of different countries or regions; Among them, obtaining the target country or region for the broadcast video delivery based on user input, and implementing different country or region-specific identification through different country or region encoding values, includes: the user completing the user input of the target country or region for the broadcast video delivery through the user interaction interface.

[0016] Furthermore, after acquiring the associated products that simultaneously match the spoken video, the target playback platform for the spoken video, and the localized spoken video script that parses the spoken language of the digital human, the method further includes: The cultural compliance rules and prohibited word databases of different countries or regions are stored on different network storage nodes; This includes storing cultural compliance rules and prohibited word databases from different countries or regions on different network storage nodes, such as different big data storage nodes or different cloud computing storage nodes.

[0017] According to a second aspect of the present invention, an automatic adaptation system for multinational digital human images and multilingual voiceovers is provided, the system comprising: Data parsing device, used to parse the multidimensional data of digital humans corresponding to the target country or region for voice-over video delivery; The first capturing device is used to acquire the associated products that simultaneously match the spoken video, the target playback platform for the spoken video, and the industrial platform-based localized spoken video script that parses the spoken language of the digital human. The second capture device is used to acquire cultural compliance rules and a database of prohibited words for the target country or region where the spoken video is delivered. The first acquisition device is used to acquire various platform playback parameters of the voice-over video corresponding to the current playback platform of the voice-over video. The second acquisition device is used to collect various audience feedback information collected during the delivery of the spoken video on the current playback platform within any time segment as the feedback associated content corresponding to the time segment. A marker detection device is used to detect lip-sync markers corresponding to any given time segment. The lip-sync generation device is connected to a data parsing device, a first capture device, a second capture device, a first acquisition device, a second acquisition device, and an identifier detection device. It is used to generate a lip-sync video for the next time segment of the set time segment by using a 3DGS model that matches the target country or region and the current playback platform. The 3DGS model is a 3D Gaussian sputtering model.

[0018] Compared with the prior art, the present invention achieves the corresponding technical effects through the following main inventive points: By adopting a 3DGS model that matches the target country or region and the current playback platform, the total number of Gaussian points is proportional to the total number of pixels on the target playback platform, and the density control threshold is correlated with the number of facial features. This enables the automatic adaptation of model parameters to platform resolution and the number of digital human facial features, thereby ensuring that the clarity of the generated audio video matches the pixel requirements of the playback platform and that the detail of the digital human face is adapted to the number of facial features. By analyzing and matching the facial features, clothing, posture, background of the spoken words, language, timbre, speech rate and emotions of the digital human based on the target country or region as multi-dimensional data of the digital human, the system obtains localized spoken video scripts for industrial platforms, including opening remarks, core content points, transitional phrases and closing guidance, for related products, playback platforms and spoken language. It also obtains cultural compliance rules, prohibited word database and platform playback parameters to achieve targeted screening of model input content, so that the generated spoken video content meets the cultural compliance requirements, language habits and platform rules of the target country or region, and the spoken script structure meets the information reception habits of the local audience. By using the play count, completion rate, average viewing time, number of likes, number of comments, number of shares, number of favorites, number of clicks on product pages, and number of order conversions from the previous time segment as feedback-related content, and setting the lip-sync subtitle matching identifier to either a mismatch value or a matching value, the model allows for the selection of two feedback-based input contents. This enables subsequent voice-over video generation to adaptively adjust based on the delivery effect and lip-sync status, gradually improving the synchronization between lip-sync and subtitles in subsequent voice-over videos and gradually aligning the content with audience preferences. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of a working scenario for an automatic adaptation method of a multinational digital human image and multilingual voiceover according to the present invention.

[0020] Figure 2 The flowchart illustrates the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover according to a first embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the internal structure of the 3DGS model used in the automatic adaptation method of multinational digital human image and multilingual voiceover according to the present invention.

[0022] Figure 4 The flowchart of the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover shown in the second embodiment is presented.

[0023] Figure 5 The flowchart of the automatic adaptation method for a multinational digital human image and multilingual voiceover shown in the third embodiment is presented.

[0024] Figure 6 The flowchart of the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover shown in the fourth embodiment is presented.

[0025] Figure 7 The fifth embodiment presents a flowchart of the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover. Detailed Implementation

[0026] The specific embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0027] Figure 1 This is a schematic diagram illustrating a working scenario of an automatic adaptation method for multinational digital human images and multilingual voiceovers according to the present invention. The digital human video generation technology proposed in this invention belongs to the fields of computer graphics and speech synthesis technology.

[0028] exist Figure 1 In this context, the automatically adapted 3DGS model is a 3DGS model that matches the target country or region and the current playback platform. Different countries or regions and / or different playback platforms correspond to 3DGS models with different customized structures. exist Figure 1 In the process, the automatic adaptation of the 3DGS model includes various basic input contents: target country or region related content, delivery platform related content, first feedback content, second feedback content, industrial platform localized spoken video script, and the duration of each time segment; Specifically, in Figure 1 In the middle, the content associated with the target country or region includes multi-dimensional data of digital humans corresponding to the target country or region where the narration video is placed, as well as cultural compliance rules and prohibited word databases for the target country or region where the narration video is placed. The content associated with the placement platform includes various platform playback parameters of the narration video corresponding to the current playback platform. The first feedback content is the feedback associated content corresponding to the set time segment, and the second feedback content is the lip-sync subtitle matching identifier corresponding to the set time segment. For example, if the timeline presents multiple time segments, and each time segment has the same duration, the duration of each time segment can be set to 2 hours, 6 hours, or 36 hours, etc. Specifically, 3DGS (3D Gaussian Splatting) is a mainstream real-time 3D rendering model. Its core uses a large number of 3D Gaussian distributions (Gaussian points) to represent the 3D spatial structure, texture, and color of objects, replacing the traditional mesh model, and can quickly generate high-definition 3D digital human images. The working principle of the 3DGS model is as follows: Scenes and characters are decomposed into a massive number of independent 3D Gaussian points. Each Gaussian point contains parameters such as position, size, rotation, transparency, and color. Using camera-view projection, the 3D Gaussian points are projected onto 2D screen pixels. Gaussian colors are mixed layer by layer to directly output a clear image. The rendering speed is far higher than traditional 3D meshes, making it suitable for short videos and real-time digital human generation. This invention has several key features: a direct correlation between the total number of Gaussian points and the total number of pixels in the 3DGS model that matches the target country or region and the current playback platform; a unidirectional correlation between the density control threshold and the number of facial features; targeted selection of various input contents for the model; and a selection of two feedback-based input contents: audience feedback information from the previous time segment and lip-sync subtitle matching markers.

[0029] This invention targets the global distribution of spoken-video content. The target countries or regions for the spoken-video distribution can be Asian countries, European countries, North American countries, or other regions; the current playback platform can be a short video platform, a content community platform, an e-commerce platform, or other video distribution platforms.

[0030] The following will describe in detail, through embodiments, a method for automatic adaptation of multinational digital human images and multilingual voiceovers according to the present invention.

[0031] Example 1 Figure 2 The flowchart illustrates a method for automatic adaptation of a multinational digital human image and multilingual voiceover according to a first embodiment of the present invention. The method is implemented on a voiceover video generation platform and includes steps S11 to S17, as detailed below: Step S11: Analyze the multidimensional data of digital humans corresponding to the target countries or regions for the broadcast video delivery; In practice, the facial features, clothing, posture, background, language, tone, speaking speed, and emotions of the digital human can be analyzed and matched based on the target country or region of the video broadcast. This data can be used as multi-dimensional data of the digital human corresponding to the target country or region of the video broadcast. Specifically, audiences in different countries or regions have differences in facial aesthetics, clothing preferences, language habits, and emotional expression. For example, narration videos targeting the Japanese market usually prefer softer digital human facial contours and low-saturation narration backgrounds, while narration videos targeting the Middle Eastern market usually prefer digital human images with strong three-dimensional facial features and warm-toned narration backgrounds. Therefore, before generating narration videos, this invention first analyzes and matches the corresponding digital human multidimensional data according to the target country or region, so that the subsequently generated digital human image, from facial features to narration tone, is consistent with the expectations of the target market audience. For example, facial features include facial contour features, eye features, nose features, lip features, skin color and texture features, hair features, and grooming features; facial contour features include face shape, cheekbone height, cheekbone width, jawline, and facial three-dimensionality; eye features include eye shape, eye fissure length, eye fissure opening, eye distance, iris color, eyebrow shape, and eyebrow color; nose features include nasal bridge height and nasal tip shape; lip features include lip shape and lip color; skin color and texture features include skin color type and skin texture data; hair features include hair color, hairstyle, hair texture data, and beard shape; grooming features include facial makeup type and facial accessory type; the above facial features total 24 sub-features, and under the baseline condition, 20 core features can be taken as the baseline number for density control threshold calculation; Specifically, clothing, posture, background of the broadcast, language of the broadcast, timbre of the broadcast, speed of the broadcast, and emotion of the broadcast are all matched according to the cultural database of the target country or region. For example, the language of the broadcast includes Chinese, English, Japanese, Korean, Arabic, etc., the timbre of the broadcast includes male voice, female voice, child's voice, and the voice characteristics of different age groups, and the emotion of the broadcast includes calm, enthusiastic, serious, friendly, etc. Step S12: Obtain the associated products that match the spoken video, the target playback platform for the spoken video, and the industrial platform-based localized spoken video script that parses the spoken language of the digital human. Specifically, the script for a spoken video is a written script that matches a spoken video. It includes an opening statement, key content points, transitional phrases, and closing prompts organized in chronological order. The opening statement is used to establish the audience's attention at the beginning of the spoken video. The key content points are used to break down the product's selling points or brand information into independent points in a parallel or progressive manner. Each independent point corresponds to a complete unit of information. The transitional phrases are used to maintain a smooth speaking rhythm between different points. The closing prompts are used to encourage the audience to take actions such as following, liking, or purchasing at the end of the spoken video. Among them, the acquired industrialized, platform-based, and localized in-stream video scripts are matched with industry terminology, product selling points, brand tone, platform rules, and local expression habits; for example, for in-stream videos targeting the Southeast Asian market for beauty products, the industry terminology in the scripts uses locally common beauty terms, the product selling points highlight sun protection and long-lasting makeup, the brand tone is lively and friendly, the platform rules meet the requirements of local short video platforms for duration and subtitle placement, and the local expression habits use colloquial sentences familiar to the local audience; Step S13: Obtain the cultural compliance rules and prohibited word database of the target country or region for the matched spoken video delivery; Specifically, cultural compliance rules include religious taboos, color taboos, gesture taboos, and gender display norms of the target country or region. The prohibited word database includes words that are prohibited from appearing in advertising or public communication by laws and regulations of the target country or region. During the generation of the spoken video, the rendering unit calls the cultural compliance rules and the prohibited word database to filter content when synthesizing the spoken video to ensure that the generated spoken video does not violate local cultural taboos and laws and regulations. Step S14: Collect various platform playback parameters of the voice-over video corresponding to the current playback platform where the voice-over video is played; In practice, the collected audio-visual videos are played on the current playback platform, including the total number of pixels, playback duration, style, and color of the audio-visual videos. Specifically, the total number of pixels can be determined by the video resolution supported by the playback platform. For example, a short video platform supports a vertical screen resolution of 1080×1920, and its total number of pixels is approximately 2.07 million; a content community platform supports a horizontal screen resolution of 1920×1080, and its total number of pixels is also approximately 2.07 million; an e-commerce platform supports a 4K vertical screen resolution of 2160×3840, and its total number of pixels is approximately 8.29 million. Meanwhile, the playback duration can be determined based on the platform's time limit or recommended duration for a single video. Styles include realistic, animated, and documentary styles, and colors include the main color and a color scheme that matches the main color. Here, steps S11-S14 embody a key inventive point of the present invention, namely: by analyzing and matching the digital human's facial features, clothing, posture, voiceover background, language, timbre, speech rate, and emotions as multidimensional data of the digital human based on the target country or region, and obtaining an industrialized, platform-based, localized voiceover video script containing an opening, core content points, transitional phrases, and closing guidance for related products, playback platforms, and voiceover language, and obtaining cultural compliance rules, prohibited word databases, and platform playback parameters, the model input content is targeted for filtering, so that the generated voiceover video content conforms to the cultural compliance requirements, language habits, and platform rules of the target country or region, and the voiceover script structure meets the information reception habits of the local audience; Step S15: Collect various audience feedback information collected during the delivery of the spoken video on the current playback platform within any time segment as the feedback associated content corresponding to the time segment; Specifically, the audience feedback information corresponding to any time segment includes the number of views, completion rate, average viewing time, number of likes, number of comments, number of shares, number of favorites, number of clicks on the product page, and number of orders converted into sales for the spoken video within that time segment. The above nine feedback data reflect the dissemination effect and commercial conversion effect of the spoken video within the corresponding time segment from different dimensions. Among them, the number of views reflects the scale of content reach, the completion rate and average viewing time reflect the attractiveness of the content, the number of likes, comments and shares reflects the audience's willingness to interact, and the number of clicks on the product page and the number of orders converted into sales reflect the commercial monetization effect. Among them, the audience feedback information of the spoken video before its first broadcast on the current video platform is all empty data, that is, the feedback related content corresponding to the first time segment is empty data. The spoken video in the first time segment is generated by the 3DGS model based on digital human multidimensional data, industrial platform localized spoken video script, cultural compliance rules and prohibited word database, and platform playback parameters, without carrying previous feedback data. Step S16: Detect the lip-sync matching identifier corresponding to any time segment; In practice, if a mismatch between lip-sync and subtitles is detected during the delivery of a spoken video on the current playback platform at any time segment, the lip-sync matching identifier corresponding to the time segment is set to the mismatch value; otherwise, it is set to the matching value. The mismatch value and the matching value can be binary values ​​of different values. Specifically, lip-sync mismatch detection is determined by comparing the actual pronunciation timestamp in the audio of the spoken video with the timestamp displayed in the subtitles. When the deviation between the start and end times of the audio corresponding to the same pronunciation unit and the start and end times of the subtitles exceeds a preset allowable deviation threshold (e.g., 100 milliseconds), a lip-sync mismatch is determined to exist in that time segment. If a mismatch is detected, the lip-sync matching flag is set to a mismatch value (e.g., 0BFF); otherwise, it is set to a matching value (e.g., 0B00). This flag serves as one of the input conditions when the 3DGS model generates the next time segment of the spoken video, and is used to trigger the reprojection correction of the Gaussian points in the lip region. Here, steps S15-S16 embody another key inventive point of the present invention, namely: by using the playback volume, completion rate, average viewing time, number of likes, number of comments, number of reposts, number of favorites, number of clicks on product pages, and number of order conversions in the previous time segment as feedback-related content, and setting the lip-sync subtitle matching identifier to a mismatch value or a matching value, the selection of two feedback model input contents is realized, so that the subsequent lip-sync video generation can be adaptively adjusted according to the delivery effect and lip-sync synchronization status, so that the lip-sync and subtitle synchronization of the subsequently generated lip-sync video gradually improves, and the content orientation gradually conforms to the audience's preferences; Step S17: Using a 3DGS model that matches the target country or region and the current playback platform, based on the digital human multidimensional data corresponding to the target country or region for the narration video, the localized narration video script for the industrial platform, the cultural compliance rules and prohibited word database of the target country or region for the narration video, the various platform playback parameters of the narration video corresponding to the current playback platform, the feedback related content corresponding to the set time segment, the lip-sync subtitle matching identifier corresponding to the set time segment, and the duration occupied by each time segment, generate a narration video with lip-sync subtitle matching for the next time segment of the set time segment, which is used to be deployed to the current playback platform in the target country or region. The 3DGS model is the 3D Gaussian sputtering model. Here, step S17 embodies the most critical inventive point of the present invention, namely: adopting a 3DGS model that matches the target country or region and the current playback platform, making the total number of Gaussian points proportional to the total number of pixels on the target playback platform, and the density control threshold correlated with the number of facial features, thereby achieving automatic adaptation of model parameters to platform resolution and the number of digital human facial features, so that the clarity of the generated spoken video picture matches the pixel requirements of the playback platform, and the fineness of digital human facial details is adapted to the number of facial features; like Figure 3The image shown is a 3DGS model used in the automatic adaptation method of multinational digital human image and multilingual voiceover of the present invention, that is, a 3DGS model that matches the target country or region and matches the current playback platform. The automatic matching 3DGS model in this application includes an initialization unit, a density control unit, a rendering unit and a parameter update unit. exist Figure 3 In the initialization unit, a Gaussian point set is generated based on the multidimensional data of the digital human corresponding to the target country or region. These Gaussian points are preferentially distributed in the face, lips, and clothing areas of the digital human. Each Gaussian point carries five network parameters: position, scaling, rotation, opacity, and color. Color is represented by spherical harmonic coefficients. The initialization unit sets the total number of baseline Gaussian points to 50,000, corresponding to a total number of baseline pixels of 2 million. The total number of Gaussian points is equal to the ratio of the total number of baseline Gaussian points multiplied by the total number of pixels on the current playback platform to the total number of baseline pixels, i.e.: N_gauss = 50000 × (P_total / 2000000) Where N_gauss is the total number of Gaussian points, and P_total is the total number of pixels corresponding to the current playback platform; Therefore, the higher the total number of pixels, the larger the total Gaussian count. For example, when the total number of pixels corresponding to the current playback platform is 2.07 million (corresponding to 1080P resolution), the total Gaussian count is approximately 51,750; when the total number of pixels corresponding to the current playback platform is 8.29 million (corresponding to 4K resolution), the total Gaussian count is approximately 207,250; and when the total number of pixels corresponding to the current playback platform is 920,000 (corresponding to 720P resolution), the total Gaussian count is approximately 23,000. In this way, by establishing a proportional relationship between the total number of Gaussian points in the 3DGS model and the total number of pixels on the target playback platform, the number of Gaussian points can be automatically scaled according to the platform resolution, avoiding blurry images due to insufficient Gaussian points on high-definition platforms, and also avoiding waste of rendering resources due to redundant Gaussian points on standard-definition platforms. Specifically, the density control unit clones or splits Gaussian points according to the density control threshold: when the position adjustment of a Gaussian point exceeds the density control threshold, it is cloned or split into smaller Gaussian points, and Gaussian points with an opacity lower than the removal threshold of 0.005 are removed.

[0032] The density control threshold is directly correlated with the number of facial features of the digital human matched to the target country or region of the broadcast video; when the number of facial features is 20, the density control threshold is set to a baseline value of 0.01; for every additional facial feature, the density control threshold increases by 5%; expressed by the formula: T_density = 0.01 × [1 + 0.05 × (N_feature - 20)] Where T_density is the density control threshold and N_feature is the number of facial features.

[0033] Therefore, the more facial features there are, the more precise the density control becomes; for example, when there are 20 facial features, the density control threshold is 0.01; when there are 21 facial features, the density control threshold increases to 0.0105; when there are 24 facial features, the density control threshold increases to 0.0120; and when there are 30 facial features, the density control threshold increases to 0.0150. In this way, by establishing a positive correlation between the density control threshold and the number of facial features, the more detailed the digital human face is, the more refined the triggering conditions for Gaussian point cloning or splitting become. This ensures the rendering accuracy of areas with rich facial details, avoids redundant Gaussian points in simple facial models due to excessively fine thresholds, and also avoids the loss of details in complex facial models due to excessively coarse thresholds.

[0034] Specifically, the rendering unit projects the cloned or split Gaussian point set into image frames in a differentiable rasterization manner, and is constrained by industrial platform localization of spoken video scripts, cultural compliance rules and prohibited word databases, platform playback parameters, feedback-related content and lip-syncing subtitle matching identifiers (i.e. Figure 3 Under the various constraints in the script, the next time segment of the spoken video is synthesized frame by frame along the timeline of the spoken video script; Among them, the differentiable rasterization method refers to the use of opacity-weighted mixing projection of Gaussian points, so that the color of each pixel is determined by the color and opacity of multiple Gaussian points covering the pixel. Moreover, the projection process is differentiable, which facilitates the subsequent backpropagation to adjust the Gaussian point parameters. Among them, the constraints of cultural compliance rules and the prohibited word database are reflected in the rendering unit's filtering of subtitle content and background elements. For example, when the script contains prohibited words, the rendering unit replaces the prohibited words with synonymous compliant words or blocks their display; when the background color violates the color taboos of the target country or region, the rendering unit automatically adjusts the background color scheme. Among them, the constraints of the platform playback parameters are reflected in the direct setting of rendering resolution, screen ratio and color style. The rendering unit determines the output frame resolution according to the total number of pixels in the platform playback parameters and determines the lighting and shadow rendering mode according to the style parameters. Among them, the constraints of feedback-related content are reflected in the fine-tuning of the orientation of the audio video content of the next time segment by the rendering unit. For example, when the completion rate of the feedback-related content of the current time segment is low, the rendering unit shortens the audio duration of a single core content point and speeds up the transition when generating the audio video of the next time segment; when the number of likes and comments is high, the rendering unit continues the audio mood and clothing style of the digital human in the current time segment. Specifically, when the lip-syncing subtitle matching identifier is a mismatch value, the Gaussian points in the lip region are reprojected and corrected to align the lip shape with the subtitle. The specific method of reprojection correction is as follows: determine the geometry of the lips based on the corresponding phonetic unit of the subtitle, calculate the deviation between the existing Gaussian point projection position in the lip region and the target geometry, and adjust the position and rotation parameters of the Gaussian points in the lip region through backpropagation so that the lip contour matches the subtitle pronunciation at the corresponding timestamp. Specifically, the parameter update unit adjusts each network parameter in reverse according to the deviation between the synthesized frame and the constraint, and the model learning rate is adjusted according to the feedback-related content. Among them, the higher the completion rate, the number of likes and the number of order conversions, the smaller the learning rate, and vice versa. The learning rate adjustment uses a segmented mapping method. For example, the baseline values ​​are set as follows: completion rate 50%, likes 1000, and order conversions 50. When all three values ​​are higher than their respective baseline values, the learning rate is set to the smallest value of 0.0001. When two of the three values ​​are higher than their baseline values, the learning rate is set to the middle value of 0.0005. When only one of the three values ​​is higher than its baseline value or all three are lower than their baseline values, the learning rate is set to the largest value of 0.001. In this way, time segments with better delivery performance correspond to smaller learning rates, allowing the model to make conservative fine-tuning based on its already good performance; time segments with poor delivery performance correspond to larger learning rates, allowing the model to quickly adjust its generation strategy to improve subsequent performance.

[0035] Example 2 Figure 4 The flowchart illustrates the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover according to a second embodiment of the present invention.

[0036] like Figure 4 As shown, with Figure 2 Unlike the previous embodiment, in the automatic adaptation method for multinational digital human images and multilingual narration, after generating narration videos with lip-sync subtitles matching the next time segment of a set time segment using a 3DGS model that matches the target country or region and the current playback platform, and then deploying them to the current playback platform in the target country or region, i.e. after step S17, the method further includes: Step S18: Receive the lip-sync video of the next time segment after the set time segment, and transmit it to the current playback platform in the target country or region for distribution via the network transmission interface; Specifically, video data for the lip-synced video, matching the lip-sync subtitles for the next time segment within a specified time segment, is embedded into the API functions of the network transmission interface. This data is then transmitted to the current playback platform in the target country or region via the network transmission interface. For example, the network transmission interface can use the HTTP / HTTPS protocol. The API functions include a video upload interface, a video publishing interface, and a delivery status callback interface. The video upload interface is responsible for transmitting the lip-synced video file to the content storage node of the current playback platform. The video publishing interface is responsible for binding the uploaded video to the target delivery account and setting the delivery timestamp. The delivery status callback interface is responsible for receiving the delivery success or failure status returned by the current playback platform.

[0037] 3rd Embodiment Figure 5 The flowchart illustrates the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover according to a third embodiment of the present invention.

[0038] like Figure 5 As shown, with Figure 2 Unlike the embodiments described above, in the automatic adaptation method for multinational digital human images and multilingual voiceovers, before parsing the multidimensional data of the digital human corresponding to the target country or region of the voiceover video, i.e. before step S11, the method further includes: Step S19: Obtain the target country or region for the user-input verbal video delivery, wherein different country or region-specific identification is performed by using the encoding values ​​of different countries or regions; Specifically, users input the target country or region for the broadcast video through a user interface. For example, country codes can be used: 01 for China, 02 for the United States, 03 for Japan, 04 for South Korea, and 05 for Saudi Arabia. After the user selects or enters the corresponding country code at the user interface, the system determines the target country or region for the broadcast video and subsequently calls the corresponding digital human multidimensional data template, cultural compliance rules database, and prohibited word database for that country or region.

[0039] Example 4 Figure 6 The flowchart illustrates the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover according to the fourth embodiment of the present invention.

[0040] like Figure 6 As shown, with Figure 2Unlike the embodiments described above, in the automatic adaptation method for multinational digital human images and multilingual voiceovers, after obtaining the associated products that simultaneously match the voiceover video, the target playback platform for the voiceover video, and the industrial platform-based localized voiceover video script for the parsed digital human's voiceover language, i.e., after step S12, the method further includes: Step S20: Store the cultural compliance rules and prohibited word databases of different countries or regions to different network storage nodes; Specifically, different network storage nodes serve as either large-scale data storage nodes or cloud computing storage nodes. For example, the first network storage node is deployed in a data center in the Asia-Pacific region to store cultural compliance rules and prohibited word databases for East Asian countries or regions such as China, Japan, and South Korea; the second network storage node is deployed in a data center in North America to store cultural compliance rules and prohibited word databases for countries or regions such as the United States and Canada; and the third network storage node is deployed in a data center in the Middle East to store cultural compliance rules and prohibited word databases for countries or regions such as Saudi Arabia and the United Arab Emirates. This allows for the storage of compliant data from different target countries or regions locally, reducing data retrieval latency and meeting the legal and regulatory requirements for localized data storage in different countries or regions.

[0041] Example 5 Figure 7 The flowchart illustrates the steps of an automatic adaptation method for a multinational digital human image and multilingual voiceover according to the fifth embodiment of the present invention.

[0042] like Figure 7 As shown, with Figure 2 Unlike the embodiments described above, in the automatic adaptation method for multinational digital human images and multilingual voiceovers, before parsing the multidimensional data of the digital human corresponding to the target country or region of the voiceover video, i.e. before step S11, the method further includes: Step S21: Construct a 3DGS model that matches any country or region and any playback platform; Specifically, a 3DGS model that matches any country or region and any playback platform includes a 3DGS model that matches the target country or region and the current playback platform. In practical applications, a family of 3DGS models covering major countries or regions and major playback platforms is pre-built, with each model identified by a corresponding set of model parameters. When an actual delivery task arrives, the corresponding model is selected from the model family based on the target country or region and the current playback platform, and its model parameters are loaded. This avoids retraining the model for each delivery task and shortens the preparation time for generating spoken videos. Specifically, the model parameters of a 3DGS model that matches the target country or region and the current playback platform can be used to represent the 3DGS model that matches the target country or region and the current playback platform. For example, the model parameters may include the total number of Gaussians, the density control threshold, the position distribution template of the initialization unit, the cloning splitting strategy parameters of the density control unit, the rasterization resolution of the rendering unit, and the baseline learning rate of the parameter update unit. All of the above parameters can be stored in the model parameter database with the target country or region encoding and the playback platform encoding as indexes.

[0043] And in the automatic adaptation method for multinational digital human figures and multilingual voiceovers shown in various embodiments of the present invention: The total Gaussian count of the 3DGS model that matches the target country or region and the current playback platform is proportional to the total number of pixels corresponding to the target playback platform. The density control threshold of the 3DGS model that matches the target country or region and the current playback platform is correlated in the same direction with the number of facial features of the digital human matched by the target country or region in the voice-over video. This includes: using a Gaussian count mapping curve to represent the numerical mapping relationship that the total Gaussian count is proportional to the total number of pixels, and using a threshold mapping transformation curve to represent the numerical mapping relationship that the density control threshold is correlated in the same direction with the number of facial features. Among them, the Gaussian total number mapping curve and the threshold mapping transformation curve have the same curve resolution. Curve resolution refers to the proportion of the curve length to the number of sampling points that can be used to take values ​​on the curve. In the Gaussian total number mapping curve, the total number of pixels corresponding to the target playback platform is the horizontal axis value, and the total number of Gaussians of the 3DGS model that matches the target country or region and the current playback platform is the vertical axis value. In the threshold mapping transformation curve, the number of facial features of the digital human matched by the target country or region for the broadcast video is the horizontal axis value, and the density control threshold of the 3DGS model that matches the target country or region and the current playback platform is the vertical axis value.

[0044] In addition, the present invention may also cite the following technical contents to highlight the significant technical advancements of the present invention: Employing a 3DGS model that matches both the target country or region and the current playback platform, this system generates lip-synced videos for the next time segment of the target period based on multi-dimensional digital human data corresponding to the target country or region, localized lip-synced video scripts for industrial platforms, cultural compliance rules and prohibited word databases for the target country or region, various platform playback parameters for the lip-synced videos on the current playback platform, feedback-related content for each time segment, lip-sync matching identifiers for each time segment, and the duration of each time segment. These videos are then used for distribution to the target country or region. The playback platform, in which the 3DGS model, namely the 3D Gaussian sputtering model, includes: multi-dimensional digital human data corresponding to the target country or region of the narrated video, localized narrated video scripts for industrialization and platformization, cultural compliance rules and prohibited word databases matching the target country or region of the narrated video, various platform playback parameters of the narrated video corresponding to the current playback platform, feedback related content corresponding to the set time segments, lip-sync subtitle matching identifiers corresponding to the set time segments, and the duration occupied by each time segment, which are then processed by binary value conversion and input in parallel into the 3DGS model that matches the target country or region and the current playback platform; The process involves using a 3DGS model that matches the target country or region and the current playback platform. Based on the target country or region's digital human multidimensional data, the localized lip-sync video script, the cultural compliance rules and prohibited word database of the target country or region, the playback parameters of the lip-sync video on the current playback platform, the feedback-related content corresponding to the set time segments, the lip-sync matching identifiers for the set time segments, and the duration of each time segment, a lip-sync video matching the next time segment is generated and used for distribution to the current playback platform in the target country or region. The 3DGS model, also known as the 3D Gaussian sputtering model, further includes executing the 3DGS model that matches the target country or region and the current playback platform to obtain the binary numerical representation of the lip-sync video matching the next time segment of the set time segment from the output of the 3DGS model that matches the target country or region and the current playback platform. For example, obtaining the lip-sync video of the next time segment of a set time segment that matches the target country or region and the current playback platform, represented by the binary numerical representation of the output of the 3DGS model, includes: the output lip-sync video of the next time segment of a set time segment that matches the target country or region and the current playback platform is a binary bitstream.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for automatic adaptation of multinational digital human avatars and multilingual voiceovers, characterized in that, The method includes: Analyze the multidimensional data of digital humans corresponding to the target countries or regions for the delivery of voice-over videos; Acquire related products that match the spoken video, the target playback platform for the spoken video, and the localized spoken video script for the parsed digital human's spoken language; Obtain a database of cultural compliance rules and prohibited words for the target countries or regions where the audio-visual videos are to be delivered; Collect various platform playback parameters of the voice-over video corresponding to the current playback platform where the voice-over video is being played; The various audience feedback information collected during the delivery of the spoken video on the current playback platform within any time segment will be used as the feedback-related content corresponding to the time segment. Detect the lip-sync matching identifier for any given time segment; Using a 3DGS model that matches the target country or region and the current playback platform, the system generates lip-synced videos for the next time segment of the target time segment based on the multi-dimensional data of digital humans corresponding to the target country or region, the localized scripts of the lip-synced videos, the cultural compliance rules and prohibited word databases of the target country or region, the various platform playback parameters of the lip-synced videos corresponding to the current playback platform, the feedback-related content corresponding to the set time segments, the lip-sync matching identifiers corresponding to the set time segments, and the duration of each time segment. These videos are then used to deliver the videos to the current playback platform in the target country or region. The 3DGS model is a 3D Gaussian sputtering model.

2. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 1, characterized in that, In the method: The analysis of digital human multidimensional data corresponding to the target country or region of the broadcast video includes: analyzing the facial features, clothing, posture, broadcast background, broadcast language, broadcast timbre, broadcast speed and broadcast emotion of the digital human matched to the target country or region of the broadcast video, so as to serve as the digital human multidimensional data corresponding to the target country or region of the broadcast video. The spoken video script is the text script matched with the spoken video, which includes an opening speech, key content points, transitional phrases and closing instructions organized in chronological order; Among them, the platform playback parameters of the voice-over video corresponding to the current playback platform where the voice-over video is played include: the total number of pixels, playback duration, style and color of the voice-over video. The detection of lip-sync matching identifiers for any time segment includes: when a mismatch between lip-sync and subtitles is detected in the broadcast video during the current playback platform within any time segment, the lip-sync matching identifier for the time segment is set to a mismatch value; otherwise, it is set to a matching value.

3. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 2, characterized in that, In the method: The process involves acquiring relevant products that match the spoken video, the target playback platform for the spoken video, and the industrialized, platform-based, and localized spoken video scripts that parse the spoken language of the digital human. This includes: the acquired industrialized, platform-based, and localized spoken video scripts that match industry terminology, product selling points, brand tone, platform rules, and local expression habits. Among them, the core content sub-points in the spoken video script refer to breaking down the main information of the spoken video script into independent points in a parallel or progressive relationship, with each independent point corresponding to a complete unit of information. Among them, the audience feedback information corresponding to any time segment includes the number of plays, completion rate, average viewing time, number of likes, number of comments, number of shares, number of favorites, number of clicks on the product page, and number of orders converted within the time segment. Moreover, all audience feedback information before the first broadcast of the video on the current video platform is empty data. Among them, facial features include facial contour features, eye features, nose features, lip features, skin color and texture features, hair features, and grooming features. Facial contour features include face shape, cheekbone height, cheekbone width, jawline, and facial three-dimensionality. Eye features include eye shape, eye fissure length, eye fissure opening and closing, eye distance, iris color, eyebrow shape, and eyebrow color. Nose features include nasal bridge height and nasal tip shape. Lip features include lip shape and lip color. Skin color and texture features include skin color type and skin texture data. Hair features include hair color, hairstyle, hair texture data, and beard shape. Grooming features include facial makeup type and facial accessory type.

4. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 3, characterized in that, In the method: The total number of Gaussian points in the 3DGS model that matches the target country or region and the current playback platform is directly proportional to the total number of pixels corresponding to the target playback platform. Furthermore, the density control threshold of the 3DGS model that matches the target country or region and the current playback platform is positively correlated with the number of facial features of the digital human that are parsed and matched in the target country or region of the voice-over video. Among them, the 3DGS model that matches the target country or region and the current playback platform includes an initialization unit, a density control unit, a rendering unit, and a parameter update unit; The initialization unit sets the total number of reference Gaussians to 50,000, which corresponds to a total number of reference pixels of 2 million. The total number of Gaussians is equal to the total number of reference Gaussians multiplied by the ratio of the total number of pixels on the current playback platform to the total number of reference pixels. The higher the total number of pixels, the larger the total number of Gaussians. When the number of facial features is 20, the density control threshold is set to a baseline value of 0.

01. For each additional facial feature, the density control threshold increases by 5%. The more facial features there are, the more precise the density control becomes.

5. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 4, characterized in that, In the method: The initialization unit generates a Gaussian point set based on the multidimensional data of the digital human corresponding to the target country or region. The Gaussian points are preferentially distributed in the face, lips and clothing areas of the digital human, and each Gaussian point carries five types of network parameters: position, scaling, rotation, opacity and color. The color is represented by spherical harmonic coefficients. Among them, the density control unit clones or splits Gaussian points according to the density control threshold: when the position adjustment of a Gaussian point exceeds the density control threshold, it is cloned or split into smaller Gaussian points, and Gaussian points with an opacity lower than the removal threshold of 0.005 are removed. The rendering unit projects the cloned or split Gaussian point set into image frames in a differentiable rasterization manner, and synthesizes the next time segment of the spoken video frame by frame along the time axis of the spoken video script under the constraints of the industrial platform localized spoken video script, cultural compliance rules and prohibited word database, platform playback parameters, feedback related content and lip-sync subtitle matching identifier. When the lip-syncing subtitle matching identifier is a mismatch value, Gaussian point reprojection correction is performed on the lip area to align the lip shape with the subtitle. The parameter update unit adjusts each network parameter in reverse according to the deviation between the synthesized frame and the constraint. The model learning rate is adjusted according to the feedback-related content. The higher the completion rate, the number of likes, and the number of order conversions, the smaller the learning rate, and vice versa.

6. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 5, characterized in that, Before analyzing the digital human multidimensional data corresponding to the target country or region for the broadcast video, the method further includes: Construct a 3DGS model that matches any country or region and any playback platform; Among them, the 3DGS model that matches any country or region and matches any playback platform includes the 3DGS model that matches the target country or region and matches the current playback platform. Among them, the model parameters of the 3DGS model that matches the target country or region and the current playback platform represent the 3DGS model that matches the target country or region and the current playback platform.

7. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 5, characterized in that, After employing a 3DGS model that matches the target country or region and the current playback platform, and based on the digital human multidimensional data corresponding to the target country or region for the narrated video, the localized narrated video script for the industrial platform, the cultural compliance rules and prohibited word database of the target country or region for the narrated video, the various platform playback parameters of the narrated video corresponding to the current playback platform, the feedback-related content corresponding to the set time segment, the lip-sync subtitle matching identifier corresponding to the set time segment, and the duration of each time segment, the method generates a narrated video with lip-sync subtitle matching for the next time segment of the set time segment, which is then used for distribution to the current playback platform in the target country or region. The method further includes: Receive the lip-sync video for the next time segment of the set time segment, and transmit it to the current playback platform in the target country or region for distribution via the network transmission interface. The process of receiving lip-synced video for the next time segment of a set time segment and transmitting it to the current playback platform in the target country or region via a network transmission interface includes: embedding video data of the lip-synced video for the next time segment of a set time segment into the API function of the network transmission interface, so as to transmit it to the current playback platform in the target country or region via the network transmission interface.

8. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 5, characterized in that, Before analyzing the digital human multidimensional data corresponding to the target country or region for the broadcast video, the method further includes: Obtain the target country or region for the user-input verbal video delivery, and perform different country or region-specific identification by using the encoding values ​​of different countries or regions; Among them, obtaining the target country or region for the broadcast video delivery based on user input, and implementing different country or region-specific identification through different country or region encoding values, includes: the user completing the user input of the target country or region for the broadcast video delivery through the user interaction interface.

9. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to claim 5, characterized in that, After acquiring the associated products that simultaneously match the spoken video, the target playback platform for the spoken video, and the localized spoken video script for the parsed digital human's spoken language, the method further includes: Store cultural compliance rules and prohibited word databases from different countries or regions on different network storage nodes; This includes storing cultural compliance rules and prohibited word databases from different countries or regions on different network storage nodes, such as different big data storage nodes or different cloud computing storage nodes.

10. The automatic adaptation method for multinational digital human images and multilingual voiceovers according to any one of claims 4-9, characterized in that, In the method: The total Gaussian count of the 3DGS model that matches the target country or region and the current playback platform is proportional to the total number of pixels corresponding to the target playback platform. The density control threshold of the 3DGS model that matches the target country or region and the current playback platform is correlated in the same direction with the number of facial features of the digital human matched by the target country or region in the voice-over video. This includes: using a Gaussian count mapping curve to represent the numerical mapping relationship that the total Gaussian count is proportional to the total number of pixels, and using a threshold mapping transformation curve to represent the numerical mapping relationship that the density control threshold is correlated in the same direction with the number of facial features. Among them, the Gaussian total number mapping curve and the threshold mapping transformation curve have the same curve resolution. Curve resolution refers to the proportion of the curve length to the number of sampling points that can be used to take values ​​on the curve. In the Gaussian total number mapping curve, the total number of pixels corresponding to the target playback platform is the horizontal axis value, and the total number of Gaussians of the 3DGS model that matches the target country or region and the current playback platform is the vertical axis value. In the threshold mapping transformation curve, the number of facial features of the digital human matched by the target country or region for the broadcast video is the horizontal axis value, and the density control threshold of the 3DGS model that matches the target country or region and the current playback platform is the vertical axis value.

Images