A method and system for generating portrait videos supporting user-defined text

Abstract

The present invention discloses a method and system for generating portrait videos supporting user-defined text, including a text parsing module, a semantic alignment module, a video generation module, a real-time processing module, a feedback adjustment module, and an early warning display module; the problems of semantic mismatch between text and image generation, inaccurate multi-language support, and poor real-time response are solved. Through multi-level text parsing and semantic alignment modules, the solution combines cultural background and multi-language characteristics to generate accurate structured data in the text parsing stage, and accurately maps behavioral instructions and facial expression descriptions in the text through deep learning algorithms to ensure that the generated video is consistent with the text semantics. At the same time, the translation processing unit combines language models and algorithms to achieve efficient and accurate multi-language translation to avoid ambiguity. The real-time processing module improves the system's response performance in high-load environments such as transportation hubs through time monitoring and speed control, and dynamically adjusts the generation strategy to ensure that the system is stable and efficient.

Description

Technical Field

[0001] The present invention relates to the field of security and monitoring technology, and in particular to a method and system for generating a portrait video supporting user-defined text. Background Art

[0002] The portrait video generation system originated from the development of computer vision and deep learning technology, and was mainly used for face recognition and expression analysis in the early days. With the maturity of technologies such as generative adversarial networks (GANs), the system has gradually evolved to be able to generate high-quality virtual portrait videos. In recent years, such systems have been widely used in various fields, including public safety, transportation, retail, community security and other industries. This technology improves monitoring efficiency, risk warning and event simulation capabilities; the wide application of the industry has not only improved the level of security protection, but also enhanced the ability to respond to emergencies;

[0003] However, the portrait video generation system has the following technical disadvantages when applied to transportation hub security inspection:

[0004] 1. Semantic mismatch between text and image generation: There may be semantic deviations between user-defined text and generated portrait videos. Especially in complex or ambiguous security inspection instructions, the generated video content may not accurately express the meaning of the text, causing monitoring personnel to misunderstand or fail to take timely action.

[0005] 2. Accuracy and consistency issues in multi-language support: Transportation hubs usually need to support security inspection instructions and warning prompts in multiple languages. However, when existing systems generate multi-language texts, inaccurate translations, incorrect grammar, or cultural differences may lead to misunderstandings of the content, affecting security inspection efficiency and safety.

[0006] 3. Poor real-time responsiveness: In high-traffic transportation hubs, the security environment changes rapidly, and videos generated by custom text usually take a long time to process, making it difficult to respond quickly to emergencies. The system may not be able to generate corresponding instructions and videos in real time in an emergency, affecting the overall response speed. Summary of the invention

[0007] The present invention aims to provide a method and system for generating a portrait video supporting user-defined text, so as to solve the above-mentioned problem.

[0008] The present invention is achieved through the following technical solutions:

[0009] A portrait video generation system supporting user-defined text, characterized by comprising a text parsing module, a semantic alignment module, a video generation module, a real-time processing module, a feedback adjustment module and an early warning display module;

[0010] The text parsing module is used to receive the custom text input by the user, and perform semantic analysis and translation processing on the text through natural language processing technology, generate structured text data, and provide accurate translation according to different languages ​​and cultural backgrounds to solve the accuracy and consistency problems of multi-language support;

[0011] The semantic alignment module is used to semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yypz, and evaluate it, and then adjust the semantic matching problem of text and image generation according to the evaluation content;

[0012] The video generation module is used to generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video;

[0013] The real-time processing module is used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the required total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity;

[0014] The warning display module is used to compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate a warning report based on the video content and actual security inspection data, so as to promptly inform security personnel of potential safety risks and take corresponding warning measures.

[0015] Preferably, the text parsing module includes a semantic analysis unit and a translation processing unit;

[0016] The semantic analysis unit is used to receive the user-defined text input by the user, convert the text information in natural language form into processable structured data, and perform word segmentation, part-of-speech tagging, syntactic analysis and semantic parsing on the text to identify the key content and instructions in the text; secondly, through natural language processing technology, the logical relationship of the sentence, the meaning of the instruction and the context association are analyzed, and after the core semantic elements are extracted, a semantic structure with a logical hierarchy is generated, and the sentences in the text are classified and annotated according to the semantic content to form a hierarchical text data structure;

[0017] The translation processing unit is used to translate the structured text data generated by the semantic analysis unit into multiple languages, and adjust the translation content according to the syntactic rules, word order, vocabulary selection and cultural background of different languages ​​to make it conform to the expression habits of the target language; the unit combines the language model and the translation algorithm to verify the semantic consistency of multiple languages, and through iterative optimization, it reduces the ambiguity and inconsistency problems caused by language conversion, and generates multi-version text data with cross-language applicability, providing a unified standardized input for subsequent semantic alignment and video generation.

[0018] Preferably, the semantic alignment module includes a semantic matching unit, a behavior mapping unit and a matching value calculation and evaluation unit;

[0019] The semantic matching unit is used to receive the structured text data generated by the text parsing module, and compare it with the preset video generation model layer by layer to achieve accurate semantic matching; secondly, the instructions, descriptions and keywords in the text are extracted through a deep learning algorithm, and correspond one by one with the character behavior and facial expression features in the video generation model, and the parsed semantic elements are mapped to executable actions or expression modules in the video generation model;

[0020] The behavior mapping unit is used to quantify the behavior description in the text based on the mapping result generated by the semantic matching unit, and map it to the video generation model in turn to generate behavior model data that can be used for video generation.

[0021] Preferably, the matching value calculation and evaluation unit is used to compare and analyze the content generated by the video with the text semantics;

[0022] The user-defined text is analyzed for vocabulary and syntax through natural language processing technology to extract semantic content; then the weight of each instruction in the overall semantics is calculated, and finally the instruction semantic weight index Zlyz is generated. The specific calculation formula is as follows:

[0023]

[0024] Where ti is a specific instruction, d is a document, N is the total number of documents, and df(ti) is the number of documents containing the instruction;

[0025] Then, when generating the video, the actual generated behavior is compared with the expected behavior to calculate the behavior mapping accuracy index Xwmz. The specific calculation formula is as follows:

[0026]

[0027] Where A and B are vector representations of text behavior features and generated behavior features;

[0028] Secondly, in the process of generating videos, the expression-related data in the generated video frames are collected and compared with the emotional description in the text semantics, and the matching degree between the two is calculated to obtain the expression matching index Bxpd. The specific calculation formula is as follows:

[0029]

[0030] Where Et is the expression feature vector in the text, and Eg is the expression feature vector in the generated video;

[0031] Finally, by extracting the instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, and calculating, the semantic matching value Yypz is obtained. The specific calculation formula is as follows:

[0032] Yypz=a1×Zlyz+a2×Xwmz+a3×Bxpd

[0033] Where a1, a2 and a3 are the weight coefficients of the extracted instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, respectively, and a1+a2+a3=1, 0<a1≤1, 0<a2≤1, 0<a3≤1;

[0034] The preset matching threshold Q is compared with the semantic matching value Yypz for evaluation. The specific evaluation contents are as follows:

[0035] When the semantic matching value Yypz ≥ matching threshold Q: it means that the semantic matching degree between the generated video content and the custom text is qualified, the preset requirements are met and the generated content meets the user's expectations;

[0036] When the semantic matching value Yypz < matching threshold Q: it means that the semantic matching degree between the generated video and the custom text is unqualified. At this time, the video generation process is adjusted, including readjustment of the text parsing.

[0037] Preferably, the video generation module is used to receive structured behavior instructions and model parameters from the semantic alignment module, parse and map them to the corresponding video generation algorithm, and generate behavior and action sequences that conform to preset semantics; then, multi-level video rendering technology is used to generate video content containing character movements, expressions and background details frame by frame, and a dynamic image enhancement algorithm is applied to process the generated video; finally, the video format is compressed and optimized to meet the requirements of different playback platforms.

[0038] Preferably, the real-time processing module includes a time monitoring unit, a speed control unit and a load evaluation unit;

[0039] The time monitoring unit is used to record the text parsing start time T1 and the video generation completion time T2, wherein the text parsing start time T1 is obtained by recording the time point when the text parsing starts, and the video generation completion time T2 is obtained by recording the time point when the video generation is completed; the total time required for the entire processing process Sxsc is calculated using the following formula:

[0040] Sxsc=T2-T1

[0041] The required total duration Sxsc is calculated by the preset reference duration Yjzc, and the duration difference Scy is output and evaluated. The specific calculation and evaluation contents are as follows:

[0042] Scy=Sxsc-Yjzc

[0043] in:

[0044] If the duration difference Scy>0, it means that the actual processing time exceeds the benchmark duration, indicating that the system response speed is abnormal. At this time, the response speed is adjusted through the speed control unit;

[0045] If the duration difference Scy≤0, it means that the actual processing time is less than or equal to the benchmark duration, indicating that the system response speed is normal.

[0046] Preferably, the speed control unit is used to adjust the response speed of each step in the video generation process, including scheduling and adjusting the processing time of links including image rendering, action generation, and expression capture;

[0047] The load evaluation unit is used to collect system load related data and external environment related data at the traffic hub peak period in real time, and calculate and evaluate the load evaluation parameter Fplz after dimensionless processing of the system load related data and the external environment related data; finally, the speed control strategy in the video generation process is corrected according to the evaluation content of the load evaluation parameter Fplz;

[0048] The specific calculation formula of the load evaluation parameter Fplz is as follows:

[0049]

[0050] In the formula, Rbm represents the area of ​​the fluctuation range of human traffic, Wyc represents the network delay index, and Zgr represents the noise interference index;

[0051] The preset load threshold W is compared with the load evaluation parameter Fplz for evaluation, and the specific contents are as follows:

[0052] If the load evaluation parameter Fplz ≤ load threshold W: it means that the system load is normal and the system processing capacity is normal;

[0053] If the load evaluation parameter Fplz> load threshold W: it means that the system load is abnormal, the system processing capacity is abnormal, and it is in an overload state;

[0054] Finally, the speed control strategy is modified as follows:

[0055] If the load is normal, maintain the current video generation speed and processing strategy, and continue normal content generation;

[0056] If the load is abnormal, reduce the processing load in the video generation process, including adjusting the video resolution, image details of each frame and processing complexity; prioritize system resources for critical tasks and reduce the priority of secondary tasks; dynamically adjust the parameters in the processing algorithm according to load changes, including rendering time and cache size.

[0057] Preferably, the warning display module includes a video comparison unit, a risk assessment unit and a warning notification unit;

[0058] The video comparison unit is used to compare the generated custom video with the real-time monitoring video in the security inspection system at the frame level, compare the changes in the behavior, expression and scene of the characters in the two videos through image analysis and behavior detection algorithms, and compare the displacement differences of key points frame by frame to obtain the character behavior difference index Rbdx through the following formula:

[0059]

[0060] Where N represents the number of key points detected in the two videos, (x i gen ,y i gen ) represents the coordinate position of the i-th key point in the generated video; (x i real ,y i real ) represents the coordinate position of the corresponding key point in the real-time monitoring video;

[0061] Then, by comparing the scene features of the generated video and the real-time surveillance video, the surveillance scene matching degree Jcpd is calculated. The specific calculation formula is as follows:

[0062]

[0063] Where M is the number of feature points in the scene, F j gen Indicates the value of the jth feature point in the generated video, F j real Indicates the value of the corresponding feature point in the real-time monitoring video.

[0064] Preferably, the risk assessment unit is used to extract the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd, calculate and evaluate to obtain the real-time risk index Rsx, and the specific calculation formula and evaluation content are as follows:

[0065] Rsx=b1×Rbdx+b2×(1-Jcpd)

[0066] Where b1 and b2 represent the weight coefficients of the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd, respectively, and b1+b2=1, 0<b1≤1, 0<b2≤1;

[0067] At the same time, the real-time risk index Rsx is evaluated by presetting the first risk threshold W1 and the second risk threshold W2, and the first risk threshold W1> the second risk threshold W2, specifically:

[0068] If the second risk threshold W2 ≥ the real-time risk index Rsx, it means that the real-time risk of the content generated by the video is 30% or less, and the warning level is marked as green;

[0069] If the first risk threshold W1 ≥ the real-time risk index Rsx > the second risk threshold W2, it means that the real-time risk of the content generated by the video is between 30% and 70%, and the warning level is marked as orange;

[0070] If the real-time risk index Rsx> the first risk threshold W1, it means that the real-time risk of the content generated by the video is above 70%, and the warning level is marked as red;

[0071] The warning notification unit informs the security personnel through interface prompts or SMS notifications according to the color status of the warning level output; at the same time, it generates a detailed warning report including specific risk sources, recommended response measures and related safety recommendations.

[0072] A method for generating a portrait video supporting user-defined text, comprising the following steps:

[0073] Step 1: Receive custom text input by the user, perform semantic analysis and translation on the text through natural language processing technology, generate structured text data, and provide accurate translation based on different languages ​​and cultural backgrounds to solve the accuracy and consistency issues of multi-language support;

[0074] Step 2: semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yyp, and evaluate it, and then adjust the semantic matching problem of text and image generation based on the evaluation content;

[0075] Step 3: Generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video;

[0076] Step 4: Used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity;

[0077] Step 5: Compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate an early warning report based on the video content and actual security inspection data, so as to promptly inform security inspectors of potential safety risks and take corresponding early warning measures.

[0078] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0079] 1. The present invention solves the semantic mismatch problem of text and image generation existing in the prior art; by introducing a multi-level text parsing and semantic alignment module, it can perform deep semantic analysis on the user-entered custom text in the text parsing stage, and generate accurate and consistent structured data in combination with cultural background and multilingual characteristics; in the semantic alignment process, the system uses a deep learning algorithm to accurately map the behavioral instructions and expression descriptions in the text to the video generation model, ensuring that the content of the generated video is highly matched with the text semantics, thereby effectively eliminating the generation deviation problem caused by semantic inconsistency, and improving the accuracy of the video content and user experience;

[0080] 2. The present invention also addresses common accuracy and consistency issues in multi-language support; by adding a translation processing unit to the text parsing module, the system can combine the language model and translation algorithm to perform multiple analyses and adjustments on the syntactic rules and cultural backgrounds of different languages, thereby achieving efficient and accurate multi-language translation; based on multi-version text generation and semantic consistency verification, the solution reduces ambiguity and errors in the translation process through iterative optimization, thereby providing standardized input data for subsequent video generation, ensuring that the needs of users of different languages ​​can be accurately expressed;

[0081] 3. The present invention significantly improves the real-time responsiveness of the system in high-load environments such as transportation hubs; through the time monitoring and response speed control of the entire video generation process by the real-time processing module, the system can timely evaluate and adjust the generation speed to ensure that efficient and stable generation performance can be maintained in complex traffic environments; at the same time, the load evaluation unit dynamically adjusts the video generation strategy by collecting external environmental data and system load-related parameters, thereby avoiding delays caused by system overload and ensuring that the system can quickly respond to user needs and generate high-quality video content during peak periods. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0083] Figure 1 It is a schematic diagram of the system framework structure of the present invention;

[0084] Figure 2 The figure is a schematic diagram of the process steps of the method of the present invention. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and drawings. The schematic implementation modes and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention. It should be noted that the present invention is already in the actual development and use stage.

[0086] Example 1

[0087] like Figure 1 As shown, this embodiment includes a portrait video generation system that supports user-defined text, which is characterized by: including a text parsing module, a semantic alignment module, a video generation module, a real-time processing module, a feedback adjustment module and an early warning display module;

[0088] The text parsing module is used to receive the custom text input by the user, and perform semantic analysis and translation processing on the text through natural language processing technology, generate structured text data, and provide accurate translation according to different languages ​​and cultural backgrounds to solve the accuracy and consistency problems of multi-language support;

[0089] The semantic alignment module is used to semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yypz, and evaluate it, and then adjust the semantic matching problem of text and image generation according to the evaluation content;

[0090] The video generation module is used to generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video;

[0091] The real-time processing module is used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the required total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity;

[0092] The warning display module is used to compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate a warning report based on the video content and actual security inspection data, so as to promptly inform security personnel of potential safety risks and take corresponding warning measures.

[0093] In this embodiment, by combining text parsing, semantic alignment and real-time processing, the semantic consistency, multilingual adaptability and system responsiveness of custom text and generated video content are significantly improved; compared with traditional solutions, the text parsing module of this system not only performs semantic analysis, but also considers accurate translation of different languages ​​and cultural backgrounds to ensure consistency of multi-language support; the semantic alignment module uses a deep learning algorithm to accurately map text and video behavior, solving the problem of semantic mismatch; the real-time processing module adjusts the video generation speed according to the dynamic environment during peak traffic hours, enhances responsiveness in complex environments, and further improves the stability and accuracy of video generation.

[0094] Example 2

[0095] The text parsing module includes a semantic analysis unit and a translation processing unit;

[0096] The semantic analysis unit is used to receive the user-defined text input by the user, convert the text information in natural language form into processable structured data, and perform word segmentation, part-of-speech tagging, syntactic analysis and semantic parsing on the text to identify the key content and instructions in the text; secondly, through natural language processing technology, the logical relationship of the sentence, the meaning of the instruction and the context association are analyzed, and after the core semantic elements are extracted, a semantic structure with a logical hierarchy is generated, and the sentences in the text are classified and annotated according to the semantic content to form a hierarchical text data structure;

[0097] The translation processing unit is used to translate the structured text data generated by the semantic analysis unit into multiple languages, and adjust the translation content according to the syntactic rules, word order, vocabulary selection and cultural background of different languages ​​to make it conform to the expression habits of the target language; the unit combines the language model and the translation algorithm to verify the semantic consistency of multiple languages, and through iterative optimization, it reduces the ambiguity and inconsistency problems caused by language conversion, and generates multi-version text data with cross-language applicability, providing a unified standardized input for subsequent semantic alignment and video generation.

[0098] In this embodiment, accurate parsing and translation optimization of multilingual custom texts are achieved through the semantic analysis unit and translation processing unit of the text parsing module; the semantic analysis unit can convert complex natural language texts into hierarchical structured data, and extract core semantics through steps such as word segmentation, syntactic analysis and semantic analysis to ensure accurate delivery of instructions; while the translation processing unit dynamically adjusts the translation results according to the syntactic rules and cultural backgrounds of different languages, reduces translation errors through semantic consistency verification, and generates standardized text data suitable for multilingual environments; this improvement effectively solves the problem of semantic inconsistency in cross-language use and enhances the multilingual adaptability and accuracy of the system.

[0099] Example 3

[0100] The semantic alignment module includes a semantic matching unit, a behavior mapping unit and a matching value calculation and evaluation unit;

[0101] The semantic matching unit is used to receive the structured text data generated by the text parsing module, and compare it with the preset video generation model layer by layer to achieve accurate semantic matching; secondly, the instructions, descriptions and keywords in the text are extracted through a deep learning algorithm, and correspond one by one with the character behavior and facial expression features in the video generation model, and the parsed semantic elements are mapped to executable actions or expression modules in the video generation model;

[0102] The behavior mapping unit is used to quantify the behavior description in the text based on the mapping result generated by the semantic matching unit, and map it to the video generation model in turn to generate behavior model data that can be used for video generation.

[0103] The matching value calculation and evaluation unit is used to compare and analyze the content generated by the video with the text semantics;

[0104] The user-defined text is analyzed for vocabulary and syntax through natural language processing technology to extract semantic content; then the weight of each instruction in the overall semantics is calculated, and finally the instruction semantic weight index Zlyz is generated. The specific calculation formula is as follows:

[0105]

[0106] Where ti is a specific instruction, d is a document, N is the total number of documents, and df(ti) is the number of documents containing the instruction;

[0107] Then, when generating the video, the actual generated behavior is compared with the expected behavior to calculate the behavior mapping accuracy index Xwmz. The specific calculation formula is as follows:

[0108]

[0109] Where A and B are vector representations of text behavior features and generated behavior features;

[0110] Secondly, in the process of generating videos, the expression-related data in the generated video frames are collected and compared with the emotional description in the text semantics, and the matching degree between the two is calculated to obtain the expression matching index Bxpd. The specific calculation formula is as follows:

[0111]

[0112] Where Et is the expression feature vector in the text, and Eg is the expression feature vector in the generated video;

[0113] Finally, by extracting the instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, and calculating, the semantic matching value Yypz is obtained. The specific calculation formula is as follows:

[0114] Yypz=a1×Zlyz+a2×Xwmz+a3×Bxpd

[0115] Where a1, a2 and a3 are the weight coefficients of the extracted instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, respectively, and a1+a2+a3=1, 0<a1≤1, 0<a2≤1, 0<a3≤1;

[0116] The preset matching threshold Q is compared with the semantic matching value Yypz for evaluation. The specific evaluation contents are as follows:

[0117] When the semantic matching value Yypz ≥ matching threshold Q: it means that the semantic matching degree between the generated video content and the custom text is qualified, the preset requirements are met, and the generated content meets the user's expectations;

[0118] When the semantic matching value Yypz < matching threshold Q: it means that the semantic matching degree between the generated video and the custom text is unqualified. At this time, the video generation process is adjusted, including readjustment of the text parsing.

[0119] In this embodiment, the instruction semantic weight index Zlyz is used to measure the importance of different instructions in the text in the overall semantics; by analyzing the weight of each instruction, it can be ensured that important instructions are executed more accurately during the video generation process, so that the generated video is more consistent with the user's expectations; this index can identify the core semantic content, optimize the priority of the generation process, and avoid the secondary content affecting the expression of key semantics;

[0120] The behavior mapping accuracy index Xwmz is used to evaluate the degree of match between the behavior instructions described in the text and the actual generated video behavior. By comparing the difference between the expected behavior and the generated behavior, the accuracy of the generated result can be quantified, and the index can be used to reflect the accuracy of the behavior mapping. This helps to evaluate whether the generated video truly restores the behavior instructions in the text, and make corresponding adjustments when it does not meet expectations.

[0121] The expression matching index Bxpd measures the degree of match between the expression of the characters in the generated video and the emotional description in the text; by collecting and comparing expression features, the accuracy of emotional expression can be quantified to ensure that the video content not only matches the text in terms of behavior, but also in terms of emotional expression. This is crucial to improving the realism and user satisfaction of video generation, especially in scenarios involving emotional expression.

[0122] Example 4

[0123] The video generation module is used to receive structured behavior instructions and model parameters from the semantic alignment module, parse and map them to the corresponding video generation algorithm, and generate behavior and action sequences that conform to the preset semantics; then, through multi-level video rendering technology, it generates video content containing character actions, expressions and background details frame by frame, and applies a dynamic image enhancement algorithm to process the generated video; finally, the video format is compressed and optimized to meet the requirements of different playback platforms.

[0124] In this embodiment, the improvement of the video generation module is reflected in receiving structured behavior instructions and model parameters, accurately parsing them and mapping them to an adaptive video generation algorithm, thereby generating action and behavior sequences that conform to preset semantics; compared with traditional video generation methods, this module optimizes frame by frame through multi-level video rendering technology, which is not only more realistic in terms of character movements and expression details, but also uses dynamic image enhancement algorithms to improve visual effects, ensuring that the generated video has high-quality expressiveness; finally, through format compression and optimization, the module can adapt to the requirements of multiple playback platforms, greatly improving the system's compatibility and video generation efficiency, and is particularly suitable for complex cross-platform application scenarios; this improvement not only improves the accuracy and visual effects of video generation, but also optimizes processing efficiency and enhances the adaptability of the system.

[0125] Example 5

[0126] The real-time processing module includes a time monitoring unit, a speed control unit and a load evaluation unit;

[0127] The time monitoring unit is used to record the text parsing start time T1 and the video generation completion time T2, wherein the text parsing start time T1 is obtained by recording the time point when the text parsing starts, and the video generation completion time T2 is obtained by recording the time point when the video generation is completed; the total time required for the entire processing process Sxsc is calculated using the following formula:

[0128] Sxsc=T2-T1

[0129] The required total duration Sxsc is calculated by the preset reference duration Yjzc, and the duration difference Scy is output and evaluated. The specific calculation and evaluation contents are as follows:

[0130] Scy=Sxsc-Yjzc

[0131] in:

[0132] If the duration difference Scy>0, it means that the actual processing time exceeds the benchmark duration, indicating that the system response speed is abnormal. At this time, the response speed is adjusted through the speed control unit;

[0133] If the duration difference Scy≤0, it means that the actual processing time is less than or equal to the benchmark duration, indicating that the system response speed is normal.

[0134] The speed control unit is used to adjust the response speed of each step in the video generation process, including scheduling and adjusting the processing time of links including image rendering, action generation, and expression capture;

[0135] The load evaluation unit is used to collect system load related data and external environment related data at the traffic hub peak period in real time, and calculate and evaluate the load evaluation parameter Fplz after dimensionless processing of the system load related data and the external environment related data; finally, the speed control strategy in the video generation process is corrected according to the evaluation content of the load evaluation parameter Fplz;

[0136] The specific calculation formula of the load evaluation parameter Fplz is as follows:

[0137]

[0138] In the formula, Rbm represents the area of ​​the fluctuation range of human traffic, Wyc represents the network delay index, and Zgr represents the noise interference index;

[0139] The preset load threshold W is compared with the load evaluation parameter Fplz for evaluation, and the specific contents are as follows:

[0140] If the load evaluation parameter Fplz ≤ load threshold W: it means that the system load is normal and the system processing capacity is normal;

[0141] If the load evaluation parameter Fplz> load threshold W: it means that the system load is abnormal, the system processing capacity is abnormal, and it is in an overload state;

[0142] Finally, the speed control strategy is modified as follows:

[0143] If the load is normal, maintain the current video generation speed and processing strategy, and continue normal content generation;

[0144] If the load is abnormal, reduce the processing load in the video generation process, including adjusting the video resolution, image details of each frame and processing complexity; prioritize system resources for critical tasks and reduce the priority of secondary tasks; dynamically adjust the parameters in the processing algorithm according to load changes, including rendering time and cache size.

[0145] In this embodiment, the real-time processing module optimizes system performance through time monitoring, speed control and load evaluation to ensure efficient and stable video generation process; by recording the start and completion time of text parsing and video generation, the processing time Sxsc is calculated, the system response speed is evaluated, and abnormal situations are adjusted in time; the speed control unit dynamically adjusts the processing time of image rendering, motion generation and expression capture to improve system processing efficiency; the load evaluation unit calculates the load parameter Fplz according to external environmental data such as peak traffic, network delay and noise, and adjusts the generation strategy in real time by comparing the load threshold; the improvement of this module lies in that through multi-level optimization, the system can still maintain efficient and stable content generation in complex environments, which is particularly suitable for high-load scenarios such as transportation hubs.

[0146] Example 6

[0147] The warning display module includes a video comparison unit, a risk assessment unit and a warning notification unit;

[0148] The video comparison unit is used to compare the generated custom video with the real-time monitoring video in the security inspection system at the frame level, compare the changes in the behavior, expression and scene of the characters in the two videos through image analysis and behavior detection algorithms, and compare the displacement differences of key points frame by frame to obtain the character behavior difference index Rbdx through the following formula:

[0149]

[0150] Where N represents the number of key points detected in the two videos, (x i gen ,y i gen) represents the coordinate position of the i-th key point in the generated video; (x i real ,y i real ) represents the coordinate position of the corresponding key point in the real-time monitoring video;

[0151] Then, by comparing the scene features of the generated video and the real-time surveillance video, the surveillance scene matching degree Jcpd is calculated. The specific calculation formula is as follows:

[0152]

[0153] Where M is the number of feature points in the scene, F j gen Indicates the value of the jth feature point in the generated video, F j real Indicates the value of the corresponding feature point in the real-time monitoring video;

[0154] The risk assessment unit is used to obtain the real-time risk index Rsx by calculating and evaluating by extracting the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd. The specific calculation formula and evaluation content are as follows:

[0155] Rsx=b1×Rbdx+b2×(1-Jcpd)

[0156] Where b1 and b2 represent the weight coefficients of the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd, respectively, and b1+b2=1, 0<b1≤1, 0<b2≤1;

[0157] At the same time, the real-time risk index Rsx is evaluated by presetting the first risk threshold W1 and the second risk threshold W2, and the first risk threshold W1> the second risk threshold W2, specifically:

[0158] If the second risk threshold W2 ≥ the real-time risk index Rsx, it means that the real-time risk of the content generated by the video is 30% or less, and the warning level is marked as green;

[0159] If the first risk threshold W1 ≥ the real-time risk index Rsx > the second risk threshold W2, it means that the real-time risk of the content generated by the video is between 30% and 70%, and the warning level is marked as orange;

[0160] If the real-time risk index Rsx> the first risk threshold W1, it means that the real-time risk of the content generated by the video is above 70%, and the warning level is marked as red;

[0161] The warning notification unit informs the security personnel through interface prompts or SMS notifications according to the color status of the warning level output; at the same time, it generates a detailed warning report including specific risk sources, recommended response measures and related safety recommendations.

[0162] In this embodiment, the character behavior difference index Rbdx reflects whether the behavior in the generated video truly and accurately matches the preset instructions and the character behavior in the actual scene; if the difference is too large, it may indicate that the behavior in the generated video is not in line with reality and there is a potential safety hazard; the monitoring scene matching degree Jcpd ensures that the generated video scene is consistent with the actual monitoring environment. If the scene matching degree is low, it may cause the video content to be inconsistent with the actual environment, affecting the accuracy of risk assessment.

[0163] Example 7

[0164] See also Figure 2 , a method for generating a portrait video supporting user-defined text, comprising the following steps:

[0165] Step 1: Receive custom text input by the user, perform semantic analysis and translation on the text through natural language processing technology, generate structured text data, and provide accurate translation based on different languages ​​and cultural backgrounds to solve the accuracy and consistency issues of multi-language support;

[0166] Step 2: semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yypz, and evaluate it, and then adjust the semantic matching problem of text and image generation based on the evaluation content;

[0167] Step 3: Generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video;

[0168] Step 4: Used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity;

[0169] Step 5: Compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate an early warning report based on the video content and actual security inspection data, so as to promptly inform security inspectors of potential safety risks and take corresponding early warning measures.

[0170] In this embodiment, the text parsing step uses natural language processing technology to perform semantic analysis and multilingual translation, ensuring that text data in different languages ​​and cultural backgrounds are accurate and consistent, solving the accuracy problem in multilingual support; secondly, by semantically aligning structured text data with the video generation model, the deep learning algorithm can accurately convert the instructions and descriptions in the text into corresponding character behaviors and facial expressions, thereby improving the consistency of the video content and text semantics and solving the semantic matching problem; thirdly, through real-time generation and image enhancement processing, the generated video not only conforms to the preset behavior model, but also optimizes the visual quality and detail performance, thereby improving the overall effect of the final video; fourthly, by real-time calculation and adjustment of the speed of the video generation process, the system response efficiency can be maintained in high-load environments such as transportation hubs, solving the problem of poor real-time responsiveness; finally, the generated video is compared and analyzed with the real-time monitoring data, potential safety risks can be discovered in a timely manner, and a detailed early warning report can be generated to provide effective early warning measures for security personnel, ensuring the safety and practicality of the generated content; these steps jointly solve the technical defects of semantic mismatch, insufficient multilingual support and slow real-time response, and improve the stability and reliability of the entire system.

[0171] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A portrait video generation system supporting user-defined text, characterized in that: It includes text parsing module, semantic alignment module, video generation module, real-time processing module and warning display module; The text parsing module is used to receive the custom text input by the user, and perform semantic analysis and translation processing on the text through natural language processing technology, generate structured text data, and provide accurate translation according to different languages ​​and cultural backgrounds to solve the accuracy and consistency problems of multi-language support; The semantic alignment module is used to semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yypz, and evaluate it, and then adjust the semantic matching problem of text and image generation according to the evaluation content; The video generation module is used to generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video; The real-time processing module is used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the required total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity; The warning display module is used to compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate a warning report based on the video content and actual security inspection data, so as to promptly inform security personnel of potential safety risks and take corresponding warning measures.

2. A portrait video generation system supporting user-defined text as claimed in claim 1, characterized in that: The text parsing module includes a semantic analysis unit and a translation processing unit; The semantic analysis unit is used to receive the user-defined text input by the user, convert the text information in natural language form into processable structured data, and perform word segmentation, part-of-speech tagging, syntactic analysis and semantic parsing on the text to identify the key content and instructions in the text; secondly, through natural language processing technology, the logical relationship of the sentence, the meaning of the instruction and the context association are analyzed, and after the core semantic elements are extracted, a semantic structure with a logical hierarchy is generated, and the sentences in the text are classified and annotated according to the semantic content to form a hierarchical text data structure; The translation processing unit is used to translate the structured text data generated by the semantic analysis unit into multiple languages, and adjust the translation content according to the syntactic rules, word order, vocabulary selection and cultural background of different languages ​​to make it conform to the expression habits of the target language; the unit combines the language model and the translation algorithm to verify the semantic consistency of multiple languages, and through iterative optimization, it reduces the ambiguity and inconsistency problems caused by language conversion, and generates multi-version text data with cross-language applicability, providing a unified standardized input for subsequent semantic alignment and video generation.

3. A portrait video generation system supporting user-defined text as claimed in claim 2, characterized in that: The semantic alignment module includes a semantic matching unit, a behavior mapping unit and a matching value calculation and evaluation unit; The semantic matching unit is used to receive the structured text data generated by the text parsing module, and compare it with the preset video generation model layer by layer to achieve accurate semantic matching; secondly, the instructions, descriptions and keywords in the text are extracted through a deep learning algorithm, and correspond one by one with the character behavior and facial expression features in the video generation model, and the parsed semantic elements are mapped to executable actions or expression modules in the video generation model; The behavior mapping unit is used to quantify the behavior description in the text based on the mapping result generated by the semantic matching unit, and map it to the video generation model in turn to generate behavior model data that can be used for video generation.

4. A portrait video generation system supporting user-defined text as claimed in claim 3, characterized in that: The matching value calculation and evaluation unit is used to compare and analyze the content generated by the video with the text semantics; The user-defined text is analyzed for vocabulary and syntax through natural language processing technology to extract semantic content; then the weight of each instruction in the overall semantics is calculated, and finally the instruction semantic weight index Zlyz is generated. The specific calculation formula is as follows: Where ti is a specific instruction, d is a document, N is the total number of documents, and df(ti) is the number of documents containing the instruction; When generating a video, the actual generated behavior is compared with the expected behavior, and the behavior mapping accuracy index Xwmz is calculated. The specific calculation formula is as follows: Where A and B are vector representations of text behavior features and generated behavior features; In the process of generating videos, the expression-related data in the generated video frames are collected and compared with the emotional description in the text semantics, and the matching degree between the two is calculated to obtain the expression matching index Bxpd. The specific calculation formula is as follows: Where Et is the expression feature vector in the text, and Eg is the expression feature vector in the generated video; Finally, by extracting the instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, and calculating, the semantic matching value Yypz is obtained. The specific calculation formula is as follows: Yypz=a1×Zlyz+a2×Xwmz+a3×Bxpd Where a1, a2 and a3 are the weight coefficients of the extracted instruction semantic weight index Zlyz, the behavior mapping accuracy parameter Xwmz and the expression matching parameter Bxpd, respectively, and a1+a2+a3=1, 0<a1≤1, 0<a2≤1, 0<a3≤1; The preset matching threshold Q is compared with the semantic matching value Yypz for evaluation. The specific evaluation contents are as follows: When the semantic matching value Yypz ≥ matching threshold Q: it means that the semantic matching degree between the generated video content and the custom text is qualified, the preset requirements are met and the generated content meets the user's expectations; When the semantic matching value Yypz is less than the matching threshold Q, it indicates that the semantic matching degree between the generated video and the custom text is unqualified. At this time, the video generation process is adjusted, including readjustment of the text parsing.

5. A portrait video generation system supporting user-defined text as claimed in claim 4, characterized in that: The video generation module is used to receive structured behavior instructions and model parameters from the semantic alignment module, parse and map them to the corresponding video generation algorithm, and generate behavior and action sequences that conform to the preset semantics; then, through multi-level video rendering technology, it generates video content containing character actions, expressions and background details frame by frame, and applies a dynamic image enhancement algorithm to process the generated video; finally, the video format is compressed and optimized to meet the requirements of different playback platforms.

6. A portrait video generation system supporting user-defined text as claimed in claim 5, characterized in that: The real-time processing module includes a time monitoring unit, a speed control unit and a load evaluation unit; The time monitoring unit is used to record the text parsing start time T1 and the video generation completion time T2, wherein the text parsing start time T1 is obtained by recording the time point when the text parsing starts, and the video generation completion time T2 is obtained by recording the time point when the video generation is completed; the total time required for the entire processing process Sxsc is calculated using the following formula: Sxsc=T2-T1 The required total duration Sxsc is calculated by the preset reference duration Yjzc, and the duration difference Scy is output and evaluated. The specific calculation and evaluation contents are as follows: Scy=Sxsc-Yjzc in: If the duration difference Scy>0, it means that the actual processing time exceeds the benchmark duration, indicating that the system response speed is abnormal. At this time, the response speed is adjusted through the speed control unit; If the duration difference Scy≤0, it means that the actual processing time is less than or equal to the benchmark duration, indicating that the system response speed is normal.

7. A portrait video generation system supporting user-defined text as claimed in claim 6, characterized in that: The speed control unit is used to adjust the response speed of each step in the video generation process, including scheduling and adjusting the processing time of links including image rendering, action generation, and expression capture; The load evaluation unit is used to collect system load related data and external environment related data at the traffic hub peak period in real time, and calculate and evaluate the load evaluation parameter Fplz after dimensionless processing of the system load related data and the external environment related data; finally, the speed control strategy in the video generation process is corrected according to the evaluation content of the load evaluation parameter Fplz; The specific calculation formula of the load evaluation parameter Fplz is as follows: In the formula, Rbm represents the area of ​​the fluctuation range of human traffic, Wyc represents the network delay index, and Zgr represents the noise interference index; The preset load threshold W is compared with the load evaluation parameter Fplz for evaluation, and the specific contents are as follows: If the load evaluation parameter Fplz ≤ load threshold W: it means that the system load is normal and the system processing capacity is normal; If the load evaluation parameter Fplz> load threshold W: it means that the system load is abnormal, the system processing capacity is abnormal, and it is in an overload state; Finally, the speed control strategy is modified as follows: If the load is normal, maintain the current video generation speed and processing strategy, and continue normal content generation; If the load is abnormal, reduce the processing load during video generation, including adjusting video resolution, image detail per frame, and processing complexity; Prioritize system resources for critical tasks and reduce the priority of secondary tasks; dynamically adjust parameters in the processing algorithm, including rendering time and cache size, according to load changes.

8. The portrait video generation system supporting user-defined text according to claim 7, characterized in that: The warning display module includes a video comparison unit, a risk assessment unit and a warning notification unit; The video comparison unit is used to compare the generated custom video with the real-time monitoring video in the security inspection system at the frame level, compare the changes in the behavior, expression and scene of the characters in the two videos through image analysis and behavior detection algorithms, and compare the displacement differences of key points frame by frame to obtain the character behavior difference index Rbdx through the following formula: Where N represents the number of key points detected in the two videos, (x i gen ,y i gen ) represents the coordinate position of the i-th key point in the generated video; (x i real ,y i real ) represents the coordinate position of the corresponding key point in the real-time monitoring video; Then, by comparing the scene features of the generated video and the real-time surveillance video, the surveillance scene matching degree Jcpd is calculated. The specific calculation formula is as follows: Where M is the number of feature points in the scene, F j gen Indicates the value of the jth feature point in the generated video, F j real Indicates the value of the corresponding feature point in the real-time monitoring video.

9. The portrait video generation system supporting user-defined text according to claim 8, characterized in that: The risk assessment unit is used to obtain the real-time risk index Rsx by calculating and evaluating by extracting the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd. The specific calculation formula and evaluation content are as follows: Rsx=b1×Rbdx+b2×(1-Jcpd) Where b1 and b2 represent the weight coefficients of the character behavior difference index Rbdx and the monitoring scene matching degree Jcpd, respectively, and b1+b2=1, 0<b1≤1, 0<b2≤1; At the same time, the real-time risk index Rsx is evaluated by presetting the first risk threshold W1 and the second risk threshold W2, and the first risk threshold W1> the second risk threshold W2, specifically: If the second risk threshold W2 ≥ the real-time risk index Rsx, it means that the real-time risk of the content generated by the video is 30% or less, and the warning level is marked as green; If the first risk threshold W1 ≥ the real-time risk index Rsx > the second risk threshold W2, it means that the real-time risk of the content generated by the video is between 30% and 70%, and the warning level is marked as orange; If the real-time risk index Rsx> the first risk threshold W1, it means that the real-time risk of the content generated by the video is above 70%, and the warning level is marked as red; The warning notification unit informs the security personnel through interface prompts or SMS notifications according to the color status of the warning level output; at the same time, it generates a detailed warning report including specific risk sources, recommended response measures and related safety recommendations.

10. A method for generating a portrait video supporting user-defined text, applied to a system for generating a portrait video supporting user-defined text as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Receive custom text input by the user, perform semantic analysis and translation on the text through natural language processing technology, generate structured text data, and provide accurate translation based on different languages ​​and cultural backgrounds to solve the accuracy and consistency issues of multi-language support; Step 2: semantically align the parsed structured text data with the video generation model, convert the instructions and descriptions in the text into matching character behaviors and facial expressions through a deep learning algorithm, calculate the consistency between the video content and the text semantics, obtain the semantic matching value Yypz, and evaluate it, and then adjust the semantic matching problem of text and image generation based on the evaluation content; Step 3: Generate a corresponding portrait video in real time according to the behavior model generated by the semantic alignment module, and perform image enhancement processing, including adjusting the visual quality and detail expression of the video; Step 4: Used to adjust the speed of the video generation process. After calculating the total time Sxsc required from text parsing to final video generation, the total time Sxsc is used to evaluate the response time of the system in the peak traffic environment of the transportation hub, and adjustments are made at the same time, including adjusting the video resolution, image details of each frame, and processing complexity; Step 5: Compare and analyze the generated video with the real-time monitoring data in the security inspection system, display the real-time effect of the user-defined content, and generate an early warning report based on the video content and actual security inspection data, so as to promptly inform security inspectors of potential safety risks and take corresponding early warning measures.

Images