Video sequence generation method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of image processing, and relates to a video sequence generation method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining text key information, and generating text description information according to the text key information; extracting spatial position relation information, and constructing an initial virtual scene according to the scene elements and the spatial position relation information; key content information is extracted, and effective virtual camera shooting parameters are acquired according to the key content information; converting a video sequence according to the effective virtual camera shooting parameters; judging whether the video sequence meets a preset quality threshold requirement or not; if not, the virtual camera shooting parameters are adjusted, and the video sequence is converted and adjusted according to the adjusted virtual camera shooting parameters; and extracting deep key information, searching deep scene elements according to the deep key information, and integrating the deep scene elements into the adjusted video sequence to obtain an effective video sequence. According to the method, the effective video sequence conforming to the content of the text information can be effectively and accurately generated according to the input text information.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, specifically to the field of digital medicine, and in particular to a method, device, computer equipment and storage medium for generating a video sequence. Background Art

[0002] In the era of rapidly developing digital medicine, the use of multimedia technology, especially video, to spread medical knowledge has become an important way to improve public health literacy and promote communication between doctors and patients. However, the knowledge system in the medical field is complex and highly professional, involving a large number of professional terms, complex physiological mechanisms and delicate surgical procedures. The accurate transmission of this information places extremely high demands on video production.

[0003] Traditionally, generating videos based on medical-related descriptive information relies on manual editing or simple text-to-video technology, but this method often faces huge challenges when processing medical texts. Medical texts contain many professional terms with precise meanings, and it is difficult for non-professionals to accurately understand their deep meanings, which directly leads to deviations in information interpretation during video production. In addition, the content of medical information is complex and varied, including disease diagnosis, treatment plans, surgical procedures, and other aspects. The accurate communication of information requires a high degree of professionalism and meticulousness. Any slight mistake may mislead the audience, affecting the correct dissemination of medical knowledge and the effectiveness of medical information query. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video sequence generation method, apparatus, computer equipment and storage medium to solve the problem that a valid video sequence that is consistent with the content of the text information cannot be accurately generated based on the input text information.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a method for generating a video sequence, which adopts the following technical solution:

[0006] Acquire text key information, acquire corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information;

[0007] Extracting spatial position relationship information of the text description information, and constructing an initial virtual scene according to the scene elements and the spatial position relationship information;

[0008] Extracting key content information of the text description information, and dynamically adjusting virtual camera parameters according to the key content information to obtain effective virtual camera parameters;

[0009] Converting the initial virtual scene into a video sequence according to the effective virtual camera parameters;

[0010] Determining whether the video sequence meets a preset quality threshold requirement;

[0011] If the video sequence does not meet the preset quality threshold requirement, extracting image defect information from the video sequence, adjusting the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, converting the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then judging whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement;

[0012] The deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from a pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence.

[0013] Furthermore, the step of obtaining text key information, obtaining corresponding scene elements according to the text key information, and generating text description information according to the scene elements and the text key information specifically includes:

[0014] Acquire input text information, perform word segmentation processing on the input text information, and obtain a text segmentation result;

[0015] Perform part-of-speech tagging and named entity recognition on the input text information according to the text segmentation result, and extract the text key information from the input text information;

[0016] Performing similarity matching in a pre-built scene element library according to the text key information to obtain the scene element;

[0017] The scene elements and the text key information are fused to obtain the text description information.

[0018] Furthermore, the step of extracting the spatial position relationship information of the text description information and constructing the initial virtual scene according to the scene elements and the spatial position relationship information specifically includes:

[0019] Performing first language processing on the text description information to extract scene element information and the spatial position relationship information;

[0020] Constructing a semantic association network between the scene elements according to the scene element information;

[0021] Calculating the three-dimensional coordinate position information of the scene element in the virtual scene according to the spatial position relationship information;

[0022] Generate a scene layout solution according to the semantic association network and the three-dimensional coordinate position information;

[0023] The initial virtual scene is constructed according to the scene layout scheme and the scene element information.

[0024] Furthermore, the step of extracting key content information of the text description information, dynamically adjusting virtual camera parameters according to the key content information, and obtaining effective virtual camera parameters specifically includes:

[0025] Performing second language processing on the text description information to extract the key content information;

[0026] Determining a scene focus area in the initial virtual scene according to the key content information;

[0027] The virtual camera parameters corresponding to the initial virtual scene are adjusted based on the scene focus area to obtain the effective virtual camera parameters.

[0028] Furthermore, the step of converting the initial virtual scene into a video sequence according to the effective virtual camera parameters specifically includes:

[0029] Acquire the effective virtual camera parameters, set a virtual camera in a preset virtual scene according to the effective virtual camera parameters, and adjust the viewing angle of the virtual camera to align with the initial virtual scene;

[0030] Performing light propagation simulation on the initial virtual scene based on a ray tracing algorithm to obtain pixel point illumination information;

[0031] Performing texture image mapping in the initial virtual scene according to the pixel illumination information to generate an illumination texture image;

[0032] The two-dimensional image frames corresponding to the virtual camera are rendered according to the illumination texture image, and the rendered two-dimensional image frames are encoded to generate the video sequence.

[0033] Furthermore, the step of determining whether the video sequence meets a preset quality threshold requirement specifically includes:

[0034] Dividing the video sequence into segments to obtain video frame segments;

[0035] Extracting key frames from the video frame segments to obtain video key frames;

[0036] Inputting the video key frames into a pre-trained neural network model for quality assessment to obtain a video quality score;

[0037] The video quality score is compared with a preset quality score threshold.

[0038] Furthermore, the step of extracting deep key information of the text key information, searching for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrating the deep scene elements into the adjusted video sequence to obtain a valid video sequence specifically includes:

[0039] Acquire the adjusted video sequence, and extract adjusted scene information from the adjusted video sequence;

[0040] Matching the adjustment scene information with the text key information, and extracting the matched information to obtain the deep key information, wherein the deep key information includes character action information, expression information, and scene change information;

[0041] According to the character action information, the expression information, and the scene change information, a matching search is performed in the scene element library to obtain the deep scene element;

[0042] Comparing the deep scene element with the video sequence to determine an insertion position and insertion time point of the deep scene element in the adjusted video sequence;

[0043] The deep scene elements are inserted into the adjusted video sequence frame by frame according to the insertion position and the insertion time point to obtain the effective video sequence.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a video sequence generation device, which adopts the following technical solution:

[0045] An information acquisition module, used to acquire text key information, acquire corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information;

[0046] A scene construction module, used to extract the spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information;

[0047] A parameter adjustment module, used for extracting key content information of the text description information, and dynamically adjusting virtual camera parameters according to the key content information to obtain effective virtual camera parameters;

[0048] A sequence conversion module, used for converting the initial virtual scene into a video sequence according to the effective virtual camera parameters;

[0049] A quality judgment module, used to judge whether the video sequence meets a preset quality threshold requirement;

[0050] an adjustment processing module, configured to extract image defect information from the video sequence if the video sequence does not meet the preset quality threshold requirement, and adjust the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, convert the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then determine whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement;

[0051] The video generation module is used to extract deep key information of the text key information, search for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrate the deep scene elements into the adjusted video sequence to obtain a valid video sequence.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0053] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any one of the above video sequence generation methods when executing the computer-readable instructions.

[0054] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0055] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video sequence generation method as described in any one of the above items are implemented.

[0056] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: the embodiments obtain text key information, obtain corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; extract spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information; extract key content information of the text description information, dynamically adjust virtual camera parameters according to the key content information, and obtain effective virtual camera parameters; convert the initial virtual scene into a video sequence according to the effective virtual camera parameters; and determine whether the video sequence meets the preset quality threshold requirements. ; If the video sequence does not meet the preset quality threshold requirement, then extract image defect information from the video sequence, and adjust the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, convert the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then determine whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; extract deep key information of the text key information, search for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrate the deep scene elements into the adjusted video sequence to obtain a valid video sequence. Thus, a valid video sequence that is consistent with the text information content is accurately generated based on the input text information. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0059] Figure 2 A flowchart of an embodiment of a method for generating a video sequence according to the present application;

[0060] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;

[0061] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;

[0062] Figure 5 yes Figure 2A flowchart of a specific implementation of step S30;

[0063] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S40;

[0064] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S50;

[0065] Figure 8 yes Figure 2 A flowchart of a specific implementation of step S80;

[0066] Fig. 9 is a structural schematic diagram of an embodiment of a video sequence generating device according to the present application;

[0067] Fig.10 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0069] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to non-related or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0070] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0071] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0072] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0073] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0074] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0075] It should be noted that the video sequence generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the video sequence generation device is generally arranged in the server / terminal device.

[0076] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0077] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating a video sequence according to the present application. The method for generating a video sequence comprises the following steps:

[0078] Step S10, acquiring text key information, acquiring corresponding scene elements according to the text key information, and generating text description information according to the scene elements and the text key information;

[0079] In this embodiment, the key information of the text is the key information extracted based on the input text information. The key information of the text includes the main characters, events, time, place, objects, etc. The scene elements include environmental elements: background, weather and lighting, time indication, props and decorations: props, decorations, character elements: character appearance, character actions, character relationships, sound effects and music: sound effects, music, visual effects: digital effects, transition effects, atmosphere and emotional elements: color and tone, light and shadow effects. The text description information includes: theme or central idea, content overview, key information points, detailed description, emotional color, background information, visual element description, etc.

[0080] Step S20, extracting the spatial position relationship information of the text description information, and constructing an initial virtual scene according to the scene elements and the spatial position relationship information;

[0081] In this embodiment, the spatial position relationship information includes: directional words (such as "above", "below", "left", "right", "in front", "behind", etc.), distance descriptions (such as "near", "far away", "close to", "several meters apart", etc.), layout descriptions (such as "center", "corner", "surround", "side by side", etc.), landmarks or reference objects (specific objects or places that can be used as references to determine the positions of other objects). The initial virtual scene is a three-dimensional virtual environment constructed based on text description information, spatial position relationship information, and scene elements. The initial virtual scene includes spatial layout: a basic spatial layout is set according to the spatial position relationship information in the text description, including the relative position, distance and direction between objects; scene elements: scene elements are the basic units that constitute the virtual scene, which can be static (such as buildings, trees, furniture, etc.) or dynamic (such as people, animals, vehicles, etc.); lighting and shadows: basic lighting and shadow effects in the virtual scene; materials and textures: the materials and textures assigned to scene elements; virtual camera perspective: the initial virtual scene will set a default camera perspective to display the whole or a specific part of the scene.

[0082] Step S30, extracting key content information of the text description information, and dynamically adjusting virtual camera parameters according to the key content information to obtain effective virtual camera parameters;

[0083] In this embodiment, the key content information includes key events: important events or plot turning points described in the text; main characters: main characters and their behaviors closely related to key events; important scenes: scenes specifically mentioned in the text that have an important impact on the development of the story; emotional climax: the part where the emotions expressed in the text reach a climax, such as tension, excitement, sadness, etc. The virtual camera parameters include the focal length and focus point parameters of the virtual camera.

[0084] Step S40, converting the initial virtual scene into a video sequence according to the effective virtual camera parameters;

[0085] In this embodiment, the initial virtual scene is converted into a video sequence using a three-dimensional rendering technology, which includes a ray tracing algorithm and a texture mapping algorithm. In this embodiment, an animation is set for the virtual camera according to effective virtual camera parameters, and the virtual scene is rendered according to the animation and viewing angle settings of the camera using the three-dimensional rendering technology by determining the starting position, target position, motion trajectory, speed curve, etc. of the camera, and a series of rendered images are exported as a video file, which is a video sequence.

[0086] Step S50, determining whether the video sequence meets a preset quality threshold requirement;

[0087] In this embodiment, the quality of the video key frames corresponding to the video sequence is evaluated by a pre-trained neural network model to obtain a video quality score, wherein the pre-trained neural network model can be a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM) model. A pre-trained CNN model is selected as a feature extractor, such as VGG, ResNet, etc., and an LSTM network is constructed after the CNN to process the key frame feature sequence extracted by the CNN, thereby constructing a neural network model, and then the model is trained and optimized by the key frames extracted by the sample data set, and finally the optimized model is used as the pre-trained neural network model. After obtaining the video quality score, it is compared with the preset quality score threshold to determine whether the video sequence meets the preset quality threshold requirement.

[0088] Step S60, if the video sequence meets the preset quality threshold requirement, extracting deep key information of the text key information, searching for corresponding deep scene elements from the scene element library according to the deep key information, and integrating the deep scene elements into the video sequence to obtain a valid video sequence;

[0089] In this embodiment, natural language processing (NLP technology) can be used to extract deep key information of text key information. The deep key information includes character action information, expression information, scene change information, etc. The deep scene elements include character action elements, expression elements, and scene change elements.

[0090] Step S70: if the video sequence does not meet the preset quality threshold requirement, extract image defect information from the video sequence, adjust the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, convert the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then determine whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement;

[0091] In this embodiment, each frame in the video sequence is analyzed by image processing technology (such as edge detection, texture analysis, color contrast, etc.) to identify defects in the video sequence image frames, including blur, noise, underexposure / overexposure, color distortion, geometric distortion, etc. The virtual camera parameters are adjusted according to the severity and type of the defect corresponding to the image defect information. For example, if the image is blurred, you can try to increase the focal length or reduce the aperture; if it is underexposed, you can increase the exposure compensation, etc., to adjust the virtual camera parameters.

[0092] Step S80, extracting deep key information of the text key information, searching for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrating the deep scene elements into the adjusted video sequence to obtain a valid video sequence.

[0093] In this embodiment, after obtaining a valid video sequence, the quality of the valid video sequence can also be evaluated to determine whether there are problems such as image distortion and discontinuous motion. If so, optimization adjustments are performed until the effective video quality score corresponding to the valid video sequence reaches a preset quality score threshold.

[0094] In this embodiment, the above method can be applied to a medical service system, in which a video sequence is generated by inputting medical text information. Specifically, in this embodiment, the medical service system can be one or more of a medical insurance system and a disease insurance system, and the text key information is the key information extracted from the medical text information. The text key information is stored in the medical insurance system and the disease insurance system and obtained from the database of the above system. The valid video sequence is generated by the above system through the method of this embodiment and stored in the system database for preservation.

[0095] The present embodiment obtains text key information, obtains corresponding scene elements according to the text key information, and generates text description information according to the scene elements and the text key information; extracts spatial position relationship information of the text description information, and constructs an initial virtual scene according to the scene elements and the spatial position relationship information; extracts key content information of the text description information, and dynamically adjusts virtual camera parameters according to the key content information to obtain effective virtual camera parameters; converts the initial virtual scene into a video sequence according to the effective virtual camera parameters; determines whether the video sequence meets the preset quality threshold requirement; if the video sequence does not meet the preset quality threshold requirement, the initial virtual scene is converted into a video sequence; and ... text key information If the preset quality threshold requirement is met, the image defect information is extracted from the video sequence, and the virtual camera parameters are adjusted according to the image defect information to obtain the adjusted virtual camera parameters, and the initial virtual scene is converted into an adjusted video sequence according to the adjusted virtual camera parameters, and then it is determined whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; the deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from the pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence. In this way, a valid video sequence that is consistent with the text information content is accurately generated according to the input text information.

[0096] Continue to refer Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:

[0097] Step S101, obtaining input text information, performing word segmentation processing on the input text information, and obtaining a text segmentation result;

[0098] In this embodiment, the input text information may be medical text information, which includes the cause of the disease, a brief description of the pathological physiological process, imaging examination results (such as descriptions and conclusions of X-rays, CT, and MRI), diagnostic basis (such as laboratory test results, imaging reports, etc.), surgical treatment plan (operation name, purpose, expected effect, risk), etc. The word segmentation processing of the input text information can be performed by a natural language processing (NLP) library or tool.

[0099] Step S102, performing part-of-speech tagging and named entity recognition on the input text information according to the text segmentation result, and extracting the text key information from the input text information;

[0100] In this embodiment, part-of-speech tagging is to add a part-of-speech tag to each word in the input text information, such as / n for nouns, / v for verbs, / a for adjectives, etc. Named entity recognition is to identify entities with specific meanings in the text, such as names of people, places, institutions, time, disease names, etc. For example, in medical text information, the disease name is recognized: acute myocardial infarction (marked as / nz, but more professional NER tools will give specific disease tags), treatment method: PCI treatment (also marked as / nz, more professional NER tools will recognize "PCI" as a treatment method or technology). After part-of-speech tagging and named entity recognition are completed, combined with the results of word segmentation, part-of-speech tagging and named entity recognition, we can extract key information from the text. For example, in medical text information, key information includes: patient symptoms (such as "persistent chest pain with dyspnea"), diagnosis results (such as "acute myocardial infarction"), and treatment plans (such as "PCI treatment").

[0101] Step S103, performing similarity matching in a pre-built scene element library according to the text key information to obtain the scene element;

[0102] In this embodiment, the pre-constructed scene element library refers to a library pre-defined according to different types of visual elements (such as backgrounds, props, character costumes), sound effects, music, etc. By determining the category and attribute of the text key information, a text key feature vector corresponding to the text key information is constructed, and then the scene elements in the pre-constructed scene element library are classified and attributed, so as to construct an element feature vector corresponding to the scene element, and by calculating the similarity between the text key feature vector and the element feature vector, the scene element corresponding to the text key information is determined according to the similarity.

[0103] Step S104: fusing the scene elements and the text key information to obtain the text description information.

[0104] In this embodiment, the fusion processing steps include: constructing a target scene that matches the text key information based on the category and attributes of the scene elements; fusing the target scene with the text key information to generate a target text description that meets the semantics of the input text information and the requirements of the target scene; and optimizing the target text description to obtain text description information.

[0105] This embodiment obtains input text information, performs word segmentation processing on the input text information, and obtains a text segmentation result; performs part-of-speech tagging and named entity recognition on the input text information according to the text segmentation result, and extracts the text key information from the input text information; performs similarity matching in a pre-built scene element library according to the text key information to obtain the scene element; and fuses the scene element and the text key information to effectively obtain text description information related to the key information and scene elements in the text, so as to facilitate the subsequent initial virtual scene construction process.

[0106] Continue to refer Figure 4 In some optional implementations of this embodiment, step S20 includes the following steps:

[0107] Step S201, performing first language processing on the text description information to extract scene element information and the spatial position relationship information;

[0108] In this embodiment, the first language processing is a natural language processing technology, and the scene element information and the spatial position relationship information in the text description information are extracted by the natural language processing technology.

[0109] Step S202, constructing a semantic association network between the scene elements according to the scene element information;

[0110] In this embodiment, knowledge graph technology is used to construct a semantic association network between scene elements. The construction of the semantic association network includes: defining scenes and elements, clarifying the scenes for constructing the semantic association network, and the elements contained in the scenes; extracting the semantic relationships between scene elements, such as "located in", "belonging to", "related to", etc.; constructing a knowledge graph based on the entities and concepts corresponding to the scene elements and the relationships between them; and obtaining the semantic association network between scene elements based on the constructed knowledge graph.

[0111] Step S203, calculating the three-dimensional coordinate position information of the scene element in the virtual scene according to the spatial position relationship information;

[0112] In this embodiment, a spatial reasoning algorithm is used to calculate the three-dimensional coordinate position information of scene elements in the virtual scene. The spatial reasoning algorithm uses geometric reasoning: the position is calculated through the principles of geometry (such as the Pythagorean theorem, trigonometric functions, etc.). The steps of calculating the three-dimensional coordinate position information by the spatial reasoning algorithm include initialization: selecting one or more elements with known positions as starting points (or anchor points), whose coordinates can be directly set or known; iterative calculation: for each element with unknown position, the spatial reasoning algorithm is used to calculate its three-dimensional coordinates according to its spatial relationship with the elements with known positions; verification and adjustment: after the calculation is completed, verify whether the result conforms to all given spatial position relationship information. If not, it may be necessary to adjust the algorithm parameters or reconsider the interpretation of the spatial relationship. When the calculation is completed, the three-dimensional coordinates finally output are used as the three-dimensional coordinate position information.

[0113] Step S204, generating a scene layout solution according to the semantic association network and the three-dimensional coordinate position information;

[0114] In this embodiment, a scene layout scheme is generated according to a scene layout optimization algorithm, and the generation of the scene layout scheme includes the following steps: generating an initial scene layout scheme according to a semantic association network and three-dimensional coordinate position information; evaluating the initial scene layout scheme using preset evaluation indicators (such as space utilization, traffic line length, visual comfort, etc.); iteratively optimizing the layout scheme using a scene layout optimization algorithm; in each iteration, the algorithm adjusts the position of elements according to the semantic association network and layout constraints to improve the evaluation indicators; judging whether the optimization process converges, that is, whether the evaluation indicators reach a preset threshold or whether the optimization progress becomes very slow; and using the optimized layout scheme as the scene layout scheme.

[0115] Step S205: constructing the initial virtual scene according to the scene layout solution and the scene element information.

[0116] In this embodiment, according to the scene layout plan, a three-dimensional model in the virtual scene is constructed by using three-dimensional modeling technology; according to the attribute information of the scene elements, an object placement optimization algorithm is used to determine the precise placement position and posture of the three-dimensional model in the virtual scene; the three-dimensional model is assembled according to the precise placement position and posture to obtain an initial virtual scene that conforms to the text description information.

[0117] This embodiment performs first language processing on the text description information to extract scene element information and the spatial position relationship information; constructs a semantic association network between the scene elements based on the scene element information; calculates the three-dimensional coordinate position information of the scene elements in the virtual scene based on the spatial position relationship information; generates a scene layout scheme based on the semantic association network and the three-dimensional coordinate position information; and constructs the initial virtual scene based on the scene layout scheme and the scene element information. This effectively realizes the construction of an initial virtual scene that conforms to the scene elements in the text based on the text description information, so as to facilitate the subsequent video sequence conversion processing.

[0118] Continue to refer Figure 5 In some optional implementations of this embodiment, step S30 includes the following steps:

[0119] Step S301, performing second language processing on the text description information to extract the key content information;

[0120] In this embodiment, the second language processing is a natural language processing technology, which extracts corresponding key content information from the text description information according to preset key content information extraction standards. The key content information extraction standards include summary statements: extracting sentences or phrases that can summarize the main theme of the text; key facts: listing the key facts, data or opinions mentioned in the text; important conclusions: if there are conclusions or suggestions in the text, they need to be specially extracted; key words: listing key words closely related to the text theme.

[0121] Step S302, determining a scene focus area in the initial virtual scene according to the key content information;

[0122] In this embodiment, the elements in the key content information are mapped to the initial virtual scene, so as to determine the position and orientation of each element in the initial virtual scene to determine the scene focus area. The determination of the scene focus area includes placing the key elements mentioned in the key content information at the visual center of the scene or at a position that is easy to notice, and organizing the related elements together to form functional areas, such as the operating area, diagnosis area, rest area, etc., according to the functions of the scene elements and the relationship between them, to determine the scene focus area.

[0123] Step S303: adjusting the virtual camera parameters corresponding to the initial virtual scene based on the scene focus area to obtain the effective virtual camera parameters.

[0124] In this embodiment, a virtual camera parameter adjustment technology is used to adjust virtual camera parameters, wherein the virtual camera parameters include the focal length and focus point parameters of the virtual camera. The adjustment of the focal length parameter is achieved through a focal length calculation algorithm, and a suitable focal length value is calculated according to the spatial position and depth information of the key content; the adjustment of the focus point parameter is achieved through a focus point positioning algorithm, and a suitable focus point is determined according to the spatial position of the key content.

[0125] This embodiment extracts the key content information by performing second language processing on the text description information; determines a scene focus area in the initial virtual scene based on the key content information; and adjusts virtual camera parameters corresponding to the initial virtual scene based on the scene focus area, thereby effectively obtaining effective virtual camera parameters for adjusting the virtual camera in the initial virtual scene, so as to facilitate subsequent video sequence conversion processing.

[0126] Continue to refer Figure 6 In some optional implementations of this embodiment, step S40 includes the following steps:

[0127] Step S401, obtaining the effective virtual camera parameters, setting a virtual camera in a preset virtual scene according to the effective virtual camera parameters, and adjusting the viewing angle of the virtual camera to align with the initial virtual scene;

[0128] In this embodiment, the effective virtual camera parameters include information such as the position, orientation, angle, focal length, etc. of the virtual camera. The steps of setting and adjusting the virtual camera include: adding a new virtual camera object to the scene through virtual scene editing software or tools; setting the position, orientation, and other parameters of the virtual camera to the effective values ​​corresponding to the effective virtual camera parameters.

[0129] Step S402, performing light propagation simulation on the initial virtual scene based on a ray tracing algorithm to obtain pixel point illumination information;

[0130] In this embodiment, the basic parameters of the ray tracer are set to initialize the ray tracer, and then for each pixel point within the field of view of the virtual camera, a ray of light is emitted from the position of the camera, so that the light propagates in the scene and interacts with objects in the scene, wherein the interaction types include reflection, refraction, scattering and absorption, etc., and the direction and intensity of the reflected, refracted or scattered light are calculated according to the material properties of the object and the type of light (such as direct light, indirect light, ambient light, etc.), and the information of all the light reaching the pixel points is collected, including their color, intensity and direction, and the comprehensive contribution of these light rays to the color of the pixel points is calculated according to the lighting model and material properties, and global lighting effects such as shadows, ambient occlusion (AO) and global illumination (GI) are considered to enhance the realism of the scene, and the above information is combined to obtain the pixel lighting information.

[0131] Step S403, performing texture image mapping in the initial virtual scene according to the pixel illumination information to generate an illumination texture image;

[0132] In this embodiment, the corresponding texture image is obtained by the attributes of the scene elements in the initial virtual scene, the pixel illumination information calculated by the ray tracing algorithm is merged with the texture image, and the texture image with the merged illumination information is mapped to the corresponding object in the initial virtual scene, and UV coordinates are defined for each object in the scene. According to the UV coordinates of the object and the viewing angle of the camera, the corresponding color values ​​are sampled from the texture image and applied to the surface of the object, thereby obtaining a lighting texture image.

[0133] Step S404: Render the two-dimensional image frame corresponding to the virtual camera according to the illumination texture image, and encode the rendered two-dimensional image frame to generate the video sequence.

[0134] In this embodiment, after rendering is completed, the generated virtual scene video can also be post-processed, such as color correction and special effects synthesis, to further improve the visual quality and artistic expression of the video, and finally output a video sequence that meets the required virtual scene.

[0135] This embodiment obtains the effective virtual camera parameters, sets a virtual camera in a preset virtual scene according to the effective virtual camera parameters, and adjusts the viewing angle of the virtual camera to align with the initial virtual scene; performs light propagation simulation on the initial virtual scene based on a ray tracing algorithm to obtain pixel illumination information; performs texture image mapping in the initial virtual scene according to the pixel illumination information to generate an illumination texture image; renders the two-dimensional image frame corresponding to the virtual camera according to the illumination texture image, and encodes the rendered two-dimensional image frame, thereby performing light simulation generation, texture image mapping, and texture image rendering according to the effective virtual camera parameters to generate an accurate and effective video sequence, which is convenient for subsequent quality threshold requirement judgment processing.

[0136] Continue to refer Figure 7 In some optional implementations of this embodiment, step S50 includes the following steps:

[0137] Step S501, segmenting the video sequence to obtain video frame segments;

[0138] In this embodiment, the video sequence can be segmented based on a fixed time interval (such as every 5 seconds), or the video segments can be dynamically segmented according to changes in video content (such as scene switching, motion changes, etc.), and the dynamic segmentation is implemented based on image comparison technology, such as histogram difference, feature point matching, etc. In this embodiment, each video frame segment includes at least one frame of image.

[0139] Step S502, extracting key frames from the video frame segments to obtain video key frames;

[0140] In this embodiment, key frames may be extracted from video frame segments based on clustering, by clustering the video frames and selecting the central frame of each cluster as the video key frame.

[0141] Step S503, inputting the video key frame into a pre-trained neural network model for quality assessment to obtain a video quality score;

[0142] In this embodiment, the pre-trained neural network model can be extracted from the database through the model extraction identifier, and the model extraction identifier is the unique identification information corresponding to the pre-trained neural network model, and the model extraction identifier can be a data ID or a UUID.

[0143] Step S504: compare the video quality score with a preset quality score threshold.

[0144] In this embodiment, the preset quality score threshold is a comparison standard value preset according to actual needs and application scenarios. By comparing the score value corresponding to the video quality score with the quality score threshold, it is determined whether the video sequence meets the preset quality threshold requirement. When the video quality score is greater than or equal to the quality score threshold, it means that the video sequence meets the preset quality threshold requirement. Otherwise, it means that the video sequence does not meet the preset quality threshold requirement.

[0145] This embodiment divides the video sequence into segments to obtain video frame segments; extracts key frames from the video frame segments to obtain video key frames; inputs the video key frames into a pre-trained neural network model for quality assessment to obtain a video quality score; and compares the video quality score with a preset quality score threshold. This effectively implements accurate quality assessment of the video sequence, making it convenient to subsequently implement corresponding processing steps based on the assessment results of the video sequence.

[0146] Continue to refer Figure 8 In some optional implementations of this embodiment, step S80 includes the following steps:

[0147] Step S801, obtaining the adjusted video sequence, and extracting the adjusted scene information from the adjusted video sequence;

[0148] In this embodiment, different scenes in the video may be identified by using video analysis technology (such as frame difference analysis, object detection, scene recognition algorithm, etc.), thereby extracting the identified scenes and obtaining adjustment scene information.

[0149] Step S802, matching the adjustment scene information with the text key information, and extracting the matched information to obtain the deep key information, wherein the deep key information includes character action information, expression information, and scene change information;

[0150] In this embodiment, a match search is performed in the text key information based on the information content corresponding to the adjusted scene information to extract key information such as character actions, expression changes, and scene changes related to the scene, thereby obtaining deep key information including character action information, expression information, and scene change information.

[0151] Step S803, performing a matching search in the scene element library according to the character action information, the expression information, and the scene change information to obtain the deep scene element;

[0152] In this embodiment, a search is performed in a pre-built scene element library based on the character action information, expression information, and scene change information to obtain character action elements, expression elements, and scene change elements corresponding to the character action information, expression information, and scene change information. The character action elements, expression elements, and scene change elements are the deep scene elements.

[0153] Step S804, comparing the deep scene element with the video sequence to determine the insertion position and insertion time point of the deep scene element in the adjusted video sequence;

[0154] In this embodiment, the deep scene elements are screened and the elements that do not match or conflict with the scenes in the generated video sequence are eliminated to obtain the final set of scene elements that need to be fused. The screened set of deep scene elements is compared with the generated video sequence to determine the insertion position and time point of each deep scene element in the video sequence.

[0155] Step S805 , inserting the deep scene elements into the adjusted video sequence frame by frame according to the insertion position and the insertion time point to obtain the effective video sequence.

[0156] In this embodiment, the acquisition of a valid video sequence includes the following steps: according to the determined insertion position and time point, the deep scene elements are inserted frame by frame into the generated video sequence using animation fusion technology, and smooth transition processing is performed to ensure seamless connection between the elements and the original video sequence; the quality of the fused video sequence is evaluated to determine whether there are problems such as image distortion and discontinuous motion. If so, local optimization adjustments are performed until a preset quality threshold is reached; and the final optimized video sequence is output as a valid video sequence.

[0157] This embodiment obtains the adjusted video sequence and extracts the adjusted scene information from the adjusted video sequence; matches the adjusted scene information with the text key information and extracts the matched information to obtain the deep key information, wherein the deep key information includes character action information, expression information, and scene change information; performs matching and searching in the scene element library according to the character action information, the expression information, and the scene change information to obtain the deep scene elements; compares the deep scene elements with the video sequence to determine the insertion position and insertion time point of the deep scene elements in the adjusted video sequence; inserts the deep scene elements frame by frame into the adjusted video sequence according to the insertion position and the insertion time point, thereby effectively merging the deep scene elements and the adjusted video sequence to obtain a valid video sequence.

[0158] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0159] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0160] Further references Fig. 9 , as a response to the above Figure 1 The present application provides an embodiment of a video sequence generation device, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0161] like Fig. 9 As shown, the video sequence generation device 900 described in this embodiment includes: an information acquisition module 901, a scene construction module 902, a parameter adjustment module 903, a sequence conversion module 904, a quality judgment module 905, an adjustment processing module 906, a video output module 907, and a video generation module 908. Among them:

[0162] The information acquisition module 901 is used to acquire text key information, acquire corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information;

[0163] A scene construction module 902 is used to extract the spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information;

[0164] A parameter adjustment module 903, used to extract key content information of the text description information, and dynamically adjust virtual camera parameters according to the key content information to obtain effective virtual camera parameters;

[0165] A sequence conversion module 904, configured to convert the initial virtual scene into a video sequence according to the effective virtual camera parameters;

[0166] The quality judgment module 905 is used to judge whether the video sequence meets the preset quality threshold requirement;

[0167] The video output module 906 is used to extract the deep key information of the text key information if the video sequence meets the preset quality threshold requirement, search for the corresponding deep scene element from the scene element library according to the deep key information, and integrate the deep scene element into the video sequence to obtain a valid video sequence;

[0168] The adjustment processing module 907 is used to extract image defect information from the video sequence if the video sequence does not meet the preset quality threshold requirement, and adjust the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, convert the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then determine whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement;

[0169] The video generation module 908 is used to extract deep key information of the text key information, search for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrate the deep scene elements into the adjusted video sequence to obtain a valid video sequence.

[0170] By adopting the above-mentioned video sequence generation device, this embodiment can obtain text key information, obtain corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; extract spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information; extract key content information of the text description information, and dynamically adjust virtual camera parameters according to the key content information to obtain effective virtual camera parameters; convert the initial virtual scene into a video sequence according to the effective virtual camera parameters; determine whether the video sequence meets the preset quality threshold requirement; if the video sequence meets the preset quality threshold requirement, the video sequence can be converted into a video sequence according to the effective virtual camera parameters; and ... If the video sequence does not meet the preset quality threshold requirement, image defect information is extracted from the video sequence, and the virtual camera parameters are adjusted according to the image defect information to obtain adjusted virtual camera parameters, and the initial virtual scene is converted into an adjusted video sequence according to the adjusted virtual camera parameters, and then it is determined whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from the pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence. In this way, a valid video sequence that is consistent with the text information content is accurately generated according to the input text information.

[0171] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0172] The computer device 9 includes a memory 91, a processor 92, and a network interface 93 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 9 with components 91-93, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0173] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0174] The memory 91 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 91 can be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 91 can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 91 can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 91 is generally used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions of the video sequence generation method, etc. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or are to be output.

[0175] The processor 92 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 92 is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 92 is used to run computer-readable instructions stored in the memory 91 or process data, such as computer-readable instructions for running the video sequence generation method.

[0176] The network interface 93 may include a wireless network interface or a wired network interface. The network interface 93 is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0177] By adopting the above-mentioned computer device, this embodiment can obtain text key information, obtain corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; extract spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information; extract key content information of the text description information, and dynamically adjust virtual camera parameters according to the key content information to obtain effective virtual camera parameters; convert the initial virtual scene into a video sequence according to the effective virtual camera parameters; determine whether the video sequence meets the preset quality threshold requirement; if the video If the sequence does not meet the preset quality threshold requirement, the image defect information is extracted from the video sequence, and the virtual camera parameters are adjusted according to the image defect information to obtain the adjusted virtual camera parameters, and the initial virtual scene is converted into an adjusted video sequence according to the adjusted virtual camera parameters, and then it is determined whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; the deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from the pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence. In this way, a valid video sequence that is consistent with the text information content is accurately generated based on the input text information.

[0178] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the video sequence generation method as described above.

[0179] By adopting the above-mentioned computer-readable storage medium, this embodiment can obtain text key information, obtain corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; extract spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information; extract key content information of the text description information, and dynamically adjust virtual camera parameters according to the key content information to obtain effective virtual camera parameters; convert the initial virtual scene into a video sequence according to the effective virtual camera parameters; determine whether the video sequence meets the preset quality threshold requirement; if the If the video sequence does not meet the preset quality threshold requirement, the image defect information is extracted from the video sequence, and the virtual camera parameters are adjusted according to the image defect information to obtain the adjusted virtual camera parameters, and the initial virtual scene is converted into an adjusted video sequence according to the adjusted virtual camera parameters, and then it is determined whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; the deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from the pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence. In this way, a valid video sequence that is consistent with the text information content is accurately generated based on the input text information.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0181] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0182] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A video sequence generation method, characterized in that: The steps include: Acquire text key information, acquire corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; Extracting spatial position relationship information of the text description information, and constructing an initial virtual scene according to the scene elements and the spatial position relationship information; Extracting key content information of the text description information, and dynamically adjusting virtual camera parameters according to the key content information to obtain effective virtual camera parameters; Converting the initial virtual scene into a video sequence according to the effective virtual camera parameters; Determining whether the video sequence meets a preset quality threshold requirement; If the video sequence does not meet the preset quality threshold requirement, extracting image defect information from the video sequence, adjusting the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, converting the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then judging whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; The deep key information of the text key information is extracted, and the corresponding deep scene elements are searched from a pre-built scene element library according to the deep key information, and the deep scene elements are integrated into the adjusted video sequence to obtain a valid video sequence.

2. The video sequence generation method according to claim 1, characterized in that: The step of acquiring text key information, acquiring corresponding scene elements according to the text key information, and generating text description information according to the scene elements and the text key information specifically includes: Acquire input text information, perform word segmentation processing on the input text information, and obtain a text segmentation result; Perform part-of-speech tagging and named entity recognition on the input text information according to the text segmentation result, and extract the text key information from the input text information; Performing similarity matching in a pre-built scene element library according to the text key information to obtain the scene element; The scene elements and the text key information are fused to obtain the text description information.

3. The video sequence generation method according to claim 1, characterized in that: The step of extracting the spatial position relationship information of the text description information and constructing an initial virtual scene according to the scene elements and the spatial position relationship information specifically includes: Performing first language processing on the text description information to extract scene element information and the spatial position relationship information; Constructing a semantic association network between the scene elements according to the scene element information; Calculating the three-dimensional coordinate position information of the scene element in the virtual scene according to the spatial position relationship information; Generate a scene layout solution according to the semantic association network and the three-dimensional coordinate position information; The initial virtual scene is constructed according to the scene layout scheme and the scene element information.

4. The video sequence generation method according to claim 1, characterized in that: The step of extracting key content information of the text description information, dynamically adjusting virtual camera parameters according to the key content information, and obtaining effective virtual camera parameters specifically includes: Performing second language processing on the text description information to extract the key content information; Determining a scene focus area in the initial virtual scene according to the key content information; The virtual camera parameters corresponding to the initial virtual scene are adjusted based on the scene focus area to obtain the effective virtual camera parameters.

5. The video sequence generation method according to claim 1, characterized in that: The step of converting the initial virtual scene into a video sequence according to the effective virtual camera parameters specifically includes: Acquire the effective virtual camera parameters, set a virtual camera in a preset virtual scene according to the effective virtual camera parameters, and adjust the viewing angle of the virtual camera to align with the initial virtual scene; Performing light propagation simulation on the initial virtual scene based on a ray tracing algorithm to obtain pixel point illumination information; Performing texture image mapping in the initial virtual scene according to the pixel illumination information to generate an illumination texture image; The two-dimensional image frames corresponding to the virtual camera are rendered according to the illumination texture image, and the rendered two-dimensional image frames are encoded to generate the video sequence.

6. The video sequence generation method according to claim 1, characterized in that: The step of determining whether the video sequence meets a preset quality threshold requirement specifically includes: Dividing the video sequence into segments to obtain video frame segments; Extracting key frames from the video frame segments to obtain video key frames; Inputting the video key frames into a pre-trained neural network model for quality assessment to obtain a video quality score; The video quality score is compared with a preset quality score threshold.

7. The video sequence generation method according to claim 1, characterized in that: The step of extracting deep key information of the text key information, searching for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrating the deep scene elements into the adjusted video sequence to obtain a valid video sequence specifically includes: Acquire the adjusted video sequence, and extract adjusted scene information from the adjusted video sequence; Matching the adjustment scene information with the text key information, and extracting the matched information to obtain the deep key information, wherein the deep key information includes character action information, expression information, and scene change information; According to the character action information, the expression information, and the scene change information, a matching search is performed in the scene element library to obtain the deep scene element; Comparing the deep scene element with the video sequence to determine an insertion position and insertion time point of the deep scene element in the adjusted video sequence; The deep scene elements are inserted into the adjusted video sequence frame by frame according to the insertion position and the insertion time point to obtain the effective video sequence.

8. A video sequence generating device, characterized in that: include: An information acquisition module, used to acquire text key information, acquire corresponding scene elements according to the text key information, and generate text description information according to the scene elements and the text key information; A scene construction module, used to extract the spatial position relationship information of the text description information, and construct an initial virtual scene according to the scene elements and the spatial position relationship information; A parameter adjustment module, used for extracting key content information of the text description information, and dynamically adjusting virtual camera parameters according to the key content information to obtain effective virtual camera parameters; A sequence conversion module, used for converting the initial virtual scene into a video sequence according to the effective virtual camera parameters; A quality judgment module, used to judge whether the video sequence meets a preset quality threshold requirement; an adjustment processing module, configured to extract image defect information from the video sequence if the video sequence does not meet the preset quality threshold requirement, and adjust the virtual camera parameters according to the image defect information to obtain adjusted virtual camera parameters, convert the initial virtual scene into an adjusted video sequence according to the adjusted virtual camera parameters, and then determine whether the adjusted video sequence meets the preset quality threshold requirement, until the adjusted video sequence meets the preset quality threshold requirement; The video generation module is used to extract deep key information of the text key information, search for corresponding deep scene elements from a pre-built scene element library according to the deep key information, and integrate the deep scene elements into the adjusted video sequence to obtain a valid video sequence.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the video sequence generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the video sequence generation method according to any one of claims 1 to 7 are implemented.

Images