Three-dimensional scene sequence generation method and device, equipment and storage medium

Abstract

The invention discloses a three-dimensional scene sequence generation method and device, equipment and a storage medium, and relates to the technical field of three-dimensional digital twinning, and the method comprises the steps: obtaining a preset target document created by a target user, and extracting a corresponding movement type and a target object name of a virtual camera in a preset virtual three-dimensional space from the target document; obtaining feature information of the virtual three-dimensional space, matching information of the name of the target object, a key object in the target object and a camera motion path of the virtual camera in the virtual three-dimensional space by using a large language model; and constructing target sequence data based on the target object, the key object, the moving type, the matching information and the camera motion path, and generating a target three-dimensional scene sequence according to the target sequence data. The target sequence data is constructed by using the commentary document created by the user and the large language model, and the user can create the three-dimensional scene sequence only by the text editing ability, so that the editing difficulty of the user is reduced.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional digital twin technology, and in particular to a three-dimensional scene sequence generation method, device, equipment and storage medium. Background Art

[0002] A sequence is a data set consisting of bound objects, their tracks, and one or more keyframes on the tracks. It can be edited or played using the sequence engine. Unlike videos, 3D world sequences can be paused or stopped during playback to switch to normal roaming operation mode. Sequence playback is driven in real time by the sequence engine in the 3D engine, while videos are driven by the video player.

[0003] At present, the method for obtaining a 3D scene sequence is to generate it using a 3D non-linear sequence editor. The specific process of this method is to add a 3D object track on the basis of a 2D non-linear video editor, and edit the 3D scene sequence to finally obtain the target 3D scene sequence. Commonly used 3D non-linear editors include the non-linear sequence editor (Sequencer) that comes with the Unreal Engine and the animation (Animation) editor that comes with the Unity engine. Due to the numerous and complex functions of the 3D non-linear sequence editor, obtaining a 3D scene sequence in this way is difficult to get started with and takes a long time to create a sequence. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a 3D scene sequence generation method, device, equipment and storage medium, which can obtain a target document edited by a user and generate a 3D scene sequence based on the target document and a virtual 3D space, so that the user only needs to edit the document and can generate a 3D scene sequence without operating professional software, thereby reducing the operational threshold for users to generate 3D scene sequences. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a method for generating a three-dimensional scene sequence, comprising:

[0006] Acquire a preset target document created by a target user based on a preset virtual three-dimensional space, and extract a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document;

[0007] Acquire feature information of the preset virtual three-dimensional space, input the feature information and the target name into a preset large language model to obtain matching information between the feature information and the target name, and input the target name and a target description corresponding to the target object in the preset target document into the preset large language model to obtain a key object in the target object;

[0008] Determining a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and the preset order of the target objects in the preset target document;

[0009] Target sequence data corresponding to the target object is constructed according to the target object, the key object, the movement type, the matching information and the camera motion path, and a target three-dimensional scene sequence is generated according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

[0010] Optionally, extracting the movement type of the preset virtual camera in the preset virtual three-dimensional space from the preset target document includes:

[0011] Inputting the target interpretation words in the preset target document into the preset large language model, and setting the output format of the preset large language model by using the large language model prompt word mechanism;

[0012] The movement type of the preset virtual camera in the preset virtual three-dimensional space is extracted by utilizing the preset large language model according to the output format and the target commentary.

[0013] Optionally, the determining, based on the matching information and a preset order of the target objects in the preset target document, a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space includes:

[0014] adding a tag including the target name and the preset order to the target object in the preset virtual three-dimensional space according to the matching information and the preset order of the target objects in the preset target document;

[0015] Placing a target vector in the preset virtual three-dimensional space based on the mark, and sequentially generating a target direction indicating geometric body and a connecting line between the target direction geometric body and a previous target direction geometric body according to a vector starting point and a vector direction of the target vector;

[0016] The camera motion path of the preset virtual camera for recording the target object in the preset virtual three-dimensional space is determined according to the target direction indicating geometric body and the connecting line.

[0017] Optionally, constructing target sequence data corresponding to the target object includes:

[0018] Determining a moving speed of the preset virtual camera in the preset virtual three-dimensional space on the camera motion path based on the movement type corresponding to the preset virtual camera;

[0019] The moving speed and the length of the camera movement path are used to determine the moving time of the preset virtual camera on the camera movement path, so as to construct the target sequence data corresponding to the preset virtual three-dimensional space according to the moving time.

[0020] Optionally, constructing target sequence data corresponding to the target object includes:

[0021] Determine the audio duration corresponding to a single commentary word according to the file duration of the target audio file and the number of words in the target commentary word;

[0022] The playing time required for playing the target audio file to the position of the key commentary is determined according to the audio duration corresponding to the single commentary and the position of the key commentary corresponding to the key object in the target commentary, and the target sequence data corresponding to the preset virtual three-dimensional space is constructed according to the playing time.

[0023] Optionally, the three-dimensional scene sequence generation method further includes:

[0024] Determining whether the camera motion path overlaps with the model in the preset virtual three-dimensional space;

[0025] If the camera motion path overlaps with a preset collision model in the preset virtual three-dimensional space, when the preset virtual camera moves to the position of the preset collision model, the preset collision model is set to be transparent;

[0026] If the camera motion path overlaps with a model other than the preset collision model in the preset virtual three-dimensional space, a new camera motion path is calculated using a preset three-dimensional path-finding algorithm.

[0027] Optionally, generating a target three-dimensional scene sequence according to the target sequence data includes:

[0028] A corresponding sequence playback engine is developed based on the target sequence data, and the target sequence data is parsed using the sequence playback engine to generate the target three-dimensional scene sequence corresponding to the target sequence data according to the corresponding parsing result.

[0029] In a second aspect, the present application provides a three-dimensional scene sequence generation device, comprising:

[0030] A data extraction module is used to obtain a preset target document created by a target user based on a preset virtual three-dimensional space, and extract a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document;

[0031] A key object acquisition module is used to acquire feature information of the preset virtual three-dimensional space, input the feature information and the target name into a preset large language model to acquire matching information between the feature information and the target name, and input the target name and a target description corresponding to the target object in the preset target document into the preset large language model to acquire key objects in the target object;

[0032] A motion path determination module, configured to determine a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and a preset order of the target objects in the preset target document;

[0033] A three-dimensional scene sequence generation module is used to construct target sequence data corresponding to the target object according to the target object, the key object, the movement type, the matching information and the camera motion path, and generate a target three-dimensional scene sequence according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

[0034] In a third aspect, the present application provides an electronic device, including:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the aforementioned three-dimensional scene sequence generation method.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned three-dimensional scene sequence generation method when executed by a processor.

[0038] The present application first obtains a preset target document created by a target user based on a preset virtual three-dimensional space, and extracts the movement type of a preset virtual camera in the preset virtual three-dimensional space and the target name of a target object in the preset virtual three-dimensional space from the preset target document, then obtains feature information of the preset virtual three-dimensional space, inputs the feature information and the target name into a preset large language model to obtain matching information between the feature information and the target name, and inputs the target name and a target commentary corresponding to the target object in the preset target document into the preset large language model to obtain a key object in the target object, and then determines a camera motion path of the preset virtual camera for recording the target object in the preset virtual three-dimensional space based on the matching information and a preset order of the target objects in the preset target document, and finally constructs target sequence data corresponding to the target object according to the target object, the key object, the movement type, the matching information and the camera motion path, and generates a target three-dimensional scene sequence according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document. It can be seen that the present application obtains the target document edited by the user, constructs target sequence data according to the target object, key object, transition type, matching information and camera motion path obtained by the target document and the large language model, and constructs a three-dimensional scene sequence through the target sequence data, so that the user does not need to use a three-dimensional non-linear sequence editor to generate a three-dimensional scene sequence, but only needs to edit the target document to generate a three-dimensional scene sequence using the large language model, which reduces the user's operating difficulty and improves the efficiency of generating three-dimensional scene sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 A schematic diagram of a three-dimensional scene sequence generation method disclosed in this application;

[0041] Figure 2 A flow chart of a three-dimensional scene sequence generation method disclosed in this application;

[0042] Figure 3 A flowchart of a specific three-dimensional scene sequence generation method disclosed in this application;

[0043] Figure 4A flow chart of a large language model training method disclosed in this application;

[0044] Figure 5 This is a schematic diagram of the structure of a three-dimensional scene sequence generation device disclosed in this application;

[0045] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] At present, the method of generating a 3D scene sequence is to generate it manually using a 3D non-linear sequence editor. Since the functions of a 3D non-linear sequence editor are numerous and complex, it is difficult to get started with this method and it takes a long time to create a sequence. To this end, the present application provides a 3D scene sequence generation method, which generates a 3D scene sequence by using a user to edit a large document and a large language model, thereby reducing the operational threshold for users to generate a 3D scene sequence.

[0048] See also Figure 1 As shown, an embodiment of the present invention discloses a method for generating a three-dimensional scene sequence, comprising:

[0049] Step S11, obtaining a preset target document created by a target user based on a preset virtual three-dimensional space, and extracting a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document.

[0050] In this embodiment, the process of extracting the movement type of the virtual camera in the virtual three-dimensional space from the target document specifically includes: inputting the target commentary in the target document into the large language model, and using the large language model prompt word mechanism to set the output format of the large language model; using the large language model and extracting the movement type of the virtual camera in the preset virtual three-dimensional space, that is, the operation type, according to the output format and the target commentary, and the result is temporarily stored after confirmation by the user; wherein the movement type of the virtual camera includes flying, walking and teleporting. In addition, the commentary document contains a series of theme scenes and corresponding commentaries, and the format is as follows:

[0051] [{title:“Theme Scene 1”,description:“Explanation of Theme Scene 1”},

[0052] {title:“Theme Scene 2”, description:“Explanation of Theme Scene 2”},

[0053] {title:"Theme Scene 3",description:"Explanation of Theme Scene 3"},……]

[0054] The system directly extracts all the target objects involved in the document, that is, the theme scene in the virtual three-dimensional space, that is, the title part of the explanation document, from the explanation document created by the user, that is, the target document. By obtaining the movement type of the virtual camera based on the target document and using the large language model, the operation of extracting the movement type of the virtual camera is simplified and the work efficiency is improved.

[0055] Step S12, obtaining feature information of the preset virtual three-dimensional space, inputting the feature information and the target name into a preset large language model to obtain matching information between the feature information and the target name, and inputting the target name and the target description corresponding to the target object in the preset target document into the preset large language model to obtain key objects in the target object.

[0056] In this embodiment, it is first necessary to obtain the feature information of the virtual three-dimensional space, that is, the attribute information of each model in the three-dimensional scene, including classification information (floor, roof, wall, HVAC, water supply and drainage, electrical, equipment, air conditioning, support and hanger, etc.), name (air conditioning room, pipe gallery, power center, coiling workshop, silk making workshop, raw material warehouse, finished product warehouse, etc.); send the classification information and name in the scene information and the target name of the target object in the target document to the large language model for matching, so as to obtain the matching information between the scene information and the target name of the target object. Before matching, it is also necessary to set the prompt word of the large language model to determine the output format of the large language model according to the prompt word; wherein, the above matching information refers to which part of the three-dimensional scene the explanation target corresponding to each segment of the explanation word is. For example, the part of the explanation word describing the "forging workshop" actually corresponds to the "hot processing" area in the three-dimensional scene. Similarly, we first need to use the prompt word mechanism of the large language model to set the output format of the large language model, and then input the target name of the target object and the commentary corresponding to the target object in the target document into the large language model to obtain the key object in the target object. Specifically, we obtain all the model names in the specified scene in the three-dimensional scene, and then send the model name and commentary in this scene into the large model to obtain the key model in this scene; in each scene, there may be a key model that needs to be displayed, and the above key object is the object that the virtual camera needs to emphasize during the recording process. By using the large language model to obtain relevant information, the manual workload is reduced and the work efficiency is improved.

[0057] Step S13: determining a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and the preset order of the target objects in the preset target document.

[0058] In this embodiment, in order to allow users to select sequence path key points more smoothly and clearly, necessary prompt information is now given in the three-dimensional scene. The corresponding model positions are marked with their serial numbers and titles in the explanation text. The significance of these marks to users is to make the order in the user's explanation document consistent with the order of the path key points marked by them. Prevent the occurrence of the explanation document order being scene A, scene B, scene C, but the path is from scene C to scene A to scene B; accordingly, the process of determining the virtual camera motion path according to the matching information and the sequence of the target objects in the explanation document specifically includes: adding a mark including the target name and the preset order to the target object in the virtual three-dimensional space according to the matching information and the preset order of the target object in the target document; placing the target vector in the virtual three-dimensional space based on the above mark, and generating the target direction indicating geometry and the connection line between the target direction geometry and the previous target direction geometry in turn according to the vector starting point and vector direction of the target vector; determining the camera motion path of the virtual camera recording the target object in the virtual three-dimensional space according to the target direction indicating geometry and the connection line. Specifically, the user selects a series of vector starting points and directions according to the mark, and records these vectors in sequence. In the 3D engine that edits the virtual 3D scene, each time the user selects a path point, the system will generate two actions in sequence, generating a visual direction indicating geometry and a line connecting the previous (if any) indicating geometry at the selected point, and recording the key point position position and orientation target of the geometry. The user can then fine-tune the position and orientation of this key point or delete this indicating geometry by clicking on this indicating geometry; the main data structure of the above target vector is: {Name:””, Indicators:[MeshId:”Indicator model id”, Position:Vector3, Target:Vector3},...]}, where Name is used to identify a path name in the 3D scene, Indicators records the key points of this path, MeshId records the visual indicating geometry of the key points, and Position and Target record the position and orientation of this key point. It can be understood that in order to facilitate the identification of target objects in the virtual 3D scene, in addition to adding marks, the corresponding model in the 3D scene can also be highlighted and stroked. It is understandable that the order of the path point vectors selected by the user reflects the path of the camera in the sequence. The starting point of the vector is the observation reference position of the virtual camera, and the direction of the vector is the observation reference direction of the corresponding observation position. The system generates the sequence camera motion path based on the path points. There are two main steps in the generation here: the first step is to generate two spline curves of the camera motion and its target point based on the path points, and the second step is to constrain the sequence camera and its target point to their respective curves. The generation of spline curves based on path points is common knowledge in the industry and will not be repeated here.It should be noted that after determining the camera motion path, in order to further improve the rationality of the generated motion path, it is also necessary to determine whether the motion path overlaps with certain models. Specifically, determine whether the camera motion path overlaps with the model in the virtual three-dimensional space; if the camera motion path overlaps with the preset collision model in the virtual three-dimensional space, when the virtual camera moves to the position of the preset collision model, the preset collision model is set to transparent; if the camera motion path overlaps with a model other than the preset collision model in the virtual three-dimensional space, a new camera motion path is calculated using a preset three-dimensional path-finding algorithm; in a specific implementation, the system detects whether the camera motion path collides with the model, and if there is no collision, no processing is performed. In another specific implementation, if the model colliding is classified as a door (or other types of preset models, i.e., preset collision models), the collision door is automatically added as a key model, the special effect is transparent, and the special effect display time is the camera approaching; the method for judging the camera approaching in the sequence is to have an invisible collision body larger than the camera at the camera, and the camera is approaching when the collision body detects a collision with this key model. In the third specific implementation, if the camera path collides with other types of models, a new motion path is calculated using comprehensive path finding to avoid the camera passing through the wall in the final sequence. The rationality of the generated camera motion path is ensured by generating the camera motion path according to the order of the target objects and judging whether the camera motion path overlaps with the model.

[0059] Step S14, constructing target sequence data corresponding to the target object according to the target object, the key object, the movement type, the matching information and the camera motion path, and generating a target three-dimensional scene sequence according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

[0060] In this embodiment, the process of constructing the target sequence data corresponding to the target object includes: determining the moving speed of the preset virtual camera in the virtual three-dimensional space on the camera motion path based on the movement type corresponding to the preset virtual camera; determining the moving time of the preset virtual camera on the camera motion path using the moving speed and the length of the camera motion path, so as to construct the target sequence data corresponding to the preset virtual three-dimensional space according to the moving time; it can be understood that the camera motion time between path points is determined by the path point distance and the control type, that is, the movement type of the virtual camera. For example, if the distance is 100 meters and the control type is walking (1.25m / s), the time is 100 / 1.25=80s. It can be understood that the target sequence data also includes a target audio file corresponding to the target commentary in the target document; the system uses the TTS (Text-to-Speech, i.e., text-to-speech model) model to convert the commentary in the commentary document into commentary voice in sequence. The voice file corresponding to the commentary is obtained. Accordingly, the process of constructing the target sequence data corresponding to the target object includes: determining the audio duration corresponding to a single commentary word according to the file duration of the target audio file and the number of words in the target commentary word; determining the playback time required for playing the target audio file to the position of the key commentary word according to the audio duration corresponding to the single commentary word and the position of the key commentary word corresponding to the key object in the target commentary word, and constructing the target sequence data corresponding to the virtual three-dimensional space according to the above playback time. Figure 2, "x": 10, "y": 2, "z": 3}, { "x": 20, "y": 2, "z": 3}],"secondFromStart": [0,10, 20],"Interpolation":"liner"},{"key": "target","values": [{"x":0,"y":0,"z": 1}, {"x":10, "y": 2, "z": 3}],"secondFromStart": [0,10, 20],"Interpolation":"liner"},{"key": "target","values": [{"x":0,"y":0,"z": 1}]}, "z":0}],"timeFromStart": [0]},{"key": "outline","values": [true,false],"timeFromStart": [9,11]}],"settings": {}}. Based on the above functional description and sequence data introduction, it is not difficult to develop a sequence playback engine. It should be noted that the sequence camera is a newly created camera and does not use the original camera. The advantages are high flexibility, easy management, and no damage to the original control mode. In addition, before the sequence engine plays, the initial data of all bound objects is recorded so that it can be restored at the end of the playback; accordingly, the process of generating the target 3D scene sequence according to the target sequence data, such as Figure 3 As shown, it includes: developing a corresponding sequence playback engine based on the target sequence data, parsing the target sequence data using the sequence playback engine to generate a target three-dimensional scene sequence corresponding to the target sequence data according to the corresponding parsing result, and after the three-dimensional scene sequence is generated, the user previews the three-dimensional scene sequence and determines to save the data.

[0061] It can be seen that the embodiment of the present application simplifies the operation of extracting the movement type of the virtual camera and improves work efficiency by obtaining the movement type of the virtual camera according to the target document and using the large language model; generates a camera motion path according to the order of target objects and determines whether the camera motion path overlaps with the model, thereby ensuring the rationality of the generated camera motion path; obtains the target document edited by the user, constructs target sequence data according to the target document and the target object, key object, transition type, matching information and camera motion path obtained by the large language model, and constructs a three-dimensional scene sequence through the target sequence data, so that the user does not need to use a three-dimensional non-linear sequence editor to generate a three-dimensional scene sequence, thereby reducing the user's operating difficulty.

[0062] The above embodiments disclose the related contents of generating target sequence data and using the target sequence data to generate a three-dimensional scene sequence. In order to make the process of using a large language model to output data in this application more complete, the process of using prompt words to make the large language model output related data is described in detail below. Figure 4 As shown, the embodiment of the present invention discloses a large language model training process, including:

[0063] Step S21: extracting the movement type of a preset virtual camera in a preset virtual three-dimensional space from a preset target document using a preset large language model.

[0064] In this embodiment, the specific process of using prompt words to train a large language model to extract the movement type of the virtual camera is shown in Table 1 below:

[0065] Table 1. Movement types of virtual cameras extracted by large language model

[0066] Prompt word You are a tour guide in a virtual 3D space. Based on the explanation content input by the user, choose the one you think is suitable for viewing and explaining in the provided travel methods. You can provide a variety of different travel methods corresponding to the places at scenic spots and between two scenic spots. The provided travel methods: teleportation, flying, walking. Reply in the following json format, do not reply with redundant content: ```{"Method of travel within the scenic spot": <Replace with your suggestion>,"Method of travel between scenic spots": <Replace with your suggestion>}```Explanation content: Send example Welcome to the power center, the heart of our industrial park. It is not only the core of the park's energy supply, but also the key to supporting the stable operation of the entire production process. The power center provides continuous and stable electricity to meet the needs of each production workshop. In response to the call for energy conservation and emission reduction, we have adopted efficient energy conversion technology and waste heat recovery system, which greatly improves energy utilization efficiency and reduces environmental pollution. Next, please allow me to take you to visit our office area, where wisdom and creativity converge. The spacious and bright open office space not only promotes communication and collaboration between teams, but also creates a comfortable working environment for every staff member. From modern meeting rooms to quiet and independent thinking rooms, every detail has been carefully designed to inspire employees' creativity and improve work efficiency. In addition to the needs of daily work, the office area also has a rest area, a tea room and fitness facilities, reflecting the company's attention to the health and welfare of employees. Output example {"Travel method within the attraction": "Walking","Travel method between attractions": "Teleport"}

[0067] Step S22: Acquire feature information of the preset virtual three-dimensional space, and use the prompt word mechanism of the large language model to acquire a matching relationship between the feature information and the target name of the target object.

[0068] In this embodiment, when using a large language model to obtain a matching relationship between feature information and a target name, the prompt words input into the large language model are shown in Table 2 below:

[0069] Table 2 Matching information between the output feature information of the large language model and the target name

[0070] Prompt word The following user input is the commentary of the scene. Please find the model scene category or name corresponding to the commentary and output the json object. The title of the commentary is the key of the output object, and the matching scene category or scene name is the value of the output object. Only the scene names associated with the commentary are considered, and unrelated scene names should not appear in the output. For example, user input: """Scene name: ["Power Center", "Office Area", "Warehouse", "Cutting Workshop", "Forging Workshop"]Scene category: ["Floor", "Roof", "Wall"]Commentary: [{title: "Power Center", description: "Welcome to the heart of our industrial park - the power center. This is not only the core of the park's energy supply, but also the key to supporting the stable operation of the entire production process. The power center provides continuous and stable electricity to meet the needs of various production workshops. In response to the call for energy conservation and emission reduction, we have adopted efficient energy conversion technology and waste heat recovery system, which greatly improved energy utilization efficiency and reduced environmental pollution. ”},{title:"Office Area",description:"Next, please allow me to show you our office area, where wisdom and creativity converge. The spacious and bright open office space not only promotes communication and collaboration among teams, but also creates a comfortable working environment for every staff member. From modern conference rooms to quiet and independent thinking rooms, every detail has been carefully designed to inspire employees' creativity and improve work efficiency. In addition to the needs of daily work, the office area also has a rest area, a tea room and fitness facilities, reflecting the company's concern for the health and welfare of employees. ”}]”””You need to output the scene name corresponding to each commentary in the format. For example, the output for the above input should be: {"Power Center":"Power Center","Office Area":"Office Area"} Input format Scene name: ["Scene name 1", "Scene name 2", "Scene name 3", ...] Scene category: ["Category 1", "Category 2", ...] Commentary: [{title:"Theme 1", description:"Commentary of Theme 1"}, {title:"Theme 2", description:"Commentary of Theme 2"}, {title:"Theme 3", description:"Commentary of Theme 3"}, ... Output example {"Forging Workshop":"Hot Processing","Office Area":"Office Area"}

[0071] Step S23, extracting the target name of the target object in the preset virtual three-dimensional space from the preset target document, inputting the target name and the target description corresponding to the target object in the preset target document into the preset large language model, and using the preset large language model to output the key objects in the target object.

[0072] In this embodiment, when the key objects in the target object are output using the large language model, the prompt words input into the large language model are shown in Table 3 below:

[0073] Table 3 Key objects in the output target object of the large language model

[0074] Prompt word Now there are some 3D model names in a scene and the description of this scene. Please analyze the description and identify the model name that is highlighted in the description from the given 3D model name. Output the description and the corresponding key model name in the form of json. Do not output redundant content. For example: [<description keyword 1>:<key model name 1>,<description keyword 2>:<key model name 2>...] Send example Model name: ["Conference Room", "Manager's Office", "Open Office Area", "Lounge", "Reception Room", "Tea Room", "Training Room"] Narration: Next, please allow me to show you our office area, where wisdom and creativity converge. The spacious and bright open office space not only promotes communication and collaboration among teams, but also creates a comfortable working environment for every staff member. From modern conference rooms to quiet and independent thinking rooms, every detail has been carefully designed to inspire employees' creativity and improve work efficiency. In addition to the needs of daily work, the office area also has a rest area, tea room and fitness facilities, reflecting the company's attention to the health and welfare of employees. Output example ["Open Office Space":"Open Office Area", "Meeting Room":"Meeting Room", "Lounge Area":"Lounge", "Tea Room":"Tea Room"]

[0075] It can be seen that the embodiment of the present application uses different prompt words to train the output format of the large language model according to the prompt word mechanism of the large language model, obtains corresponding output data, reduces manual workload, and improves work efficiency.

[0076] See also Figure 5 As shown, the embodiment of the present application also discloses a three-dimensional scene sequence generation device, including:

[0077] The data extraction module 11 is used to obtain a preset target document created by a target user based on a preset virtual three-dimensional space, and extract a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document;

[0078] The key object acquisition module 12 is used to acquire feature information of the preset virtual three-dimensional space, input the feature information and the target name into a preset large language model to acquire matching information between the feature information and the target name, and input the target name and the target description corresponding to the target object in the preset target document into the preset large language model to acquire key objects in the target object;

[0079] A motion path determination module 13, configured to determine a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and a preset order of the target objects in the preset target document;

[0080] The three-dimensional scene sequence generation module 14 is used to construct target sequence data corresponding to the target object according to the target object, the key object, the movement type, the matching information and the camera motion path, and generate a target three-dimensional scene sequence according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

[0081] It can be seen that the present application obtains the target document edited by the user, constructs target sequence data according to the target object, key object, transition type, matching information and camera motion path obtained by the target document and the large language model, and constructs a three-dimensional scene sequence through the target sequence data, so that the user does not need to use a three-dimensional non-linear sequence editor to generate a three-dimensional scene sequence, but only needs to edit the target document to generate a three-dimensional scene sequence using the large language model, which reduces the user's operating difficulty and improves the efficiency of generating three-dimensional scene sequences.

[0082] In some specific embodiments, the data extraction module 11 specifically includes:

[0083] an output format setting unit, configured to input the target interpretation words in the preset target document into the preset large language model, and set the output format of the preset large language model using a large language model prompt word mechanism;

[0084] A movement type extraction unit is used to extract the movement type of the preset virtual camera in the preset virtual three-dimensional space by using the preset large language model and according to the output format and the target commentary.

[0085] In some specific embodiments, the motion path determination module 13 specifically includes:

[0086] a tag adding unit, configured to add a tag including the target name and the preset order to the target object in the preset virtual three-dimensional space according to the matching information and the preset order of the target objects in the preset target document;

[0087] a geometry generating unit, configured to place a target vector in the preset virtual three-dimensional space based on the mark, and sequentially generate a target direction indicating geometry and a connecting line between the target direction geometry and a previous target direction geometry according to a vector starting point and a vector direction of the target vector;

[0088] A motion path determination unit is used to determine the camera motion path of the preset virtual camera for recording the target object in the preset virtual three-dimensional space according to the target direction indicating geometric body and the connecting line.

[0089] In some specific embodiments, the three-dimensional scene sequence generation module 14 specifically includes:

[0090] A moving speed determining unit, configured to determine a moving speed of the preset virtual camera in the preset virtual three-dimensional space on the camera motion path based on the movement type corresponding to the preset virtual camera;

[0091] The first sequence data construction unit is used to determine the movement time of the preset virtual camera on the camera movement path by using the movement speed and the length of the camera movement path, so as to construct the target sequence data corresponding to the preset virtual three-dimensional space according to the movement time.

[0092] In some specific embodiments, the three-dimensional scene sequence generation module 14 specifically includes:

[0093] An audio duration determination unit, configured to determine an audio duration corresponding to a single commentary word according to the file duration of the target audio file and the number of words in the target commentary word;

[0094] The second sequence data construction unit is used to determine the playback time required for the target audio file to play to the position of the key commentary according to the audio duration corresponding to the single commentary and the position of the key commentary corresponding to the key object in the target commentary, and to construct the target sequence data corresponding to the preset virtual three-dimensional space according to the playback time.

[0095] In some specific embodiments, the three-dimensional scene sequence generating device further includes:

[0096] A motion path determination module, used to determine whether the camera motion path overlaps with the model in the preset virtual three-dimensional space;

[0097] A model setting module, configured to set the preset collision model to be transparent when the preset virtual camera moves to the position of the preset collision model if the camera motion path overlaps with the preset collision model in the preset virtual three-dimensional space;

[0098] The motion path calculation module is used to calculate a new camera motion path by using a preset three-dimensional path-finding algorithm if the camera motion path overlaps with a model other than the preset collision model in the preset virtual three-dimensional space.

[0099] In some specific embodiments, the three-dimensional scene sequence generation module 14 specifically includes:

[0100] The playback engine development module is used to develop a corresponding sequence playback engine based on the target sequence data, and use the sequence playback engine to parse the target sequence data to generate the target three-dimensional scene sequence corresponding to the target sequence data according to the corresponding parsing results.

[0101] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0102] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional scene sequence generation method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0103] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0104] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0105] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the three-dimensional scene sequence generation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0106] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed method for generating a three-dimensional scene sequence. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0107] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0108] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0110] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0111] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating a three-dimensional scene sequence, characterized in that: include: Acquire a preset target document created by a target user based on a preset virtual three-dimensional space, and extract a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document; Acquire feature information of the preset virtual three-dimensional space, input the feature information and the target name into a preset large language model to obtain matching information between the feature information and the target name, and input the target name and a target description corresponding to the target object in the preset target document into the preset large language model to obtain a key object in the target object; Determining a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and the preset order of the target objects in the preset target document; Target sequence data corresponding to the target object is constructed according to the target object, the key object, the movement type, the matching information and the camera motion path, and a target three-dimensional scene sequence is generated according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

2. The three-dimensional scene sequence generation method according to claim 1, characterized in that: The step of extracting the movement type of the preset virtual camera in the preset virtual three-dimensional space from the preset target document includes: Inputting the target interpretation words in the preset target document into the preset large language model, and setting the output format of the preset large language model by using the large language model prompt word mechanism; The movement type of the preset virtual camera in the preset virtual three-dimensional space is extracted by utilizing the preset large language model according to the output format and the target commentary.

3. The three-dimensional scene sequence generation method according to claim 1, characterized in that: The step of determining a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and the preset order of the target objects in the preset target document includes: adding a tag including the target name and the preset order to the target object in the preset virtual three-dimensional space according to the matching information and the preset order of the target objects in the preset target document; Placing a target vector in the preset virtual three-dimensional space based on the mark, and sequentially generating a target direction indicating geometric body and a connecting line between the target direction geometric body and a previous target direction geometric body according to a vector starting point and a vector direction of the target vector; The camera motion path of the preset virtual camera for recording the target object in the preset virtual three-dimensional space is determined according to the target direction indicating geometric body and the connecting line.

4. The method for generating a three-dimensional scene sequence according to claim 3, characterized in that: The constructing target sequence data corresponding to the target object includes: Determining a moving speed of the preset virtual camera in the preset virtual three-dimensional space on the camera motion path based on the movement type corresponding to the preset virtual camera; The moving speed and the length of the camera movement path are used to determine the moving time of the preset virtual camera on the camera movement path, so as to construct the target sequence data corresponding to the preset virtual three-dimensional space according to the moving time.

5. The three-dimensional scene sequence generation method according to claim 1, characterized in that: The constructing target sequence data corresponding to the target object includes: Determine the audio duration corresponding to a single commentary word according to the file duration of the target audio file and the number of words in the target commentary word; The playing time required for playing the target audio file to the position of the key commentary is determined according to the audio duration corresponding to the single commentary and the position of the key commentary corresponding to the key object in the target commentary, and the target sequence data corresponding to the preset virtual three-dimensional space is constructed according to the playing time.

6. The three-dimensional scene sequence generation method according to any one of claims 1 to 5, characterized in that: Also includes: Determining whether the camera motion path overlaps with the model in the preset virtual three-dimensional space; If the camera motion path overlaps with a preset collision model in the preset virtual three-dimensional space, when the preset virtual camera moves to the position of the preset collision model, the preset collision model is set to be transparent; If the camera motion path overlaps with a model other than the preset collision model in the preset virtual three-dimensional space, a new camera motion path is calculated using a preset three-dimensional path-finding algorithm.

7. The method for generating a three-dimensional scene sequence according to claim 1, characterized in that: Generating a target three-dimensional scene sequence according to the target sequence data includes: A corresponding sequence playback engine is developed based on the target sequence data, and the target sequence data is parsed using the sequence playback engine to generate the target three-dimensional scene sequence corresponding to the target sequence data according to the corresponding parsing result.

8. A three-dimensional scene sequence generation device, characterized in that: include: A data extraction module is used to obtain a preset target document created by a target user based on a preset virtual three-dimensional space, and extract a movement type of a preset virtual camera in the preset virtual three-dimensional space and a target name of a target object in the preset virtual three-dimensional space from the preset target document; A key object acquisition module is used to acquire feature information of the preset virtual three-dimensional space, input the feature information and the target name into a preset large language model to acquire matching information between the feature information and the target name, and input the target name and a target description corresponding to the target object in the preset target document into the preset large language model to acquire key objects in the target object; A motion path determination module, configured to determine a camera motion path for the preset virtual camera to record the target object in the preset virtual three-dimensional space based on the matching information and a preset order of the target objects in the preset target document; A three-dimensional scene sequence generation module is used to construct target sequence data corresponding to the target object according to the target object, the key object, the movement type, the matching information and the camera motion path, and generate a target three-dimensional scene sequence according to the target sequence data; wherein the target sequence data also includes a target audio file corresponding to the target commentary in the preset target document.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the three-dimensional scene sequence generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the three-dimensional scene sequence generation method as described in any one of claims 1 to 7.

Images