3D virtual model control method, device, equipment and medium based on large model
By training a large language model to receive voice commands and perform control actions in the 3D editor, the problems of high difficulty and low efficiency of 3D virtual models are solved, the convenience and accuracy of voice control are achieved, and the user operation process is simplified.
Patent Information
- Application Number
- CN202410925121.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-11
AI Technical Summary
The existing 3D virtual model control is problematic in the high operation difficulty and low efficiency, especially in the management and search of complex cards, which leads to increased user operation complexity and reduced efficiency.
By training a large language model to receive user's voice commands, analyze and output control commands, directly enter control commands in the specified class of the 3D editor, use the parameter monitoring program to monitor and trigger actions, set and display the corresponding cards, and realize voice control Actions of 3D virtual models.
It realizes convenient and efficient control of 3D virtual models through voice, improves operation accuracy and efficiency, reduces visual perspective errors and complex operations, and simplifies user operation flow.
Smart Images

Figure CN118470277B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of 3D virtual models, and in particular to a 3D virtual model control method, device, equipment and medium based on a large model. Background Art
[0002] Lingchuang Editor is an online 3D editor that uses ThreeJS and "card" programming. This editor defines a set of conversion specifications between ThreeJS and cards. Each card has a set of clear function definitions and can implement specified functional logic. Users can perform low-code programming to create 3D interactive online courses by setting "cards".
[0003] Compared with code programming, the efficiency of creation is much higher with the card-based approach. However, as the number of steps in the 3D interactive course increases, the complexity of the cards also increases. A large number of cards will be produced, which is not convenient for users to find and modify, making the operation more difficult and reducing efficiency. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a control method for a 3D virtual model in a 3D editor based on a large model to solve the technical problems of difficult operation and low efficiency in the prior art of 3D virtual model control. The method includes:
[0005] The command input model receives the user's voice command, analyzes the voice command and outputs a control command, wherein the control command is used to control the 3D virtual model in the 3D editor to perform a corresponding action, and the command input model is obtained by training a large language model;
[0006] Inputting the control instruction into a designated class of the 3D editor;
[0007] When the parameter transmission monitoring program detects that the control instruction is input in the designated class, the 3D editor is triggered to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction;
[0008] The 3D editor is triggered to set and display a card corresponding to the action according to the control instruction.
[0009] The embodiment of the present invention also provides a control device for a 3D virtual model in a 3D editor based on a large model, so as to solve the technical problems of difficult operation and low efficiency in the prior art of 3D virtual model control. The device includes:
[0010] An instruction receiving module is used to receive a user's voice instruction through an instruction input model, and analyze the voice instruction to output a control instruction, wherein the control instruction is used to control the 3D virtual model in the 3D editor to perform a corresponding action, and the instruction input model is obtained by training a large language model;
[0011] An instruction input module, used for inputting the control instruction into a specified class of the 3D editor;
[0012] A control module, configured to trigger the 3D editor to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction when the control instruction is input in the designated class through the parameter transmission monitoring program;
[0013] The card setting module is used to trigger the 3D editor to set and display the card of the corresponding action according to the control instruction.
[0014] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned control methods for a 3D virtual model in a 3D editor based on a large model when executing the computer program, so as to solve the technical problems of difficult operation and low efficiency of 3D virtual model control in the prior art.
[0015] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned control methods of a 3D virtual model in a 3D editor based on a large model, so as to solve the technical problems of difficult operation and low efficiency in the prior art of 3D virtual model control.
[0016] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least: it is proposed that a command input model obtained by training a large language model receives a user's voice command, and then analyzes the voice command and outputs a control command, so that the voice command can be conveniently and efficiently input by voice to control the 3D virtual model to perform a corresponding action; in a 3D editor, the control command is input into a specified class of the 3D editor, and a parameter transmission monitoring program is used to monitor whether the control command is input in the specified class, so as to trigger the 3D editor to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, so that the control command can be captured in real time, and the 3D virtual model can be accurately controlled to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, which is equivalent to the 3D editor directly outputting the control command based on the large language model to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor. The 3D editor does not obtain the command to control the 3D virtual model through the graphical display layer, so that the accuracy of the output of the control command and the action control of the 3D virtual model is guaranteed, which is conducive to conveniently and accurately controlling the 3D virtual model to perform an action. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 is a flow chart of a control method of a 3D virtual model in a large model-based 3D editor provided by an embodiment of the present invention;
[0019] Figure 2 is a display schematic diagram of a 3D editor provided by an embodiment of the present invention;
[0020] Figure 3 is a structural block diagram of a computer device provided by an embodiment of the present invention;
[0021] Figure 4 It is a structural block diagram of a control device for a 3D virtual model in a large model-based 3D editor provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0023] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.
[0024] In an embodiment of the present invention, a control method for a 3D virtual model in a 3D editor based on a large model is provided, such as Figure 1 As shown, the method includes:
[0025] Step S101: receiving a user's voice command through a command input model, analyzing the voice command and outputting a control command, wherein the control command is used to control a 3D virtual model in a 3D editor to perform a corresponding action, and the command input model is obtained by training a large language model;
[0026] Step S102: inputting the control instruction into a designated class of the 3D editor;
[0027] Step S103: when the parameter transmission monitoring program detects that the control instruction is input in the designated class, the 3D editor is triggered to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction;
[0028] Step S104: triggering the 3D editor to set and display a card corresponding to the action according to the control instruction.
[0029] Depend on Figure 1As can be seen from the process shown, in the embodiment of the present invention, it is proposed to receive the user's voice command with the command input model trained by the large language model, and then analyze the voice command and output the control command, so that the voice command can be input conveniently and efficiently by voice to control the 3D virtual model to perform the corresponding action; in the 3D editor, the control command is input into the specified class of the 3D editor, and the parameter transmission monitoring program is used to monitor whether the control command is input in the specified class, so as to trigger the 3D editor to control the 3D virtual model to perform the corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, so that the control command can be captured in real time, and the 3D virtual model can be accurately controlled to perform the corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, which is equivalent to the 3D editor directly outputting the control command based on the large language model to control the 3D virtual model to perform the corresponding action in the three-dimensional virtual space of the 3D editor. The 3D editor does not obtain the command to control the 3D virtual model through the graphical display layer, so that the accuracy of the output of the control command and the action control of the 3D virtual model is guaranteed, which is conducive to conveniently and accurately controlling the 3D virtual model to perform the action.
[0030] In specific implementation, the general large language model cannot solve the problem of controlling the 3D virtual model in the 3D editor in this specific field. It is necessary to deploy the LLM large language model and train the large language model. For example, various 3D virtual model actions and corresponding descriptions are used as part of the training data, so that the large language model can learn the association between different descriptions and specific actions. Then, when the user issues a command through voice, the trained large language model analyzes and understands the text after voice conversion. If the instruction clearly mentions the specific model name or some information that can uniquely identify a specific model, combined with keywords describing the model action, such as "move", "rotate", "zoom", etc., as well as specific directions, values, etc., the trained large language model can comprehensively judge which action should correspond to which model based on this information to output control instructions.
[0031] In specific implementation, in order to achieve enhanced feature training of a large language model and improve the accuracy of the large language model, this embodiment proposes the following method for training a large language model, for example,
[0032] A plurality of sample instructions are defined for each voice instruction, wherein each voice instruction corresponds to a control action of the 3D virtual model, each voice instruction includes a noun related to the 3D virtual model, an action verb, and an end position of the action, and the plurality of sample instructions of each voice instruction are different descriptions of the noun related to the same 3D virtual model, the same action verb, and the end position of the same action;
[0033] Combining a data set with multiple sample instructions of each control action and the corresponding voice command;
[0034] The data set is used to train a large language model to obtain the instruction input model.
[0035] Specifically, voice commands are first defined. Each voice command corresponds to a control action of the 3D virtual model. Each voice command includes a noun related to the 3D virtual model, an action verb, and the end position of the action. For example,
[0036] The nouns related to the 3D virtual model can be subject keywords: model name {$Model_Name} or noun information that can uniquely identify a specific model;
[0037] Action verbs can be verb keywords: move, move to, move to, move, drag, drag, drag to, drag to, drag to, etc.;
[0038] The end point of the action can be a predicate keyword: X{$Num}, Y{$Num}, Z{$Num}. X, Y, Z are followed by Arabic numerals (1234567890). They correspond to the spoken Chinese 1234567890, 10 ...
[0039] Secondly, each voice command is diversified and enhanced to generate multiple sample commands for each voice command. For example, for a voice command including "material box", "move" and "X11 Y22 Z30", the following sample commands with different descriptions can be expanded:
[0040] Move the model material box to the position of X11 Y22 Z30;
[0041] Move the material box to the position of X11 Y22 Z30;
[0042] Move the material box to X11 Y22 Z30;
[0043] The material box moves to X11 Y22 Z30;
[0044] Material box moves X11 Y22 Z30;
[0045] In this way, a data set consisting of multiple sample instructions of each control action and the corresponding voice command is used to train a large language model. This can make the command input model obtained by training the large language model applicable to more diverse voice commands and better simulate user input training scenarios, so as to improve the applicability and accuracy of the command input model.
[0046] In specific implementation, in order to further improve the accuracy of the instruction input model, this embodiment also proposes to train a large language model based on a reward mechanism, for example,
[0047] In the process of training the large language model, the output result of each voice instruction output by the large language model is compared with the corresponding control action, and a score is determined according to the comparison result, wherein the size of the score represents the correctness and accuracy of the output result, and the size of the score is proportional to the correctness and accuracy of the output result;
[0048] The score is input into the large language model. In the subsequent training process, the large language model adjusts the number of iterations with reference to the score until the score reaches a preset threshold, stops training, and obtains the instruction input model.
[0049] In specific implementation, during the training of the large language model, the output result of each voice command output by the large language model is compared with the corresponding control action, and the first level is determined based on the comparison result: correct or wrong, and the second level: quality score is given on the basis of correctness: for example, 5 points: very good accuracy; 3 points: average accuracy; 1 point: very poor accuracy, etc. Ultimately, the large language model can know and be familiar with distinguishing which are correct, wrong, high-quality, and low-quality outputs based on many training terms, and constantly adjust the number of training iterations.
[0050] Specifically, the process of comparing the output result of each voice command output by the large language model with the corresponding control action and obtaining a score can be achieved by training a neural network model, or the score can be calculated by other comparison methods, for example, calculating the score based on the matching degree and similarity between the two.
[0051] In specific implementation, in order to improve the control convenience and operability of the 3D virtual model in the 3D editor, this embodiment proposes an output format of the control instruction, which is a string that conforms to a preset format, and the string includes the name of the 3D virtual model and the coordinates of the terminal position after the 3D virtual model executes the corresponding control action. For example, {"Model_Name":"Material Box","X":"11","Y":"22","Z":"30"}.
[0052] During specific implementation, the character string in the preset format may be a Json statement.
[0053] Specifically, in the Lingchuang editor, the original monitoring mechanism needs to regularize the Json data generated by the graphical card and then give it to the behavior engine for processing.
[0054] Graphical cards are generated using open source software. After adjusting the card content in the web browser, the background will automatically generate the corresponding Json data, but the Json format data cannot meet the parsing requirements of the Lingchuang editor behavior engine. Therefore, the Json data needs to be regularized. The implementation of Json regularization includes:
[0055] A) Add or remove statements: Delete redundant statements and automatically complete insufficient or missing statements;
[0056] B) Sorting statements: adjust the order of statements;
[0057] C) Adjust the structure: Simplify and adjust the statement structure that does not conform to the behavior engine.
[0058] However, this application uses a command input model based on a large language model to receive voice commands and output control commands. Since the step of "enhanced feature training" is performed in the process of training the large language model, the corresponding natural language defines the Json rules, so that the command input model directly outputs control commands that conform to the Json statements that can be parsed by the behavior engine. On the one hand, the control commands can be directly given to the behavior engine, and on the other hand, the control commands can be given to the graphical JavaScript class of the Lingchuang editor to directly generate cards in the interface. Compared with the original monitoring mechanism, this method improves the execution efficiency and does not require all operations in the regularization link.
[0059] In specific implementation, in order to improve the real-time control of the 3D virtual model in the 3D editor, this embodiment proposes a specific method of monitoring whether the control instruction is input in the specified class (for example, a class used to drive a graphical JavaScript) through a parameter passing monitoring program. For example, the parameter passing monitoring program monitors whether the control instruction is input in the driving graphical JavaScript of the 3D editor; the behavior parsing engine of the 3D editor is triggered to control the 3D virtual model to perform corresponding actions in the three-dimensional virtual space of the 3D editor according to the control instruction, so as to realize that the instruction input model is directly connected to the parsing engine from the code layer to pass the control instruction.
[0060] Specifically, since the card-based monitoring form in the prior art is: the user operates the model by dragging the mouse, etc., and the model transmits the attribute information after the operation to the card. This interaction method is very dependent on the user's operation. In 3D space, the cost of frequent user operations is relatively high, and because of the visual perspective error, it is not intuitive and requires repeated debugging. In addition, although cardization is simpler than the WebUI toolbar method and has more unified operation characteristics. However, when the card content is too much (such as more than 30 items), because the card itself has logical nesting and inclusion relationships, user operations and interactions are more complicated, and the card needs to be dragged repeatedly to move.
[0061] However, the present application uses a large language model (i.e., the above-mentioned command input model) to input voice control commands, which solves the problem of users operating the model by dragging the mouse, etc., and greatly simplifies the number of operations and the complexity of operations. It can be controlled by voice. Even if the ideal position is not obtained for the first time, it is sufficient to give the "command" through voice to fine-tune. In addition, after the model control is realized through the large language model, the control command output by the large language model is directly output to the driving graphical JavaScript. When the control command input is detected in the driving graphical JavaScript through the parameter transmission monitoring program, the behavior parsing engine of the 3D editor is triggered to control the 3D virtual model according to the control command to perform the corresponding action in the three-dimensional virtual space of the 3D editor, thereby realizing that the large language model is directly connected to the behavior parsing engine from the bottom layer of the code to transmit the control command. Its accuracy is guaranteed, which is equivalent to directly using the large language model to output standard formatted data to the behavior parsing engine, and the behavior parsing engine does not receive instructions through the graphical display layer.
[0062] In specific implementation, in order to improve the visualization of the 3D virtual model control in the 3D editor and facilitate users to observe and understand the scale of the current courseware production, it is proposed to trigger the 3D editor to set the corresponding card according to the control instruction, and then graphically display the card on the screen of the 3D editor. That is, while controlling the 3D virtual model to perform the corresponding action in the three-dimensional virtual space of the 3D editor, the graphics of the corresponding card can also be displayed. For example, Figure 2 As shown, a "voice" entry is added to the editor to create a practice course interface. After the user clicks the "voice" entry and speaks a command, the editor will generate a corresponding card puzzle in the graphical area of the screen based on the control command output by the command input model. Each action step of the 3D virtual model corresponds to a card puzzle, which may include the name of the 3D virtual model, the action, the coordinates of the end position after executing the action, and the time taken to execute the action (e.g., within 1 second).
[0063] In specific implementation, the control instruction can also be used to control the shape change of the 3D virtual model in the 3D editor, such as proportional enlargement, proportional reduction, deformation based on any X, Z, Y axis, etc.;
[0064] The control instruction may also be used to control the transparency change of the 3D virtual model in the 3D editor, for example, the model may be placed in a semi-transparent state;
[0065] The control instructions can also be used to control the playback animation of the 3D virtual model in the 3D editor: the model can be allowed to play its own animation, for example, playing a schematic animation of the model structure decomposition process;
[0066] The control instruction can also be used to control the model decomposition of the 3D virtual model in the 3D editor: multiple sub-models in the model can be moved.
[0067] The operation process of the control instructions for controlling the shape change, transparency change, animation playback, model decomposition and other scenes of the 3D virtual model in the 3D editor is similar to the operation process of controlling the 3D virtual model to perform corresponding actions. Both involve inputting the control instructions into the specified class of the 3D editor, thereby triggering the 3D editor to set the corresponding card according to the control instructions and perform corresponding control.
[0068] The control method of the 3D virtual model in the above 3D editor can define a variety of 3D virtual model actions in a voice interactive manner through a large language model, and allow it to execute any action of the 3D virtual model in the ThreeJS development environment.
[0069] Specifically, the physical deployment of the large language model and the process of the 3D editor controlling the actions of the 3D virtual model based on the behavior parsing engine are not specifically limited in this application and can be implemented using existing technologies.
[0070] In this embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302 and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned control methods for the 3D virtual model in the large model-based 3D editor is implemented.
[0071] Specifically, the computer device may be a computer terminal, a server or a similar computing device.
[0072] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned methods for controlling a 3D virtual model in a 3D editor based on a large model.
[0073] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0074] Based on the same inventive concept, an embodiment of the present invention also provides a control device for a 3D virtual model in a 3D editor based on a large model, as described in the following embodiments. Since the principle of solving the problem by the control device for a 3D virtual model in a 3D editor based on a large model is similar to the control method for a 3D virtual model in a 3D editor based on a large model, the implementation of the control device for a 3D virtual model in a 3D editor based on a large model can refer to the implementation of the control method for a 3D virtual model in a 3D editor based on a large model, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0075] Figure 4 is a structural block diagram of a control device for a 3D virtual model in a 3D editor based on a large model according to an embodiment of the present invention, such as Figure 4 As shown, the device comprises:
[0076] The instruction receiving module 401 is used to receive the user's voice instruction through the instruction input model, and analyze the voice instruction to output the control instruction, wherein the control instruction is used to control the 3D virtual model in the 3D editor to perform a corresponding action, and the instruction input model is obtained by training a large language model;
[0077] An instruction input module 402, used for inputting the control instruction into a specified class of the 3D editor;
[0078] The control module 403 is used to trigger the 3D editor to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction when the parameter transmission monitoring program detects that the control instruction is input in the designated class;
[0079] The card setting module 404 is used to trigger the 3D editor to set and display the card corresponding to the action according to the control instruction.
[0080] In one embodiment, the above device further comprises:
[0081] A model training module is used to define multiple sample instructions for each voice instruction, wherein each voice instruction corresponds to a control action of a 3D virtual model, each voice instruction includes relevant nouns, action verbs and the end position of the action of the 3D virtual model, and the multiple sample instructions of each voice instruction are different descriptions of the relevant nouns, the same action verbs and the end position of the same action of the same 3D virtual model; each control action and the multiple sample instructions of the corresponding voice instruction are combined into a data set; the data set is used to train a large language model to obtain the instruction input model.
[0082] In one embodiment, the model training module is also used to compare the output result of the large language model for each voice instruction with the corresponding control action during the training of the large language model, and determine a score based on the comparison result, wherein the size of the score indicates the correctness and accuracy of the output result; input the score into the large language model, and in the subsequent training process, the large language model adjusts the number of iterations with reference to the score until the score reaches a preset threshold, stops training, and obtains the instruction input model.
[0083] In one embodiment, the control instruction is a character string conforming to a preset format, and the character string includes the name of the 3D virtual model and the coordinates of the end point position after the 3D virtual model performs the corresponding control action.
[0084] In one embodiment, the character string in the preset format is a Json statement.
[0085] In one embodiment, a control module is used to monitor the input of the control instruction in the driving graphical JavaScript of the 3D editor through a parameter listening program; trigger the behavior parsing engine of the 3D editor to control the 3D virtual model to perform corresponding actions in the three-dimensional virtual space of the 3D editor according to the control instruction, so as to realize that the instruction input model is directly connected with the parsing engine from the code layer to pass the control instruction.
[0086] In one embodiment, the above device further comprises:
[0087] The graphic module is used to trigger the 3D editor to set the corresponding card according to the control instruction, and then graphicize the card on the screen of the 3D editor.
[0088] The embodiments of the present invention achieve the following technical effects: it proposes to use a command input model trained by a large language model to receive a user's voice command, and then analyze the voice command and output a control command, so that the voice command can be input conveniently and efficiently by voice to control the 3D virtual model to perform a corresponding action; in a 3D editor, the control command is input into a specified class of the 3D editor, and a parameter transmission monitoring program is used to monitor whether the control command is input in the specified class, so as to trigger the 3D editor to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, so that the control command can be captured in real time, and the 3D virtual model can be accurately controlled to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control command, which is equivalent to the 3D editor directly outputting the control command based on the large language model to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor. The 3D editor does not obtain the command to control the 3D virtual model through a graphical display layer, so that the accuracy of the output of the control command and the action control of the 3D virtual model is guaranteed, which is conducive to conveniently and accurately controlling the 3D virtual model to perform an action.
[0089] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.
[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A control method for a 3D virtual model in a 3D editor based on a large model, characterized in that: include: The command input model receives the user's voice command, analyzes the voice command and outputs a control command, wherein the control command is used to control the 3D virtual model in the 3D editor to perform a corresponding action, and the command input model is obtained by training a large language model; Inputting the control instruction into a designated class of the 3D editor; When the parameter transmission monitoring program detects that the control instruction is input in the designated class, the 3D editor is triggered to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction; Triggering the 3D editor to set and display a card corresponding to the action according to the control instruction; The method further comprises: A plurality of sample instructions are defined for each voice instruction, wherein each voice instruction corresponds to a control action of the 3D virtual model, each voice instruction includes a noun related to the 3D virtual model, an action verb, and an end position of the action, and the plurality of sample instructions of each voice instruction are different descriptions of the noun related to the same 3D virtual model, the same action verb, and the end position of the same action; Combining a data set with multiple sample instructions of each control action and the corresponding voice command; Using the data set to train a large language model to obtain the instruction input model; The large language model is trained using the data set to obtain the instruction input model, including: In the process of training the large language model, the output result of each voice instruction output by the large language model is compared with the corresponding control action, and a score is determined according to the comparison result, wherein the size of the score indicates the correctness and accuracy of the output result; The score is input into the large language model, and in the subsequent training process, the large language model adjusts the number of iterations with reference to the score until the score reaches a preset threshold, stops training, and obtains the instruction input model; When the parameter transmission monitoring program detects that the control instruction is input in the designated class, the 3D editor is triggered to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction, including: The parameter listening program detects that the control command is input in the graphical JavaScript driving the 3D editor; The behavior parsing engine of the 3D editor is triggered to control the 3D virtual model to perform corresponding actions in the three-dimensional virtual space of the 3D editor according to the control instruction, so that the instruction input model is directly connected to the behavior parsing engine from the code layer to transmit the control instruction. The instruction input model directly outputs the control instruction that conforms to the Json statement that can be parsed by the behavior engine, and the behavior parsing engine does not receive the control instruction through the graphical display layer.
2. The control method of a 3D virtual model in a 3D editor based on a large model as claimed in claim 1, characterized in that: The control instruction is a character string conforming to a preset format, and the character string includes the name of the 3D virtual model and the coordinates of the end point position after the 3D virtual model executes the corresponding control action.
3. The control method of a 3D virtual model in a large model-based 3D editor according to any one of claims 1 to 2, characterized in that: Also includes: After the 3D editor is triggered to set the corresponding card according to the control instruction, the card is graphically displayed on the screen of the 3D editor.
4. A control device for a 3D virtual model in a 3D editor based on a large model, characterized in that: include: An instruction receiving module is used to receive a user's voice instruction through an instruction input model, and analyze the voice instruction to output a control instruction, wherein the control instruction is used to control the 3D virtual model in the 3D editor to perform a corresponding action, and the instruction input model is obtained by training a large language model; An instruction input module, used for inputting the control instruction into a specified class of the 3D editor; A control module, configured to trigger the 3D editor to control the 3D virtual model to perform a corresponding action in the three-dimensional virtual space of the 3D editor according to the control instruction when the control instruction is input in the designated class through the parameter transmission monitoring program; A card setting module, used to trigger the 3D editor to set and display a card corresponding to the action according to the control instruction; The above device also includes: A model training module is used to define multiple sample instructions for each voice instruction, wherein each voice instruction corresponds to a control action of a 3D virtual model, each voice instruction includes relevant nouns of the 3D virtual model, action verbs and the end position of the action, and the multiple sample instructions of each voice instruction are different descriptions of the relevant nouns of the same 3D virtual model, the same action verb and the end position of the same action; the multiple sample instructions of each control action and the corresponding voice instruction are formed into a data set; the large language model is trained using the data set to obtain the instruction input model; The model training module is further used to compare the output result of each voice instruction output by the large language model with the corresponding control action during the training of the large language model, and determine a score according to the comparison result, wherein the size of the score indicates the correctness and accuracy of the output result; input the score into the large language model, and in the subsequent training process, the large language model adjusts the number of iterations with reference to the score until the score reaches a preset threshold, stops training, and obtains the instruction input model; The control module is used to monitor the input of the control instruction in the driving graphical JavaScript of the 3D editor through the parameter transmission monitoring program; trigger the behavior parsing engine of the 3D editor to control the 3D virtual model to perform corresponding actions in the three-dimensional virtual space of the 3D editor according to the control instruction, so as to realize that the instruction input model is directly connected with the behavior parsing engine from the code layer to transmit the control instruction, and the instruction input model directly outputs the control instruction that conforms to the Json statement that can be parsed by the behavior engine, and the behavior parsing engine does not receive the control instruction through the graphical display layer.
5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for controlling a 3D virtual model in a large model-based 3D editor according to any one of claims 1 to 3 is implemented.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method for controlling a 3D virtual model in a large model-based 3D editor according to any one of claims 1 to 3.
