Large language model structured generation system and method

Abstract

The present invention discloses a large language model structured generation system and method, relating to the field of artificial intelligence technology. The system comprises: a front end for receiving a domain-specific language input by a user and converting the domain-specific language into a back-end recognizable language; a back end for receiving the back-end recognizable language and converting the back-end recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved based on the wait operator and the write operator; an inference framework for receiving the domain-specific language input by a user and converting the domain-specific language into a word-unit ID state machine; and a word-unit ID state machine for constraining the large language model to generate content based on the basic operators corresponding to the word-unit ID state machine, so as to ensure that the content generated by the large language model conforms to the corresponding grammatical structure. This system helps to solve the problems of the existing technology that it is unable to achieve efficient and accurate processing of complex large model structured requirements, has high time complexity, weak structured requirement description capabilities, and a high threshold for use.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a system and method for generating a large language model structure. Background Art

[0002] In the field of natural language processing, large-model structured generation technology is crucial to improving the standardization and accuracy of content generation. At present, the principle of large-model structured generation mainly includes that in the sequence generation process, the algorithm needs to maintain a word state machine to track the list of illegal words corresponding to the current moment, etc., and during single-step generation, the algorithm determines the list of illegal words at the current moment through a certain data structure and word state machine, and then reduces the score corresponding to the illegal word to a small enough number to ensure that no illegal words appear in the generation of this step.

[0003] Common large-scale language model structured generation frameworks include outlines and xgrammar; outline uses a deterministic finite state machine as the word state machine, and single-step generation uses data structures such as lists and tensors;

[0004] However, the process of obtaining a deterministic finite state machine from outlines is extremely complex and requires multiple conversions, with a time complexity of up to O(n 2 V), where n is approximately the number of characters in the regular expression entered by the user, and V is the size of the word segmenter vocabulary. In practice, V is often much larger than n. Furthermore, outlines are based on the underlying description method of regular languages, and their ability to describe structural requirements is insufficient, making them unable to meet the general structural requirements of context-free grammars.

[0005] xgrammar uses a nondeterministic pushdown automaton as its lemma state machine, and single-step generation utilizes a bit set data structure. However, xgrammar's time complexity in maintaining the lemma state machine is extremely high. Furthermore, xgrammar's underlying description language, based on context-free grammars, while improved, still cannot cover common requirements, such as avoiding duplicate output strings. Furthermore, the EBNF description language used by xgrammar is limited in popularity and has a high barrier to entry, requiring users to learn it before using it.

[0006] Therefore, an innovative large-scale structured generation system is urgently needed to overcome the shortcomings of the existing large-scale structured generation framework. Summary of the Invention

[0007] In view of this, the present invention proposes a large language model structured generation system and method, which can solve the above-mentioned problems in the prior art.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A large language model structured generation system, comprising:

[0010] The front end is used to receive domain-specific language input by the user and convert the domain-specific language into a language recognizable by the back end;

[0011] a backend for receiving the backend-recognizable language and converting the backend-recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved from the wait operator and the write operator, wherein the wait operator and the write operator are used to record a current list of legal or illegal word units, the wait operator is used to convert a word unit ID state machine state when a specific word unit appears, and the write operator is used to output a specific word unit at the current moment;

[0012] An inference framework for receiving domain-specific language input by a user and converting the domain-specific language into a word-unit ID state machine;

[0013] The word-gram ID state machine is used to constrain the large language model to generate content based on the basic operator corresponding to the word-gram ID state machine, so as to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

[0014] On the basis of the above technical solution, the present invention can also be improved as follows:

[0015] Optionally, when converting the domain-specific language into a language recognizable by the backend, the front end limits the time complexity of the compilation algorithm to O(n), where n is the number of characters in the domain-specific language.

[0016] Optionally, the domain specific language received by the front end as input by the user includes a format specifying part and a parameter inputting part.

[0017] Optionally, the large language model structured generation system further includes a vocabulary mask, which interacts with the reasoning framework to filter or constrain the vocabulary during the large model generation process.

[0018] Optionally, the wait operator and the write operator update the current legal word list or illegal word list based on the word list mask.

[0019] Optionally, the word-unit ID state machine is further configured to:

[0020] Limit the final word sequence to the language space described by the word ID state machine;

[0021] Optionally, the word-unit ID state machine is further configured to:

[0022] When the basic operator corresponding to the word-unit ID state machine is the wait operator, the wait operator will generate a word list mask constraint large language model through the legal word list or illegal word list recorded by itself;

[0023] When the basic operator corresponding to the word-gram ID state machine is the write operator, the write operator will generate a large language model by outputting a vocabulary mask constraint based on the expected word-gram or word-gram list at the current moment recorded by itself.

[0024] A structured generation method for a large language model, comprising:

[0025] The front end receives the domain-specific language input by the user and converts the domain-specific language into a language recognizable by the back end;

[0026] Receiving the backend-recognizable language through the backend and converting the backend-recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved based on the wait operator and the write operator, wherein the wait operator and the write operator are used to record a current legal word list or an illegal word list, the wait operator is used to convert a word ID state machine state when a specific word appears, and the write operator is used to output the specific word at the current moment;

[0027] Receive domain-specific language input by the user through the reasoning framework and convert the domain-specific language into a word-unit ID state machine;

[0028] The word-gram ID state machine is used to constrain the large language model to generate content based on the basic operator corresponding to the word-gram ID state machine, so as to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

[0029] An electronic device comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0030] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0031] The present invention has the following advantages:

[0032] The structured generation system of large language models in the present invention avoids operations with high time complexity through orderly processing of the front-end and back-end. It does not need to go through multiple complex conversions to obtain the word ID state machine like outlines. It directly converts the domain-specific language into a back-end recognizable language and then converts the back-end recognizable language into basic operators through the back-end, reducing unnecessary calculation steps, thereby significantly improving generation efficiency and reducing time costs.

[0033] The large language model structured generation system in the present invention, the domain-specific language includes a format specification part and a parameter input part, which can meet a wider range and more complex structural requirements. Whether it is context-free grammar requirements or special requirements such as requiring the output string to be non-repeated, it can be achieved through flexible format specification and parameter input, which greatly expands the application scenarios of the system. Users do not need to specifically learn complex description languages. They only need to input requirements according to the system's preset format specification and parameter input methods, and the system can complete the generation of structured content, reducing the user's learning cost and difficulty of use.

[0034] The large language model structured generation system in the present invention has an inference framework that can quickly convert the domain-specific language input by the user into a word-gram ID state machine. The word-gram ID state machine constrains the large language model to generate content based on the corresponding basic operators to ensure that the content generated by the large language model conforms to the corresponding grammatical structure. The entire process is closely connected, allowing the system to quickly respond to user needs and complete the generation of structured content in a short time.

[0035] The large language model structured generation system in this invention uses wait and write operators to record and control lists of valid and invalid tokens, as well as output tokens. This allows for flexible adjustment of generation rules based on the characteristics and needs of different fields. For example, in fields such as finance and healthcare, different domain-specific languages ​​and parameters can be set to generate structured content that meets the standards and requirements of that field. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] For purposes of illustration and not limitation, the present invention will now be described with reference to embodiments thereof and the accompanying drawings, in which:

[0037] Figure 1 Schematic diagram of the main components of the large language model structured generation system in an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of the process of the structured generation method of a large language model in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram comparing the relative time consumption of each framework in an embodiment of the present invention;

[0040] Figure 4 A comparative diagram of the total time consumption for inference of each framework in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of the time consumption ratio of each framework in the three stages of state machine compilation, state tracking, and tensor operation in the embodiment of the present invention;

[0042] Figure 6 Schematic diagram of the absolute time consumption of each framework in the three stages of state machine compilation, state tracking, and tensor operation in an embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "first," "second," and the like in the description of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate for the embodiments of the present invention described herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features thereof can be combined with each other. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Figure 1 Schematic diagram of the main components of the large language model structured generation system in the embodiment of the present invention. Figure 1 As shown, the large language model structured generation system 1 provided by the embodiment of the present invention includes a front end 10, a back end 20, an inference framework 30, a word unit ID state machine 40 and a vocabulary mask 50.

[0048] "Structured generation," also known as "constrained generation," refers to restricting the sampling space during the Large Language Model (LLM) generation process so that the resulting text conforms to a certain pattern, such as regular expressions or JSON Schema.

[0049] The LLM generation process is regarded as an intermediate process. Only when its output conforms to a certain pattern can it serve as a stable and reliable "interface". Only when "structured generation" is guaranteed can the scalability of the overall system be guaranteed.

[0050] The process of generating text using a large language model is essentially a sequential multi-class classification problem; this multi-class classification problem involves selecting the most likely word from a vocabulary. Generating a large language model is a sequential decision-making process; the number of "invalid words" changes at each moment. Therefore, a state machine is required to track invalid words in a given state.

[0051] The front end 10 is used to receive a domain-specific language (DSL) input by a user and convert the domain-specific language into a language recognizable by the back end. The domain-specific language includes a format specification part and a parameter input part. The format specification part is formed by filling in a format type in a preset specific identifier, the specific identifier is FORMAT[...], and the format type is SECTION or SUBSECTION. The SECTION is composed of a state machine segment, SECTION, and PART_END. The parameter input part is formed by adding specific parameter values ​​after the specific identifier in a preset connection method.

[0052] A preferred embodiment of a domain specific language is:

[0053] How to pass in specific parameters. A typical front-end DSL is as follows:

[0054] FORMAT["SECTION"]_"1"

[0055] FORMAT["SECTION"]_"2"

[0056] FORMAT["SUBSECTION"]

[0057] FORMAT["SUBSECTION"]

[0058] FORMAT["SUBSECTION"]

[0059] FORMAT["SUBSECTION"]

[0060] The SECTION or SUBSECTION filled in FORMAT[...] corresponds to a certain format; and the underscore in FORMAT[...]_"..." specifies the specific parameters to be passed in.

[0061] When the front-end converts a domain-specific language into a language recognizable by the back-end, the time complexity of the compilation algorithm is limited to O(n), where n is the number of characters in the domain-specific language. This constraint is achieved by limiting the expressive power of the DSL. For example, only keywords such as FORMAT can be used to determine the format, and complex judgments cannot be made.

[0062] The backend 20 (backend) is used to receive the backend-recognizable language and convert the backend-recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved based on the wait operator and the write operator. The wait operator (wait operator) and the write operator (write operator) are used to record the current legal word list or illegal word list. The wait operator is used to convert the word ID state machine state when a specific word appears. The write operator is used to output a specific word at the current moment; the update is based on the word list mask. The word list mask interacts with the reasoning framework to filter or constrain the word list range based on which the word legitimacy is judged. The wait operator and the write operator update the current legal word list or illegal word list based on the word list mask.

[0063] A preferred embodiment of a backend-recognizable language is:

[0064] TOKEN_"."=13

[0065] TOKEN_"0"=15

[0066] TOKEN_"1"=16

[0067] TOKEN_"2"=17

[0068] TOKEN_"3"=18

[0069] TOKEN_"4"=19

[0070] TOKEN_"5"=20

[0071] TOKEN_"6"=21

[0072] TOKEN_"7"=22

[0073] TOKEN_"8"=23

[0074] TOKEN_"9"=24

[0075] TOKEN_"<|box_start|>"=151648

[0076] TOKEN_"<|box_end|>"=151649

[0077] TOKEN_"####"=820

[0078] TOKEN_"<|fim_pad|>"=151662

[0079] TOKEN_"<|sep|>"=151665

[0080] TOKEN_"<|im_end|>"=151645

[0081] TOKEN_"\t"=197

[0082] TOKEN_"-"=12

[0083] TOKEN_""=220

[0084] TOKEN_"<|endoftext|>"=151643

[0085] IDSET_"EOS"=TOKEN_"<|endoftext|>",TOKEN_"<|im_end|>"

[0086] IDSET_"SECTION_START"=TOKEN_"####",TOKEN_"",TOKEN_"<|box_start|>"

[0087] IDSEQ_"SUBSECTION_START"=TOKEN_"-",TOKEN_"",TOKEN_"<|box_start|>"

[0088] IDSEQ_"SUBSUBSECTION_START"=TOKEN_"\t",TOKEN_"-"

[0089] SNIPPET_"SECTION"=write([IDSEQ_"SECTION_START",ARGS_"SE CTION",TOKEN_"."]),waits({TOKEN_"<|fim_pad|>"},deny={IDSET_"EOS"

[0090] }),write([TOKEN_"<|box_end|>"])

[0091] SNIPPET_"SUBSECTION"=write([IDSEQ_"SUBSECTION_START"]),waits({TOKEN_"<|fim_pad|>"},deny={IDSET_"EOS"}),write([TOKEN_"<|box_end|>"])

[0092] SNIPPET_"SUBSUBSECTION"write([IDSEQ_"SUBSUBSECTION_ST ART"]),waits({TOKEN_"<|fim_pad|>"},deny={IDSET_"EOS"})

[0093] SNIPPET_"PART_END"=waits({TOKEN_"<|sep|>"},deny={IDSET_"EOS"})

[0094] FORMAT["SECTION"]=sequence(SNIPPET_"SECTION",SNIPPET_"PART_END")

[0095] FORMAT["SUBSECTION"]=sequence(SNIPPET_"SUBSECTION",SNI PPET_"PART_END")

[0096] FORMAT["SUBSUBSECTION"]sequence(SNIPPET_"SUBSUBSECTI ON",SNIPPET_"PART_END")

[0097] The wait operator knows the current list of valid and invalid tokens and transitions state when it receives certain token ids. For example, after <|box_end|>, it waits for <|sep|>. Reject token ids include <|im_end|> and line break.

[0098] The write operator knows which token ID should be output. For example, in this embodiment, the first token ID must be 820 ("####").

[0099] The reasoning framework 30 is used to receive domain-specific language input by a user and convert the domain-specific language into a word-unit ID state machine;

[0100] The word-unit ID state machine 40 is used to constrain the large language model to generate content based on the corresponding basic operators, so as to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

[0101] The word unit ID state machine is also used to:

[0102] The final word sequence is restricted to the language space described by the word ID state machine.

[0103] The word unit ID state machine is also used to:

[0104] When the basic operator corresponding to the word-unit ID state machine is the wait operator, the wait operator will generate a word list mask constraint large language model through the legal word list or illegal word list recorded by itself;

[0105] When the basic operator corresponding to the word-gram ID state machine is the write operator, the write operator will generate a large language model by outputting a vocabulary mask constraint based on the expected word-gram or word-gram list at the current moment recorded by itself.

[0106] The time consumption of structured generation mainly comes from three stages: state machine compilation, state tracking, and tensor operations. State machine compilation refers to the process of converting the user-entered DSL into the internal running word unit state machine; state tracking refers to the related actions of maintaining the internal word unit state machine; tensor operations refer to the conversion of the final invalid word unit list and other items into tensors before they can be input into the large model generation framework to constrain the large model generation process.

[0107] For "state machine compilation", this application customizes the front-end language to ensure that the input DSL is concise while also having sufficient expressiveness. The decoupling design of the front-end and back-end ensures low time complexity during the compilation phase. The operators corresponding to the back-end only need to be run once and can be globally shared. At the same time, separating the back-end from the front-end ensures that wgrammar only accepts specific operators, laying the foundation for subsequent performance improvements in "state tracking" and "tensor operations".

[0108] Regarding "state machine tracking", this application achieves high performance by constraining the backend compilation process - the final word state machine only allows the internal state to be changed by changing the pointer state (the time complexity is at a constant level).

[0109] Regarding "tensor operations", this application ensures high performance from two aspects: first, the data structure maintained at each moment only contains illegal words and legal words, and the construction cost is low; second, there are only two types of underlying operators in wgrammar: wait & write. The two operators wait & write essentially record the legal words or illegal words that should be produced in the current state. The wait operator records the current legal word list or illegal word list; the write operator records the words that need to be output at the current moment. However, it is worth noting that the word lists produced by both are context-free, so they can be shared globally, which helps to limit the complexity of the DSL; it cannot lead to features that affect global sharing, such as the stack in the push-down automaton, so this design further amortizes the cost of constructing tensors; write can support jump-forward, which is expected to reduce 10.89% of LLM decoding calculations (obtained by sampling 5048 instances), opening up space for further performance optimization.

[0110] Experimental setup: Taking the single-document structured generation scenario as an example, 76 data points are sampled and SGLang is used as the framework base; the performance of outlines, xgrammar, and wgrammar are compared.

[0111] Figure 3 This chart compares the relative runtimes of the three frameworks: outlines, xgrammar, and wgrammar. The vertical axis shows the runtime, using a logarithmic scale, while the horizontal axis shows the framework names. It's clear that the outlines framework has the highest relative runtime, reaching 62877.65 seconds; the xgrammar framework has a relative runtime of 144.45 seconds; and the wgrammar framework has the lowest relative runtime, at just 1.01 seconds. This indicates that the wgrammar framework is the most efficient, while the outlines framework takes the longest and is relatively inefficient.

[0112] Figure 4 This table shows the total time taken for each framework, including inference, and the performance of four different categories: outlines, xgrammar, baseline, and wgrammar. The vertical axis represents the total time taken (in milliseconds) and uses a logarithmic scale, with values ​​from bottom to top being 1, 1000, and 1,000,000, reflecting the magnitude of the time taken.

[0113] Horizontal axis: From left to right, there are four categories: outlines, xgrammar, baseline, and wgrammar.

[0114] Data comparison: outlines takes the longest time, reaching 8395423.83ms, xgrammar takes 10899.97ms, baseline takes 133.52ms, and wgrammar takes the shortest time, at 120.39ms.

[0115] Figure 5 The figure shows the time consumption of the three frameworks, outlines, xgrammar, and wgrammar, in the three stages of state machine compilation, state tracking, and tensor operation. The vertical axis is the name of each framework, and the horizontal axis is the percentage of time consumption.

[0116] Outlines framework: State machine compilation accounts for about 67.83% of the time, tensor operations account for about 32.15%, and state tracking accounts for a very small proportion, only 0.01%.

[0117] xgrammar framework: Tensor operations account for the largest proportion of time consumption, approximately 55.62%, state tracking accounts for approximately 34.12%, and state machine compilation accounts for 10.25%.

[0118] wgrammar framework: State machine compilation accounts for 40.59% of the time, state tracking accounts for approximately 32.67%, and tensor operations account for 25.74%. It can be seen that the time consumption distribution of each stage varies greatly among different frameworks.

[0119] Figure 6 This chart shows the absolute timings of the three frameworks, outlines, xgrammar, and wgrammar, for state machine compilation, state tracking, and tensor operations. The vertical axis represents the name of each stage, and the horizontal axis represents the duration (in milliseconds), using a logarithmic scale.

[0120] During the state machine compilation phase, the outlines framework takes the longest time, reaching 42651.92 seconds, followed by the wgrammar framework at 40.59 seconds, and the xgrammar framework takes the shortest time, at only 0.41 seconds.

[0121] In the state tracking phase, the xgrammar framework takes the longest time, 49.29 seconds, the wgrammar framework takes 32.67 seconds, and the outlines framework takes the least time, 6.83 seconds.

[0122] During the tensor operation phase, the outlines framework took 20218.85 seconds, the xgrammar framework took 80.35 seconds, and the wgrammar framework took 25.74 seconds. The absolute time consumption of each framework varies significantly across phases, which helps us better understand the performance of each framework at different operation stages.

[0123] Figure 2FIG. 1 is a flow chart of a method for generating a large language model structure in an embodiment of the present invention. Figure 2 As shown, the large language model structured generation method provided by the embodiment of the present invention includes the following steps S101 to S104.

[0124] S101, receiving domain-specific language input by a user through a front end, and converting the domain-specific language into a language recognizable by a back end.

[0125] The domain-specific language includes a format specification part and a parameter input part;

[0126] When converting the domain-specific language into a language recognizable by the backend, the frontend limits the compilation algorithm's time complexity to O(n), where n is the number of characters in the domain-specific language. The format specification is accomplished by filling in the format type within a pre-defined identifier. The pre-defined identifier is FORMAT[...], and the format type is either SECTION or SUBSECTION. A SECTION consists of a state machine segment, SECTION, and PART_END. Parameters are passed by adding specific parameter values ​​after the pre-defined identifier using a pre-defined connection method.

[0127] S102, receiving backend-recognizable language through the backend and converting the backend-recognizable language into basic operators, which include wait operators, write operators, and operators evolved based on the wait operators and write operators. The wait operator and the write operator are used to record the current legal word list or illegal word list. The wait operator is used to convert the word ID state machine state when a specific word appears, and the write operator is used to output a specific word at the current moment.

[0128] S103 , receiving domain-specific language input by the user through the reasoning framework, and converting the domain-specific language into a word-unit ID state machine.

[0129] S104 , constraining the large language model to generate content based on the basic operator corresponding to the word-gram ID state machine through the word-gram ID state machine to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

[0130] Figure 7 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 7 As shown, the electronic device 60 includes: a processor 601 (processor), a memory 602 (memory) and a bus 603;

[0131] The processor 601 and the memory 602 communicate with each other via the bus 603.

[0132] The processor 601 is used to call the program instructions in the memory 602 to execute the methods provided by the above-mentioned method embodiments, so as to execute the methods provided by the implementation methods of the present invention.

[0133] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the method provided by the embodiment of the present invention.

[0134] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various storage media that can store program codes.

[0135] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A large language model structured generation system, characterized by: include: The front end is used to receive domain-specific language input by the user and convert the domain-specific language into a language recognizable by the back end; a backend for receiving the backend-recognizable language and converting the backend-recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved from the wait operator and the write operator, wherein the wait operator and the write operator are used to record a current list of legal or illegal word units, the wait operator is used to convert a word unit ID state machine state when a specific word unit appears, and the write operator is used to output a specific word unit at the current moment; An inference framework for receiving domain-specific language input by a user and converting the domain-specific language into a word-unit ID state machine; The word-unit ID state machine is used to constrain the large language model to generate content based on the corresponding basic operators to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

2. The large language model structured generation system according to claim 1, characterized in that: When the front end converts the domain-specific language into a language recognizable by the back end, the time complexity of the compilation algorithm is limited to O(n), where n is the number of characters in the domain-specific language.

3. The large language model structured generation system according to claim 1, characterized in that: The domain specific language received by the front end as input by the user includes a format specifying part and a parameter inputting part.

4. The large language model structured generation system according to claim 1, characterized in that: The large language model structured generation system also includes a vocabulary mask, which interacts with the reasoning framework to filter or constrain the vocabulary during the large model generation process.

5. The large language model structured generation system according to claim 4, characterized in that: The wait operator and the write operator update the current legal word list or the illegal word list based on the word list mask.

6. The large language model structured generation system according to claim 1, characterized in that: The word unit ID state machine is also used to: The final word sequence is restricted to the language space described by the word ID state machine.

7. The large language model structured generation system according to claim 1, characterized in that: The word unit ID state machine is also used to: When the basic operator corresponding to the word-unit ID state machine is the wait operator, the wait operator will generate a word list mask constraint large language model through the legal word list or illegal word list recorded by itself; When the basic operator corresponding to the word-gram ID state machine is the write operator, the write operator will generate a large language model by outputting a vocabulary mask constraint based on the expected word-gram or word-gram list at the current moment recorded by itself.

8. A method for generating a large language model structured system, applied to the system according to any one of claims 1 to 7, characterized in that: The method includes: The front end receives the domain-specific language input by the user and converts the domain-specific language into a language recognizable by the back end; Receiving the backend-recognizable language through the backend and converting the backend-recognizable language into basic operators, wherein the basic operators include a wait operator, a write operator, and operators evolved based on the wait operator and the write operator, wherein the wait operator and the write operator are used to record a current legal word list or an illegal word list, the wait operator is used to convert a word ID state machine state when a specific word appears, and the write operator is used to output the specific word at the current moment; Receive domain-specific language input by the user through the reasoning framework and convert the domain-specific language into a word-unit ID state machine; The word-gram ID state machine is used to constrain the large language model to generate content based on the basic operator corresponding to the word-gram ID state machine, so as to ensure that the content generated by the large language model conforms to the corresponding grammatical structure.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to claim 8 are implemented.

10. A non-transitory computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 8 are implemented.

Images