Large model structured output method for heuristic beam search

Abstract

The invention relates to a large model structured output method for heuristic beam search. The method comprises the following steps: S1, designing a structured data representation template; s2, defining a real data structure output by the large model based on the structured data representation template; s3, transmitting the real data structure as an additional parameter, and performing coding and KV-cache calculation on the input data through the large model; s4, a round of decoding is carried out through the large model, the width of beam search is set by using a heuristic function, and sampling is carried out to obtain a candidate set; s5, traversing the candidate set through a large model, calculating a matching degree through a similarity measurement function, and performing screening; s6, iteration is continuously carried out through the large model, if an output ending symbol is encountered, the optimal path is reserved through comprehensive similarity calculation, and the method is ended, otherwise, the step S3 is executed again. According to the method, a heuristic search strategy and a beam search technology are combined, the efficiency and accuracy of a large model in a structured output task are improved, and the method is widely applied to the fields of automatic translation, text generation, classification and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning and natural language processing, in particular to a heuristic beam search large model structured output method. BACKGROUND

[0002] With the rapid development of deep learning technology, neural network-based generative models have made remarkable achievements in natural language processing, image generation, speech recognition, etc. In particular, large language models such as GPT and BERT have achieved excellent performance in various tasks, and structured data generation based on large models has become a research hotspot. However, although large models perform well in generation, they also face some unique challenges in generating structured data.

[0003] In structured output tasks, the generated results not only need to meet the rationality of grammar and semantics, but also must follow strict format requirements, such as key-value pair matching, accurate generation of nested structures, etc. This format constraint greatly increases the difficulty of structured output generation, especially when facing complex, multi-level structured data, traditional output sampling methods (such as maximum likelihood estimation) are prone to format errors, and cannot guarantee the integrity and correctness of the data structure. The output stability of large models is poor, and is easily affected by input disturbances, leading to more format errors or inconsistencies in the generated structured data. Secondly, compared to traditional models, large models consume more computing resources in the reasoning process, and the reasoning speed is slower, which may become a bottleneck in application scenarios that require real-time generation of large-scale structured data.

[0004] Therefore, there is an urgent need for a large model structured output method to solve the problems of low model reasoning efficiency, poor output accuracy, and lack of diversity. SUMMARY

[0005] To solve the technical problems existing in the prior art, the purpose of the present application is to provide a heuristic beam search large model structured output method, which combines heuristic search strategy and beam search technology, aiming to improve the efficiency and accuracy of large models in structured output tasks, and is widely used in automatic translation, text generation, classification, etc.

[0006] To achieve the above application purpose, the present application provides a heuristic beam search large model structured output method, comprising the following steps:

[0007] Step S1, based on object-oriented design, design a structured data representation template;

[0008] Step S2, according to the actual use scene of the large model, define the real data structure output by the large model based on the structured data representation template;

[0009] Step S3: pass the real data structure as an additional parameter, and encode the input data and perform KV-cache calculation through the large model;

[0010] Step S4: performing a round of decoding on the encoded input data and the calculated KV-cache using the large model, setting the width of the beam search using a heuristic function, and sampling according to the width of the beam search to obtain a candidate set;

[0011] Step S5: traverse the tokens in the candidate set using the large model, determine the matching degree between the path sequence corresponding to the token and the real data structure using a similarity measurement function, retain the token sequences with the best matching degree, and complete one iteration;

[0012] Step S6: Continue iterating through the large model until an end symbol is output, then retain the optimal path through comprehensive similarity calculation and end, otherwise return to step S3.

[0013] According to a technical solution of the present invention, in step S1, the structured data representation template at least includes:

[0014] Structure declaration key;

[0015] Structure declaration value: used to define the structure of a class; the structure declaration value includes several class keyword fields; the class keyword fields include mandatory class keyword fields and optional class keyword fields, the mandatory class keyword fields include class name field, type field, fields that must be initialized to construct the class, and class attribute fields; the optional class keyword fields include nested structure definition fields.

[0016] According to a technical solution of the present invention, the class attribute field is in the form of a Key-Value pair, the key value of the class attribute field is the attribute name, and the Value value of the class attribute field is used to define the structure of the attribute; the Value value of the class attribute field includes several attribute keywords, and the attribute keywords include necessary keywords and optional keywords. The necessary keywords include the attribute name and the attribute type, and the attribute type is used to define the type of the attribute keyword; the optional keywords are used to constrain the data structure, including one or more of the description class attribute field, the example class attribute field, the numerical constraint field, and the string length constraint field.

[0017] According to a technical solution of the present invention, the structured data representation template is represented using a Json structure.

[0018] According to a technical solution of the present invention, in step S4, the heuristic function is expressed as:

[0019] W=W base f(F, L, D)

[0020]

[0021] Among them, W base represents the basic beam search width; f(F,L,D) represents the dynamic adjustment factor, where F represents the number of defined data structure fields, L represents the nesting level of the data structure; D represents the depth of the current beam search path; α represents the factor that controls the impact of path depth on search width; β represents the factor that controls the impact of the number of fields on search width; γ represents the factor that controls the impact of nesting level on search width.

[0022] According to a technical solution of the present invention, in step S5, determining the matching degree between the path sequence corresponding to each token in the candidate set and the real data structure by using a similarity measurement function specifically includes:

[0023] Step S41: Based on the field name, type and constraint of the data in each token, perform field name similarity, type similarity and constraint similarity matching calculations with the real data structure respectively;

[0024] Step S42: perform weighted averaging on the similarity functions of the M data in the path to calculate the comprehensive similarity;

[0025] The calculation formula of the comprehensive similarity is:

[0026] Sim(s,D)=w1*name_sim(s,D)+w2*type_match(s,D)+w3*constraint_matc h(s,D)

[0027] Among them, s represents a path sequence where a token in the candidate set is located, D represents the real data structure, Sim(s,D) represents the comprehensive similarity between the path sequence s and the real data structure D, name_sim(s,D) is the field name similarity between the path sequence s and the real data structure D, type_match(s,D) is the similarity between the data type of the path sequence s and the real data structure D, constraint_match(s,D) is the similarity between the data value in the path sequence s and the constraints defined in the real data structure D; w1, w2 and w3 are the coefficients of field name similarity, type similarity and constraint similarity, respectively.

[0028] According to one aspect of the present invention, an electronic device includes: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to enable the electronic device to perform the above-mentioned large model structured output method of heuristic beam search.

[0029] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions, wherein when the computer instructions are executed by a processor, the large model structured output method of the heuristic beam search is implemented.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This invention provides a large-scale structured output method for heuristic beam search, significantly improving inference efficiency and generation diversity while optimizing computing resource utilization. Specifically, the invention first designs a set of universal structured data representation templates, based on which users can customize arbitrarily complex structured data. This flexible template design not only allows users to efficiently organize and store data but also allows them to quickly obtain structured data that meets their needs during the generation process, thereby achieving more efficient data processing and adaptation to application scenarios.

[0032] In addition, the present invention also designs a set of heuristic functions that can dynamically adjust the search width of the beam search algorithm based on the defined structured data. Traditional beam search algorithms typically use a fixed width during the search process, which can lead to inefficient or insufficiently diverse results in some scenarios. However, the heuristic functions of the present invention can flexibly adjust the search width based on the characteristics and constraints of the current data structure, thereby significantly improving search efficiency and enhancing the diversity of generated results while ensuring generation quality. This innovative heuristic adjustment mechanism not only improves generation efficiency but also effectively avoids overcomputation and the generation of redundant paths when processing complex data structures.

[0033] To further improve inference efficiency, the present invention also provides a similarity calculation function for measuring the correlation between the generated path sequences and the defined structured data. This calculation function can accurately identify which path sequences are closely related to the target data structure and which are invalid paths that deviate from the target structure. By filtering the generated paths during the inference process and eliminating irrelevant or low-quality path sequences, this effectively saves memory overhead, reduces unnecessary computation, and improves inference speed. It also helps optimize the precision and accuracy of the generation process, further enhancing the intelligence of the generation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0035] Figure 1 The flowchart of the method for outputting a large model structure by heuristic beam search according to one embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0036] The description of the embodiments in this specification should be combined with the corresponding drawings, which should be considered a complete part of this specification. In the drawings, the shapes and thicknesses of the embodiments may be exaggerated and indicated for simplicity or convenience. Furthermore, the various structural components in the drawings will be described separately. It is worth noting that components not shown in the drawings or not described in words are known to those of ordinary skill in the art.

[0037] The description of the embodiments herein and any references to directions and orientations are for ease of description only and are not to be construed as limiting the scope of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination. The present invention is not specifically limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0038] like Figure 1 As shown, a large model structured output method of a heuristic beam search of the present invention includes the following steps:

[0039] Step S1: designing a structured data representation template based on object-oriented design;

[0040] In step S1, the structured data representation template includes at least:

[0041] Structure declaration key;

[0042] Structure declaration value: used to define the structure of a class; the structure declaration value includes several class keyword fields; the class keyword fields include required class keyword fields and optional class keyword fields, the required class keyword fields include the class name field, type field, fields that must be initialized when building the class, and class attribute fields; the optional class keyword fields include nested structure definition fields.

[0043] The class attribute field adopts the form of Key-Value key-value pairs. The key value of the class attribute field is the attribute name, and the Value value of the class attribute field is used to define the structure of the attribute; the Value value of the class attribute field includes several attribute keywords, and the attribute keywords include required keywords and optional keywords. The required keywords include the attribute name and attribute type, and the attribute type is used to define the type of the attribute keyword; the optional keywords are used to constrain the data structure, including one or more of the description class attribute field, example class attribute field, numerical constraint field, and string length constraint field.

[0044] In one embodiment of the present invention, the structured data representation template may be represented using a Json structure, such as:

[0045] {

[0046] '$defs':

[0047] {'Class A':

[0048] {'properties':{'properties':{'title':'Property','description':','type':'string'},

[0049] 'required':['attribute'],

[0050] 'title':'Class A','type':'object'}},

[0051] 'properties':{'properties':{'items':{'$ref':'# / $defs / Class A'},

[0052] 'title':'Metrics','type':'array'}},

[0053] 'required':['attribute'],

[0054] 'title':'Class B','type':'object'

[0055] }

[0056] In this embodiment, the structure declaration must use $defs as the key and the value in the form of a Json object, which is used to define the structure of the class. Its keywords are: title, required, type, and properties; title defines the class name, type defines the type (fixed to be object), required defines the fields that must be initialized to build the class structure, and properties defines the attributes (fields) of the class.

[0057] properties defines various attributes. Each attribute is in the form of a Key-Value pair. The key value is the attribute name, and the structure of the attribute is defined in the Value. The attribute structure is also defined in the form of a Json object. The required keywords are: title and type. When defining an attribute, the value range of type is string (string type), int (integer value type), float (floating point value type), bool (Boolean type), bytes (byte type), list (array type), object (object type), dict (dictionary type), date (date type), time (timestamp type), and enum (enumeration type), which basically covers common data types.

[0058] Optional keywords for attribute definition: description describes the attribute field; examples provides examples of attribute values ​​to inspire large model reasoning; the keywords gt (greater than), lt (less than), gte (greater than or equal to), and lte (less than or equal to) for numeric types are used to constrain the value size obtained during reasoning; and max_length and min_length for string types are used to constrain the length of the string.

[0059] If there is a nested structure, such as the attribute type of class B is class A, use {'$ref':'# / $defs / class A'} to define it.

[0060] Step S2: Define the real data structure output by the large model based on the structured data representation template according to the actual usage scenario of the large model;

[0061] Step S3: pass the real data structure as an additional parameter, and encode the input data and perform KV-cache calculation through the large model;

[0062] The large model service encodes the prompt and input, calculates the KV-cach, and then enters the Decode stage.

[0063] Step S4: performing a round of decoding on the encoded input data and the calculated KV-cache using the large model, setting the width of the beam search using a heuristic function, and sampling according to the width of the beam search to obtain a candidate set;

[0064] In step S4, in the Decode stage, a round of Decode output is performed through the large model to obtain the word probability distribution of the Token table, call the heuristic function to set the width of the beam search, and sample according to the set width to generate a new token table as the candidate set for the next round of iteration.

[0065] In step S4, the heuristic function is expressed as:

[0066] W=W base f(F, L, D)

[0067]

[0068] Among them, W base Indicates the basic beam search width, which can usually be set to an initial value (such as 5 or 10);

[0069] f(F, L, D) represents a dynamic adjustment factor that determines how the search width adjusts based on the number of fields, nesting level, and path depth. F represents the number of fields in the defined data structure. L represents the nesting level of the data structure. The deeper the nesting level, the more complex the structure, requiring more width to cover each level. D represents the depth of the current beam search path, typically the length of the generated path sequence.

[0070] α: controls the effect of path depth on search width. A larger α will make the path depth have a greater impact on the search width, resulting in a gradual decrease in width when searching for a deeper path. The default value is 0.3.

[0071] β: controls the effect of the number of fields on the search width. A larger β will make the number of fields have a greater impact on the search width. When there are more fields, the search width will increase. The default value is 0.1.

[0072] γ: controls the effect of nesting levels on search width. A larger γ will make the nesting level have a greater impact on search width. When the nesting level is deep, the width will increase. The default value is 0.6.

[0073] Step S5: traverse the tokens in the candidate set, use the similarity measurement function Sim(s,D) to determine the matching degree between the path sequence corresponding to each token in the candidate set and the real data structure, and retain the best several token sequences for the next round of iteration of the large model;

[0074] In step S5, the large model uses a similarity measurement function to determine the matching degree between the path sequence corresponding to each token in the candidate set and the real data structure, specifically including:

[0075] Step S51: Based on the field name, type and constraint of the data in each token, perform field name similarity, type similarity and constraint similarity matching calculations with the real data structure respectively;

[0076] In step S51, similarity matching calculation is performed by a Jaccard-based field name, type, constraint matching function. Taking the field name matching function match(fj, vij, D) as an example, when calculating the field name matching degree, fj represents the field name list of the defined real data structure D, vij represents the field name list of the i-th data in the path corresponding to the j-th Token sequence in the beam search, when vij exists in fj, it is 1, otherwise it is 0; the matching calculation logic of type and constraint is consistent.

[0077] Step S52, weighted average is performed on each similarity function of the M data in the path, and the comprehensive similarity is calculated;

[0078] The calculation formula of the comprehensive similarity is:

[0079] Sim(s, D) = w1*name_sim(s, D) + w2*type_match(s, D) + w3*constraint_match(s, D)

[0080] Wherein, s represents a path sequence in which a token in the candidate set is located, D represents a real data structure, Sim(s, D) represents the comprehensive similarity of the path sequence s and the real data structure D, name_sim(s, D) is the field name similarity of the path sequence s and the real data structure D, type_match(s, D) is the data type similarity of the path sequence s and the real data structure D, constraint_match(s, D) is the similarity of the data value in the path sequence s and the constraint defined in the real data structure D; w1, w2 and w3 are respectively the coefficients of the field name similarity, the type similarity and the constraint similarity.

[0081] Step S6, continuously iterate through the large model until the output end symbol, then retain the optimal path through the comprehensive similarity calculation and end, otherwise return to step S3.

[0082] According to one aspect of the present application, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected with the memory, the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a heuristic beam search large model structured output method according to any one of the above technical solutions.

[0083] The processor may be a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0084] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a large model structured output method of a heuristic beam search as described in any one of the above technical solutions is implemented.

[0085] Computer-readable storage media may include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments may be downloaded via a computer network such as the Internet, an intranet, and the like.

[0086] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0087] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0089] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0090] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A large model structured output method based on heuristic beam search, characterized in that: The following steps are involved: Step S1: designing a structured data representation template based on object-oriented design; Step S2: defining the real data structure output by the large model based on the structured data representation template according to the actual usage scenario of the large model; Step S3: pass the real data structure as an additional parameter, and encode the input data and perform KV-cache calculation through the large model; Step S4: performing a round of decoding on the encoded input data and the calculated KV-cache using the large model, setting the width of the beam search using a heuristic function, and sampling according to the width of the beam search to obtain a candidate set; Step S5: traverse the tokens in the candidate set using the large model, determine the matching degree between the path sequence corresponding to the token and the real data structure using a similarity measurement function, retain the token sequences with the best matching degree, and complete one iteration; Step S6: Continue to iterate through the large model. If an output end symbol is encountered, retain the optimal path through comprehensive similarity calculation and end. Otherwise, return to step S3.

2. The large model structured output method of heuristic beam search according to claim 1, characterized in that: In step S1, the structured data representation template includes at least: Structure declaration key; Structure declaration value: used to define the structure of a class; the structure declaration value includes several class keyword fields; the class keyword fields include mandatory class keyword fields and optional class keyword fields, the mandatory class keyword fields include class name field, type field, fields that must be initialized to construct the class, and class attribute fields; the optional class keyword fields include nested structure definition fields.

3. The large model structured output method of heuristic beam search according to claim 2, characterized in that: The class attribute field is in the form of a Key-Value pair, the key value of the class attribute field is the attribute name, and the Value value of the class attribute field is used to define the structure of the attribute; the Value value of the class attribute field includes several attribute keywords, and the attribute keywords include necessary keywords and optional keywords. The necessary keywords include the attribute name and the attribute type, and the attribute type is used to define the type of the attribute keyword; the optional keywords are used to constrain the data structure, including one or more of the description class attribute field, the example class attribute field, the numerical constraint field, and the string length constraint field.

4. The large model structured output method of heuristic beam search according to claim 3, characterized in that: The structured data representation template is represented using a Json structure.

5. The large model structured output method of heuristic beam search according to claim 1, characterized in that: In step S4, the heuristic function is expressed as: W=W base ·f(F,L,D) Among them, W base represents the basic beam search width; f(F,L,D) represents the dynamic adjustment factor, where F represents the number of defined data structure fields, L represents the nesting level of the data structure; D represents the depth of the current beam search path; α represents the factor that controls the impact of path depth on search width; β represents the factor that controls the impact of the number of fields on search width; γ represents the factor that controls the impact of nesting level on search width.

6. The large model structured output method of heuristic beam search according to claim 5, characterized in that: In step S5, the similarity measurement function is used to determine the matching degree between the path sequence corresponding to each token in the candidate set and the real data structure, specifically including: Step S41: Based on the field name, type and constraint of the data in each token, perform field name similarity, type similarity and constraint similarity matching calculations with the real data structure respectively; Step S42: perform weighted averaging on the similarity functions of the M data in the path to calculate the comprehensive similarity; The calculation formula of the comprehensive similarity is: Sim(s,D)=w1*name_sim(s,D)+w2*type_match(s,D)+w3*constraint_matc h(s,D) Among them, s represents a path sequence where a token in the candidate set is located, D represents the real data structure, Sim(s,D) represents the comprehensive similarity between the path sequence s and the real data structure D, name_sim(s,D) is the field name similarity between the path sequence s and the real data structure D, type_match(s,D) is the similarity between the data type of the path sequence s and the real data structure D, constraint_match(s,D) is the similarity between the data value in the path sequence s and the constraints defined in the real data structure D; w1, w2 and w3 are the coefficients of field name similarity, type similarity and constraint similarity, respectively.

7. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to enable the electronic device to perform the large model structured output method of heuristic beam search as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, implement the large model structured output method of heuristic beam search according to any one of claims 1 to 6.

Images