Data structured large model training and reasoning method, device and equipment
By constructing tree structure sequences for large language models and improving training methods, the difficulty of capturing subtle information in medical data processing and pattern crashes of large language models is solved, the accuracy and stability of data structured processing are improved, and the Its application scope.
Patent Information
- Application Number
- CN202411356441.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-14
AI Technical Summary
Existing large language models are difficult to capture subtle information in medical data processing, resulting in the granularity of data structured results being too coarse and frequent pattern crashes, affecting their stability and reliability, limiting their application effect and scope in medical data analysis. .
By obtaining the text to be structured, the preset processing method is marked and segmented, the tree structure sequence after word frequency sorting is constructed, and large language models are trained using improved instruction fine-tuning methods, and mask maskgen is introduced to improve Model stability and processing quality.
Refine the granularity of data structured results, improve the accuracy of processing results, reduce the loss rate of important medical information, ensure the stability and reliability of large language models, and expand its application scope in data structured processing.
Smart Images
Figure CN119312920A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer information processing technology, and in particular to a data structured large model training and reasoning method, device and equipment. Background Art
[0002] With the widespread application of artificial intelligence technology in the medical field, large language models (LLM) play an increasingly important role in medical data processing.
[0003] In the existing technology, large language models are used to structure data related to medical tests and examinations. Although good data structuring results can be obtained to a certain extent, there are still challenges. For example, the granularity of the data structuring results is too coarse, making it difficult to capture subtle but potentially crucial medical information; and the excessive number of identical entity names in the medical data that causes mode collapse, which greatly reduces the stability and reliability of large language models. The above problems seriously restrict the application effect and scope of large language models in medical data analysis.
[0004] Therefore, how to improve the processing quality of large language models while applying them for structured data processing is an urgent problem to be solved. Summary of the invention
[0005] Based on the above-mentioned problems existing in the prior art, a technical solution of a data-structured large model training and reasoning method, device, and equipment is now provided, aiming to solve the problem in the prior art that the processing quality of the large language model cannot be improved while applying the large language model for data structured processing.
[0006] To achieve the above-mentioned purpose and other related purposes, the present invention provides a data-structured large model training and reasoning method, device, and equipment. The data-structured large model training and reasoning method includes the following steps: obtaining the text to be structured, annotating and segmenting it according to a preset processing method, and obtaining a data set to be structured; sorting the content in the data set according to the annotations; sorting the content based on the content in the data set according to a preset word frequency sorting method; constructing a tree structure sequence based on the sorting results; using the text as the input of a large language model and the tree structure sequence as the output of the large language model to train the large language model.
[0007] In one embodiment of the present invention, annotation and segmentation are performed according to a preset processing method to obtain a data set to be structured, including: based on the text, according to the preset field columns corresponding to the text, the text is annotated by applying an inclusion detection method; based on the annotated text, according to the association relationship between the annotated contents and the logic of sentence segmentation, the text is segmented to obtain a data set; wherein the field column includes an entity name, a feature name corresponding to the entity name, a feature value corresponding to the feature name, and a unit corresponding to the feature name.
[0008] In one embodiment of the present invention, based on the content in the data set, the content is sorted according to a preset word frequency sorting method, including: counting the word frequency P of the first content in the column where the first field to which the first content belongs is located; the first content is any content belonging to the first field; the first field is any field in the field column; setting a first threshold M, based on the word frequency P, determining the pattern low The corresponding field set when it is 1; pattern low The expression is:
[0009]
[0010] Among them, len(C i ) represents the total number of categories of all contents in the first field of column i, pattern low Indicates the word frequency attribute of the first field, pattern low 1 indicates that the first field is a low mode field; pattern low 0 indicates that the first field is a non-low mode field; is the distribution function of the word frequency P, which indicates the word frequency when the first content in the first field of the i-th column is X, and n indicates the maximum column value of the column where the first field is located; based on the field set, according to Sort the first content in descending order.
[0011] In one embodiment of the present invention, a tree structure sequence is constructed based on the sorting result, including: based on the result of descending sorting of the word frequency of the first content, the first content is arranged and organized according to the hierarchy of the tree structure to construct the tree structure sequence.
[0012] In one embodiment of the present invention, before constructing the tree structure sequence, the method further includes: Determine the pattern complexity of the first field; modify the pattern complexity, and the expression of the modified pattern complexity is:
[0013]
[0014] Among them, P numnerIndicates the ratio of the number of words with the first content X in the column where the first field is located and the frequency of which is greater than the first threshold M, to the number of words with the frequency of which is greater than the first threshold M in all the contents of the field column; F number Representing the corrected pattern complexity; arranging and organizing the first content according to the hierarchy of the tree structure; when the difference between two corrected pattern complexities is less than a preset second threshold, arranging and organizing the two fields corresponding to the two corrected pattern complexities as brother nodes of the same hierarchy of the tree structure.
[0015] In one embodiment of the present invention, a large language model is trained, including: applying an improved instruction fine-tuning method to train the large language model, and introducing a mask into the training corpus. gen ; Mask gen The expression is:
[0016]
[0017] Among them, mask gen 1 indicates the mask set when inputting the instruction part of the training corpus gen value;mask gen A value of 0 indicates the mask corresponding to the desired output setting. gen Value; set mask gen The corresponding learning rate l r The expression is:
[0018]
[0019] Among them, mask gen When the learning rate is 1, r Less than mask gen The corresponding learning rate l is 0 r .
[0020] In one embodiment of the present invention, after training the large language model, it also includes: obtaining the text to be processed; based on the text to be processed, applying the trained large language model to determine the corresponding tree structure sequence; based on the tree structure sequence, applying deserialization and decoding processing to determine the data structuring result corresponding to the text to be processed.
[0021] Correspondingly, the present invention provides a data-structured large model training and reasoning device, including: a first processing module, used to obtain the text to be structured, annotate and segment it according to a preset processing method, and obtain the data set to be structured; the content in the data set is sorted according to the annotations; a second processing module, used to sort the content based on the content in the data set according to a preset word frequency sorting method; a construction module, used to construct a tree structure sequence based on the sorting result; a training module, used to use the text as the input of the large language model and the tree structure sequence as the output of the large language model to train the large language model.
[0022] The present invention provides a device, including a processor and a memory storing a computer program. When the computer program is executed by the processor, a large model training and reasoning method is implemented.
[0023] The present invention provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned large model training and reasoning method is implemented.
[0024] As described above, the data structured large model training and reasoning method, device, and apparatus of the present invention have the following beneficial effects:
[0025] (1) It can refine the granularity of data structured results, improve the accuracy of processing results, and reduce the loss rate of important medical information.
[0026] (2) Ensure the stability and reliability of large-scale language models.
[0027] (3) Expand the application scope of large language models in structured data processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0029] Figure 1 Shown is a flow chart of a data-structured large model training and inference method in one embodiment of the present invention.
[0030] Figure 2 Shown is a schematic diagram of the structure of a data-structured large model training and reasoning device in one embodiment of the present invention.
[0031] Figure 3 Shown is a device in one embodiment of the data structured large model training and inference apparatus of the present invention.
[0032] Component number description
[0033] 21 First processing module 22 Second processing module 23 Building Blocks 24 Training Module 31 processor 32 Memory DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0036] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0037] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0038] The data structured large model training and reasoning method, device, and equipment of the present invention can refine the granularity of data structured results, improve the accuracy of processing results, and reduce the loss rate of important medical information; and can ensure the stability and reliability of the operation of large language models; further, it can expand the application scope of large language models in data structured processing.
[0039] like Figure 1 As shown, in one embodiment, the data structured large model training and reasoning method of the present invention includes the following steps:
[0040] Step S1, obtain the text to be structured, annotate and segment it according to a preset processing method, and obtain the data set to be structured; the content in the data set is sorted according to the annotations.
[0041] Specifically, based on the text, the text is annotated by applying the inclusion detection method according to the preset field columns of the corresponding text; based on the annotated text, the text is segmented according to the association relationship between the annotated contents and the logic of sentence segmentation to obtain a data set; wherein the field column includes the entity name, the feature name corresponding to the entity name, the feature value corresponding to the feature name, and the unit corresponding to the feature name.
[0042] Specifically, the text is annotated with the feature name, feature value, and unit associated with the specific content in the field column as a reference, and the annotated specific content and the corresponding feature name, feature value, and unit are taken as a subsequence. Based on the subsequence, the shortest subsequence matching it in the text is determined; based on the shortest subsequence, the punctuation marks at both ends of the shortest subsequence are queried according to the sentence segmentation logic formed by the punctuation marks; based on the punctuation marks at both ends, the text is segmented to obtain a data set.
[0043] For example, the employees of Company A undergo physical examinations, and the ultrasound examination reports of Ms. Zhang, Xiao Wang (male), and Ms. Li are sampled, including thyroid color ultrasound and breast ultrasound.
[0044] Ms. Zhang's thyroid color Doppler ultrasound described the following as follows: the thyroid gland is normal in shape and size, with a low-echo nodule in the right lobe, about 2*1mm in size, regular in shape, clear in borders, and uniform in internal echoes. No obvious abnormal mass echoes were found in the left lobe. Breast ultrasound described the following as follows: the hierarchical structure of the breast glands on both sides is clear, with a low-echo nodule in the right upper inner gland, regular in shape, smooth in margins, and about 1*1mm in size. No obvious abnormal mass echoes were found in the left breast.
[0045] Xiao Wang's thyroid color Doppler ultrasound showed that the thyroid gland was normal in shape and size, with a low-echo nodule in the left lobe, about 4*3mm in size, with regular shape, clear boundaries, and uneven internal echoes. No obvious abnormal mass echo was found in the right lobe.
[0046] Ms. Li's thyroid color Doppler ultrasound described the following as follows: the thyroid gland is normal in shape and size, with a low-echo nodule in the right lobe, about 1*1mm in size, regular in shape, clear in borders, and uniform in internal echoes. No obvious abnormal mass echoes were found in the left lobe. Breast ultrasound described the following as follows: the hierarchical structure of the breast glands on both sides is clear, and a low-echo nodule can be seen in the right upper inner gland, with regular shape, smooth edges, and about 3*2mm in size. No obvious abnormal mass echoes were found in the left breast.
[0047] After collecting the thyroid color ultrasound and breast ultrasound description information of Ms. Zhang, Xiao Wang (male), and Ms. Li, the specific content corresponding to the entity name in the field column here is thyroid and breast; with the thyroid and breast as references, the corresponding feature names, feature values, and units are marked in the above description information. For example, Ms. Zhang's physical examination report is marked with: thyroid, right lobe, nodule, 2*1, mm, left lobe, not seen; breast, right inner upper side, nodule, 1*1, mm, left side, not seen. Xiao Wang's physical examination report is marked with: thyroid, left lobe, nodule, 4*3, mm, right lobe, not seen. Ms. Li's physical examination report is marked with: thyroid, right lobe, nodule, 1*1, mm, left lobe, not seen; breast, right inner upper side, nodule, 3*2, mm, left side, not seen.
[0048] Take thyroid, right lobe, nodule, 2*1, mm, left lobe, not seen as a subsequence; breast, right inner upper side, nodule, 1*1, mm, left side, not seen as a subsequence, and so on. Based on the above subsequences, determine the shortest subsequence that matches it in the text, and query the punctuation marks at both ends of the shortest subsequence according to the sentence segmentation logic formed by the punctuation marks to segment the text and obtain the data set. For example, for the subsequence of thyroid, right lobe, nodule, 2*1, mm, left lobe, not seen, the data set obtained includes: thyroid, right lobe nodule, 2*1, mm; thyroid, left lobe, not seen. And so on, the data set obtained in the above example is shown in Table 1 below.
[0049] Table 1
[0050] Entity Name Feature Name Eigenvalue unit thyroid Right lobe nodule 2*1 mm thyroid Left lobe Not seen Breast Right superior medial tubercle 1*1 mm Breast Left Not seen thyroid Right lobe Not seen thyroid Left lobe nodule 4*3 mm thyroid Right lobe nodule 1*1 mm thyroid Left lobe Not seen Breast Right superior medial tubercle 3*2 mm Breast Left Not seen
[0051] Through the above processing, it can be ensured that the necessary complete information in the text is not truncated, and the content in the data set comprehensively and accurately reflects the part of the text to be structured. In addition, in actual applications, the number of samples is at least in the thousands. This example only shows the processing method because the sampling volume is small.
[0052] Step S2: sort the content based on the content in the data set according to a preset word frequency sorting method.
[0053] Specifically, the word frequency P of the first content in the column where the first field belongs to the first content is counted; the first content is any content belonging to the first field; the first field is any field in the field column; a first threshold M is set, and based on the word frequency P, a pattern is determined low The corresponding field set when it is 1; pattern low The expression is:
[0054]
[0055] Among them, len(C i) represents the total number of categories of all contents in the first field of column i, pattern low Indicates the word frequency attribute of the first field, pattern low 1 indicates that the first field is a low mode field; pattern low 0 indicates that the first field is a non-low mode field; is the distribution function of the word frequency P, which indicates the word frequency when the first content in the first field of the i-th column is X, and n indicates the maximum column value of the column where the first field is located; based on the field set, according to Sort the first content in descending order.
[0056] Step S3: construct a tree structure sequence based on the sorting results.
[0057] Specifically, based on the result of descending sorting of the word frequencies of the first contents, the first contents are arranged and organized according to the levels of the tree structure to construct a tree structure sequence.
[0058] For example, after the first content is sorted in descending order according to the frequency of words, the first content corresponding to the first word frequency is used as the root node content of the tree structure sequence. The child node content of the next level of the root node is to exclude the first content attribution field, and sort the feature name, feature value, and unit content corresponding to the first content in other fields in descending order. According to the sorting result, the sibling nodes of this level from left to right are determined, and so on, to construct the next level nodes of the tree structure until the leaf node. In addition, for the other contents of the first content attribution field, each is used as a separate root node to arrange and construct the tree structure.
[0059] More specifically, based on Determine the pattern complexity of the first field; modify the pattern complexity, and the expression of the modified pattern complexity is:
[0060]
[0061] Among them, P number Indicates the ratio of the number of words with the first content X in the column where the first field is located and the frequency of which is greater than the first threshold M, to the number of words with the frequency of which is greater than the first threshold M in all the contents of the field column; F number Representing the corrected pattern complexity; arranging and organizing the first content according to the hierarchy of the tree structure; when the difference between two corrected pattern complexities is less than a preset second threshold, arranging and organizing the two fields corresponding to the two corrected pattern complexities as brother nodes of the same hierarchy of the tree structure.
[0062] Through the above processing, the tree structure sequence constructed by the first content is adjusted according to the corrected pattern complexity, so that the data structuring result presented by the tree structure sequence corresponding to the text can be ensured to be more accurate.
[0063] Step S4: Use the text as the input of the large language model and the tree structure sequence as the output of the large language model to train the large language model.
[0064] Specifically, we apply an improved instruction fine-tuning method to train large language models and introduce masks into the training corpus. gen ; Mask gen The expression is:
[0065]
[0066] Among them, mask gen 1 indicates the mask set when inputting the instruction part of the training corpus gen value;mask gen A value of 0 indicates the mask corresponding to the desired output setting. gen value.
[0067] More specifically, set the mask gen The corresponding learning rate l r The expression is:
[0068]
[0069] Among them, mask gen When the learning rate is 1, r Less than mask gen The corresponding learning rate l is 0 r .
[0070] In addition, there are no requirements for parameter selection of large language models. The scale of large language models is usually determined based on actual data and inference costs, and there is no standard limitation, so it will not be elaborated here.
[0071] More specifically, after the large language model is trained, it also includes: obtaining the text to be processed; based on the text to be processed, applying the trained large language model to determine the corresponding tree structure sequence; based on the tree structure sequence, applying deserialization and decoding processing to determine the data structuring result corresponding to the text to be processed.
[0072] For example, using a trained large language model, the text data to be structured is input into the large language model to obtain the corresponding tree structure sequence; deserialization and decoding processing are applied to the tree structure sequence to obtain the final data structuring result.
[0073] Through the processing of the above steps S1-S4, the single field problem existing in the traditional application of large language models for data structured processing can be greatly improved, the quality and efficiency of data structured processing can be improved, and medical decision-making and research can be better supported.
[0074] like Figure 2 As shown, in one embodiment, the data structured large model training and reasoning device of the present invention includes:
[0075] The first processing module 21 is used to obtain the text to be structured, annotate and segment it according to a preset processing method, and obtain the data set to be structured; the content in the data set is sorted according to the annotations.
[0076] The second processing module 22 is used to sort the content based on the content in the data set according to a preset word frequency sorting method.
[0077] The construction module 23 is used to construct a tree structure sequence based on the sorting result.
[0078] The training module 24 is used to train the large language model by taking the text as the input of the large language model and the tree structure sequence as the output of the large language model.
[0079] The technical features specifically implemented in the data structured large model training and reasoning device of this embodiment are basically the same as the principles of each step in the data structured large model training and reasoning method in Example 1, and the technical contents that are common between the method and the device will not be repeated.
[0080] The storage medium of the present invention stores a computer program, which, when executed by a processor, implements the above-mentioned data structured large model training and reasoning method.
[0081] like Figure 3 As shown, in one embodiment, the data structured large model training and reasoning device of the present invention includes: a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, the steps in the above-mentioned data structured large model training and reasoning method embodiment are implemented. Alternatively, when the processor 31 executes the computer program, the functions of each module / unit in the above-mentioned data structured large model training and reasoning device embodiment are implemented.
[0082] For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 32, and executed by the processor 31 to complete the present disclosure. One or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program in the data structured large model training and reasoning device.
[0083] The data structured large model training and reasoning device can be composed of computer devices such as desktop computers, notebooks, PDAs, and cloud servers. The data structured large model training and reasoning device can include but is not limited to a processor 31 and a memory 32. Those skilled in the art can understand that Figure 3 It is only an example of a data structured big model training and reasoning device and does not constitute a limitation of the data structured big model training and reasoning device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the data structured big model training and reasoning device may also include input and output devices, network cable access equipment, buses, etc.
[0084] The processor 31 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP); or a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.
[0085] The memory 32 may be an internal storage unit of the data structured large model training and reasoning device, for example, a hard disk or memory of the data structured large model training and reasoning device. The memory 32 may also be an external storage device of the data structured large model training and reasoning device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the data structured large model training and reasoning device. Furthermore, the memory 32 may also include both an internal storage unit and an external storage device of the data structured large model training and reasoning device. The memory 32 is used to store computer programs and other programs and data required by the computer device. The memory 32 may also be used to temporarily store data that has been output or is to be output.
[0086] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0087] In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.
[0088] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software development medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.
[0089] In addition, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or equipment including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or equipment.
[0090] In addition, in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.
Claims
1. A data structured large model training and reasoning method, characterized in that: include: Acquire the text to be structured, annotate and segment it according to a preset processing method, and obtain a data set to be structured; the content in the data set is sorted according to the annotations; Based on the content in the data set, sorting the content according to a preset word frequency sorting method; Based on the sorting results, a tree structure sequence is constructed; The text is used as the input of a large language model, and the tree structure sequence is used as the output of the large language model to train the large language model.
2. The large model training and reasoning method according to claim 1, characterized in that: The labeling and segmentation are performed according to the preset processing method to obtain the data set to be structured, including: Based on the text, according to the preset field columns corresponding to the text, an inclusion detection method is applied to annotate the text; Based on the annotated text, the text is segmented according to the association relationship between the annotated contents and the logic of sentence segmentation to obtain the data set; The field column includes an entity name, a feature name corresponding to the entity name, a feature value corresponding to the feature name, and a unit corresponding to the feature name.
3. The large model training and reasoning method according to claim 2, characterized in that: The step of sorting the content based on the content in the data set according to a preset word frequency sorting method includes: Counting the word frequency P of the first content in the column where the first field to which the first content belongs is located; the first content is any content belonging to the first field; the first field is any field in the field column; Set a first threshold M, and determine the pattern based on the word frequency P low The corresponding field set when it is 1; the pattern low The expression is: Among them, the len(C i ) represents the total number of categories of all contents in the first field of the i-th column, the pattern low represents the word frequency attribute of the first field, the pattern low is 1, indicating that the first field is a low mode field; the pattern low 0 indicates that the first field is a non-low mode field; is a distribution function of the word frequency P, indicating the word frequency when the first content in the first field of the i-th column is X, and n indicates the maximum column value of the column where the first field is located; Based on the field set, The first contents are sorted in descending order.
4. The large model training and reasoning method according to claim 3, characterized in that: The step of constructing a tree structure sequence based on the sorting result includes: Based on the result of the descending sorting of the word frequencies of the first contents, the first contents are arranged and organized according to the levels of the tree structure to construct the tree structure sequence.
5. The large model training and reasoning method according to claim 4, characterized in that: Before constructing the tree structure sequence, the method further includes: Based on the determining a pattern complexity of the first field; The pattern complexity is corrected, and the expression of the corrected pattern complexity is: Among them, the P number Indicates that the first content of the column where the first field is located is the ratio of the number of words whose frequency of the word X is greater than the first threshold M to the number of words whose frequency of the word in all the contents of the field column is greater than the first threshold M; the F num ber represents the complexity of the modified pattern; Arrange and organize the first content according to the hierarchy of the tree structure; when the difference between two corrected pattern complexities is less than a preset second threshold, arrange and organize the two fields corresponding to the two corrected pattern complexities as brother nodes of the same hierarchy of the tree structure.
6. The large model training and reasoning method according to claim 1, characterized in that: The training of the large language model comprises: Apply the improved instruction fine-tuning method to train the large language model and introduce mask into the training corpus gen ; The mask mask gen The expression is: Among them, the mask gen 1 indicates that the mask is set when the input belongs to the instruction part of the training corpus gen Value; the mask gen 0 indicates the mask corresponding to the desired output setting gen value; Set the mask gen The corresponding learning rate l r The expression is: Among them, the mask gen When the learning rate is 1, r Smaller than the mask gen The learning rate l corresponding to 0 r .
7. The large model training and reasoning method according to claim 1, characterized in that: After the large language model is trained, the method further includes: Get the text to be processed; Based on the text to be processed, applying the trained large language model to determine the corresponding tree structure sequence; Based on the tree structure sequence, deserialization and decoding processing are applied to determine the data structuring result corresponding to the text to be processed.
8. A data structured large model training and reasoning device, characterized in that: include: The first processing module is used to obtain the text to be structured, annotate and segment it according to a preset processing method, and obtain a data set to be structured; the content in the data set is sorted according to the annotation; A second processing module, configured to sort the content in the data set according to a preset word frequency sorting method; A construction module is used to construct a tree structure sequence based on the sorting results; The training module is used to use the text as the input of a large language model and the tree structure sequence as the output of the large language model to train the large language model.
9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the large model training and reasoning method as described in any one of claims 1 to 7 is executed.
10. A computer-readable storage medium, characterized in that: Store program instructions, and when the program instructions are executed on a computer, the computer executes the large model training and reasoning method as described in any one of claims 1 to 7.