Training method of large language model based on mortise and tenon building drawing and text alignment

Abstract

The application discloses a large language model training method based on mortise and tenon building drawing and text alignment, relates to the technical field of model training, and comprises the following steps: acquiring a mortise and tenon structure library and a conventional structure library; acquiring structure characteristic parameters of mortise and tenon structures by using a structure disassembly analysis method; acquiring structure conventional parameters of the mortise and tenon structures by using a conventional correlation method; constructing a mortise and tenon text training method, and training a large language model by using the mortise and tenon text training method; in the aspect of mortise and tenon recognition and discrimination, the application lacks an analysis method for point feature and geometric position constraint of mortise and tenon structures, cannot accurately recognize different mortise and tenon structures and judge the positions of the mortise and tenon structures, and thus problems of recognition confusion and text annotation errors and omissions exist when recognizing mortise and tenon in drawings.

Description

Technical Field

[0001] This invention relates to the field of model training technology, specifically to a method for training large language models based on the alignment of mortise and tenon architectural drawings with text. Background Technology

[0002] Aligning mortise and tenon architectural drawings with text refers to precisely matching and aligning the dimensions of tenons and mortises, as well as process descriptions, with the graphic structure of the drawings in the design drawings of mortise and tenon architecture, ensuring a one-to-one correspondence between information and component positions. The large language model training for mortise and tenon architectural drawings with text alignment is a multimodal specialized training for the traditional mortise and tenon architectural field, enabling the large language model to accurately understand the visual features of mortise and tenon drawings and the semantic relationship between the corresponding professional text descriptions.

[0003] Existing methods for training large language models to align mortise and tenon joint architectural drawings with text typically rely on image-text comparison training methods. They use the entire drawing and complete specification text as training samples and employ a single training method to achieve global semantic alignment. While this improved training method can achieve semantic alignment between modal data, it lacks analytical methods for the positional features and geometric constraints of mortise and tenon joints in terms of identification and discrimination. This makes it difficult to accurately identify different mortise and tenon joint structures and determine their positions, leading to identification confusion and errors in image and text annotation when recognizing mortise and tenon joints in drawings. For example, patent application CN116304307A discloses a cross-modal image-text retrieval network training method and its application... The proposed solution uses image-text matching and semantic alignment operations to calculate the semantic connections between entities and relationships, thus achieving semantic alignment between modal data at the entity and relationship levels. However, improvements to other large language model training methods for aligning mortise and tenon architectural drawings with text typically focus on component classification and retrieval. In mortise and tenon recognition, there is a lack of analysis methods for the point features and geometric constraints of mortise and tenon structures, making it impossible to accurately identify different mortise and tenon structures and determine their positions. This leads to confusion and errors in text and image annotation when recognizing mortise and tenon structures in drawings. Therefore, it is necessary to improve existing large language model training methods for aligning mortise and tenon architectural drawings with text. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art. By proposing a large language model training method based on the alignment of mortise and tenon architectural drawings and text, this invention addresses the shortcomings of existing methods for mortise and tenon identification. These methods lack analysis methods on the point features and geometric constraints of mortise and tenon structures, making it impossible to accurately identify different mortise and tenon structures and determine their positions. This results in problems such as confusion and errors in the annotation of mortise and tenon in drawings.

[0005] To achieve the above objectives, this application provides a method for training a large language model based on the alignment of mortise and tenon architectural drawings and text, including the following steps: Obtain text-annotated mortise and tenon architectural drawings, and obtain a mortise and tenon structure library and a conventional structure library based on the mortise and tenon ancient building multimodal corpus; use structural decomposition analysis to obtain the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; Based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, conventional association methods are used to obtain the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. A mortise and tenon text training method is constructed based on structural decomposition analysis and conventional association methods, and the mortise and tenon text training method is used to train a large language model.

[0006] Furthermore, text-annotated architectural drawings of mortise and tenon joints are obtained, and a mortise and tenon structure library and a conventional structure library are obtained based on the multimodal corpus of ancient mortise and tenon joint architecture, including: Obtain the mortise and tenon architectural drawings that have been annotated with text, and label all the drawings sequentially as Mortise and Tenon Annotation Drawing ST1 to Mortise and Tenon Annotation Drawing ST2. u , where u is the number of mortise and tenon architectural drawings obtained; Based on the multimodal corpus of ancient mortise and tenon joint architecture, the mortise and tenon structures that need to be annotated in the architectural drawings are obtained and denoted as mortise and tenon structure SG1 to mortise and tenon structure SG. n ; from mortise and tenon structure SG1 to mortise and tenon structure SG n Store in the mortise and tenon structure library.

[0007] Furthermore, text-annotated mortise and tenon architectural drawings are obtained, and a mortise and tenon structure library and a conventional structure library are acquired based on the mortise and tenon ancient building multimodal corpus. Based on the multimodal corpus of ancient mortise and tenon joint architecture, the structures that need to be annotated in the architectural drawings of mortise and tenon joints, other than the mortise and tenon joint structure itself, are obtained and denoted as conventional structure CG1 to conventional structure CG. m ;Transfer conventional structure CG1 to conventional structure CG m Store in a regular structure library; For any mortise and tenon structure in any mortise and tenon structure library: obtain the standard 3D model corresponding to the mortise and tenon structure based on big data, and record it as the standard model of the mortise and tenon structure; obtain the standard models of all mortise and tenon structures in the mortise and tenon structure library.

[0008] Furthermore, structural decomposition analysis includes: For any mortise and tenon structure in the mortise and tenon structure library: Place a standard model of the mortise and tenon structure in a spatial rectangular coordinate system where the coordinate axes are all in cm; denote the models where the tenon and mortise are located in the standard model as the tenon model and the mortise model, respectively. The tenon is the protruding part when the mortise and tenon structure is fitted together, and the mortise is the recessed part when the mortise and tenon structure is fitted together. In the tenon model, the plane that fits when the tenon and mortise are engaged is denoted as the tenon engagement plane; in the mortise model, the plane that fits when the tenon and mortise are engaged is denoted as the mortise engagement plane.

[0009] Furthermore, structural decomposition analysis also includes: Obtain all vertices in the tenon mating plane and record them as tenon mating vertices; record the position of the tenon mating vertices in the tenon model as the tenon vertex position; obtain all vertices in the mortise mating plane and record them as mortise mating vertices; record the position of the mortise mating vertices in the mortise model as the mortise vertex position; All tenon mating vertices and their positions, as well as all mortise mating vertices and their positions, are recorded as structural characteristic parameters of the mortise and tenon structure.

[0010] Furthermore, common association methods include: For any mortise and tenon structure α in the mortise and tenon structure library: record all mortise and tenon annotation drawings that contain mortise and tenon structure α as mortise and tenon special drawings; For any mortise and tenon special drawing: the area where the mortise and tenon structure α is located in the mortise and tenon special drawing is recorded as the mortise and tenon area; based on the standard model of the mortise and tenon structure α, the position of the mortise and tenon area in the standard model is obtained and marked as the regular analysis area in the mortise and tenon model; based on the regular structure library, all the marked regular structures in the mortise and tenon special drawing are obtained.

[0011] Furthermore, conventional association methods also include: When there are tenon fitting vertices or mortise fitting vertices in the regular analysis area, the tenon fitting vertices and mortise fitting vertices in the regular analysis area are recorded as special calibration points, and the positions of the feature calibration points are marked in the tenon and mortise area; For any special calibration point in the mortise and tenon area: the conventional structure with the closest straight-line distance to the special calibration point in the mortise and tenon drawing is recorded as the neighboring conventional structure of the special calibration point; obtain the neighboring conventional structures of all special calibration points; All adjacent conventional structures corresponding to the tenon mating vertex and all adjacent conventional structures corresponding to the mortise mating vertex are stored in the tenon adjacent library and the mortise adjacent library, respectively. The tenon adjacent library and the mortise adjacent library are both databases used to store parameters.

[0012] Furthermore, conventional association methods also include: When there are no tenon or mortise mating vertices in the regular analysis area, the tenon and mortise mating vertices closest to the regular analysis area among all tenon and mortise mating vertices are recorded as the tenon-related point and mortise-related point, respectively. Perpendicular lines are drawn from the tenon-related point and mortise-related point to the regular analysis area, and the points corresponding to the perpendicular feet are recorded as the tenon-marking point and mortise-marking point, respectively. Based on the analysis of special calibration points, the adjacent conventional structures corresponding to the tenon calibration points and mortise calibration points are obtained from the tenon and mortise-specific drawings, and stored in the tenon adjacent library and mortise adjacent library, respectively. Record the structural parameters of mortise and tenon structure α as the adjacent tenon library and adjacent mortise library corresponding to all mortise and tenon drawings; obtain the structural parameters of all mortise and tenon structures.

[0013] Furthermore, the mortise and tenon text training method includes: When the input content of the large language model is mortise and tenon architectural drawings and accompanying text, and the text generation training is performed on the mortise and tenon architectural drawings: based on the accompanying text of the mortise and tenon architectural drawings, the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are annotated, and the areas corresponding to the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are stored in the drawing comparison library of mortise and tenon structures and conventional structures respectively. The drawing comparison library is used to store the images corresponding to the mortise and tenon structures and conventional structures, and each mortise and tenon structure and conventional structure is independent of each other. For any labeled mortise and tenon structure: obtain the corresponding structural parameters based on structural decomposition analysis and conventional association methods; based on the positional relationship between the mortise and tenon structure and adjacent conventional structures in the structural parameters within the mortise and tenon architectural drawings, use AI to generate text on the positional relationship between the mortise and tenon structure and adjacent conventional structures, and record it as the training generated text; Using the accompanying text of mortise and tenon architectural drawings as the standard, the AI ​​was trained to convert the text generated during training into the accompanying text of mortise and tenon architectural drawings, and the text generation method of the AI ​​was modified based on the training.

[0014] Furthermore, the mortise and tenon text training method also includes: When the input content in the large language model is mortise and tenon professional text and its accompanying drawings, and when training the mortise and tenon professional text to generate drawings: the mortise and tenon structure described in the mortise and tenon professional text is recorded as the structure to be generated, and the conventional structure described in the mortise and tenon professional text is recorded as the auxiliary generation structure. For any structure to be generated: among all the mortise and tenon drawings obtained by the conventional association method for the structure to be generated, the mortise and tenon drawings that contain at least one auxiliary generating structure in the conventional parameters of the structure are recorded as the mortise and tenon drawings to be generated. Obtain all the mortise and tenon drawings corresponding to the structures to be generated, and denote the mortise and tenon drawing β that has been recorded the most times as the mortise and tenon drawing to be generated as the training drawing. Using the accompanying drawings of the mortise and tenon professional text as the standard, the AI ​​is trained to convert training drawings into accompanying drawings of the mortise and tenon professional text, and the drawing modification parameters obtained from the training are recorded as the drawing improvement parameters of the AI ​​for the mortise and tenon professional drawing β.

[0015] The beneficial effects of this invention are as follows: This application first obtains mortise and tenon architectural drawings that have been annotated with text, and then obtains a mortise and tenon structure library and a conventional structure library based on the mortise and tenon ancient building multimodal corpus; it uses structural decomposition analysis to obtain the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. The advantage of this is that by constructing the mortise and tenon structure library and the conventional structure library, the structures that appear in the mortise and tenon architectural drawings can be decomposed, and the mortise and tenon structures used for identification and position determination and the conventional structures used to assist in the determination of mortise and tenon structures can be classified separately, so as to improve the data analysis efficiency when performing the structural decomposition analysis method, conventional association method and mortise and tenon text training method in the future; by obtaining the structural feature parameters corresponding to each mortise and tenon structure, the special interlocking areas in the mortise and tenon structure can be extracted, which is convenient for identifying different mortise and tenon structures based on the structural feature parameters when training a large language model, so as to ensure that the trained large language model can effectively align mortise and tenon architectural drawings with text; This application also uses conventional association methods to obtain the conventional structural parameters corresponding to each type of mortise and tenon structure in the mortise and tenon structure library, based on mortise and tenon architectural drawings, a conventional structure library, and the structural feature parameters of all mortise and tenon structures in the library. Finally, a mortise and tenon text training method is constructed based on structural decomposition analysis and conventional association methods, and the mortise and tenon text training method is used to train a large language model. The advantage of this is that by obtaining the conventional structural parameters, the conventional structures around the mortise and tenon structures in the mortise and tenon architectural drawings can be analyzed, thereby obtaining the joint positions that constrain the mortise and tenon structures. This allows the large language model to judge whether the position of the mortise and tenon structure is reasonable based on the conventional structural parameters, thereby improving the accuracy of aligning the mortise and tenon structures in the mortise and tenon architectural drawings with the text, and avoiding problems such as recognition confusion and errors in graphic and text annotation. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of the method of the present invention; Figure 2 This is a schematic diagram of the mortise and tenon structure of the present invention; Figure 3 This is a schematic diagram showing the location of the special calibration points of the present invention; Figure 4 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1 As shown, this application provides a method for training a large language model based on the alignment of mortise and tenon architectural drawings and text, including the following steps: Step S1: Obtain the mortise and tenon architectural drawings that have been annotated with text, and obtain the mortise and tenon structure library and the conventional structure library based on the mortise and tenon ancient building multimodal corpus; use the structural decomposition analysis method to obtain the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; Step S1 includes: Step S101, obtaining the mortise and tenon architectural drawings that have been annotated with text, and sequentially labeling all drawings as mortise and tenon annotation drawings ST1 to mortise and tenon annotation drawings ST. u , where u is the number of mortise and tenon architectural drawings obtained; Step S102: Based on the multimodal corpus of mortise and tenon ancient architecture, obtain the mortise and tenon structures that need to be annotated in the mortise and tenon architectural drawings, and denote them as mortise and tenon structure SG1 to mortise and tenon structure SG. n ; from mortise and tenon structure SG1 to mortise and tenon structure SG n Store in the mortise and tenon structure library; In practical implementation, mortise and tenon structures can include corner tenons, mortise and tenon joints, and shoulder tenons; dovetail tenons, flat tenons, and tongue and groove tenons for board splicing; hook tenons and dragon and phoenix tenons for doors and windows, etc. The mortise and tenon structure can be set more specifically based on the mortise and tenon architectural drawings that are used for text alignment training in the large language model during actual data analysis.

[0019] Step S1 also includes: Step S103, based on the multimodal corpus of mortise and tenon ancient buildings, obtaining the structures that need to be annotated in the mortise and tenon architectural drawings other than the mortise and tenon structure, and denoting them as conventional structure CG1 to conventional structure CG respectively. m;Transfer conventional structure CG1 to conventional structure CG m Store in a regular structure library; In the specific implementation process, the conventional structure may include the parts of the structure that can be connected by mortise and tenon structure in the mortise and tenon architectural drawings, such as beams, columns, grid frames and lintels. Step S104: For any mortise and tenon structure in any mortise and tenon structure library: obtain the standard 3D model corresponding to the mortise and tenon structure based on big data, and record it as the standard model of the mortise and tenon structure; obtain the standard models of all mortise and tenon structures in the mortise and tenon structure library.

[0020] Step S105, the structural disassembly and analysis method includes: Step S1051, for any mortise and tenon structure in the mortise and tenon structure library: place a standard model of the mortise and tenon structure in a spatial rectangular coordinate system where the coordinate axis unit is cm; denote the models where the tenon and mortise are located in the standard model as the tenon model and the mortise model, respectively, where the tenon is the protruding part when the mortise and tenon structure is fitted together, and the mortise is the recessed part when the mortise and tenon structure is fitted together; Step S1052: In the tenon model, the plane in which the tenon and mortise fit together is denoted as the tenon fitting plane; in the mortise model, the plane in which the tenon and mortise fit together is denoted as the mortise fitting plane. In the data analysis of this embodiment, for example, in a single data analysis, a mortise and tenon structure is obtained as follows: Figure 2 As shown, SU1 is the tenon model in the mortise and tenon structure, SU2 is the mortise model in the mortise and tenon structure, and planes SY1 and SY2 in SU2 are the mortise fitting planes. Therefore, for point MD in SU2, since point MD is a vertex of the mortise fitting plane, point MD can be recorded as the mortise fitting vertex.

[0021] The structural disassembly and analysis method also includes: step S1053, obtaining all vertices in the tenon mating plane and recording them as tenon mating vertices; recording the position of the tenon mating vertices in the tenon model as the tenon vertex position; obtaining all vertices in the mortise mating plane and recording them as mortise mating vertices; recording the position of the mortise mating vertices in the mortise model as the mortise vertex position; Step S1054: Record all the tenon mating vertices and their tenon vertices, as well as all the mortise mating vertices and their mortise vertices, as structural feature parameters of the mortise and tenon structure.

[0022] Step S2: Based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, use conventional association methods to obtain the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. The conventional association method includes: Step S201, for any tenon structure α in the tenon structure library: record all tenon annotation drawings that contain tenon structure α as tenon-specific drawings; Step S202: For any mortise and tenon special drawing: the area where the mortise and tenon structure α is located in the mortise and tenon special drawing is recorded as the mortise and tenon area; based on the standard model of the mortise and tenon structure α, the position of the mortise and tenon area in the standard model is obtained, and marked as the regular analysis area in the mortise and tenon model; based on the regular structure library, all the marked regular structures in the mortise and tenon special drawing are obtained.

[0023] The conventional association method also includes: step S203, when there is a tenon fitting vertex or a mortise fitting vertex in the conventional analysis area, the tenon fitting vertex and the mortise fitting vertex in the conventional analysis area are recorded as special calibration points, and the position of the feature calibration point is marked in the tenon and mortise area; Step S204: For any special calibration point in the mortise and tenon area: Record the conventional structure with the closest straight-line distance to the special calibration point in the mortise and tenon special drawing as the neighboring conventional structure of the special calibration point; obtain the neighboring conventional structures of all special calibration points; In the data analysis of this embodiment, for example Figure 2 The mortise and tenon structure in the drawing corresponds to the mortise and tenon area in a specific mortise and tenon drawing, such as... Figure 3 As shown, through spatial geometric analysis, we obtain... Figure 3 Point MD0 in the middle is Figure 2 Point MD in the mortise and tenon area can be marked as a special calibration point, and the conventional structure that is closest to point MD0 in the mortise and tenon special drawing can be obtained and marked as the adjacent conventional structure of point MD0. Step S205: Store all adjacent conventional structures corresponding to the tenon mating vertex and all adjacent conventional structures corresponding to the mortise mating vertex into the tenon adjacent library and the mortise adjacent library, respectively. The tenon adjacent library and the mortise adjacent library are databases used to store parameters.

[0024] The conventional association method also includes: step S206, when there is no tenon fitting vertex or mortise fitting vertex in the conventional analysis area, the tenon fitting vertex and mortise fitting vertex closest to the conventional analysis area among all tenon fitting vertex and mortise fitting vertex are recorded as tenon association point and mortise association point respectively; perpendicular lines are drawn from the tenon association point and mortise association point to the conventional analysis area respectively, and the points corresponding to the perpendicular feet are recorded as tenon calibration point and mortise calibration point respectively; Step S207: Based on the analysis method of special calibration points, obtain the adjacent conventional structures corresponding to the tenon calibration points and mortise calibration points in the tenon and mortise special drawings respectively, and store them in the tenon adjacent library and mortise adjacent library respectively. In the specific implementation process, by acquiring adjacent conventional structures and storing them in the tenon adjacent library and the mortise adjacent library respectively, it is possible to analyze the conventional structures around each tenon structure based on the unique special points in the tenon structure. This enables the constraint of the position of the tenon structure in the tenon architectural drawings, thereby improving the accuracy of the comparison between the tenon structure and the text. Step S208: Record the tenon adjacent library and mortise adjacent library corresponding to all tenon and mortise special drawings as the structural conventional parameters of tenon and mortise structure α; obtain the structural conventional parameters of all tenon and mortise structures.

[0025] Step S3: Construct a mortise and tenon text training method based on structural decomposition analysis and conventional association methods, and use the mortise and tenon text training method to train the large language model; The mortise and tenon text training method includes: Step S301, when the input content in the large language model is mortise and tenon architectural drawings and accompanying text, and when the mortise and tenon architectural drawings are trained for text generation: based on the accompanying text of the mortise and tenon architectural drawings, the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are annotated, and the areas corresponding to the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are stored in the drawing comparison library of mortise and tenon structures and conventional structures respectively. The drawing comparison library is used to store the images corresponding to the mortise and tenon structures and conventional structures, and each mortise and tenon structure and conventional structure are independent of each other. Step S302: For any labeled mortise and tenon structure: Obtain the structural conventional parameters corresponding to the mortise and tenon structure based on the structural decomposition analysis method and conventional association method; Based on the positional relationship between the mortise and tenon structure and the adjacent conventional structure in the structural conventional parameters within the mortise and tenon architectural drawings, use AI to generate text on the positional relationship between the mortise and tenon structure and the adjacent conventional structure, and record it as the training generated text; In the specific implementation process, by acquiring the conventional structural parameters, constraints can be imposed on the components surrounding the mortise and tenon structure. This ensures that after training, when using the large language model to generate text from mortise and tenon architectural drawings, the model can compare the obtained conventional structural parameters with the existing conventional structural parameters of the mortise and tenon structure in the large language model after obtaining the mortise and tenon structure from the drawing comparison library. This allows for the determination of the constraints on the location of the mortise and tenon structure, improving the accuracy of mortise and tenon structure recognition. In addition, the AI's text generation process during training can be retained, which can be used to train the AI's text generation method and improve the accuracy of the AI ​​in the next text generation. Step S303: Using the accompanying text of the mortise and tenon architectural drawings as the standard, train the AI ​​to convert the text generated during training into the accompanying text of the mortise and tenon architectural drawings, and modify the AI's text generation method based on the training.

[0026] The mortise and tenon text training method also includes: step S304, when the input content in the large language model is mortise and tenon professional text and its accompanying drawings, and when the mortise and tenon professional text is used for drawing generation training: the mortise and tenon structure described in the mortise and tenon professional text is recorded as the structure to be generated, and the conventional structure described in the mortise and tenon professional text is recorded as the auxiliary generation structure. Step S305: For any structure to be generated: among all the mortise and tenon drawings obtained by the conventional association method for the structure to be generated, the mortise and tenon drawings that contain at least one auxiliary generating structure in the conventional structural parameters are recorded as mortise and tenon drawings to be generated. Step S306: Obtain the mortise and tenon drawings corresponding to all structures to be generated, and record the mortise and tenon drawing β that is recorded as the mortise and tenon drawing with the most times as the mortise and tenon drawing to be generated as the training drawing. Step S307: Using the accompanying drawings of the mortise and tenon professional text as the standard, train the AI ​​to convert the training drawings into the accompanying drawings of the mortise and tenon professional text, and record the drawing modification parameters obtained from the training as the drawing improvement parameters of the AI ​​for the mortise and tenon professional drawing β. In the specific implementation process, by retaining the training records and drawing improvement parameters of the AI ​​based on the professional text on mortise and tenon joints, the accuracy of AI in optimizing the training drawings after obtaining the training drawings can be improved. Since the training drawings are all obtained by combining the professional text on mortise and tenon joints, when using a large language model to generate drawings from the professional text on mortise and tenon joints, if the training records and drawing improvement parameters retained by the AI ​​are combined, it can be ensured that the training drawings optimized by the AI ​​are more in line with the description of the professional text on mortise and tenon joints, thereby achieving a more accurate match between the generated mortise and tenon joint architectural drawings and the professional text on mortise and tenon joints.

[0027] Example 2, please refer to Figure 4 As shown, Figure 4A schematic diagram of an electronic device is provided, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call these instructions. When the processor executes a computer-readable instruction, it performs steps similar to those in a large language model training method based on the alignment of mortise and tenon architectural drawings with text, to achieve the following functions: First, it acquires mortise and tenon architectural drawings annotated with text, and obtains a mortise and tenon structure library and a conventional structure library based on a multimodal corpus of ancient mortise and tenon architecture. It then uses structural decomposition analysis to obtain the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. Next, based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, it uses a conventional association method to obtain the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. Finally, it constructs a mortise and tenon text training method based on the structural decomposition analysis method and the conventional association method, and uses this method to train the large language model.

[0028] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0029] Example 3: This application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by the computer, the computer can execute the large language model training method based on the alignment of mortise and tenon architectural drawings and text provided by the above methods. The method includes: firstly, acquiring mortise and tenon architectural drawings that have been annotated with text, and acquiring a mortise and tenon structure library and a conventional structure library based on the mortise and tenon ancient building multimodal corpus; using structural decomposition analysis to acquire the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; then, based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, using conventional association methods to acquire the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; finally, constructing a mortise and tenon text training method based on the structural decomposition analysis method and the conventional association method, and using the mortise and tenon text training method to train the large language model.

[0030] Example 4: This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps in the above-mentioned large language model training method based on the alignment of mortise and tenon architectural drawings and text to achieve the following functions: First, it acquires mortise and tenon architectural drawings that have been annotated with text, and acquires a mortise and tenon structure library and a conventional structure library based on the mortise and tenon ancient building multimodal corpus; it uses structural decomposition analysis to acquire the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; then, based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, it uses conventional association methods to acquire the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; finally, it constructs a mortise and tenon text training method based on the structural decomposition analysis method and the conventional association method, and uses the mortise and tenon text training method to train the large language model.

[0031] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0032] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training a large language model based on the alignment of mortise and tenon architectural drawings and text, characterized in that, Includes the following steps: Obtain text-annotated mortise and tenon architectural drawings, and obtain a mortise and tenon structure library and a conventional structure library based on the mortise and tenon ancient building multimodal corpus; use structural decomposition analysis to obtain the structural feature parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library; Based on the mortise and tenon architectural drawings, the conventional structure library, and the structural feature parameters of all mortise and tenon structures in the mortise and tenon structure library, conventional association methods are used to obtain the conventional structural parameters corresponding to each mortise and tenon structure in the mortise and tenon structure library. A mortise and tenon text training method is constructed based on structural decomposition analysis and conventional association methods, and the mortise and tenon text training method is used to train a large language model.

2. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 1, characterized in that, Obtain text-annotated mortise and tenon architectural drawings, and based on the mortise and tenon ancient building multimodal corpus, obtain a mortise and tenon structure library and a conventional structure library, including: Obtain the mortise and tenon architectural drawings that have been annotated with text, and label all the drawings sequentially as Mortise and Tenon Annotation Drawing ST1 to Mortise and Tenon Annotation Drawing ST2. u , where u is the number of mortise and tenon architectural drawings obtained; Based on the multimodal corpus of ancient mortise and tenon joint architecture, the mortise and tenon structures that need to be annotated in the architectural drawings are obtained and denoted as mortise and tenon structure SG1 to mortise and tenon structure SG. n ; from mortise and tenon structure SG1 to mortise and tenon structure SG n Store in the mortise and tenon structure library.

3. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 2, characterized in that, Obtain text-annotated mortise and tenon architectural drawings, and based on the mortise and tenon ancient building multimodal corpus, obtain a mortise and tenon structure library and a conventional structure library, including: Based on the multimodal corpus of ancient mortise and tenon joint architecture, the structures that need to be annotated in the architectural drawings of mortise and tenon joints, other than the mortise and tenon joint structure itself, are obtained and denoted as conventional structure CG1 to conventional structure CG. m ;Transfer conventional structure CG1 to conventional structure CG m Store in a regular structure library; For any mortise and tenon structure in any mortise and tenon structure library: obtain the standard 3D model corresponding to the mortise and tenon structure based on big data, and record it as the standard model of the mortise and tenon structure; obtain the standard models of all mortise and tenon structures in the mortise and tenon structure library.

4. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 3, characterized in that, Structural disassembly analysis methods include: For any mortise and tenon structure in the mortise and tenon structure library: Place a standard model of the mortise and tenon structure in a spatial rectangular coordinate system where the coordinate axes are all in cm; denote the models where the tenon and mortise are located in the standard model as the tenon model and the mortise model, respectively. The tenon is the protruding part when the mortise and tenon structure is fitted together, and the mortise is the recessed part when the mortise and tenon structure is fitted together. In the tenon model, the plane that fits when the tenon and mortise are engaged is denoted as the tenon engagement plane; in the mortise model, the plane that fits when the tenon and mortise are engaged is denoted as the mortise engagement plane.

5. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 4, characterized in that, Structural disassembly analysis also includes: Obtain all vertices in the tenon mating plane and record them as tenon mating vertices; record the position of the tenon mating vertices in the tenon model as the tenon vertex position; obtain all vertices in the mortise mating plane and record them as mortise mating vertices; record the position of the mortise mating vertices in the mortise model as the mortise vertex position; All tenon mating vertices and their positions, as well as all mortise mating vertices and their positions, are recorded as structural characteristic parameters of the mortise and tenon structure.

6. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 5, characterized in that, Common association methods include: For any mortise and tenon structure α in the mortise and tenon structure library: record all mortise and tenon annotation drawings that contain mortise and tenon structure α as mortise and tenon special drawings; For any mortise and tenon special drawing: the area where the mortise and tenon structure α is located in the mortise and tenon special drawing is recorded as the mortise and tenon area; based on the standard model of the mortise and tenon structure α, the position of the mortise and tenon area in the standard model is obtained and marked as the regular analysis area in the mortise and tenon model; based on the regular structure library, all the marked regular structures in the mortise and tenon special drawing are obtained.

7. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 6, characterized in that, Common association methods also include: When there are tenon fitting vertices or mortise fitting vertices in the regular analysis area, the tenon fitting vertices and mortise fitting vertices in the regular analysis area are recorded as special calibration points, and the positions of the feature calibration points are marked in the tenon and mortise area; For any special calibration point in the mortise and tenon area: the conventional structure with the closest straight-line distance to the special calibration point in the mortise and tenon drawing is recorded as the neighboring conventional structure of the special calibration point; obtain the neighboring conventional structures of all special calibration points; All adjacent conventional structures corresponding to the tenon mating vertex and all adjacent conventional structures corresponding to the mortise mating vertex are stored in the tenon adjacent library and the mortise adjacent library, respectively. The tenon adjacent library and the mortise adjacent library are both databases used to store parameters.

8. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 7, characterized in that, Common association methods also include: When there are no tenon or mortise mating vertices in the regular analysis area, the tenon and mortise mating vertices closest to the regular analysis area among all tenon and mortise mating vertices are recorded as the tenon-related point and mortise-related point, respectively. Perpendicular lines are drawn from the tenon-related point and mortise-related point to the regular analysis area, and the points corresponding to the perpendicular feet are recorded as the tenon-marking point and mortise-marking point, respectively. Based on the analysis of special calibration points, the adjacent conventional structures corresponding to the tenon calibration points and mortise calibration points are obtained from the tenon and mortise-specific drawings, and stored in the tenon adjacent library and mortise adjacent library, respectively. Record the structural parameters of mortise and tenon structure α as the adjacent tenon library and adjacent mortise library corresponding to all mortise and tenon drawings; obtain the structural parameters of all mortise and tenon structures.

9. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 8, characterized in that, The training methods for mortise and tenon text include: When the input content of the large language model is mortise and tenon architectural drawings and accompanying text, and the text generation training is performed on the mortise and tenon architectural drawings: based on the accompanying text of the mortise and tenon architectural drawings, the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are annotated, and the areas corresponding to the mortise and tenon structures and conventional structures in the mortise and tenon architectural drawings are stored in the drawing comparison library of mortise and tenon structures and conventional structures respectively. The drawing comparison library is used to store the images corresponding to the mortise and tenon structures and conventional structures, and each mortise and tenon structure and conventional structure is independent of each other. For any labeled mortise and tenon structure: obtain the corresponding structural parameters based on structural decomposition analysis and conventional association methods; based on the positional relationship between the mortise and tenon structure and adjacent conventional structures in the structural parameters within the mortise and tenon architectural drawings, use AI to generate text on the positional relationship between the mortise and tenon structure and adjacent conventional structures, and record it as the training generated text; Using the accompanying text of mortise and tenon architectural drawings as the standard, the AI ​​was trained to convert the text generated during training into the accompanying text of mortise and tenon architectural drawings, and the text generation method of the AI ​​was modified based on the training.

10. The method for training a large language model based on the alignment of mortise and tenon architectural drawings and text according to claim 9, characterized in that, The mortise and tenon text training method also includes: When the input content in the large language model is mortise and tenon professional text and its accompanying drawings, and when training the mortise and tenon professional text to generate drawings: the mortise and tenon structure described in the mortise and tenon professional text is recorded as the structure to be generated, and the conventional structure described in the mortise and tenon professional text is recorded as the auxiliary generation structure. For any structure to be generated: among all the mortise and tenon drawings obtained by the conventional association method for the structure to be generated, the mortise and tenon drawings that contain at least one auxiliary generating structure in the conventional parameters of the structure are recorded as the mortise and tenon drawings to be generated. Obtain all the mortise and tenon drawings corresponding to the structures to be generated, and denote the mortise and tenon drawing β that has been recorded the most times as the mortise and tenon drawing to be generated as the training drawing. Using the accompanying drawings of the mortise and tenon professional text as the standard, the AI ​​is trained to convert training drawings into accompanying drawings of the mortise and tenon professional text, and the drawing modification parameters obtained from the training are recorded as the drawing improvement parameters of the AI ​​for the mortise and tenon professional drawing β.

Images