Drawing splitting method and system based on computer vision and large language model

Abstract

The invention discloses a drawing splitting method and system based on computer vision and a large language model. The method comprises the steps that image information of a to-be-split drawing is acquired; identifying picture frame information in the image information through a computer vision model; obtaining character information in the picture frame information; obtaining keyword information from the text information through a large language model; and splitting the to-be-split drawing into two or more sub-drawings according to the drawing frame information, wherein the keyword information corresponds to the sub-drawings. The drawing splitting method and system based on the computer vision and the large language model are high in recognition anti-interference capacity, and the problem that drawing splitting fails due to the fact that recognition markers are modified can be avoided. Meanwhile, after the characters in the drawing frame are recognized, keyword information of character information in the drawing frame is extracted through a large language model, the content of the sub-drawings is accurately summarized, the sub-drawings obtained through splitting are named, and the quality of drawing splitting is improved.

Description

Technical Field

[0001] The present invention relates to a drawing splitting method and system based on computer vision and large language models, and belongs to the field of intelligent construction of civil engineering. Background Art

[0002] During the construction process of civil engineering, there may be a situation where a drawing contains multiple sub-drawings. Sub-drawings in a drawing usually refer to relatively independent graphic units that are divided or separated within a complete drawing. These sub-drawings each represent a relatively complete piece of information or design part, such as elevation views and sectional views of beams and columns on each floor in architectural engineering drawings.

[0003] During the storage and use of drawings, it is more convenient to treat each sub-drawing as a separate file. Therefore, it is often necessary to split the sub-drawings in a drawing. Currently, splitting an entire drawing mainly relies on specific markers to identify each drawing, such as specific drawing frames. However, this kind of marker has too high requirements for the drawing. When the user modifies, deletes, or copies the marker during the process of modifying the drawing, it will affect the splitting process. Summary of the Invention

[0004] Aiming at the problem that the existing drawing splitting method has too high requirements for markers and will affect the splitting process when the markers are modified, deleted, or copied, the present invention provides a drawing splitting method and system based on computer vision and large language models.

[0005] To solve the above technical problems, the present invention includes the following technical solutions: A drawing splitting method based on computer vision and large language models includes the following steps: Step 1, obtaining image information of the drawing to be split; Step 2, identifying the drawing frame information in the image information through a computer vision model; Step 3, obtaining the text information in the drawing frame information; Step 4, obtaining keyword information from the text information through a large language model; Step 5, splitting the drawing to be split into two or more sub-drawings according to the drawing frame information, and the keyword information corresponds to the sub-drawings.

[0006] Further, the step of obtaining the text information in the drawing frame information in step 3 includes: Obtaining the characters in the text information in the order from top to bottom and from left to right; Combining the characters into a sentence.

[0007] Further, the step four of obtaining keyword information from the text information through the large language model includes: Input the statement into the large language model to obtain the keyword information.

[0008] Further, the text includes the project name, drawing name, drawing number, drafter, reviewer, approver, and the text in the drawing.

[0009] Further, the step four of obtaining text information from the drawing frame information through the large language model includes: The drawing frame information includes two or more drawing frames, and the number of drawing frames corresponds to the number of sub-drawings, and the sub-drawings correspond to the keyword information.

[0010] Further, the step five of splitting the drawing to be split into two or more sub-drawings according to the drawing frame information, and the step where the keyword information corresponds to the sub-drawings includes: The keyword information corresponds to the title of the sub-drawing.

[0011] Further, the method for establishing the computer vision model includes the following steps: Establish a neural network architecture model; Obtain a data set, and the data set is a drawing picture containing drawing frame information; Input the drawing picture with labeled drawing frame information into the neural network architecture model for model training to obtain the computer vision model.

[0012] The above-mentioned drawing splitting system based on computer vision and large language model includes: An image acquisition module for acquiring image information of the drawing to be split; A drawing frame information recognition module for recognizing the drawing frame information in the image information through the computer vision model; A drawing content acquisition module for acquiring text information in the drawing frame information; A drawing content analysis module for obtaining keyword information from the text information through the large language model; A drawing splitting module for splitting the drawing to be split into two or more sub-drawings according to the drawing frame information, and the keyword information corresponds to the sub-drawings.

[0013] Due to the adoption of the above technical solutions, the present invention has the following advantages and positive effects compared with the prior art: (1) Strong anti-interference ability in recognition. The drawing splitting method based on computer vision and large language model of the present invention uses computer vision to intelligently recognize various types of drawing frames, which can avoid the problem of drawing splitting failure caused by the modification of recognition markers. Even if the original markers are copied, modified or deleted, the splitting results can still be stably and accurately output through the computer vision model.

[0014] (2) Good splitting quality. By combining computer vision and large language model, after the computer vision recognizes the drawing frame, the keywords of the text information inside the drawing frame are extracted through the large language model after recognizing the text inside the drawing frame, and the sub-drawings obtained by splitting are named, which improves the accuracy of the content summary of the sub-drawings, avoids omissions and repetitions during the splitting process, and ensures the quality of drawing splitting.

[0015] (3) High splitting efficiency. According to the splitting method of the present invention, the drawing to be split can be quickly split, improving the splitting efficiency. Brief Description of the Drawings

[0016] Figure 1 is a flowchart of the drawing splitting method based on computer vision and large language model in an embodiment; Figure 2 is a structural schematic diagram of the drawing splitting system based on computer vision and large language model in an embodiment. Detailed Embodiments

[0017] The following further elaborates on the drawing splitting method and system based on computer vision and large language model provided by the present invention in combination with the drawings and specific embodiments. In combination with the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are all in a very simplified form and use non-precise scales, only for the purpose of conveniently and clearly assisting in explaining the embodiments of the present invention.

[0018] The existing drawing splitting methods rely on specific markers to identify each drawing, and the robustness during the splitting process is poor. The drawing splitting method based on computer vision and large language model of the present invention performs splitting through a trained computer vision model that can recognize a type of drawing frame. This model can still preferably output the splitting results stably and accurately through visual methods even when the original markers are copied, modified or deleted, and has stronger anti-interference ability than the original methods. The present invention will be described below through specific examples. Embodiment 1

[0019] As Figure 1 shown, the drawing splitting method based on computer vision and large language model in an embodiment includes the following steps: Step 1 S10: Obtain the image information of the drawing to be split. Convert the format of the drawing to be split into a format suitable for computer vision model recognition. In this embodiment, methods such as CAD secondary development and drawing scanning are used to convert the drawing to be split into information in a picture format that can be used for computer vision recognition. For example, JEPG format, PNG format, etc.

[0020] Step 2 S20: Identify the drawing frame information in the image information through a computer vision model. A drawing frame refers to the boundary line on a drawing that defines the scope and content of the drawing and contains some basic information. Among them, the drawing frame information includes the drawing scope and the content in the drawing. Generally, the drawing to be split will include two or more sub-drawings, and each sub-drawing will correspond to a drawing frame, that is, the number of drawing frames corresponds to the number of sub-drawings. The scope of each drawing frame can be obtained through the computer vision model.

[0021] The computer vision model is used to identify the drawing frame information in the drawing to be split. The image information of the drawing to be split is input into the computer vision model to obtain the scope and content of each drawing frame. For example, the boundary line information in each drawing frame is identified to obtain the information of each drawing frame. Among them, the computer vision model can be pre-trained, the input information is the image information of the drawing to be split, and the output is the drawing frame information of the drawing to be split.

[0022] In this embodiment, the method for establishing the computer vision model includes the following steps: Establish a neural network architecture model. The form of the pre-trained model is not limited, and models such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and ResNet (Residual Network) can all be used as the model for drawing frame recognition.

[0023] Obtain a data set, and the data set is a drawing picture containing drawing frame information. The training data set is manually labeled. For example, at least 500 drawing pictures containing drawing frames are obtained according to Step 1 S10, and the positions of the drawing frames are marked on these drawing pictures as the drawing frame labels of the drawing pictures. The number of drawing pictures can be selected according to the actual situation. The more drawing pictures are selected, the relatively higher the accuracy of the training model will be.

[0024] Input the drawing pictures with the labeled drawing frame information into the neural network architecture model for model training to obtain the computer vision model. Algorithms such as stochastic gradient descent and adaptive gradient algorithm can be used to train the model until the loss function is less than the specified threshold or the number of training times exceeds the specified number of times to end the training and obtain the computer vision model.

[0025] After the drawing frame information in the image information is identified through the computer vision model, enter Step 3 S30: Step S30: Obtain the text information in the frame information. The text information in each frame can be obtained through the functions built in CAD software or other intelligent recognition software. In this embodiment, obtaining the text information in the frame information includes the following steps: Obtain the text in the text information in the order from top to bottom and from left to right. The drawing to be split generally includes multiple frames, that is, frame information. The text information in each frame information may also vary. First, select one of the frames and obtain the text in the frame in a certain order. In this embodiment, the obtained text includes the project name, drawing name, drawing number, drafter, reviewer, approver, drawing date, and the text in the drawing, etc.

[0026] Merge the text into sentences. After all the text in the frame information is obtained, merge the text into sentences. Certain merging rules can be set. For example, organize and merge the above text according to the merging rule of project name - drawing name - drawing number - drafter - drawing date to obtain a sentence, or a whole paragraph.

[0027] After the text information of the above frame information is processed, process the text information in the remaining frame information in the same way to obtain the text information of all frames.

[0028] Step S40: Obtain keyword information from the text information through a large language model.

[0029] A large language model (LLM for short) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model adopted in this embodiment is a common large language model in this field, such as GPT series, Wenxin Yiyan, Pangu, Hunyuan, etc., which are not limited here.

[0030] After obtaining the text information in the frame information, input the sentence or whole paragraph formed by merging the text information into the large language model. The large language model extracts the keywords therein to form keyword information. The rules for the large language model to extract keywords can be selected according to the actual situation. For example, extract the project name, drawing name, etc. in the text information as keywords.

[0031] The keyword information corresponds to the corresponding frame. For example, when there are 3 frames D1, D2, and D3, the keyword information corresponding to the text information obtained from frame D1 is L1, the keyword information corresponding to the text information obtained from frame D2 is L2, and the keyword information corresponding to the text information obtained from frame D3 is L3. That is, when the frame information includes two or more frames, the frames respectively correspond to the corresponding keywords.

[0032] Step Five S50: Split the drawing to be split into two or more sub-drawings according to the frame information, and the keyword information corresponds to the sub-drawings.

[0033] In Step Two S20, the frame information in the image information of the drawing to be split is identified, that is, each frame in the drawing to be split is identified. In Step Four S40, the text information in each frame is obtained, and the keyword information is obtained. Each frame is split from the drawing to be split, and then the corresponding keyword information is used as the title of the split sub-drawing.

[0034] Taking the drawing to be split including 3 sub-drawings as an example, the 3 sub-drawings respectively correspond to frame D1, frame D2, and frame D3. The computer vision model identifies that the drawing to be split includes frame D1, frame D2, and frame D3, obtains the range of each frame, and then respectively obtains the text information E1, E2, and E3 in frame D1, frame D2, and frame D3. These text information are respectively combined into a sentence or a whole paragraph, and then these sentences are respectively input into the large language model to obtain keyword information K1, K2, and K3, where keyword information K1 corresponds to frame D1, keyword information K2 corresponds to frame D2, and keyword information K3 corresponds to frame D3. Finally, according to the identified frames D1, D2, and D3, the drawing to be split is divided into sub-drawing P1, sub-drawing P2, and sub-drawing P3. At the same time, keyword information K1 is used as the title or name of sub-drawing P1, keyword information K2 is used as the title or name of sub-drawing P2, and keyword information K3 is used as the title or name of sub-drawing P3.

[0035] According to the steps from Step One S10 to Step Five S50, a drawing containing multiple independent sub-drawings with separate names can be obtained. Users can batch print or use the sub-drawings according to their usage requirements. When there are modifications, deletions, or duplications of the markers in the drawing to be split, it will not affect the splitting process. Embodiment Two

[0036] As Figure 2 shown, a drawing splitting system based on computer vision and large language model in an embodiment includes an image acquisition module 20, a frame information recognition module 22, a drawing content acquisition module 24, a drawing content analysis module 26, and a drawing splitting module 28.

[0037] The image acquisition module 20 is used to acquire the image information of the drawing to be split; The frame information recognition module 22 is used to recognize the frame information in the image information through a computer vision model; The drawing content acquisition module 24 is used to acquire the text information in the frame information; The drawing content analysis module 26 is used to obtain keyword information from the text information through a large language model; The drawing splitting module 28 is configured to split the drawing to be split into two or more sub-drawings according to the frame information, and the keyword information corresponds to the sub-drawings.

[0038] The drawing splitting method and system based on computer vision and large language model of the present invention have the following advantages: (1) Strong anti-interference ability in recognition. The drawing splitting method based on computer vision and large language model of the present invention can intelligently recognize various types of drawing frames by using computer vision, and can avoid the problem of splitting failure caused by the modification of recognition markers.

[0039] (2) Good splitting quality. By combining computer vision and large language model, after the computer vision recognizes the drawing frame, the keyword information of the text information in the drawing frame is extracted through the large language model after recognizing the text in the drawing frame, accurately summarizes the content of the sub-drawing, and names the sub-drawing obtained by splitting, thereby improving the quality of drawing splitting.

[0040] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0041] The above embodiments only represent several implementation manners of the present invention, and the description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A drawing segmentation method based on computer vision and large language model, characterized in that: The steps include: Step 1, obtaining image information of the drawing to be split; Step 2: identifying the frame information in the image information through a computer vision model; Step 3, obtaining text information in the frame information; Step 4, obtaining keyword information from the text information through a large language model; Step five: split the drawing to be split into two or more sub-drawings according to the drawing frame information, and the keyword information corresponds to the sub-drawings.

2. The drawing segmentation method based on computer vision and large language model according to claim 1, characterized in that: The step three of obtaining the text information in the frame information includes: Acquire the text in the text information in order from top to bottom and from left to right; Combine the text into sentences.

3. The drawing segmentation method based on computer vision and large language model as claimed in claim 2, characterized in that: The step 4 of obtaining keyword information in the text information through a large language model includes: The sentence is input into the large language model to obtain the keyword information.

4. The drawing segmentation method based on computer vision and large language model as claimed in claim 2, characterized in that: The text includes the project name, drawing title, drawing number, draftsman, reviewer, approver and the text in the drawing.

5. The drawing segmentation method based on computer vision and large language model according to claim 1, characterized in that: The step 4 of acquiring text information in the frame information through a large language model includes: The drawing frame information includes two or more drawing frames, the number of the drawing frames corresponds to the number of the sub-drawings, and the sub-drawings correspond to the keyword information.

6. The drawing segmentation method based on computer vision and large language model according to claim 5, characterized in that: The step 5, splitting the drawing to be split into two or more sub-drawings according to the drawing frame information, wherein the keyword information corresponds to the sub-drawings, comprises: The keyword information corresponds to the title of the sub-drawing.

7. The drawing segmentation method based on computer vision and large language model as claimed in claim 1, characterized in that: The method for establishing the computer vision model comprises the following steps: Build a neural network architecture model; Acquire a data set, wherein the data set is a drawing image containing drawing frame information; The drawing image with marked frame information is input into the neural network architecture model to perform model training to obtain the computer vision model.

8. A drawing segmentation system based on computer vision and large language model according to any one of claims 1 to 7, characterized in that: include: An image acquisition module is used to acquire image information of the drawing to be split; A frame information recognition module, used to recognize the frame information in the image information through a computer vision model; A drawing content acquisition module, used to acquire text information in the drawing frame information; A drawing content analysis module, used for obtaining keyword information from the text information through a large language model; The drawing splitting module is used to split the drawing to be split into two or more sub-drawings according to the drawing frame information, and the keyword information corresponds to the sub-drawings.

Images