Station wiring diagram link relation automatic identification method and system, medium and equipment

Abstract

The invention discloses a station wiring diagram link relation automatic identification method, system, medium and equipment, and belongs to the technical field of electrical engineering drawing digitization. The method comprises the steps that a large number of CAD station wiring images are acquired for text detection and identification, and a list containing a textbox and text content is generated; the method comprises the following steps: constructing a primitive category recognition data set and a primitive relationship recognition data set through a labeling tool, generating a language instruction data set and a CAD station wiring image instruction data set, constructing a primitive category fine-tuning data set and a primitive link relationship fine-tuning data set, and respectively training multi-modal large models for recognizing primitive categories and primitive relationships; and outputting the primitive type and the connection relationship between the primitives, and generating a power grid station wiring structure diagram. According to the method, the problems that the primitive category and the link relation in the CAD station wiring image cannot be accurately identified and the power grid station wiring structure diagram cannot be automatically generated in the prior art are solved.

Description

Technical Field

[0001] The invention relates to a method, system, medium and equipment for automatically identifying link relations of a station wiring diagram, and belongs to the technical field of digitalization of electrical engineering drawings. Background Art

[0002] Existing methods for identifying connection relationships in electrical drawings mainly rely on machine learning technology to initially identify line segments in drawings, usually through image processing and feature extraction technology, which can more accurately identify basic elements in drawings. After identifying the line segments, existing methods turn to using various non-machine learning and non-intelligent methods to construct the connection relationships between these line segments.

[0003] However, the above methods are often based on preset rules or templates, lacking flexibility and adaptability, and are difficult to cope with complex and changeable electrical drawings. There are several significant defects in the recognition of connection relationships in electrical drawings. First, the existing methods mainly rely on image processing technology, ignoring the rich text information and domain knowledge in the drawings, resulting in the inability to fully utilize all the information in the drawings when identifying the connection relationship, thus affecting the accuracy and completeness of the recognition. Secondly, electrical drawings usually contain complex connection relationships and a large amount of detailed information. Traditional methods are easily disturbed when facing complex scenes such as noise and occlusion, resulting in a decrease in recognition accuracy. In addition, it is difficult for traditional methods to deeply understand the complex logical topology and functional relationship between devices in electrical drawings, which limits their application in complex electrical systems. On the one hand, the existing methods mainly rely on image features for recognition, while ignoring the importance of domain knowledge, so that the recognition results often lack depth and accuracy. On the other hand, electrical drawings contain multiple types of information such as images, text, and symbols. Existing methods often find it difficult to effectively integrate these heterogeneous data, thus affecting the comprehensiveness and accuracy of recognition. These defects limit the application and development of existing methods in the recognition of connection relationships in electrical drawings.

[0004] In summary, the existing methods generally show the characteristics of insufficient data-driven capabilities and low intelligence. They are unable to accurately identify the types of graphics elements and link relationships in CAD station wiring images and automatically generate power grid station wiring structure diagrams. Summary of the invention

[0005] The purpose of the present invention is to provide a method, system, medium and equipment for automatically identifying link relationships in a station wiring diagram. By constructing a graphic element category recognition data set and a graphic element relationship recognition data set, further constructing a graphic element category fine-tuning data set and a graphic element connection relationship fine-tuning data set, training a multi-modal large model to generate a power grid station wiring structure diagram, so as to solve the problem that the prior art cannot accurately identify the graphic element category and link relationship in a CAD station wiring image, and automatically generate a power grid station wiring structure diagram.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] In a first aspect, the present invention provides a method for automatically identifying link relationships in a station wiring diagram, comprising: Acquire a CAD station wiring image to be identified, wherein the CAD station wiring image includes a plurality of electrical components and connecting wires; The CAD station wiring image to be identified is input into the multimodal large model for identifying the type of graphic element and the multimodal large model for identifying the relationship between graphic elements, respectively, and the graphic element type and the connection relationship between the graphic elements are output respectively to generate the power grid station wiring structure diagram; Among them, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories; Generate a second language instruction dataset and positive and negative CAD station wiring image instruction datasets based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multi-modal large model for identifying primitive relationships; The method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: Acquire a large number of CAD station wiring images for text detection and recognition, and generate a list including text boxes and text content, wherein the text box marks the position of the text content of each electrical component in the CAD station wiring image; According to the list including text boxes and text contents, a graphic element category recognition dataset and a graphic element relationship recognition dataset are constructed by using annotation tools.

[0008] Furthermore, the method also includes, after acquiring a large number of CAD station wiring images, performing image preprocessing on the CAD station wiring images including grayscale, Gaussian smoothing, sharpening and data enhancement operations.

[0009] Furthermore, after generating a list including text boxes and text contents, determining the accuracy of text detection and recognition, if the accuracy is lower than a preset accuracy threshold, annotating the text boxes and text contents and then fine-tuning them using an annotation platform, wherein the annotation platform includes a Label Studio annotation platform.

[0010] Furthermore, according to the list containing text boxes and text contents, a primitive category recognition dataset is constructed by a annotation tool, including: Input the CAD station wiring image that has undergone text detection and recognition into the annotation platform; On the annotation platform, the text boxes in the CAD station wiring image are associated with the corresponding graphic element categories to obtain a graphic element category recognition data set, where the graphic element categories include electrical component types.

[0011] Furthermore, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories, including: According to the graphic element category recognition data set, a first language instruction data set and a first CAD station wiring image instruction data set are generated; The first language instruction data set includes an input first language instruction and a first output language instruction, wherein the first input language instruction is a question asking about the category of the graphic element to be classified, and the first output language instruction is an answer including the category of the graphic element to be classified; The first CAD station wiring image instruction data set includes a first global image instruction and a first local image instruction; The first global image instruction is a global CAD station wiring image containing only one text box to be classified; The first local image instruction is a local CAD site wiring image centered on the text box to be classified, wherein the local CAD site wiring image centered on the text box to be classified can be randomly segmented; The instructions in the first language instruction data set and the first CAD site wiring image instruction data set are randomly paired to ensure that a group of instructions contains an input first language instruction, a first global image instruction and a first local image instruction, wherein the output of each group of instructions corresponds to a first output language instruction.

[0012] Furthermore, a second language instruction dataset and positive and negative CAD station wiring image instruction datasets are generated based on the primitive relationship recognition dataset, and a primitive link relationship fine-tuning dataset is constructed to train a multimodal large model for identifying primitive relationships, including: Generate a second language instruction dataset based on the graph element relationship recognition dataset; The second language instruction data set includes an input second language instruction and a second output language instruction, wherein the second input language instruction is a question asking whether the connection relationship between the graphic elements is correct, and the second output language instruction is an answer including the connection relationship between the graphic elements; According to the primitive relationship recognition data set, a second CAD station wiring image instruction data set is generated; The second CAD station wiring image instruction data set includes a second global image instruction and a second local image instruction; The second global image instruction is a global CAD station wiring image including two text boxes; The second local image instruction is a local CAD site wiring image centered on the text box, wherein the local CAD site wiring image centered on the text box can be randomly segmented; Randomly pair the instructions in the second language instruction data set and the second CAD station wiring image instruction data set to ensure that a group of instructions contains an input language instruction, a global image instruction and two local image instructions, wherein the output of each group of instructions corresponds to a second output language instruction, and the output value of the second output language instruction is "yes" or "no"; Among them, if the electrical components corresponding to the two text boxes are connected, the second CAD site wiring image instruction data set is a positive CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "yes"; if the electrical components corresponding to the two text boxes are not connected, the second CAD site wiring image instruction data set is a negative CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "no".

[0013] Furthermore, it also includes, before generating the power grid station wiring structure diagram, submitting the graphic element types output by the multimodal large model for identifying graphic element categories and the multimodal large model for identifying graphic element relationships and the connection relationships between graphic elements to the annotation review platform for manual review confirmation and correction, obtaining the corrected data set feedback from the annotation review platform, re-inputting the multimodal large model for identifying graphic element categories and the multimodal large model for identifying graphic element relationships, and using an iterative method to train the multimodal large model for identifying graphic element categories and the multimodal large model for identifying graphic element relationships.

[0014] In a second aspect, the present invention provides a station wiring diagram link relationship automatic identification system, comprising: A data acquisition module, used to acquire a CAD station wiring image to be identified, wherein the CAD station wiring image includes a plurality of electrical components and connecting wires; An image recognition module is used to input the CAD station wiring image to be recognized into the multimodal large model for recognizing the type of graphic element and the multimodal large model for recognizing the relationship between graphic elements for recognition; A result output module is used to output the type of graphic elements and the connection relationship between the graphic elements and generate a wiring structure diagram of a power grid station; Among them, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories; Generate a second language instruction dataset and positive and negative CAD station wiring image instruction datasets based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multi-modal large model for identifying primitive relationships; The method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: Acquire a large number of CAD station wiring images for text detection and recognition, and generate a list including text boxes and text content, wherein the text box marks the position of the text content of each electrical component in the CAD station wiring image; According to the list including text boxes and text contents, a graphic element category recognition dataset and a graphic element relationship recognition dataset are constructed by using annotation tools.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatically identifying link relationships of a station wiring diagram as described in the first aspect.

[0016] In a fourth aspect, the present invention provides a computer device, comprising: A memory for storing instructions; The processor is used to execute the instructions so that the device performs operations to implement the method for automatically identifying the link relationship of the site wiring diagram as described in the first aspect.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention realizes the automatic recognition of electrical components and their connection relationships in CAD field wiring images by constructing and training a multimodal large model for identifying graphic element categories and for identifying graphic element relationships. It does not need to rely on preset rules or templates, significantly improving the flexibility and adaptability of recognition. It can not only accurately identify the graphic element categories and connection relationships in complex electrical drawings, but also automatically generate power grid field wiring structure diagrams, providing an efficient and accurate tool for the design and maintenance of electrical systems; 2. The present invention adopts a strict labeling process when constructing the primitive category recognition dataset and the primitive relationship recognition dataset. First, a list containing text boxes and text content is generated through text detection and recognition, and its accuracy is judged. If the accuracy is not up to standard, manual labeling and fine-tuning are performed using a labeling platform such as Label Studio to ensure the accuracy and completeness of the data, which greatly improves the quality of the dataset and provides a strong guarantee for training efficient and accurate recognition models. At the same time, the recognition ability of the model is further optimized through random pairing and fine-tuning of the language instruction and image instruction datasets generated by the dataset; 3. Before generating the wiring structure diagram of the power grid station, the present invention also adds a manual review confirmation and correction link. Through the annotation review platform, the graphic element types output by the multimodal large model and the connection relationship between the graphic elements are manually reviewed to obtain the corrected data, which not only ensures the accuracy and reliability of the recognition results, but also inputs the corrected data set into the model again for training through an iterative method, continuously improving the recognition accuracy and generalization ability of the model, so that the method of the present invention can show stronger adaptability and robustness when dealing with complex and changeable electrical drawings; 4. The present invention realizes the fusion training of multimodal information by generating a first language instruction data set and a first CAD station wiring image instruction data set, and constructing a graphic element category fine-tuning data set. It not only makes full use of the visual features in the CAD station wiring image, but also integrates the semantic information in the language instructions, thereby significantly improving the recognition ability of the multimodal large model for identifying the graphic element type to the graphic element category; 5. The present invention realizes paired training of positive and negative samples by generating a second language instruction data set and positive and negative CAD site wiring image instruction data sets, and constructing a graphic element link relationship fine-tuning data set. This not only improves the recognition accuracy of the multimodal large model for identifying graphic element relationships for correct connection relationships, but also enhances the discrimination ability of the multimodal large model for identifying graphic element relationships for incorrect connection relationships, thereby improving the overall recognition accuracy and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure is a schematic flow chart of a method for automatically identifying link relationships of a station wiring diagram provided by an embodiment of the present invention; Figure 2 The figure is a flow chart of text detection and recognition of a conventional method provided by an embodiment of the present invention; Figure 3 FIG. 1 is a schematic diagram showing the working principle of a multi-modal large model for identifying primitive categories provided by an embodiment of the present invention; Figure 4 The figure is a schematic diagram of an application example of a multi-modal large model for identifying primitive categories provided by an embodiment of the present invention; Figure 5 The figure is a schematic diagram of the working principle of a multi-modal large model for identifying primitive relationships provided by an embodiment of the present invention; Figure 6 The figure is a schematic diagram of an application example of a multi-modal large model for identifying primitive relationships provided by an embodiment of the present invention; Figure 7 FIG. 1 is a schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0020] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship.

[0021] Example 1

[0022] like Figure 1 As shown, this embodiment introduces a method for automatically identifying link relationships of a station wiring diagram, including: Step 1: Obtain a CAD site wiring image to be identified, wherein the CAD site wiring image includes a plurality of electrical components and connecting wires.

[0023] Step 2: Input the CAD station wiring image to be identified into the multimodal large model for identifying the graphic element category and the multimodal large model for identifying the graphic element relationship for identification.

[0024] Step 2.1: Construct a primitive category recognition dataset and a primitive relationship recognition dataset.

[0025] In this embodiment, the method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: The flowchart of text detection and recognition of traditional methods is as follows Figure 2 As shown, this embodiment uses mature OCR tools such as PaddleOCR to perform text detection and recognition, including the following steps: A large number of CAD site wiring images are obtained for text detection and recognition, and a list including text boxes and text content is generated, wherein the text box marks the position of the text content of each electrical component in the CAD site wiring image.

[0026] In some embodiments, after acquiring a large number of CAD station wiring images, image preprocessing including grayscale, Gaussian smoothing, sharpening and data enhancement operations is performed on the CAD station wiring images.

[0027] Image preprocessing is an essential part of the digitization process. It removes noise and highlights edges through basic image processing such as grayscale, Gaussian smoothing, and sharpening, thereby improving subsequent recognition accuracy. In addition, data enhancement technology enhances the robustness of the model through random transformation and expansion of images, allowing the model to adapt to more image changes.

[0028] In some embodiments, according to the list including the text box and the text content, constructing a primitive category recognition dataset and a primitive relationship recognition dataset by using a annotation tool includes: Input the CAD station wiring image that has undergone text detection and recognition into the annotation platform; On the annotation platform, the text boxes in the CAD station wiring image are associated with the corresponding graphic element categories to obtain a graphic element category recognition data set, where the graphic element categories include electrical component types, including circuit breakers, busbars, transformers, etc.

[0029] In some embodiments, after generating a list including text boxes and text contents, determining the accuracy of text detection and recognition, if the accuracy is lower than a preset accuracy threshold, annotating the text boxes and text contents and then fine-tuning them using an annotation platform, wherein the annotation platform includes a Label Studio annotation platform.

[0030] Step 2.2: Generate a first language instruction dataset and a first CAD station wiring image instruction dataset based on the graphic element category recognition dataset, and construct a graphic element category fine-tuning dataset to train a multimodal large model for identifying graphic element categories. The working principle of the multimodal large model for identifying graphic element categories is as follows: Figure 3 shown.

[0031] Step 2.2.1: generating a first language instruction dataset and a first CAD station wiring image instruction dataset according to the graphic element category identification dataset; The first language instruction data set includes an input first language instruction and a first output language instruction, wherein the first input language instruction is a question asking about the category of the graphic element to be classified, and the first output language instruction is an answer including the category of the graphic element to be classified; The first CAD station wiring image instruction data set includes a first global image instruction and a first local image instruction; The first global image instruction is a global CAD station wiring image containing only one text box to be classified; The first local image instruction is a local CAD site wiring image centered on the text box to be classified, wherein the local CAD site wiring image centered on the text box to be classified can be randomly segmented; Step 2.2.2: Randomly pair the instructions in the first language instruction data set and the first CAD site wiring image instruction data set to ensure that a group of instructions contains an input first language instruction, a first global image instruction and a first local image instruction, wherein the output of each group of instructions corresponds to a first output language instruction.

[0032] The schematic diagram of the application example of the multi-modal large model for identifying the category of graphic elements provided in this embodiment is as follows: Figure 4 shown.

[0033] Step 2.3: Generate a second language instruction dataset and a positive and negative CAD station wiring image instruction dataset based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multimodal large model for identifying primitive relationships. The working principle of the multimodal large model for identifying primitive relationships is as follows: Figure 5 shown.

[0034] Step 2.3.1: Generate a second language instruction dataset based on the graph primitive relationship recognition dataset; The second language instruction data set includes an input second language instruction and a second output language instruction, wherein the second input language instruction is a question asking whether the connection relationship between the graphic elements is correct, and the second output language instruction is an answer including the connection relationship between the graphic elements; Step 2.3.2: Generate a second CAD station wiring image instruction dataset based on the primitive relationship recognition dataset; The second CAD station wiring image instruction data set includes a second global image instruction and a second local image instruction; The second global image instruction is a global CAD station wiring image including two text boxes; The second local image instruction is a local CAD site wiring image centered on the text box, wherein the local CAD site wiring image centered on the text box can be randomly segmented; Step 2.3.3: Randomly pair the instructions in the second language instruction data set and the second CAD station wiring image instruction data set to ensure that a group of instructions contains an input language instruction, a global image instruction and two local image instructions, wherein the output of each group of instructions corresponds to a second output language instruction, and the output value of the second output language instruction is "yes" or "no"; Among them, if the electrical components corresponding to the two text boxes are connected, the second CAD site wiring image instruction data set is a positive CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "yes"; if the electrical components corresponding to the two text boxes are not connected, the second CAD site wiring image instruction data set is a negative CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "no".

[0035] The schematic diagram of the application example of the multi-modal large model for identifying the relationship between graphic elements provided in this embodiment is as follows: Figure 6 shown.

[0036] Step 3: Output the graphic element types and the connection relationships between the graphic elements respectively and generate the power grid station wiring structure diagram.

[0037] In some embodiments, it also includes, before generating the power grid station wiring structure diagram, submitting the graphic element types and the connection relationships between the graphic elements output by the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships to the annotation review platform for manual review confirmation and correction, obtaining the corrected data set feedback from the annotation review platform, re-inputting the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships, and using an iterative method to train the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships.

[0038] In some embodiments, based on the correction data fed back by the annotation review platform, the multimodal large model is continuously optimized and fine-tuned to achieve data-driven model iteration. Through multiple iterations, the model continuously improves recognition accuracy and gradually achieves automatic recognition and accurate processing of complex wiring diagrams.

[0039] The power grid station wiring structure diagram provided by the present invention can provide dispatching and operation and maintenance personnel with accurate graphic element information and topological relationships, support real-time dispatching decisions and production management, and can be applied to specific scenarios such as equipment maintenance planning, operation mode changes, and fault analysis. Technical personnel in related fields can also use the present invention to combine the power grid station wiring structure diagram with specific business needs, support the intelligent construction of power plants / power grids, and apply it to specific scenarios such as power grid primary equipment information management, secondary circuit diagram analysis, and equipment ledger management.

[0040] Example 2

[0041] Based on the same inventive concept as Example 1, this embodiment introduces a station wiring diagram link relationship automatic identification system, including: A data acquisition module, used to acquire a CAD station wiring image to be identified, wherein the CAD station wiring image includes a plurality of electrical components and connecting wires; An image recognition module is used to input the CAD station wiring image to be recognized into the multimodal large model for recognizing the type of graphic element and the multimodal large model for recognizing the relationship between graphic elements for recognition; A result output module is used to output the type of graphic elements and the connection relationship between the graphic elements and generate a wiring structure diagram of a power grid station; Among them, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories; Generate a second language instruction dataset and positive and negative CAD station wiring image instruction datasets based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multi-modal large model for identifying primitive relationships; The method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: Acquire a large number of CAD station wiring images for text detection and recognition, and generate a list including text boxes and text content, wherein the text box marks the position of the text content of each electrical component in the CAD station wiring image; According to the list including text boxes and text contents, a graphic element category recognition dataset and a graphic element relationship recognition dataset are constructed by using annotation tools.

[0042] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.

[0043] Example 3

[0044] Based on the same inventive concept as other embodiments, Figure 1 The method shown, accordingly, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the big data-driven consumer preference analysis method of any of the above embodiments are implemented.

[0045] Example 4

[0046] Based on the same inventive concept as other embodiments, Figure 1 The method shown in the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Figure 7 As shown, the computer device may include a communication bus, a processor, a memory and a communication interface, and may also include an input / output interface and a display device, wherein each functional unit may communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the steps of the automatic identification method of the link relationship of the station wiring diagram described in the above embodiment.

[0047] In summary, the present invention realizes automatic recognition of electrical components and their connection relationships in CAD station wiring images by constructing and training a multimodal large model for identifying graphic element categories and for identifying graphic element relationships. There is no need to rely on preset rules or templates, which significantly improves the flexibility and adaptability of recognition. It can not only accurately identify the graphic element categories and connection relationships in complex electrical drawings, but also automatically generate power grid station wiring structure diagrams, providing an efficient and accurate tool for the design and maintenance of electrical systems.

[0048] The present invention adopts a strict annotation process when constructing the primitive category recognition dataset and the primitive relationship recognition dataset. First, a list containing text boxes and text content is generated through text detection and recognition, and its accuracy is judged. If the accuracy is not up to standard, manual annotation and fine-tuning are performed using an annotation platform such as Label Studio to ensure the accuracy and completeness of the data, greatly improving the quality of the dataset and providing a strong guarantee for training efficient and accurate recognition models. At the same time, the recognition ability of the model is further optimized through random pairing and fine-tuning of the language instruction and image instruction datasets generated by the dataset.

[0049] Before generating the power grid station wiring structure diagram, the present invention also adds a manual review confirmation and correction link. Through the annotation review platform, the graphic element types output by the multimodal large model and the connection relationship between the graphic elements are manually reviewed to obtain corrected data. This not only ensures the accuracy and reliability of the recognition results, but also re-inputs the corrected data set into the model for training through an iterative method, continuously improving the recognition accuracy and generalization ability of the model, so that the method of the present invention can show stronger adaptability and robustness when dealing with complex and changeable electrical drawings.

[0050] The present invention realizes the fusion training of multimodal information by generating a first language instruction dataset and a first CAD site wiring image instruction dataset, and constructing a graphic element category fine-tuning dataset. It not only makes full use of the visual features in the CAD site wiring image, but also integrates the semantic information in the language instructions, thereby significantly improving the recognition ability of the multimodal large model used to identify the graphic element type for the graphic element category.

[0051] The present invention realizes paired training of positive and negative samples by generating a second language instruction dataset and positive and negative CAD site wiring image instruction datasets, and constructing a graphic element link relationship fine-tuning dataset. This not only improves the recognition accuracy of the multimodal large model for identifying graphic element relationships for correct connection relationships, but also enhances the discrimination ability of the multimodal large model for identifying graphic element relationships for incorrect connection relationships, thereby improving the overall recognition accuracy and reliability.

[0052] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0054] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0056] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for automatically identifying link relationships in a station wiring diagram, characterized in that: include: Acquire a CAD station wiring image to be identified, wherein the CAD station wiring image includes a plurality of electrical components and connecting wires; The CAD station wiring image to be identified is input into the multimodal large model for identifying the type of graphic element and the multimodal large model for identifying the relationship between graphic elements, respectively, and the graphic element type and the connection relationship between the graphic elements are output respectively to generate the power grid station wiring structure diagram; Among them, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories; Generate a second language instruction dataset and positive and negative CAD station wiring image instruction datasets based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multi-modal large model for identifying primitive relationships; The method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: Acquire a large number of CAD station wiring images for text detection and recognition, and generate a list including text boxes and text content, wherein the text box marks the position of the text content of each electrical component in the CAD station wiring image; According to the list including text boxes and text contents, a graphic element category recognition dataset and a graphic element relationship recognition dataset are constructed by using annotation tools.

2. The method for automatically identifying link relationships of a station wiring diagram according to claim 1 is characterized in that: The method also includes, after acquiring a large number of CAD station wiring images, performing image preprocessing on the CAD station wiring images, including grayscale conversion, Gaussian smoothing, sharpening and data enhancement operations.

3. The method for automatically identifying link relationships of a station wiring diagram according to claim 1, characterized in that: It also includes, after generating a list including text boxes and text contents, determining the accuracy of text detection and recognition, and if the accuracy is lower than a preset accuracy threshold, annotating the text boxes and text contents and then fine-tuning them using an annotation platform, wherein the annotation platform includes a Label Studio annotation platform.

4. The method for automatically identifying link relationships of a station wiring diagram according to claim 1, characterized in that: According to the list containing text boxes and text content, a primitive category recognition dataset is constructed by using annotation tools, including: Input the CAD station wiring image that has undergone text detection and recognition into the annotation platform; On the annotation platform, the text boxes in the CAD station wiring image are associated with the corresponding graphic element categories to obtain a graphic element category recognition data set, where the graphic element categories include electrical component types.

5. The method for automatically identifying link relationships of a station wiring diagram according to claim 1, characterized in that: The first language instruction dataset and the first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and the graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories, including: According to the graphic element category recognition data set, a first language instruction data set and a first CAD station wiring image instruction data set are generated; The first language instruction data set includes an input first language instruction and a first output language instruction, wherein the first input language instruction is a question asking about the category of the graphic element to be classified, and the first output language instruction is an answer including the category of the graphic element to be classified; The first CAD station wiring image instruction data set includes a first global image instruction and a first local image instruction; The first global image instruction is a global CAD station wiring image containing only one text box to be classified; The first local image instruction is a local CAD site wiring image centered on the text box to be classified, wherein the local CAD site wiring image centered on the text box to be classified can be randomly segmented; The instructions in the first language instruction data set and the first CAD site wiring image instruction data set are randomly paired to ensure that a group of instructions contains an input first language instruction, a first global image instruction and a first local image instruction, wherein the output of each group of instructions corresponds to a first output language instruction.

6. The method for automatically identifying link relationships of a station wiring diagram according to claim 1, characterized in that: Based on the primitive relationship recognition dataset, a second language instruction dataset and positive and negative CAD station wiring image instruction datasets are generated, and a primitive link relationship fine-tuning dataset is constructed to train a multimodal large model for identifying primitive relationships, including: Generate a second language instruction dataset based on the graph element relationship recognition dataset; The second language instruction data set includes an input second language instruction and a second output language instruction, wherein the second input language instruction is a question asking whether the connection relationship between the graphic elements is correct, and the second output language instruction is an answer including the connection relationship between the graphic elements; According to the primitive relationship recognition data set, a second CAD station wiring image instruction data set is generated; The second CAD station wiring image instruction data set includes a second global image instruction and a second local image instruction; The second global image instruction is a global CAD station wiring image including two text boxes; The second local image instruction is a local CAD site wiring image centered on the text box, wherein the local CAD site wiring image centered on the text box can be randomly segmented; Randomly pair the instructions in the second language instruction data set and the second CAD station wiring image instruction data set to ensure that a group of instructions contains an input language instruction, a global image instruction and two local image instructions, wherein the output of each group of instructions corresponds to a second output language instruction, and the output value of the second output language instruction is "yes" or "no"; Among them, if the electrical components corresponding to the two text boxes are connected, the second CAD site wiring image instruction data set is a positive CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "yes"; if the electrical components corresponding to the two text boxes are not connected, the second CAD site wiring image instruction data set is a negative CAD site wiring image instruction data set, and the corresponding second output language instruction output value is "no".

7. The method for automatically identifying link relationships of a station wiring diagram according to claim 1, characterized in that: It also includes, before generating the power grid station wiring structure diagram, submitting the graphic element types and the connection relationships between the graphic elements output by the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships to the annotation review platform for manual review, confirmation and correction, obtaining the corrected data set feedback from the annotation review platform, re-inputting the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships, and using an iterative method to train the multimodal large model for identifying the graphic element categories and the multimodal large model for identifying the graphic element relationships.

8. A station wiring diagram link relationship automatic identification system, characterized in that: include: A data acquisition module, used to acquire a CAD station wiring image to be identified, wherein the CAD station wiring image includes a plurality of electrical components and connecting wires; An image recognition module is used to input the CAD station wiring image to be recognized into the multimodal large model for recognizing the type of graphic element and the multimodal large model for recognizing the relationship between graphic elements for recognition; A result output module is used to output the type of graphic elements and the connection relationship between the graphic elements and generate a wiring structure diagram of a power grid station; Among them, a first language instruction dataset and a first CAD station wiring image instruction dataset are generated according to the graphic element category recognition dataset, and a graphic element category fine-tuning dataset is constructed to train a multimodal large model for identifying graphic element categories; Generate a second language instruction dataset and positive and negative CAD station wiring image instruction datasets based on the primitive relationship recognition dataset, and construct a primitive link relationship fine-tuning dataset to train a multi-modal large model for identifying primitive relationships; The method for constructing the primitive category recognition dataset and the primitive relationship recognition dataset includes: Acquire a large number of CAD station wiring images for text detection and recognition, and generate a list including text boxes and text content, wherein the text box marks the position of the text content of each electrical component in the CAD station wiring image; According to the list including text boxes and text contents, a graphic element category recognition dataset and a graphic element relationship recognition dataset are constructed by using annotation tools.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for automatically identifying link relationships of a station wiring diagram as described in any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device performs the operation of the method for automatically identifying the link relationship of the station wiring diagram as described in any one of claims 1 to 7.

Images